# gpt4-x-alpaca-13b-native-4bit-128g-cuda

**Repository Path**: modelee/gpt4-x-alpaca-13b-native-4bit-128g-cuda

## Basic Information

- **Project Name**: gpt4-x-alpaca-13b-native-4bit-128g-cuda
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2023-05-23
- **Last Updated**: 2024-04-02

## Categories & Tags

**Categories**: llm

**Tags**: None

## README

Quant of https://huggingface.co/chavinlo/gpt4-x-alpaca

There's already one located at https://huggingface.co/anon8231489123/gpt4-x-alpaca-13b-native-4bit-128g, but neither the triton nor cuda version they uploaded seem to want to work on older versions of GPTQ-for-LLaMA such as the one currently used with KoboldAI for 4bit support on 0cc4m's fork.

This was quantized with cuda, not triton.

python llama.py ./gpt4-x-alpaca c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors gpt-x-alpaca-13b-native-4bit-128g-cuda.safetensors