# LLAMA_TOOLS

**Repository Path**: yhyu13/llama_-tools

## Basic Information

- **Project Name**: LLAMA_TOOLS
- **Description**: LLaMA 工具和脚本
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 3
- **Forks**: 0
- **Created**: 2023-05-12
- **Last Updated**: 2023-12-22

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

Submodules

https://github.com/yhyu13/GPTQ-for-LLaMa.git
https://github.com/shawwn/llama-dl.git
https://github.com/ggerganov/llama.cpp.git
https://github.com/PanQiWei/AutoGPTQ.git
https://github.com/casper-hansen/AutoAWQ.git
https://github.com/Cornell-RelaxML/quip-sharp.git


# LLaMA original weights

[13B]https://huggingface.co/decapoda-research/llama-13b-hf/tree/main
[30B]https://huggingface.co/decapoda-research/llama-13b-hf/tree/main

Just one caveat thing

> CAUTION : you need to replace LlamaTokenizer in tokenizer config json into LlamaTokenizer in the original weight repo, otherwise the pypl transformers would fail to read the original weight ([see issue here](https://github.com/huggingface/transformers/issues/22222))

# Convert to HF Scripts

convert_llama_weights_to_hf.py

https://github.com/huggingface/transformers/blob/28f26c107b4a1c5c7e32ed4d9575622da0627a40/src/transformers/models/llama/convert_llama_weights_to_hf.py

# Combo Scripts

> CAUTION : you need to use the correct script with corresponding delta weights, there is generally no combo script fits all. Every model release their own combo script.

fastchat (vicuna):

https://raw.githubusercontent.com/lm-sys/FastChat/main/fastchat/model/apply_delta.py

stable-vicuna:

https://huggingface.co/CarperAI/stable-vicuna-13b-delta/raw/main/apply_delta.py

LLMZoo (chimera):

https://raw.githubusercontent.com/FreedomIntelligence/LLMZoo/main/tools/apply_delta.py

## Usage

```
conda env create -n llama_tool -f llama_tool.yml
conda activate llama_tool

# code for autogptq a model
./call_autogptq.sh

# code for quip sharp a model
./call_quip_sharp.sh
```