将tensor parallel大于1的megatron模型转换为huggingFace报错

一、问题现象（附报错日志上下文）：
Process Process-1:
Traceback (most recent call last):
  File "/home/miniconda3/envs/ml/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/miniconda3/envs/ml/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/modellink/ModelLink/tools/checkpoint/saver_megatron.py", line 645, in save_model_checkpoint
    save_huggingface_llama(args, models[tp_rank], md)
  File "/home/modellink/ModelLink/tools/checkpoint/saver_megatron.py", line 145, in save_huggingface_llama
    name_param_h[1].data.copy_(hf2mg_map[name_param_h[0]])
RuntimeError: The size of tensor a (32000) must match the size of tensor b (4000) at non-singleton dimension 0

二、软件版本:
-- CANN 版本 (e.g., CANN 3.0.x，5.x.x):  7.0.RC1.10
--Tensorflow/Pytorch/MindSpore 版本:  Pytorch2.1.0
--Python 版本 (e.g., Python 3.7.5):  python3.8
-- MindStudio版本 (e.g., MindStudio 2.0.0 (beta3)):
--操作系统版本 (e.g., Ubuntu 18.04):  4.19.90-vhulk2211.3.0.h1543.eulerosv2r10.aarch64

三、测试步骤：
1. 将llama-7b-hf转换为megatron
python tools/checkpoint/convert_ckpt.py --model-type GPT --loader llama2_hf --saver megatron --target-tensor-parallel-size 8 --target-pipeline-parallel-size 1 --save-dir /home/model_weights/llama-7b-tp8-pp1 --load-dir /home/model_from_hf/llama-7b-hf --tokenizer-model /home/model_from_hf/llama-7b-hf/tokenizer.model

2. 将转换后的megatron重新转为llama-7b-hf
python tools/checkpoint/convert_ckpt.py --model-type GPT --loader megatron --saver megatron --save-model-type save_huggingface_llama --load-dir /home/model_weights/llama-7b-tp8-pp1/ --target-tensor-parallel-size 8 --target-pipeline-parallel-size 1 --save-dir /home/model_from_hf/llama-7b-hf

四、日志信息:
Process Process-1:
Traceback (most recent call last):
  File "/home/miniconda3/envs/ml/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/miniconda3/envs/ml/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/modellink/ModelLink/tools/checkpoint/saver_megatron.py", line 645, in save_model_checkpoint
    save_huggingface_llama(args, models[tp_rank], md)
  File "/home/modellink/ModelLink/tools/checkpoint/saver_megatron.py", line 145, in save_huggingface_llama
    name_param_h[1].data.copy_(hf2mg_map[name_param_h[0]])
RuntimeError: The size of tensor a (32000) must match the size of tensor b (4000) at non-singleton dimension 0

Ascend/MindSpeed-LLM
暂停

内容风险标识

评论 (3)

Ascend/MindSpeed-LLM暂停 .gitee-modal { width: 500px !important; }

内容风险标识