atb_speed_sdk baichuan2——13B-chat模型加载报错

一、问题现象（附报错日志上下文）：
atb_speed_sdk baichuan2——13B-chat模型加载报错

二、软件版本:
-- CANN 版本 (8.0.T5):  
--Python 版本 （3.10):
--操作系统版本 (ky v10):

三、测试步骤：

https://gitee.com/ascend/ModelLink/blob/4db99bfd0c929c4a6fe63da96da66b698f063635/mindie_ref/mindie_llm/atb_models/pytorch/examples/baichuan2/13b/README.md

四、日志信息:
/opt/atb-models/pytorch/examples/baichuan2/13b/main.py:203: DeprecationWarning: The section_name [ceval] is deprecated, please refer to readme and use [precision] instead
  atb_speed_config.init_config("config.ini")
lspci: Unable to load libkmod resources: error -2
2024-03-29 01:27:12,228 [INFO] pid: 692 cpu_binding.py-142: rank_id: 0, device_id: 2, numa_id: 4, shard_devices: [2], cpus: [96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119]
2024-03-29 01:27:12,230 [INFO] pid: 692 cpu_binding.py-167: process 692, new_affinity is [96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119], cpu count 24
2024-03-29 01:27:12,231 [INFO] pid: 692 npu.py-100: Bind cpu successfully!
/home/weight/baichuan2_13B_chat
Loading checkpoint shards:   0%|                                                                     | 0/3 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/opt/atb-models/pytorch/examples/baichuan2/13b/main.py", line 218, in <module>
    main()
  File "/opt/atb-models/pytorch/examples/baichuan2/13b/main.py", line 213, in main
    launcher = BaichuanLM()
  File "/usr/local/python3.10.9/lib/python3.10/site-packages/atb_speed/common/launcher/npu.py", line 44, in __init__
    super().__init__(device_ids, model_path, options)
  File "/usr/local/python3.10.9/lib/python3.10/site-packages/atb_speed/common/launcher/base.py", line 68, in __init__
    self.model, self.tokenizer = self.init_model()
  File "/opt/atb-models/pytorch/examples/baichuan2/13b/main.py", line 127, in init_model
    model = AutoModelForCausalLM.from_pretrained(self.model_path, trust_remote_code=True).half().to(self._device)
  File "/usr/local/python3.10.9/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 558, in from_pretrained
    return model_class.from_pretrained(
  File "/root/.cache/huggingface/modules/transformers_modules/baichuan2_13B_chat/modeling_baichuan_ascend.py", line 785, in from_pretrained
    return super(BaichuanForCausalLM, cls).from_pretrained(pretrained_model_name_or_path, *model_args,
  File "/usr/local/python3.10.9/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3175, in from_pretrained
    ) = cls._load_pretrained_model(
  File "/usr/local/python3.10.9/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3548, in _load_pretrained_model
    state_dict = load_state_dict(shard_file)
  File "/usr/local/python3.10.9/lib/python3.10/site-packages/transformers/modeling_utils.py", line 467, in load_state_dict
    with safe_open(checkpoint_file, framework="pt") as f:
safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge

Ascend/MindSpeed-LLM
Paused

Content Risk Flag

Comments (2)

Ascend/MindSpeed-LLMPaused .gitee-modal { width: 500px !important; }

Content Risk Flag