模型转换报错 qwen15 32

![输入图片说明](https://foruda.gitee.com/images/1720167359046387950/efafda6f_8539033.png "屏幕截图")

Traceback (most recent call last):
  File "/usr/local/python3.8.19/lib/python3.8/site-packages/transformers/utils/import_utils.py", line 1390, in _get_module
Loader exited, exiting saver
    return importlib.import_module("." + module_name, self.__name__)
  File "/usr/local/python3.8.19/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/python3.8.19/lib/python3.8/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 48, in <module>
    if is_flash_attn_2_available():
  File "/usr/local/python3.8.19/lib/python3.8/site-packages/transformers/utils/import_utils.py", line 643, in is_flash_attn_2_available
    if not _is_package_available("flash_attn"):
  File "/usr/local/python3.8.19/lib/python3.8/site-packages/transformers/utils/import_utils.py", line 43, in _is_package_available
    package_exists = importlib.util.find_spec(pkg_name) is not None
  File "/usr/local/python3.8.19/lib/python3.8/importlib/util.py", line 114, in find_spec
    raise ValueError('{}.__spec__ is None'.format(name))
ValueError: flash_attn.__spec__ is None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "tools/checkpoint/convert_ckpt.py", line 99, in <module>
    main()
  File "tools/checkpoint/convert_ckpt.py", line 92, in main
    loader.load_checkpoint(queue, args)
  File "/job/code/tools/checkpoint/loader_llama2_hf.py", line 437, in load_checkpoint
    _load_checkpoint(queue, args)
  File "/job/code/tools/checkpoint/loader_llama2_hf.py", line 331, in _load_checkpoint
    model = load_checkpoint_to_model(margs)
  File "/job/code/tools/checkpoint/loader_llama2_hf.py", line 187, in load_checkpoint_to_model
    hf_model = AutoModelForCausalLM.from_pretrained(args.load, device_map="cpu", trust_remote_code=True)
  File "/usr/local/python3.8.19/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 560, in from_pretrained
    model_class = _get_model_class(config, cls._model_mapping)
  File "/usr/local/python3.8.19/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 381, in _get_model_class
    supported_models = model_mapping[type(config)]
  File "/usr/local/python3.8.19/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 732, in __getitem__
    return self._load_attr_from_module(model_type, model_name)
  File "/usr/local/python3.8.19/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 746, in _load_attr_from_module
    return getattribute_from_module(self._modules[module_name], attr)
  File "/usr/local/python3.8.19/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 690, in getattribute_from_module
    if hasattr(module, attr):
  File "/usr/local/python3.8.19/lib/python3.8/site-packages/transformers/utils/import_utils.py", line 1380, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/python3.8.19/lib/python3.8/site-packages/transformers/utils/import_utils.py", line 1392, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.qwen2.modeling_qwen2 because of the following error (look up to see its traceback):
flash_attn.__spec__ is None
/usr/local/python3.8.19/lib/python3.8/tempfile.py:827: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpnipqney3'>
  _warnings.warn(warn_message, ResourceWarning)

命令：
python3 tools/checkpoint/convert_ckpt.py \
       --model-type GPT \
       --loader llama2_hf \
       --saver megatron \
       --target-tensor-parallel-size 8 \
       --target-pipeline-parallel-size 4 \
       --num-layers-per-virtual-pipeline-stage 2 \
       --params-dtype bf16 \
       --load-dir ./model_from_hf/Qwen1.5-32B/ \
       --save-dir ./model_weights/Qwen1.5-32B-v0.1-tp8-pp4-vpp2/ \
       --tokenizer-model ./model_from_hf/Qwen1.5-32B/tokenizer.json \
       --add-qkv-bias

Ascend/MindSpeed-LLM
暂停

内容风险标识

评论 (3)

Ascend/MindSpeed-LLM暂停 .gitee-modal { width: 500px !important; }

内容风险标识