deepseek-r1量化执行quant_deepseek_w4a8.py报错Float8_e4m3fn has not been supported

一、问题现象（附报错日志上下文）：
File "/root/.cache/huggingface/modules/transformers_modules/DeepSeek-R1/modeling_deepseeky", line 769, in forward
    q = self.q_b_proj(self.q_a_layernorm(self.q_a_proj(hidden_states)))
  File "/home/conda/envs/vllm_0.8.5/lib/python3.10/site-packages/torch/nn/modules/module. line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/conda/envs/vllm_0.8.5/lib/python3.10/site-packages/torch/nn/modules/module. line 1844, in _call_impl
    return inner()
  File "/home/conda/envs/vllm_0.8.5/lib/python3.10/site-packages/torch/nn/modules/module. line 1790, in inner
    result = forward_call(*args, **kwargs)
  File "/home/conda/envs/vllm_0.8.5/lib/python3.10/site-packages/accelerate/hooks.py", li75, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/home/conda/envs/vllm_0.8.5/lib/python3.10/site-packages/torch/nn/modules/linear. line 125, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: Float8_e4m3fn has not been supported
[ERROR] 2025-06-23-21:16:12 (PID:54987, Device:0, RankID:-1) ERR01007 OPS feature not suped

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/wzn/msit/msmodelslim/example/DeepSeek/quant_deepseek_w4a8.py", line 222, indule>
    main()
  File "/home/wzn/msit/msmodelslim/example/DeepSeek/quant_deepseek_w4a8.py", line 163, inn
    anti_outlier.process()
  File "/home/conda/envs/vllm_0.8.5/lib/python3.10/site-packages/torch/utils/_contextlib. line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/conda/envs/vllm_0.8.5/lib/python3.10/site-packages/msmodelslim/pytorch/llm_anti_outlier/anti_outlier.py", line 411, in process
    raise Exception("Please check your config, model and input!", e) from e
Exception: ('Please check your config, model and input!', RuntimeError('Float8_e4m3fn has been supported\n[ERROR] 2025-06-23-21:16:12 (PID:54987, Device:0, RankID:-1) ERR01007 OPature not supported'))
Total Process:  20%|█████▍                     | 1/5 [2:05:46<8:23:07, 7546.84s/it]

二、软件版本:
-- CANN 版本 (e.g., CANN 3.0.x，5.x.x):  8.1.RC1
--Tensorflow/Pytorch/MindSpore 版本: 2.5.1
--Python 版本 (e.g., Python 3.7.5): 3.10.18
-- MindStudio版本 (e.g., MindStudio 2.0.0 (beta3)):
--操作系统版本 (e.g., Ubuntu 18.04): openEuler release 22.03 (LTS-SP4)

环境上用的vLLM 0.8.5和vllm-ascend 0.8.5rc1。没有找到openEuler22.03的mindie镜像，所以直接跑的脚本

三、测试步骤：
按照README注释掉modeling_deepseek.py、config.json中的代码，执行脚本python3 quant_deepseek_w4a8.py --model_path /mnt/lustre/DeepSeek-R1 --save_path /mnt/lustre/DeepSeek-R1-w4a8/。
第一步转bf16完成之后报的错。

Ascend/msit
暂停

内容风险标识

评论 (2)

Ascend/msit暂停 .gitee-modal { width: 500px !important; }

内容风险标识