quant_qwen_moe_w8a8.py 量化出的qwen3 30b a3b vllm 无法使用

一、问题现象（附报错日志上下文）：
使用 quant_qwen_moe_w8a8.py 量化后的Qwen3-30b-a3b 使用vllm 启动 出现下列错误：
Loading safetensors checkpoint shards:   0% Completed | 0/8 [00:00<?, ?it/s]
ERROR 07-28 11:41:51 [engine.py:458] 'layers.0.mlp.experts.w2_weight_offset'
ERROR 07-28 11:41:51 [engine.py:458] Traceback (most recent call last):
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/engine/multiprocessing/engine.py", line 446, in run_mp_engine
ERROR 07-28 11:41:51 [engine.py:458]     engine = MQLLMEngine.from_vllm_config(
ERROR 07-28 11:41:51 [engine.py:458]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/engine/multiprocessing/engine.py", line 133, in from_vllm_config
ERROR 07-28 11:41:51 [engine.py:458]     return cls(
ERROR 07-28 11:41:51 [engine.py:458]            ^^^^
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/engine/multiprocessing/engine.py", line 87, in __init__
ERROR 07-28 11:41:51 [engine.py:458]     self.engine = LLMEngine(*args, **kwargs)
ERROR 07-28 11:41:51 [engine.py:458]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 265, in __init__
ERROR 07-28 11:41:51 [engine.py:458]     self.model_executor = executor_class(vllm_config=vllm_config)
ERROR 07-28 11:41:51 [engine.py:458]                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/executor/executor_base.py", line 287, in __init__
ERROR 07-28 11:41:51 [engine.py:458]     super().__init__(*args, **kwargs)
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/executor/executor_base.py", line 53, in __init__
ERROR 07-28 11:41:51 [engine.py:458]     self._init_executor()
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/executor/mp_distributed_executor.py", line 126, in _init_executor
ERROR 07-28 11:41:51 [engine.py:458]     self._run_workers("load_model",
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/executor/mp_distributed_executor.py", line 186, in _run_workers
ERROR 07-28 11:41:51 [engine.py:458]     driver_worker_output = run_method(self.driver_worker, sent_method,
ERROR 07-28 11:41:51 [engine.py:458]                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/utils.py", line 2674, in run_method
ERROR 07-28 11:41:51 [engine.py:458]     return func(*args, **kwargs)
ERROR 07-28 11:41:51 [engine.py:458]            ^^^^^^^^^^^^^^^^^^^^^
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib64/python3.11/site-packages/vllm_ascend/worker/worker.py", line 240, in load_model
ERROR 07-28 11:41:51 [engine.py:458]     self.model_runner.load_model()
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib64/python3.11/site-packages/vllm_ascend/worker/model_runner.py", line 997, in load_model
ERROR 07-28 11:41:51 [engine.py:458]     self.model = get_model(vllm_config=self.vllm_config)
ERROR 07-28 11:41:51 [engine.py:458]                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/model_loader/__init__.py", line 59, in get_model
ERROR 07-28 11:41:51 [engine.py:458]     return loader.load_model(vllm_config=vllm_config,
ERROR 07-28 11:41:51 [engine.py:458]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/model_loader/base_loader.py", line 41, in load_model
ERROR 07-28 11:41:51 [engine.py:458]     self.load_weights(model, model_config)
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/model_loader/default_loader.py", line 273, in load_weights
ERROR 07-28 11:41:51 [engine.py:458]     loaded_weights = model.load_weights(
ERROR 07-28 11:41:51 [engine.py:458]                      ^^^^^^^^^^^^^^^^^^^
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_moe.py", line 535, in load_weights
ERROR 07-28 11:41:51 [engine.py:458]     return loader.load_weights(weights)
ERROR 07-28 11:41:51 [engine.py:458]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 278, in load_weights
ERROR 07-28 11:41:51 [engine.py:458]     autoloaded_weights = set(self._load_module("", self.module, weights))
ERROR 07-28 11:41:51 [engine.py:458]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 236, in _load_module
ERROR 07-28 11:41:51 [engine.py:458]     yield from self._load_module(prefix,
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 209, in _load_module
ERROR 07-28 11:41:51 [engine.py:458]     loaded_params = module_load_weights(weights)
ERROR 07-28 11:41:51 [engine.py:458]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 07-28 11:41:51 [engine.py:458]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_moe.py", line 440, in load_weights
ERROR 07-28 11:41:51 [engine.py:458]     param = params_dict[name]
ERROR 07-28 11:41:51 [engine.py:458]             ~~~~~~~~~~~^^^^^^
ERROR 07-28 11:41:51 [engine.py:458] KeyError: 'layers.0.mlp.experts.w2_weight_offset'
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239] Exception in worker VllmWorkerProcess while processing method load_model.
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239] Traceback (most recent call last):
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]   File "/usr/local/lib/python3.11/site-packages/vllm/executor/multiproc_worker_utils.py", line 233, in _run_worker_process
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]     output = run_method(worker, method, args, kwargs)
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]   File "/usr/local/lib/python3.11/site-packages/vllm/utils.py", line 2674, in run_method
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]     return func(*args, **kwargs)
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]            ^^^^^^^^^^^^^^^^^^^^^
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]   File "/usr/local/lib64/python3.11/site-packages/vllm_ascend/worker/worker.py", line 240, in load_model
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]     self.model_runner.load_model()
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]   File "/usr/local/lib64/python3.11/site-packages/vllm_ascend/worker/model_runner.py", line 997, in load_model
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]     self.model = get_model(vllm_config=self.vllm_config)
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/model_loader/__init__.py", line 59, in get_model
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]     return loader.load_model(vllm_config=vllm_config,
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/model_loader/base_loader.py", line 41, in load_model
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]     self.load_weights(model, model_config)
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/model_loader/default_loader.py", line 273, in load_weights
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]     loaded_weights = model.load_weights(
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]                      ^^^^^^^^^^^^^^^^^^^
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_moe.py", line 535, in load_weights
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]     return loader.load_weights(weights)
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 278, in load_weights
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]     autoloaded_weights = set(self._load_module("", self.module, weights))
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 236, in _load_module
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]     yield from self._load_module(prefix,
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 209, in _load_module
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]     loaded_params = module_load_weights(weights)
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]   File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_moe.py", line 440, in load_weights
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]     param = params_dict[name]
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239]             ~~~~~~~~~~~^^^^^^
(VllmWorkerProcess pid=6059) ERROR 07-28 11:41:51 [multiproc_worker_utils.py:239] KeyError: 'layers.0.mlp.experts.w2_weight_offset'
ERROR 07-28 11:41:53 [multiproc_worker_utils.py:121] Worker VllmWorkerProcess pid 6059 died, exit code: -15
INFO 07-28 11:41:53 [multiproc_worker_utils.py:125] Killing local vLLM worker processes
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/usr/lib64/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib64/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.11/site-packages/vllm/engine/multiprocessing/engine.py", line 460, in run_mp_engine
    raise e from None
  File "/usr/local/lib/python3.11/site-packages/vllm/engine/multiprocessing/engine.py", line 446, in run_mp_engine
    engine = MQLLMEngine.from_vllm_config(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/vllm/engine/multiprocessing/engine.py", line 133, in from_vllm_config
    return cls(
           ^^^^
  File "/usr/local/lib/python3.11/site-packages/vllm/engine/multiprocessing/engine.py", line 87, in __init__
    self.engine = LLMEngine(*args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 265, in __init__
    self.model_executor = executor_class(vllm_config=vllm_config)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/vllm/executor/executor_base.py", line 287, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.11/site-packages/vllm/executor/executor_base.py", line 53, in __init__
    self._init_executor()
  File "/usr/local/lib/python3.11/site-packages/vllm/executor/mp_distributed_executor.py", line 126, in _init_executor
    self._run_workers("load_model",
  File "/usr/local/lib/python3.11/site-packages/vllm/executor/mp_distributed_executor.py", line 186, in _run_workers
    driver_worker_output = run_method(self.driver_worker, sent_method,
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/vllm/utils.py", line 2674, in run_method
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/vllm_ascend/worker/worker.py", line 240, in load_model
    self.model_runner.load_model()
  File "/usr/local/lib64/python3.11/site-packages/vllm_ascend/worker/model_runner.py", line 997, in load_model
    self.model = get_model(vllm_config=self.vllm_config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/model_loader/__init__.py", line 59, in get_model
    return loader.load_model(vllm_config=vllm_config,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/model_loader/base_loader.py", line 41, in load_model
    self.load_weights(model, model_config)
  File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/model_loader/default_loader.py", line 273, in load_weights
    loaded_weights = model.load_weights(
                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_moe.py", line 535, in load_weights
    return loader.load_weights(weights)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 278, in load_weights
    autoloaded_weights = set(self._load_module("", self.module, weights))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 236, in _load_module
    yield from self._load_module(prefix,
  File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/utils.py", line 209, in _load_module
    loaded_params = module_load_weights(weights)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/qwen3_moe.py", line 440, in load_weights
    param = params_dict[name]
            ~~~~~~~~~~~^^^^^^
KeyError: 'layers.0.mlp.experts.w2_weight_offset'
Loading safetensors checkpoint shards:   0% Completed | 0/8 [00:02<?, ?it/s]

二、软件版本:
-- CANN 版本 (e.g., CANN 3.0.x，5.x.x):  
--Tensorflow/Pytorch/MindSpore 版本:
--Python 版本 (e.g., Python 3.7.5):
-- MindStudio版本 (e.g., MindStudio 2.0.0 (beta3)):
--操作系统版本 (e.g., Ubuntu 18.04):

三、测试步骤：
xxxx

四、日志信息:
xxxx
请根据自己的运行环境参考以下方式搜集日志信息，如果涉及到算子开发相关的问题，建议也提供UT/ST测试和单算子集成测试相关的日志。

日志提供方式:
将日志打包后作为附件上传。若日志大小超出附件限制，则可上传至外部网盘后提供链接。

获取方法请参考wiki：
https://gitee.com/ascend/modelzoo/wikis/%E5%A6%82%E4%BD%95%E8%8E%B7%E5%8F%96%E6%97%A5%E5%BF%97%E5%92%8C%E8%AE%A1%E7%AE%97%E5%9B%BE?sort_id=4097825

Ascend/msit
暂停

内容风险标识

评论 (2)

Ascend/msit暂停 .gitee-modal { width: 500px !important; }

内容风险标识