msmodelslim对qwen3-32B进行w8a8量化后，使用vllm-ascned部署报错

一、问题现象（附报错日志上下文）：
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487] WorkerProc failed to start.
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487] Traceback (most recent call last):
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]   File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 461, in worker_main
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]     worker = WorkerProc(*args, **kwargs)
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]   File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 358, in __init__
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]     self.worker.load_model()
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/worker_v1.py", line 240, in load_model
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]     self.model_runner.load_model()
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 1748, in load_model
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]     self.model = get_model(vllm_config=self.vllm_config)
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]   File "/vllm-workspace/vllm/vllm/model_executor/model_loader/__init__.py", line 59, in get_model
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]     return loader.load_model(vllm_config=vllm_config,
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]   File "/vllm-workspace/vllm/vllm/model_executor/model_loader/base_loader.py", line 41, in load_model
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]     self.load_weights(model, model_config)
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]   File "/vllm-workspace/vllm/vllm/model_executor/model_loader/default_loader.py", line 269, in load_weights
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]     loaded_weights = model.load_weights(
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]   File "/vllm-workspace/vllm/vllm/model_executor/models/qwen3.py", line 322, in load_weights
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]     return loader.load_weights(weights)
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]   File "/vllm-workspace/vllm/vllm/model_executor/models/utils.py", line 291, in load_weights
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]     autoloaded_weights = set(self._load_module("", self.module, weights))
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]   File "/vllm-workspace/vllm/vllm/model_executor/models/utils.py", line 249, in _load_module
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]     yield from self._load_module(prefix,
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]   File "/vllm-workspace/vllm/vllm/model_executor/models/utils.py", line 222, in _load_module
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]     loaded_params = module_load_weights(weights)
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]   File "/vllm-workspace/vllm/vllm/model_executor/models/qwen2.py", line 417, in load_weights
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487]     param = params_dict[name]
(VllmWorker rank=0 pid=2259) ERROR 08-18 13:13:29 [multiproc_executor.py:487] KeyError: 'layers.46.mlp.down_proj.deq_scale'
Loading safetensors checkpoint shards:   0% Completed | 0/9 [00:01<?, ?it/s]
(VllmWorker rank=0 pid=2259) 
ERROR 08-18 13:13:33 [core.py:586] EngineCore failed to start.
ERROR 08-18 13:13:33 [core.py:586] Traceback (most recent call last):
ERROR 08-18 13:13:33 [core.py:586]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 577, in run_engine_core
ERROR 08-18 13:13:33 [core.py:586]     engine_core = EngineCoreProc(*args, **kwargs)
ERROR 08-18 13:13:33 [core.py:586]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 404, in __init__
ERROR 08-18 13:13:33 [core.py:586]     super().__init__(vllm_config, executor_class, log_stats,
ERROR 08-18 13:13:33 [core.py:586]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 75, in __init__
ERROR 08-18 13:13:33 [core.py:586]     self.model_executor = executor_class(vllm_config)
ERROR 08-18 13:13:33 [core.py:586]   File "/vllm-workspace/vllm/vllm/executor/executor_base.py", line 53, in __init__
ERROR 08-18 13:13:33 [core.py:586]     self._init_executor()
ERROR 08-18 13:13:33 [core.py:586]   File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 93, in _init_executor
ERROR 08-18 13:13:33 [core.py:586]     self.workers = WorkerProc.wait_for_ready(unready_workers)
ERROR 08-18 13:13:33 [core.py:586]   File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 422, in wait_for_ready
ERROR 08-18 13:13:33 [core.py:586]     raise e from None
ERROR 08-18 13:13:33 [core.py:586] Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
Process EngineCore_0:
Traceback (most recent call last):
  File "/usr/local/python3.10.17/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/python3.10.17/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 590, in run_engine_core
    raise e
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 577, in run_engine_core
    engine_core = EngineCoreProc(*args, **kwargs)
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 404, in __init__
    super().__init__(vllm_config, executor_class, log_stats,
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 75, in __init__
    self.model_executor = executor_class(vllm_config)
  File "/vllm-workspace/vllm/vllm/executor/executor_base.py", line 53, in __init__
    self._init_executor()
  File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 93, in _init_executor
    self.workers = WorkerProc.wait_for_ready(unready_workers)
  File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 422, in wait_for_ready
    raise e from None
Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
Traceback (most recent call last):
  File "/usr/local/python3.10.17/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/python3.10.17/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1495, in <module>
    uvloop.run(run_server(args))
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/uvloop/__init__.py", line 82, in run
    return loop.run_until_complete(wrapper())
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
  File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1431, in run_server
    await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
  File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1451, in run_server_worker
    async with build_async_engine_client(args, client_config) as engine_client:
  File "/usr/local/python3.10.17/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 158, in build_async_engine_client
    async with build_async_engine_client_from_engine_args(
  File "/usr/local/python3.10.17/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 194, in build_async_engine_client_from_engine_args
    async_llm = AsyncLLM.from_vllm_config(
  File "/vllm-workspace/vllm/vllm/v1/engine/async_llm.py", line 162, in from_vllm_config
    return cls(
  File "/vllm-workspace/vllm/vllm/v1/engine/async_llm.py", line 124, in __init__
    self.engine_core = EngineCoreClient.make_async_mp_client(
  File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 96, in make_async_mp_client
    return AsyncMPClient(*client_args)
  File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 666, in __init__
    super().__init__(
  File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 403, in __init__
    with launch_core_engines(vllm_config, executor_class,
  File "/usr/local/python3.10.17/lib/python3.10/contextlib.py", line 142, in __exit__
    next(self.gen)
  File "/vllm-workspace/vllm/vllm/v1/engine/utils.py", line 434, in launch_core_engines
    wait_for_engine_startup(
  File "/vllm-workspace/vllm/vllm/v1/engine/utils.py", line 484, in wait_for_engine_startup
    raise RuntimeError("Engine core initialization failed. "
RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
[ERROR] 2025-08-18-13:13:35 (PID:1980, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
root@bms-hdk24-1-0-3-8:/mnt# /usr/local/python3.10.17/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

二、软件版本:
-- CANN 版本:8.2.RC1
-- Pytorch 版本:
--Python 版本 (e.g., Python 3.7.5):
-- MindStudio版本 (e.g., MindStudio 2.0.0 (beta3)):
--操作系统版本 (e.g., Ubuntu 18.04):

三、测试步骤：
1.安装msit-tag_MindStudio_8.2.RC1.B020_001
2.安装vllm-ascned
3.pip list
msmodelslim                 7.0.0rc912
vllm_ascend                 0.9.2rc1
4.量化命令：msmodelslim quant --model_path /pretrained_models/Qwen3-32B/ --save_path /mnt/quantization/models/Qwen3-32B-w8a8/ --device npu --model_type Qwen3-32B --quant_type w8a8 --trust_remote_code True
5.vllm启动命令：python3 -m vllm.entrypoints.openai.api_server --model=/mnt/quantization/models/Qwen3-32B-w8a8/ --max-model-len=32768 --max-num-batched-tokens=65536 --tensor-parallel-size=2 --host=0.0.0.0 --port=8089

Ascend/msit
暂停

内容风险标识

评论 (2)

Ascend/msit暂停 .gitee-modal { width: 500px !important; }

内容风险标识