[Qwen3.5VL-3B][MonkeyOCR]Atlas 300I duo 部署qwen2.5vl-3b过程中断，提示RuntimeError: status == aclmdlRICaptureStatus::ACL_MODEL_RI_CAPTURE_STATUS_ACTIVE INTERNAL ASSERT FAILED

一、问题现象：
kunpeng 920s + atlas 300i dou的硬件环境，已安装npu对应版本的必要驱动程序（cann），通过ascend/vllm-ascend-310p镜像部署qwen2.5-vl-3b模型提示torch_npu错误并中断。

二、软件版本:
![npu-smi info](https://foruda.gitee.com/images/1756107064884488748/63f0170c_2044730.png "屏幕截图")
-- CANN 版本 (e.g., CANN 3.0.x，5.x.x):

--Python 版本:
3.11.13
--Pytorch版本：
torch==2.7.0+cpu
torch_npu==2.7.1.dev20250724
torchvision==0.22.1
--操作系统版本:
PRETTY_NAME="Ubuntu 22.04.5 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

三、测试步骤：
使用ascend/vllm-ascend-310p镜像部署monkeyocr，安装流程参考monkeyocr readme并正常完成安装。启动过程中，模型已加载至npu（有占用，约12gb），模型基座为qwen2.5vl-3b。但是最后出现torch相关错误。日志提示反馈至pytorch。

四、日志信息:
Loading safetensors checkpoint shards:   0% Completed | 0/2 [00:00<?, ?it/s]
Loading safetensors checkpoint shards:  50% Completed | 1/2 [00:00<00:00,  2.36it/s]
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:01<00:00,  1.30it/s]
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:01<00:00,  1.39it/s]

INFO 08-25 06:27:53 [default_loader.py:262] Loading weights took 1.45 seconds
INFO 08-25 06:27:54 [model_runner_v1.py:2114] Loading model weights took 7.4149 GB
INFO 08-25 06:28:10 [backends.py:530] Using cache directory: /root/.cache/vllm/torch_compile_cache/7618974dee/rank_0_0/backbone for vLLM's torch.compile
INFO 08-25 06:28:10 [backends.py:541] Dynamo bytecode transform time: 13.78 s
INFO 08-25 06:28:17 [backends.py:215] Compiling a graph for dynamic shape takes 6.94 s
INFO 08-25 06:28:20 [monitor.py:34] torch.compile takes 20.72 s in total
.INFO 08-25 06:28:23 [worker_v1.py:186] Available memory: 12710423756, total memory: 46431260672
INFO 08-25 06:28:23 [kv_cache_utils.py:833] GPU KV cache size: 344,704 tokens
INFO 08-25 06:28:23 [kv_cache_utils.py:837] Maximum concurrency for 128,000 tokens per request: 2.69x
[WARN]operator(),build/CMakeFiles/torch_npu.dir/compiler_depend.ts:156:Feature is not supportted and the possible cause is that driver and firmware packages do not match.
[WARN]operator(),build/CMakeFiles/torch_npu.dir/compiler_depend.ts:159:Feature is not supportted and the possible cause is that driver and firmware packages do not match.
ERROR 08-25 06:28:24 [core.py:632] EngineCore failed to start.
ERROR 08-25 06:28:24 [core.py:632] Traceback (most recent call last):
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 623, in run_engine_core
ERROR 08-25 06:28:24 [core.py:632]     engine_core = EngineCoreProc(*args, **kwargs)
ERROR 08-25 06:28:24 [core.py:632]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 441, in __init__
ERROR 08-25 06:28:24 [core.py:632]     super().__init__(vllm_config, executor_class, log_stats,
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 86, in __init__
ERROR 08-25 06:28:24 [core.py:632]     self._initialize_kv_caches(vllm_config)
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 190, in _initialize_kv_caches
ERROR 08-25 06:28:24 [core.py:632]     self.model_executor.initialize_from_config(kv_cache_configs)
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm/vllm/v1/executor/abstract.py", line 66, in initialize_from_config
ERROR 08-25 06:28:24 [core.py:632]     self.collective_rpc("compile_or_warm_up_model")
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm/vllm/executor/uniproc_executor.py", line 58, in collective_rpc
ERROR 08-25 06:28:24 [core.py:632]     answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 08-25 06:28:24 [core.py:632]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm/vllm/utils/__init__.py", line 2985, in run_method
ERROR 08-25 06:28:24 [core.py:632]     return func(*args, **kwargs)
ERROR 08-25 06:28:24 [core.py:632]            ^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/worker_v1.py", line 261, in compile_or_warm_up_model
ERROR 08-25 06:28:24 [core.py:632]     self.model_runner.capture_model()
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 2430, in capture_model
ERROR 08-25 06:28:24 [core.py:632]     self._dummy_run(num_tokens)
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
ERROR 08-25 06:28:24 [core.py:632]     return func(*args, **kwargs)
ERROR 08-25 06:28:24 [core.py:632]            ^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 1975, in _dummy_run
ERROR 08-25 06:28:24 [core.py:632]     hidden_states = model(
ERROR 08-25 06:28:24 [core.py:632]                     ^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
ERROR 08-25 06:28:24 [core.py:632]     return self._call_impl(*args, **kwargs)
ERROR 08-25 06:28:24 [core.py:632]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
ERROR 08-25 06:28:24 [core.py:632]     return forward_call(*args, **kwargs)
ERROR 08-25 06:28:24 [core.py:632]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm/vllm/model_executor/models/qwen2_5_vl.py", line 1145, in forward
ERROR 08-25 06:28:24 [core.py:632]     hidden_states = self.language_model.model(
ERROR 08-25 06:28:24 [core.py:632]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm/vllm/compilation/decorators.py", line 279, in __call__
ERROR 08-25 06:28:24 [core.py:632]     model_output = self.forward(*args, **kwargs)
ERROR 08-25 06:28:24 [core.py:632]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm/vllm/model_executor/models/qwen2.py", line 336, in forward
ERROR 08-25 06:28:24 [core.py:632]     def forward(
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
ERROR 08-25 06:28:24 [core.py:632]     return self._call_impl(*args, **kwargs)
ERROR 08-25 06:28:24 [core.py:632]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
ERROR 08-25 06:28:24 [core.py:632]     return forward_call(*args, **kwargs)
ERROR 08-25 06:28:24 [core.py:632]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/_dynamo/eval_frame.py", line 838, in _fn
ERROR 08-25 06:28:24 [core.py:632]     return fn(*args, **kwargs)
ERROR 08-25 06:28:24 [core.py:632]            ^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/fx/graph_module.py", line 830, in call_wrapped
ERROR 08-25 06:28:24 [core.py:632]     return self._wrapped_call(self, *args, **kwargs)
ERROR 08-25 06:28:24 [core.py:632]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/fx/graph_module.py", line 406, in __call__
ERROR 08-25 06:28:24 [core.py:632]     raise e
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/fx/graph_module.py", line 393, in __call__
ERROR 08-25 06:28:24 [core.py:632]     return super(self.cls, obj).__call__(*args, **kwargs)  # type: ignore[misc]
ERROR 08-25 06:28:24 [core.py:632]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
ERROR 08-25 06:28:24 [core.py:632]     return self._call_impl(*args, **kwargs)
ERROR 08-25 06:28:24 [core.py:632]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
ERROR 08-25 06:28:24 [core.py:632]     return forward_call(*args, **kwargs)
ERROR 08-25 06:28:24 [core.py:632]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "<eval_with_key>.74", line 261, in forward
ERROR 08-25 06:28:24 [core.py:632]     submod_0 = self.submod_0(l_inputs_embeds_, s0, l_self_modules_layers_modules_0_modules_input_layernorm_parameters_weight_, l_self_modules_layers_modules_0_modules_self_attn_modules_qkv_proj_parameters_weight_, l_self_modules_layers_modules_0_modules_self_attn_modules_qkv_proj_parameters_bias_, l_self_modules_layers_modules_0_modules_self_attn_modules_rotary_emb_buffers_cos_sin_cache_, l_positions_, s2);  l_self_modules_layers_modules_0_modules_input_layernorm_parameters_weight_ = l_self_modules_layers_modules_0_modules_self_attn_modules_qkv_proj_parameters_weight_ = l_self_modules_layers_modules_0_modules_self_attn_modules_qkv_proj_parameters_bias_ = None
ERROR 08-25 06:28:24 [core.py:632]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-25 06:28:24 [core.py:632]   File "/vllm-workspace/vllm-ascend/vllm_ascend/compilation/piecewise_backend.py", line 192, in __call__
ERROR 08-25 06:28:24 [core.py:632]     with torch.npu.graph(aclgraph, pool=self.graph_pool):
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/npu/graphs.py", line 310, in __enter__
ERROR 08-25 06:28:24 [core.py:632]     self.npu_graph.capture_begin(
ERROR 08-25 06:28:24 [core.py:632]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/npu/graphs.py", line 210, in capture_begin
ERROR 08-25 06:28:24 [core.py:632]     super().capture_begin(pool=pool, capture_error_mode=capture_error_mode)
ERROR 08-25 06:28:24 [core.py:632] RuntimeError: status == aclmdlRICaptureStatus::ACL_MODEL_RI_CAPTURE_STATUS_ACTIVE INTERNAL ASSERT FAILED at "build/CMakeFiles/torch_npu.dir/compiler_depend.ts":160, please report a bug to PyTorch. 
Process EngineCore_0:
Traceback (most recent call last):
  File "/usr/local/python3.11.13/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/python3.11.13/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 636, in run_engine_core
    raise e
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 623, in run_engine_core
    engine_core = EngineCoreProc(*args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 441, in __init__
    super().__init__(vllm_config, executor_class, log_stats,
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 86, in __init__
    self._initialize_kv_caches(vllm_config)
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 190, in _initialize_kv_caches
    self.model_executor.initialize_from_config(kv_cache_configs)
  File "/vllm-workspace/vllm/vllm/v1/executor/abstract.py", line 66, in initialize_from_config
    self.collective_rpc("compile_or_warm_up_model")
  File "/vllm-workspace/vllm/vllm/executor/uniproc_executor.py", line 58, in collective_rpc
    answer = run_method(self.driver_worker, method, args, kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/vllm-workspace/vllm/vllm/utils/__init__.py", line 2985, in run_method
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/worker_v1.py", line 261, in compile_or_warm_up_model
    self.model_runner.capture_model()
  File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 2430, in capture_model
    self._dummy_run(num_tokens)
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 1975, in _dummy_run
    hidden_states = model(
                    ^^^^^^
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/vllm-workspace/vllm/vllm/model_executor/models/qwen2_5_vl.py", line 1145, in forward
    hidden_states = self.language_model.model(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/vllm-workspace/vllm/vllm/compilation/decorators.py", line 279, in __call__
    model_output = self.forward(*args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/vllm-workspace/vllm/vllm/model_executor/models/qwen2.py", line 336, in forward
    def forward(
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/_dynamo/eval_frame.py", line 838, in _fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/fx/graph_module.py", line 830, in call_wrapped
    return self._wrapped_call(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/fx/graph_module.py", line 406, in __call__
    raise e
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/fx/graph_module.py", line 393, in __call__
    return super(self.cls, obj).__call__(*args, **kwargs)  # type: ignore[misc]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<eval_with_key>.74", line 261, in forward
    submod_0 = self.submod_0(l_inputs_embeds_, s0, l_self_modules_layers_modules_0_modules_input_layernorm_parameters_weight_, l_self_modules_layers_modules_0_modules_self_attn_modules_qkv_proj_parameters_weight_, l_self_modules_layers_modules_0_modules_self_attn_modules_qkv_proj_parameters_bias_, l_self_modules_layers_modules_0_modules_self_attn_modules_rotary_emb_buffers_cos_sin_cache_, l_positions_, s2);  l_self_modules_layers_modules_0_modules_input_layernorm_parameters_weight_ = l_self_modules_layers_modules_0_modules_self_attn_modules_qkv_proj_parameters_weight_ = l_self_modules_layers_modules_0_modules_self_attn_modules_qkv_proj_parameters_bias_ = None
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/vllm-workspace/vllm-ascend/vllm_ascend/compilation/piecewise_backend.py", line 192, in __call__
    with torch.npu.graph(aclgraph, pool=self.graph_pool):
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/npu/graphs.py", line 310, in __enter__
    self.npu_graph.capture_begin(
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch_npu/npu/graphs.py", line 210, in capture_begin
    super().capture_begin(pool=pool, capture_error_mode=capture_error_mode)
RuntimeError: status == aclmdlRICaptureStatus::ACL_MODEL_RI_CAPTURE_STATUS_ACTIVE INTERNAL ASSERT FAILED at "build/CMakeFiles/torch_npu.dir/compiler_depend.ts":160, please report a bug to PyTorch. 
2025-08-25 06:28:37.273 | INFO     | api.main:lifespan:170 - ❌ Failed to initialize MonkeyOCR model: Engine core initialization failed. See root cause above. Failed core proc(s): {}

--问题：
大神们好，如果需要基于vllm-ascend部署qwen2.5vl-3b，目前torch-npu是否支持？根据error日志来看，我是否存在某些配置遗漏？该如何调整修复？

Ascend/pytorch
Paused

Content Risk Flag

Comments (2)

Ascend/pytorchPaused .gitee-modal { width: 500px !important; }

Content Risk Flag