登录
注册
开源
企业版
高校版
搜索
帮助中心
使用条款
关于我们
开源
企业版
高校版
私有云
模力方舟
AI 队友
登录
注册
轻量养虾,开箱即用!低 Token + 稳定算力,Gitee & 模力方舟联合出品的 PocketClaw 正式开售!点击了解详情~
代码拉取完成,页面将自动刷新
仓库状态说明
开源项目
>
人工智能
>
AI-人工智能
&&
捐赠
捐赠前请先登录
取消
前往登录
扫描微信二维码支付
取消
支付完成
支付提示
将跳转至支付宝完成支付
确定
取消
Watch
不关注
关注所有动态
仅关注版本发行动态
关注但不提醒动态
59
Star
196
Fork
467
Ascend
/
msit
暂停
代码
Issues
11
Pull Requests
77
Wiki
统计
流水线
服务
质量分析
Jenkins for Gitee
腾讯云托管
腾讯云 Serverless
悬镜安全
阿里云 SAE
Codeblitz
SBOM
开发画像分析
我知道了,不再自动展开
更新失败,请稍后重试!
移除标识
内容风险标识
本任务被
标识为内容中包含有代码安全 Bug 、隐私泄露等敏感信息,仓库外成员不可访问
采用自动回退量化Qwen3-235B-A22B-Thinking-2507的W8A8版本装载模型报错
TODO
#ICVJPB
缺陷
ponyioy
创建于
2025-08-30 12:30
一、问题现象(附报错日志上下文): 因为quant_qwen_moe_w8a8.py默认配置量化出来的Qwen3-235B-A22B-Thinking-2507有比较明显的精度问题,会时不时出现“游戏副本”的字样,因此尝试使用回退linear层的方式重新量化,设置自动回退层级为L5。量化可以成功,但是装载模型报错。 二、软件版本: 运行镜像为 mindie:2.1.RC1.B152-800I-A2-py3.11-openeuler24.03-lts-aarch64 三、测试步骤: 采用镜像:mindie:2.0.T18.B010-800I-A2-py3.11-openeuler24.03-lts-aarch64 修改:example/Qwen3-MOE/quant_qwen_moe_w8a8.py 自动回退设置为L5 calibrator = Calibrator(model, quant_config, calib_data=dataset_calib, disable_level="L5", mix_cfg={"*.mlp.*": "w8a8_dynamic", "*": "w8a8"}) 执行 python3 quant_qwen_moe_w8a8.py --model_path /Model/Qwen3-235B-A22B-Thinking-2507 --save_path /Model/Qwen3-235B-A22B-Thinking-2507-L5 --trust_remote_code True 四、日志信息: 生成模型后运行报错,运行镜像为 mindie:2.1.RC1.B152-800I-A2-py3.11-openeuler24.03-lts-aarch64 机器为1台A800 I2, 加载模型报错,日志为: [2025-08-30 11:59:33.518] [47420] [281470512787808] [llmmodels] [ERROR] [acl_nn_operation.cpp:142] gmmNode call SetAclNNWorkspaceExecutor fail, error:161002 [2025-08-30 11:59:33.518] [47420] [281470512787808] [llmmodels] [ERROR] [acl_nn_operation.cpp:115] gmmNode call CreateAclNNOpCache fail, error:12 [2025-08-30 11:59:33.518] [47420] [281470512787808] [llmmodels] [ERROR] [acl_nn_operation.cpp:59] gmmNode call UpdateAclNNOpCache, error:12 [2025-08-30 11:59:33.520] [47445] [281471276740960] [llmmodels] [ERROR] [acl_nn_operation.cpp:142] gmmNode call SetAclNNWorkspaceExecutor fail, error:161002 [2025-08-30 11:59:33.520] [47445] [281471276740960] [llmmodels] [ERROR] [acl_nn_operation.cpp:115] gmmNode call CreateAclNNOpCache fail, error:12 [2025-08-30 11:59:33.520] [47445] [281471276740960] [llmmodels] [ERROR] [acl_nn_operation.cpp:59] gmmNode call UpdateAclNNOpCache, error:12 [2025-08-30 11:59:33.030+0800] [47445] [281473340731744] [batchscheduler] [ERROR] [model.py:61] : [Model] >>> Exception:Setup fail, enable log: export ASDOPS_LOG_LEVEL=ERROR, export ASDOPS_LOG_TO_STDOUT=1 to find the first error. For more details, see the MindIE official document. Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 59, in initialize return self.python_model.initialize(config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 133, in initialize self.generator = Generator( ^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 299, in __init__ self.cache_manager = self.warm_up( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 456, in warm_up npu_mem = self.__warmup_standard(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 626, in __warmup_standard npu_mem = self.__warmup_prefill(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 614, in __warmup_prefill npu_mem = self.__auto_warmup(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times, is_prefill=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 708, in __auto_warmup self.__execute_warm_up(cache_manager, input_metadata, dummy=True) File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 508, in __execute_warm_up raise e File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 499, in __execute_warm_up self.generator_backend._warm_up(model_inputs, inference_mode=self.inference_mode, File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 570, in _warm_up super()._warm_up(model_inputs, **kwargs) File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_backend.py", line 247, in _warm_up logits = self.forward(model_inputs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/utils/decorators/time_decorator.py", line 69, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 230, in forward logits = self._forward(model_inputs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 621, in _forward logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 125, in forward result = self.forward_from_model_inputs(model_inputs, npu_cache, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 197, in forward_from_model_inputs result = self.forward_tensor( ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 237, in forward_tensor result = self.model_runner.forward( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/Ascend/atb-models/atb_llm/runner/model_runner.py", line 310, in forward res = self.model.forward(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/Ascend/atb-models/atb_llm/models/base/flash_causal_lm.py", line 536, in forward logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/Ascend/atb-models/atb_llm/models/base/flash_causal_lm.py", line 467, in execute_ascend_operator acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Setup fail, enable log: export ASDOPS_LOG_LEVEL=ERROR, export ASDOPS_LOG_TO_STDOUT=1 to find the first error. For more details, see the MindIE official document. [2025-08-30 11:59:33.030+0800] [47420] [281472577433952] [batchscheduler] [ERROR] [model.py:64] : [MIE04E13030A] [Model] >>> return initialize error result: {'status': 'error', 'npuBlockNum': '0', 'cpuBlockNum': '0', 'memPoolId': '-1'} [2025-08-30 11:59:33.030+0800] [47445] [281473340731744] [batchscheduler] [ERROR] [model.py:64] : [MIE04E13030A] [Model] >>> return initialize error result: {'status': 'error', 'npuBlockNum': '0', 'cpuBlockNum': '0', 'memPoolId': '-1'}
一、问题现象(附报错日志上下文): 因为quant_qwen_moe_w8a8.py默认配置量化出来的Qwen3-235B-A22B-Thinking-2507有比较明显的精度问题,会时不时出现“游戏副本”的字样,因此尝试使用回退linear层的方式重新量化,设置自动回退层级为L5。量化可以成功,但是装载模型报错。 二、软件版本: 运行镜像为 mindie:2.1.RC1.B152-800I-A2-py3.11-openeuler24.03-lts-aarch64 三、测试步骤: 采用镜像:mindie:2.0.T18.B010-800I-A2-py3.11-openeuler24.03-lts-aarch64 修改:example/Qwen3-MOE/quant_qwen_moe_w8a8.py 自动回退设置为L5 calibrator = Calibrator(model, quant_config, calib_data=dataset_calib, disable_level="L5", mix_cfg={"*.mlp.*": "w8a8_dynamic", "*": "w8a8"}) 执行 python3 quant_qwen_moe_w8a8.py --model_path /Model/Qwen3-235B-A22B-Thinking-2507 --save_path /Model/Qwen3-235B-A22B-Thinking-2507-L5 --trust_remote_code True 四、日志信息: 生成模型后运行报错,运行镜像为 mindie:2.1.RC1.B152-800I-A2-py3.11-openeuler24.03-lts-aarch64 机器为1台A800 I2, 加载模型报错,日志为: [2025-08-30 11:59:33.518] [47420] [281470512787808] [llmmodels] [ERROR] [acl_nn_operation.cpp:142] gmmNode call SetAclNNWorkspaceExecutor fail, error:161002 [2025-08-30 11:59:33.518] [47420] [281470512787808] [llmmodels] [ERROR] [acl_nn_operation.cpp:115] gmmNode call CreateAclNNOpCache fail, error:12 [2025-08-30 11:59:33.518] [47420] [281470512787808] [llmmodels] [ERROR] [acl_nn_operation.cpp:59] gmmNode call UpdateAclNNOpCache, error:12 [2025-08-30 11:59:33.520] [47445] [281471276740960] [llmmodels] [ERROR] [acl_nn_operation.cpp:142] gmmNode call SetAclNNWorkspaceExecutor fail, error:161002 [2025-08-30 11:59:33.520] [47445] [281471276740960] [llmmodels] [ERROR] [acl_nn_operation.cpp:115] gmmNode call CreateAclNNOpCache fail, error:12 [2025-08-30 11:59:33.520] [47445] [281471276740960] [llmmodels] [ERROR] [acl_nn_operation.cpp:59] gmmNode call UpdateAclNNOpCache, error:12 [2025-08-30 11:59:33.030+0800] [47445] [281473340731744] [batchscheduler] [ERROR] [model.py:61] : [Model] >>> Exception:Setup fail, enable log: export ASDOPS_LOG_LEVEL=ERROR, export ASDOPS_LOG_TO_STDOUT=1 to find the first error. For more details, see the MindIE official document. Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/model_wrapper/model.py", line 59, in initialize return self.python_model.initialize(config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/model_wrapper/standard_model.py", line 133, in initialize self.generator = Generator( ^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 299, in __init__ self.cache_manager = self.warm_up( ^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 456, in warm_up npu_mem = self.__warmup_standard(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 626, in __warmup_standard npu_mem = self.__warmup_prefill(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 614, in __warmup_prefill npu_mem = self.__auto_warmup(max_prefill_tokens, max_seq_len, max_input_len, max_iter_times, is_prefill=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 708, in __auto_warmup self.__execute_warm_up(cache_manager, input_metadata, dummy=True) File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 508, in __execute_warm_up raise e File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/generator.py", line 499, in __execute_warm_up self.generator_backend._warm_up(model_inputs, inference_mode=self.inference_mode, File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 570, in _warm_up super()._warm_up(model_inputs, **kwargs) File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_backend.py", line 247, in _warm_up logits = self.forward(model_inputs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/utils/decorators/time_decorator.py", line 69, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 230, in forward logits = self._forward(model_inputs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/text_generator/adapter/generator_torch.py", line 621, in _forward logits = self.model_wrapper.forward(model_inputs, self.cache_pool.npu_cache, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 125, in forward result = self.forward_from_model_inputs(model_inputs, npu_cache, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 197, in forward_from_model_inputs result = self.forward_tensor( ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mindie_llm/modeling/model_wrapper/atb/atb_model_wrapper.py", line 237, in forward_tensor result = self.model_runner.forward( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/Ascend/atb-models/atb_llm/runner/model_runner.py", line 310, in forward res = self.model.forward(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/Ascend/atb-models/atb_llm/models/base/flash_causal_lm.py", line 536, in forward logits = self.execute_ascend_operator(acl_inputs, acl_param, is_prefill) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/Ascend/atb-models/atb_llm/models/base/flash_causal_lm.py", line 467, in execute_ascend_operator acl_model_out = self.acl_encoder_operation.execute(acl_inputs, acl_param) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Setup fail, enable log: export ASDOPS_LOG_LEVEL=ERROR, export ASDOPS_LOG_TO_STDOUT=1 to find the first error. For more details, see the MindIE official document. [2025-08-30 11:59:33.030+0800] [47420] [281472577433952] [batchscheduler] [ERROR] [model.py:64] : [MIE04E13030A] [Model] >>> return initialize error result: {'status': 'error', 'npuBlockNum': '0', 'cpuBlockNum': '0', 'memPoolId': '-1'} [2025-08-30 11:59:33.030+0800] [47445] [281473340731744] [batchscheduler] [ERROR] [model.py:64] : [MIE04E13030A] [Model] >>> return initialize error result: {'status': 'error', 'npuBlockNum': '0', 'cpuBlockNum': '0', 'memPoolId': '-1'}
评论 (
3
)
登录
后才可以发表评论
状态
TODO
TODO
WIP
DONE
CLOSED
REJECTED
负责人
未设置
标签
未设置
项目
未立项任务
未立项任务
里程碑
未关联里程碑
未关联里程碑
Pull Requests
未关联
未关联
关联的 Pull Requests 被合并后可能会关闭此 issue
分支
未关联
分支 (
-
)
标签 (
-
)
开始日期   -   截止日期
-
置顶选项
不置顶
置顶等级:高
置顶等级:中
置顶等级:低
优先级
不指定
严重
主要
次要
不重要
预计工期
(小时)
参与者(3)
Python
1
https://gitee.com/ascend/msit.git
git@gitee.com:ascend/msit.git
ascend
msit
msit
点此查找更多帮助
搜索帮助
Git 命令在线学习
如何在 Gitee 导入 GitHub 仓库
Git 仓库基础操作
企业版和社区版功能对比
SSH 公钥设置
如何处理代码冲突
仓库体积过大,如何减小?
如何找回被删除的仓库数据
Gitee 产品配额说明
GitHub仓库快速导入Gitee及同步更新
什么是 Release(发行版)
将 PHP 项目自动发布到 packagist.org
评论
仓库举报
回到顶部
登录提示
该操作需登录 Gitee 帐号,请先登录后再操作。
立即登录
没有帐号,去注册