pytorch迁移训练STEM模型，NPU error,NPU error code is:500002

一、问题现象（附报错日志上下文）：
STEM模型训练时报错：
![输入图片说明](https://foruda.gitee.com/images/1688612671755263073/84958eac_12123878.png "屏幕截图")
~/anaconda3/envs/MindSpore/lib/python3.7/site-packages/torch_npu/utils/module.py in cast_weight(self, device)
    144 
    145     current_class = self.__class__
--> 146     _format_cast(self, current_class)
    147 
    148     if not self.children:

~/anaconda3/envs/MindSpore/lib/python3.7/site-packages/torch_npu/utils/module.py in _format_cast(module, class_name)
    101         if issubclass(class_name, torch.nn.Linear) and not torch.npu.get_mm_bmm_format_nd():
    102             module.weight.data = module.weight.data.to(device)
--> 103             module.weight.data = torch_npu.npu_format_cast(module.weight.data, 29) # ACL_FORMAT_FRACTAL_NZ
    104         if "MultiheadAttention" in str(class_name) and \
    105                 hasattr(module,"q_proj_weight") and module.q_proj_weight is not None and \
 **RuntimeError: Run:/usr1/workspace/FPTA_Daily_Plugin_open_date/CODE/torch_npu/csrc/framework/OpParamMaker.cpp:136 NPU error,NPU error code is:500002
E49999: Inner Error, Please contact support engineer!
E49999  Pre-compile op[trans_TransData_60,] failed, oppath[/usr/local/Ascend/ascend-toolkit/6.0.1/opp/built-in/op_impl/ai_core/tbe/impl/dynamic/trans_data.py], optype[TransData], taskID[226]. Please check op's compilation error message.** [FUNC:ReportBuildErrMessage][FILE:fusion_op.cc][LINE:858]
        TraceBack (most recent call last):
        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281462913479136] recompile single op[trans_TransData_60] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:907]

二、软件版本:
-- CANN 版本 :  6.0.1
--Pytorch版本: 1.11.0
--Python 版本 :  3.7.10
-- MindStudio版本 : 无
--Modelarts平台

Ascend/pytorch
暂停

内容风险标识

评论 (2)

Ascend/pytorch暂停 .gitee-modal { width: 500px !important; }

内容风险标识