78 Star 599 Fork 1.2K

Ascend/pytorch

RuntimeError: Initialize:torch_npu/csrc/core/npu/sys_ctrl/npu_sys_ctrl.cpp:169 NPU error, error code is 500000

DONE
需求
创建于  
2024-06-12 16:03

无法启动

File "/data/miniconda3/envs/env-3.9.16/lib/python3.9/site-packages/transformers/hf_argparser.py", line 339, in parse_args_into_dataclasses
obj = dtype(**inputs)
File "", line 145, in init
File "/data/miniconda3/envs/env-3.9.16/lib/python3.9/site-packages/transformers/training_args.py", line 1593, in post_init
if torch.cuda.is_available() and not is_torch_bf16_gpu_available():
File "/data/miniconda3/envs/env-3.9.16/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 392, in is_torch_bf16_gpu_available
return torch.cuda.is_available() and torch.cuda.is_bf16_supported()
File "/data/miniconda3/envs/env-3.9.16/lib/python3.9/site-packages/torch_npu/npu/utils.py", line 329, in is_bf16_supported
torch_npu.npu._lazy_init()
File "/data/miniconda3/envs/env-3.9.16/lib/python3.9/site-packages/torch_npu/npu/init.py", line 208, in _lazy_init
torch_npu._C._npu_init()
RuntimeError: Initialize:torch_npu/csrc/core/npu/sys_ctrl/npu_sys_ctrl.cpp:169 NPU error, error code is 500000
[Error]: Unknown internal error.
Rectify the fault based on the error information in the ascend log.
EE1001: The argument is invalid.Reason: rtGetDevMsg execute failed, reason=[context pointer null]
Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
TraceBack (most recent call last):
[Set][Dump]set dump config failed, adx errorCode = -1[FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145]
[Process][DumpConfig]process HandleDumpConfig failed[FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145]
ctx is NULL![FUNC:GetDevErrMsg][FILE:api_impl.cc][LINE:4677]
The argument is invalid.Reason: rtGetDevMsg execute failed, reason=[context pointer null]

评论 (1)

jinfagang 创建了需求 1年前

麻烦提供下详细plog日志信息以及对应cann版本,torch,torch_npu版本

huangyunlong 任务状态TODO 修改为DONE 12个月前

登录 后才可以发表评论

状态
负责人
项目
里程碑
Pull Requests
关联的 Pull Requests 被合并后可能会关闭此 issue
分支
优先级
预计工期 (小时)
开始日期   -   截止日期
-
置顶选项
参与者(2)
huangyunlong-huangyunlong2022 jinfagang-jinfagang
Python
1
https://gitee.com/ascend/pytorch.git
git@gitee.com:ascend/pytorch.git
ascend
pytorch
pytorch

搜索帮助