76 Star 221 Fork 167

Ascend / modelzoo

 / 详情

【北航】【VAT】NPU 训练出错

DONE
Bug-Report
创建于  
2021-09-07 02:51

一、问题现象(附报错日志上下文):
VAT模型在不开启数据增强时正常训练,在开启数据增强时报错。

二、软件版本:
在Ascend910裸机训练.
代码文件上传到OBS中:
https://e-share.obs-website.cn-north-1.myhuaweicloud.com?token=qSok4Q03amcwVdtdl1u0tvNIJaIH6a7RALRJO257WKo0EYNvDt0cBhKqYNM085H/C0Mn3cZNwN1roLBfX/+b0Mg0wZAkfesLz7i2CsOkYBh3HilxSzBwTXarWgiKrNjJyaakXs2bvzPCPRiV02keE2CPQMPb/QlQmQhO4r6oVH2B62IbNNym9fDVs/ZJKS56hwt7+f0r9fafGWd+cg3bymPj1x4sb5CfkoKdfSKkDuJg+FveVsoQHg/rj2JMTeZksaHxaQMdmG17Tw54KzttvAuAfefzJqwSCfpYYJvdyVYgRHSn15t4+/80xE6V7UwzY8yoyz/+WXKe5haVu4uFxQS3maA8Q4VLL8p6En8tukvHnoOIlb9OFbiempTx3SXqhZ1hCf6dRtIb392nJ7BOMxKb/N/7KJ4SwOxgt0kCldxG6/NQgfgTftRQctd1SHNoC+oQvS1QeC+cLt5iL7gMvVnD7ih0RXNMBR7S1ngnXCNmaVGLwbGbXRdIs839peSPYyIcmQWfDuNfp/+xhtjmpQ==
提取码:123456
*有效期至: 2021/10/07 02:49:08 GMT+08:00

三、测试步骤:
开启图像增强训练命令:
python3 train_semisup.py --dataset cifar10 --data_url=../dataset/cifar10/ --train_url=../vat_aug --aug_flip=True --aug_trans=True
关闭图像增强训练命令(默认关闭):
python3 train_semisup.py --dataset cifar10 --data_url=../dataset/cifar10/ --train_url=../vat_noaug

四、日志信息:

WARNING:tensorflow:From /usr/local/Ascend/tfplugin/latest/tfplugin/python/site-packages/npu_bridge/estimator/npu/npu_optimizer.py:284: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

WARNING:tensorflow:From train_semisup.py:262: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.

train_url: ../vat_aug
8.0 False
WARNING:tensorflow:From train_semisup.py:124: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

W0907 02:12:41.134318 281473101354864 module_wrapper.py:139] From train_semisup.py:124: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

filenames in queue: ['seed1/labeled_train.tfrecords']
WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/dataset_utils.py:127: string_input_producer (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by `tf.data`. Use `tf.data.Dataset.from_tensor_slices(string_tensor).shuffle(tf.shape(input_tensor, out_type=tf.int64)[0]).repeat(num_epochs)`. If `shuffle=False`, omit the `.shuffle(...)`.
W0907 02:12:41.135755 281473101354864 deprecation.py:323] From /home/test_user01/Documents/VAT/code/dataset_utils.py:127: string_input_producer (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by `tf.data`. Use `tf.data.Dataset.from_tensor_slices(string_tensor).shuffle(tf.shape(input_tensor, out_type=tf.int64)[0]).repeat(num_epochs)`. If `shuffle=False`, omit the `.shuffle(...)`.
WARNING:tensorflow:From /usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/training/input.py:277: input_producer (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by `tf.data`. Use `tf.data.Dataset.from_tensor_slices(input_tensor).shuffle(tf.shape(input_tensor, out_type=tf.int64)[0]).repeat(num_epochs)`. If `shuffle=False`, omit the `.shuffle(...)`.
W0907 02:12:41.147659 281473101354864 deprecation.py:323] From /usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/training/input.py:277: input_producer (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by `tf.data`. Use `tf.data.Dataset.from_tensor_slices(input_tensor).shuffle(tf.shape(input_tensor, out_type=tf.int64)[0]).repeat(num_epochs)`. If `shuffle=False`, omit the `.shuffle(...)`.
WARNING:tensorflow:From /usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/training/input.py:189: limit_epochs (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by `tf.data`. Use `tf.data.Dataset.from_tensors(tensor).repeat(num_epochs)`.
W0907 02:12:41.149133 281473101354864 deprecation.py:323] From /usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/training/input.py:189: limit_epochs (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by `tf.data`. Use `tf.data.Dataset.from_tensors(tensor).repeat(num_epochs)`.
WARNING:tensorflow:From /usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/training/input.py:198: QueueRunner.__init__ (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
W0907 02:12:41.152537 281473101354864 deprecation.py:323] From /usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/training/input.py:198: QueueRunner.__init__ (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
WARNING:tensorflow:From /usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/training/input.py:198: add_queue_runner (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
W0907 02:12:41.154287 281473101354864 deprecation.py:323] From /usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/training/input.py:198: add_queue_runner (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/dataset_utils.py:64: TFRecordReader.__init__ (from tensorflow.python.ops.io_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by `tf.data`. Use `tf.data.TFRecordDataset`.
W0907 02:12:41.160550 281473101354864 deprecation.py:323] From /home/test_user01/Documents/VAT/code/dataset_utils.py:64: TFRecordReader.__init__ (from tensorflow.python.ops.io_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by `tf.data`. Use `tf.data.TFRecordDataset`.
WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/dataset_utils.py:66: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.

W0907 02:12:41.162452 281473101354864 module_wrapper.py:139] From /home/test_user01/Documents/VAT/code/dataset_utils.py:66: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.

WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/dataset_utils.py:70: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

W0907 02:12:41.162690 281473101354864 module_wrapper.py:139] From /home/test_user01/Documents/VAT/code/dataset_utils.py:70: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

augmentation
WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/dataset_utils.py:116: The name tf.random_crop is deprecated. Please use tf.image.random_crop instead.

W0907 02:12:41.179238 281473101354864 module_wrapper.py:139] From /home/test_user01/Documents/VAT/code/dataset_utils.py:116: The name tf.random_crop is deprecated. Please use tf.image.random_crop instead.

WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/dataset_utils.py:98: shuffle_batch (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by `tf.data`. Use `tf.data.Dataset.shuffle(min_after_dequeue).batch(batch_size)`.
W0907 02:12:41.216159 281473101354864 deprecation.py:323] From /home/test_user01/Documents/VAT/code/dataset_utils.py:98: shuffle_batch (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by `tf.data`. Use `tf.data.Dataset.shuffle(min_after_dequeue).batch(batch_size)`.
filenames in queue: ['seed1/unlabeled_train.tfrecords']
augmentation
filenames in queue: ['seed1/labeled_train.tfrecords']
augmentation
filenames in queue: ['seed1/unlabeled_train.tfrecords']
augmentation
filenames in queue: ['seed1/test.tfrecords']
WARNING:tensorflow:From train_semisup.py:147: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W0907 02:12:41.552944 281473101354864 module_wrapper.py:139] From train_semisup.py:147: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From train_semisup.py:149: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W0907 02:12:41.555109 281473101354864 module_wrapper.py:139] From train_semisup.py:149: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From train_semisup.py:62: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

W0907 02:12:41.555521 281473101354864 module_wrapper.py:139] From train_semisup.py:62: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

W0907 02:12:41.563640 281473101354864 lazy_loader.py:50]
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/layers.py:21: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W0907 02:12:41.587287 281473101354864 deprecation.py:323] From /home/test_user01/Documents/VAT/code/layers.py:21: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/layers.py:59: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.

W0907 02:12:41.622272 281473101354864 module_wrapper.py:139] From /home/test_user01/Documents/VAT/code/layers.py:59: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.

WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/layers.py:117: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

W0907 02:12:41.806411 281473101354864 module_wrapper.py:139] From /home/test_user01/Documents/VAT/code/layers.py:117: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/layers.py:86: The name tf.nn.xw_plus_b is deprecated. Please use tf.compat.v1.nn.xw_plus_b instead.

W0907 02:12:42.325828 281473101354864 module_wrapper.py:139] From /home/test_user01/Documents/VAT/code/layers.py:86: The name tf.nn.xw_plus_b is deprecated. Please use tf.compat.v1.nn.xw_plus_b instead.

WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/layers.py:124: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See `tf.nn.softmax_cross_entropy_with_logits_v2`.

W0907 02:12:42.328322 281473101354864 deprecation.py:323] From /home/test_user01/Documents/VAT/code/layers.py:124: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See `tf.nn.softmax_cross_entropy_with_logits_v2`.

WARNING:tensorflow:From train_semisup.py:72: The name tf.get_variable_scope is deprecated. Please use tf.compat.v1.get_variable_scope instead.

W0907 02:12:42.364488 281473101354864 module_wrapper.py:139] From train_semisup.py:72: The name tf.get_variable_scope is deprecated. Please use tf.compat.v1.get_variable_scope instead.

WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/vat.py:41: The name tf.random_normal is deprecated. Please use tf.random.normal instead.

W0907 02:12:42.649815 281473101354864 module_wrapper.py:139] From /home/test_user01/Documents/VAT/code/vat.py:41: The name tf.random_normal is deprecated. Please use tf.random.normal instead.

WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/vat.py:35: calling reduce_max_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
W0907 02:12:42.656598 281473101354864 deprecation.py:506] From /home/test_user01/Documents/VAT/code/vat.py:35: calling reduce_max_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/vat.py:36: calling reduce_sum_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
W0907 02:12:42.663293 281473101354864 deprecation.py:506] From /home/test_user01/Documents/VAT/code/vat.py:36: calling reduce_sum_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From /home/test_user01/Documents/VAT/code/layers.py:135: The name tf.log is deprecated. Please use tf.math.log instead.

W0907 02:12:42.960289 281473101354864 module_wrapper.py:139] From /home/test_user01/Documents/VAT/code/layers.py:135: The name tf.log is deprecated. Please use tf.math.log instead.

WARNING:tensorflow:From /usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/ops/math_grad.py:1424: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W0907 02:12:43.095123 281473101354864 deprecation.py:323] From /usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/ops/math_grad.py:1424: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From train_semisup.py:88: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

W0907 02:12:44.463106 281473101354864 module_wrapper.py:139] From train_semisup.py:88: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

WARNING:tensorflow:From train_semisup.py:89: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

W0907 02:12:44.463388 281473101354864 module_wrapper.py:139] From train_semisup.py:89: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

WARNING:tensorflow:From train_semisup.py:157: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

W0907 02:12:53.513885 281473101354864 module_wrapper.py:139] From train_semisup.py:157: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

WARNING:tensorflow:From train_semisup.py:165: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

W0907 02:12:53.516328 281473101354864 module_wrapper.py:139] From train_semisup.py:165: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

WARNING:tensorflow:From train_semisup.py:168: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

W0907 02:13:47.201998 281473101354864 module_wrapper.py:139] From train_semisup.py:168: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

WARNING:tensorflow:From train_semisup.py:168: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

W0907 02:13:47.202490 281473101354864 module_wrapper.py:139] From train_semisup.py:168: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

WARNING:tensorflow:From train_semisup.py:178: Supervisor.__init__ (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
W0907 02:13:47.377758 281473101354864 deprecation.py:323] From train_semisup.py:178: Supervisor.__init__ (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
Training...
WARNING:tensorflow:From train_semisup.py:183: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

W0907 02:13:55.856959 281473101354864 module_wrapper.py:139] From train_semisup.py:183: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

INFO:tensorflow:Running local_init_op.
I0907 02:14:04.779567 281473101354864 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I0907 02:14:05.075715 281473101354864 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Starting standard services.
I0907 02:14:15.048590 281473101354864 supervisor.py:737] Starting standard services.
INFO:tensorflow:Saving checkpoint to path ../vat_aug/model.ckpt
I0907 02:14:15.049583 281465992770048 supervisor.py:1117] Saving checkpoint to path ../vat_aug/model.ckpt
INFO:tensorflow:Starting queue runners.
I0907 02:14:15.049924 281473101354864 supervisor.py:743] Starting queue runners.
*** Error in `python3': free(): invalid pointer: 0x0000fffd781cfa50 ***
Fatal Python error: Bus error

Thread 0x0000fffd50f9f200 (most recent call first):
  File "/usr/local/python3.7.5/lib/python3.7/threading.py", line 296 in wait
  File "/usr/local/python3.7.5/lib/python3.7/thr/usr/local/Ascend/ascend-toolkit/latest/fwkacllib/lib64/liberror_manager.soeading.py", line 552 in wait
  File "/usr/local/python3.7.5/usr/local/Ascend/ascend-toolkit/latest/fwkacllib/lib64/liberror_manager.so/lib/python3.7/site-packages/tensorflow_core/p/usr/local/Ascend/ascend-toolkit/latest/fwkacllib/lib64/liberror_manager.soython/training/coordinator.py", line 311 in wait_for_stop
  File "/usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/training/queue_runner_impl.py", line 293 in _close_on_stop
  File "/usr/local/python3.7.5/lib/python3.7/threading.py", line 870 in run
  File "/usr/local/python3.7.5/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/local/python3.7.5/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x0000fffd517af200 (most recent call first):
  File "/usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1443 in _call_tf_sessionrun
  File "/usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1287 in _single_operation_run
  File "/usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/training/queue_runner_impl.py", line 257 in _run
  File "/usr/local/python3.7./usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/_pywrap_tensorflow_internal.so5/lib/python3.7/threading.py", line 870 in run
  File "/usr/local/python3.7.5/lib/python3.Fatal Python error: 7Aborted/

threading.py", line 926 in _bootstrap_inner
  File "/usr/local/python3.7.5/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x0000fffd51fbf200 (most recent call first):
  File "/usr/local/python3.7.5/lib/python3.7/threading.py", line 296 in wait
  File "/usr/local/python3.7.5/lib/python3.7/threading.py", line 552 in wait
  File "/usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/training/coordinator.py", line 311 in wait_for_stop
  File "/usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/training/queue_runner_impl.py", line 293 in _close_on_stop
  File "/usr/local/python3.7.5/lib/python3.7/threading.py", line 870 in run
  File "/usr/local/python3.7.5/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/local/python3.7.5/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x0000fffd527cf200 (most recent call first):
  File "/usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1443 in _call_tf_sessionrun
  File "/usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/clieAborted
[test_user01@localhost code]$ /usr/local/python3.7.5/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 91 leaked semaphores to clean up at shutdown
  len(cache))

评论 (1)

李永清 创建了Bug-Report
李永清 关联仓库设置为Ascend/modelzoo
wangxiaodan1103 负责人设置为张晓龙
wangxiaodan1103 任务状态TODO 修改为Analysing
展开全部操作日志

如定位结果,改问题是python残留进程导致。
运行过程中如果有ctrl+c操作,建议pkill下python后,再次执行即可。

贾正平 任务状态Analysing 修改为DONE
吴定远 关联仓库Ascend/modelzoo-his 修改为Ascend/modelzoo

登录 后才可以发表评论

状态
负责人
项目
里程碑
Pull Requests
关联的 Pull Requests 被合并后可能会关闭此 issue
分支
开始日期   -   截止日期
-
置顶选项
优先级
预计工期 (小时)
参与者(2)
1
https://gitee.com/ascend/modelzoo.git
git@gitee.com:ascend/modelzoo.git
ascend
modelzoo
modelzoo

搜索帮助