log:
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/local/python-3.7.5/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/local/python-3.7.5/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/python-3.7.5/lib/python3.7/multiprocessing/pool.py", line 470, in _handle_results
    task = get()
  File "/usr/local/python-3.7.5/lib/python3.7/multiprocessing/connection.py", line 251, in recv
    return _ForkingPickler.loads(buf.getbuffer())
ModuleNotFoundError: No module named 'tvm'

Traceback (most recent call last):
  File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/_extends/parallel_compile/akg_compiler/akg_process.py", line 128, in compile
    res.get(timeout=self.wait_time)
  File "/usr/local/python-3.7.5/lib/python3.7/multiprocessing/pool.py", line 653, in get
    raise TimeoutError
multiprocessing.context.TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/_extends/remote/kernel_build_server_gpu.py", line 87, in <module>
    messager.run()
  File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/_extends/remote/kernel_build_server.py", line 119, in run
    self.loop()
  File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/_extends/remote/kernel_build_server.py", line 116, in loop
    self.handle()
  File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/_extends/remote/kernel_build_server_gpu.py", line 56, in handle
    res = self.akg_builder.compile()
  File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/_extends/remote/kernel_build_server.py", line 33, in compile
    return self.akg_builder.compile()
  File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/_extends/parallel_compile/akg_compiler/akg_process.py", line 128, in compile
    res.get(timeout=self.wait_time)
  File "/usr/local/python-3.7.5/lib/python3.7/multiprocessing/pool.py", line 623, in __exit__
    self.terminate()
  File "/usr/local/python-3.7.5/lib/python3.7/multiprocessing/pool.py", line 548, in terminate
    self._terminate()
  File "/usr/local/python-3.7.5/lib/python3.7/multiprocessing/util.py", line 201, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/python-3.7.5/lib/python3.7/multiprocessing/pool.py", line 585, in _terminate_pool
    "Cannot have cache with result_hander not alive")
AssertionError: Cannot have cache with result_hander not alive
[ERROR] SESSION(3269,python):2021-05-12-09:48:57.778.803 [mindspore/ccsrc/backend/session/kernel_build_client.h:110] Response] Response is empty
Traceback (most recent call last):
  File "integrate_checkpoint.py", line 217, in <module>
    integrate_ckpt_file()
  File "integrate_checkpoint.py", line 141, in integrate_ckpt_file
    output_ids = generate(model_predict, input_ids, config.seq_length, 9)
  File "/userhome/pclproject/gpt/transformModelToGPU/generate.py", line 46, in generate
    logits = model.predict(ms.Tensor(input_ids, ms.int32)).asnumpy().reshape(1, seq_length, -1)
  File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/train/model.py", line 791, in predict
    result = self._predict_network(*predict_data)
  File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/nn/cell.py", line 341, in __call__
    out = self.compile_and_run(*inputs)
  File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/nn/cell.py", line 608, in compile_and_run
    self.compile(*inputs)
  File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/nn/cell.py", line 595, in compile
    _executor.compile(self, *inputs, phase=self.phase, auto_parallel_mode=self._auto_parallel_mode)
  File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/common/api.py", line 494, in compile
    result = self._executor.compile(obj, args_list, phase, use_vm)
RuntimeError: mindspore/ccsrc/backend/session/kernel_build_client.h:110 Response] Response is empty