[ST][MF][llama2_70b]TypeError: For Operator[ReshapeAndCache], slot_mapping's type 'None' does not match expected type 'Tensor'.

| name       | about	                             | labels   |
| ---------- | ------------------------------------- | -------- |
| Bug Report | Use this template for reporting a bug | kind/bug |

### Describe the current behavior / 问题描述 (Mandatory / 必填)
 llama2_70b网络在线推理、阅读理解评测都失败了
模型仓地址：https://gitee.com/mindspore/mindformers/blob/dev/docs/model_cards/llama2.md#%E8%AF%84%E6%B5%8B
 
### Environment / 环境信息 (Mandatory / 必填)
- **Hardware Environment(`Ascend`/`GPU`/`CPU`)  / 硬件环境**: 
> Please delete the backend not involved / 请删除不涉及的后端:
> /device ascend

- **Software Environment / 软件环境 (Mandatory / 必填)**:
-- MindSpore version (e.g., 1.7.0.Bxxx) :
-- Python version (e.g., Python 3.7.5) :
-- OS platform and distribution (e.g., Linux Ubuntu 16.04):
-- GCC/Compiler version (if compiled from source):

MS：r2.3_20240412121517_aa16e0b8938
MF：dev_20240412121521_5ec1709705f
CANN：Milan_C17/20240408

- **Excute Mode / 执行模式 (Mandatory / 必填)(`PyNative`/`Graph`)**: 
> Please delete the mode not involved / 请删除不涉及的模式:
> /mode pynative
> /mode graph

### Related testcase / 关联用例 (Mandatory / 必填)

用例仓地址：MindFormers_Test/cases/llama2/70b/infer/
用例：
test_mf_llama2_70b_infer_alpaca_incremental_8p_0001
test_mf_llama2_70b_infer_alpaca_batch_incremental_8p_0001
test_mf_llama2_70b_eval_squad_8p_0001
 
### Steps to reproduce the issue / 重现步骤 (Mandatory / 必填)
1. get code from mindformers
2.  cd mindformers/scripts
3. bash run_distribute.sh /home/workspace/config/hccl_8p.json ./configs/llama2/run_llama2_70b_910b.yaml [0,8] eval
4. 验证网络是否推理成功

### Describe the expected behavior / 预期结果 (Mandatory / 必填)
 网络推理成功
### Related log / screenshot / 日志 / 截图 (Mandatory / 必填)
 ```
[CRITICAL] ANALYZER(2125246,ffff9c038020,python3):2024-04-13-14:04:17.904.717 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:1263] CheckArgsSizeAndType] For Operator[ReshapeAndCache], slot_mapping's type 'None' does not match expected type 'Tensor'.
The reason may be: lack of definition of type cast, or incorrect type when creating the node.
This exception is caused by framework's unexpected error. Please create an issue at https://gitee.com/mindspore/mindspore/issues to get help.
ckpt path: %s /home/workspace/large_model_ckpt//llama2/70b/transformed_checkpoint/rank_0/checkpoint_0.ckpt
Traceback (most recent call last):
  File "./llama2_70b_predict_custom.py", line 117, in <module>
    main(args)
  File "./llama2_70b_predict_custom.py", line 68, in main
    ms.Tensor(np.ones(shape=(args.batch_size, model_config.seq_length)), ms.int32))
  File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/train/model.py", line 1903, in infer_predict_layout
    predict_net.compile(*predict_data)
  File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/nn/cell.py", line 983, in compile
    jit_config_dict=self._jit_config_dict, **kwargs)
  File "/home/miniconda3/envs/ci/lib/python3.7/site-packages/mindspore/common/api.py", line 1590, in compile
    result = self._graph_executor.compile(obj, args, kwargs, phase, self._use_vm_mode())
TypeError: For Operator[ReshapeAndCache], slot_mapping's type 'None' does not match expected type 'Tensor'.
The reason may be: lack of definition of type cast, or incorrect type when creating the node.

----------------------------------------------------
- Framework Unexpected Exception Raised:
----------------------------------------------------
This exception is caused by framework's unexpected error. Please create an issue at https://gitee.com/mindspore/mindspore/issues to get help.

----------------------------------------------------
- C++ Call Stack: (For framework developers)
----------------------------------------------------
mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:1263 CheckArgsSizeAndType

----------------------------------------------------
- The Traceback of Net Construct Code:
----------------------------------------------------
# 0 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:379
        if self.use_past:
# 1 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:380
            if not isinstance(batch_valid_length, Tensor):
# 2 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:382
        if self.training:
# 3 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:385
            tokens = input_ids
            ^
# 4 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:388
        if not self.is_first_iteration:
# 5 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:392
        if pre_gather:
# 6 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:390
        output = self.model(tokens, batch_valid_length, batch_index, zactivate_len, block_tables, slot_mapping)
                 ^
# 7 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:249
        if self.use_past:
# 8 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:250
            freqs_cis = (self.freqs_mgr.freqs_cos, self.freqs_mgr.freqs_sin, self.freqs_mgr.swap_mask)
            ^
# 9 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:390
        output = self.model(tokens, batch_valid_length, batch_index, zactivate_len, block_tables, slot_mapping)
                 ^
# 10 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:259
        for i in range(self.num_layers):
# 11 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:260
            h = self.layers[i](h, freqs_cis, mask, batch_valid_length=batch_valid_length, block_tables=block_tables,
                ^
# 12 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama_transformer.py:510
        if not self.use_past:
# 13 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama.py:260
            h = self.layers[i](h, freqs_cis, mask, batch_valid_length=batch_valid_length, block_tables=block_tables,
                ^
# 14 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama_transformer.py:517
        h = self.attention(input_x, freqs_cis, mask, batch_valid_length, block_tables, slot_mapping)
            ^
# 15 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama_transformer.py:249
        if self.qkv_concat:
# 16 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama_transformer.py:259
            query = self.cast(self.wq(x), self.dtype)  # dp, 1 -> dp, mp
            ^
# 17 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama_transformer.py:264
        if self.use_past:
# 18 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama_transformer.py:265
            freqs_cos, freqs_sin, _ = freqs_cis
# 19 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/models/llama/llama_transformer.py:266
            context_layer = self.infer_attention(query, key, value, batch_valid_length, block_tables, slot_mapping,
                            ^
# 20 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/modules/infer_attention.py:286
        if self.is_first_iteration:
# 21 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/modules/infer_attention.py:287
            position_ids = self.context_position_ids
            ^
# 22 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/modules/infer_attention.py:290
        if self.use_rope_rotary_emb:
# 23 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/modules/infer_attention.py:291
            query, key = self.apply_rotary_pos_emb(query, key, freqs_cos, freqs_sin, position_ids)
            ^
# 24 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/modules/infer_attention.py:295
        if self.is_first_iteration:
# 25 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/modules/infer_attention.py:296
            if self.input_layout == "BSH":
            ^
# 26 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/modules/infer_attention.py:297
                context_layer = self.flash_attention(query, key, value, attn_mask, alibi_mask)
                ^
# 27 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/modules/infer_attention.py:296
            if self.input_layout == "BSH":
            ^
# 28 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/modules/infer_attention.py:292
        key_out = self.paged_attention_mgr(key, value, slot_mapping)
                  ^
# 29 In file /home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/mindformers/modules/paged_attention_mgr.py:73
        key_out = self.reshape_and_cache(tmp_key, tmp_value, self.key_cache, self.value_cache, slot_mapping)
                  ^
 (See file '/home/jenkins/workspace/TDT_deployment/MindFormers_Test/cases/llama2/70b/infer/test_mf_llama2_70b_infer_alpaca_incremental_8p_0001/rank_0/om/analyze_fail.ir' for more details. Get instructions about `analyze_fail.ir` at https://www.mindspore.cn/search?inputValue=analyze_fail.ir)

```
### Special notes for this issue/备注 (Optional / 选填)
走给倪钰鑫

GVP MindSpore/mindspore

内容风险标识

评论 (14)

GVPMindSpore/mindspore

内容风险标识