amp_level为O3M,O2对dense的参数W无效,无法将其转为fp16，而是fp32

| name       | about	                             | labels   |
| ---------- | ------------------------------------- | -------- |
| Bug Report | Use this template for reporting a bug | kind/bug |

### Describe the current behavior / 问题描述 (Mandatory / 必填)
 amp_level对dense的参数W无效
 
### Environment / 环境信息 (Mandatory / 必填)
- **Hardware Environment(`Ascend`/`GPU`/`CPU`)  / 硬件环境**: GPU 3080 12G
> Please delete the backend not involved / 请删除不涉及的后端:
> /device ascend/GPU/CPU/kirin/等其他芯片

- **Software Environment / 软件环境 (Mandatory / 必填)**:
-- MindSpore version (e.g., 1.7.0.Bxxx) :mindspore 2.2.3
-- Python version (e.g., Python 3.7.5) :python 3.9.5
-- OS platform and distribution (e.g., Linux Ubuntu 16.04):WSL2 desktop docker-Ubuntu 18.04.6 LTS
-- GCC/Compiler version (if compiled from source): GCC7

- **Excute Mode / 执行模式 (Mandatory / 必填)(`PyNative`/`Graph`)**: PyNative
> Please delete the mode not involved / 请删除不涉及的模式:
> /mode pynative
> /mode graph

### Related testcase / 关联用例 (Mandatory / 必填)
 mindone/example/stable_diffusion_xl
### Steps to reproduce the issue / 重现步骤 (Mandatory / 必填)
1. 超参数amp_level为O3：python train_lokr.py \
    --config configs/training/sd_xl_base_finetune_910b.yaml \
    --data_path $data_path \
    --weight model/sd_xl_base_1.0_ms.ckpt \
    --save_path $save_path \
    --gradient_accumulation_steps 4 \
    --ms_mode 1 \
    --per_batch_size 1 \
    --device_target GPU \
    --ms_amp_level O3 \
2. 加载模型进网络
3. 开始训练程序

### Describe the expected behavior / 预期结果 (Mandatory / 必填)
 正确使用full fp16计算
### Related log / screenshot / 日志 / 截图 (Mandatory / 必填)

```
 The first step will be compiled for the graph, which may take a long time; You can come back later :)
Traceback (most recent call last):
  File "/root/mindone/examples/stable_diffusion_xl/train_lokr.py", line 712, in <module>
    train(args)
  File "/root/mindone/examples/stable_diffusion_xl/train_lokr.py", line 347, in train
    train_fn(
  File "/root/mindone/examples/stable_diffusion_xl/train_lokr.py", line 386, in train_txt2img
    loss, overflow = train_step_fn(image, *tokens)
  File "/root/mindone/examples/stable_diffusion_xl/gm/models/diffusion.py", line 170, in train_step_pynative
    vector, crossattn, concat = self.conditioner.embedding(*tokens)
  File "/root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/modules.py", line 165, in embedding
    emb_out = embedder(*token)
  File "/root/miniconda3/envs/mindspore_py39/lib/python3.9/site-packages/mindspore/nn/cell.py", line 705, in __call__
    raise err
  File "/root/miniconda3/envs/mindspore_py39/lib/python3.9/site-packages/mindspore/nn/cell.py", line 701, in __call__
    output = self._run_construct(args, kwargs)
  File "/root/miniconda3/envs/mindspore_py39/lib/python3.9/site-packages/mindspore/nn/cell.py", line 482, in _run_construct
    output = self.construct(*cast_inputs, **kwargs)
  File "/root/miniconda3/envs/mindspore_py39/lib/python3.9/site-packages/mindspore/common/api.py", line 718, in staging_specialize
    out = _MindsporeFunctionExecutor(func, hash_obj, input_signature, process_obj, jit_config)(*args, **kwargs)
  File "/root/miniconda3/envs/mindspore_py39/lib/python3.9/site-packages/mindspore/common/api.py", line 121, in wrapper
    results = fn(*arg, **kwargs)
  File "/root/miniconda3/envs/mindspore_py39/lib/python3.9/site-packages/mindspore/common/api.py", line 350, in __call__
    raise err
  File "/root/miniconda3/envs/mindspore_py39/lib/python3.9/site-packages/mindspore/common/api.py", line 344, in __call__
    phase = self.compile(self.fn.__name__, *args_list, **kwargs)
  File "/root/miniconda3/envs/mindspore_py39/lib/python3.9/site-packages/mindspore/common/api.py", line 443, in compile
    is_compile = self._graph_executor.compile(self.obj, compile_args, kwargs, phase, True)
TypeError: For primitive[Dense], the input type must be same.
name:[w]:Ref[Tensor[Float32]].
name:[x]:Tensor[Float16].
----------------------------------------------------
- C++ Call Stack: (For framework developers)
----------------------------------------------------
mindspore/core/utils/check_convert_utils.cc:1029 _CheckTypeSame

----------------------------------------------------
- The Traceback of Net Construct Code:
----------------------------------------------------
# 0 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/modules.py:362
        if self.layer == "last":
# 1 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/modules.py:364
        elif self.layer == "pooled":
# 2 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/modules.py:361
        ) = self.get_unweighted_text_embeddings_SDXL1(tokens)
            ^
# 3 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/modules.py:309
        if max_embeddings_multiples > 1:
        ^
# 4 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/modules.py:349
            (last_hidden_state, pooler_output, hidden_states, attentions) = self.embedding(
# 5 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/modules.py:373
        return self.transformer(input_ids=input_ids, output_hidden_states=output_hidden_states)
               ^
# 6 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/clip/modeling_clip.py:459
        return self.text_model(
# 7 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/clip/modeling_clip.py:299
        if input_ids is None:
# 8 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/clip/modeling_clip.py:311
        encoder_outputs = self.encoder(
# 9 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/clip/modeling_clip.py:311
        encoder_outputs = self.encoder(
                          ^
# 10 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/clip/modeling_clip.py:229
        for idx, encoder_layer in enumerate(self.layers):
        ^
# 11 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/clip/modeling_clip.py:311
        encoder_outputs = self.encoder(
                          ^
# 12 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/clip/modeling_clip.py:233
            layer_outputs = encoder_layer(
# 13 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/clip/modeling_clip.py:168
        hidden_states, attn_weights = self.self_attn(
# 14 In file /root/mindone/examples/stable_diffusion_xl/gm/modules/embedders/clip/modeling_clip.py:83
        query_states = self.q_proj(hidden_states) * self.scale
                       ^
# 15 In file /root/mindnlp/mindnlp/injection.py:598
        if LESS_MS_2_2:
        ^
# 16 In file /root/mindnlp/mindnlp/injection.py:609
        return ops.dense(x, self.weight, self.bias)
               ^
# 17 In file /root/mindnlp/mindnlp/injection.py:69
        if GLOBAL_FP16_PATCH:
        ^
# 18 In file /root/mindnlp/mindnlp/injection.py:78
        return func(*args, **kwargs)
               ^
# 19 In file /root/mindnlp/mindnlp/injection.py:210
    return dense_(input, weight, bias)
```

### Special notes for this issue/备注 (Optional / 选填)

GVP MindSpore/mindspore

内容风险标识

评论 (7)

GVPMindSpore/mindspore

内容风险标识