open-sora

环境：300duo卡
cann:8.0.RC2/
mindie:MindIE_1.0.RC2
代码：https://gitee.com/ascend/ModelZoo-PyTorch/tree/master/MindIE/MindIE-Torch/built-in/foundation/open-sora

模型运行于报错：
(MindIE_1.0.RC2) root@fb48a65a143f:/data/open-sora# python3 inference.py ./configs/opensora/inference/16x256x256.py  --ckpt-path ./OpenSora-v1-HQ-16x12x512.pth --prompt-path ./assets/texts/t2v_samples.txt --use_mindie 1 --device_id 0
[W compiler_depend.ts:623] Warning: expandable_segments currently defaults to false. You can enable this feature by `export PYTORCH_NPU_ALLOC_CONF = xpandable_segments:True`. (function operator())
/root/miniconda3/envs/MindIE_1.0.RC2/lib/python3.10/site-packages/bitsandbytes/cextension.py:34: UserWarning: The installed version of bitsandbytes ws compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
  warn("The installed version of bitsandbytes was compiled without GPU support. "
'NoneType' object has no attribute 'cadam32bit_grad_fp32'
/root/miniconda3/envs/MindIE_1.0.RC2/lib/python3.10/site-packages/colossalai/shardformer/layer/normalization.py:45: UserWarning: Please install apex rom source (https://github.com/NVIDIA/apex) to use the fused RMSNorm kernel
  warnings.warn("Please install apex from source (https://github.com/NVIDIA/apex) to use the fused RMSNorm kernel")
/root/miniconda3/envs/MindIE_1.0.RC2/lib/python3.10/site-packages/diffusers/models/transformers/transformer_2d.py:34: FutureWarning: `Transformer2DMoelOutput` is deprecated and will be removed in version 1.0.0. Importing `Transformer2DModelOutput` from `diffusers.models.transformer_2d` is deprecatd and this will be removed in a future version. Please use `from diffusers.models.modeling_outputs import Transformer2DModelOutput`, instead.
  deprecate("Transformer2DModelOutput", "1.0.0", deprecation_message)
Config (path: ./configs/opensora/inference/16x256x256.py): {'num_frames': 16, 'fps': 8, 'image_size': (256, 256), 'model': {'type': 'STDiT-XL/2', 'spce_scale': 0.5, 'time_scale': 1.0, 'enable_flashattn': False, 'enable_layernorm_kernel': False, 'from_pretrained': './OpenSora-v1-HQ-16x512x512.pth'} 'scheduler': {'type': 'iddpm', 'num_sampling_steps': 100, 'cfg_scale': 7.0}, 'dtype': 'fp32', 'batch_size': 1, 'seed': 42, 'prompt_path': './assets/exts/t2v_samples.txt', 'save_dir': './outputs/samples/', 'vae_path': 'stabilityai/sd-vae-ft-ema', 't5_path': './t5-v1_1-xxl', 'use_mindie': 1, 'devic_id': 0, 'output_dir': './models', 'multi_resolution': False}
inference by MindIE
[2024-09-11 01:49:52][warning](thread 16275)Profiling env variable isn't set.(Profiling.cpp:36)
./t5-v1_1-xxl
Loading checkpoint shards:   0%|                                                                                                | 0/2 [00:00<?, ?it/s/root/miniconda3/envs/MindIE_1.0.RC2/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed inthe future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStrage directly, use tensor.untyped_storage() instead of tensor.storage()
  return self.fget.__get__(instance, owner)()
Loading checkpoint shards:  50%|████████████████████████████████████████████                                            | 1/2 [00:10<00:10, 10.69s/itLoading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:21<00:00, 10.55s/itLoading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:21<00:00, 10.57s/it]
[2024-09-11 01:50:21][warning](thread 16275)Profiling env variable isn't set.(Profiling.cpp:36)
[2024-09-11 01:50:36][warning](thread 16275)Profiling env variable isn't set.(Profiling.cpp:36)
[2024-09-11 01:50:37][warning](thread 16275)Get optimization cache failed!(OmParser.cpp:201)
Missing keys: ['pos_embed', 'pos_embed_temporal']
Unexpected keys: []
Traceback (most recent call last):
  File "/data/open-sora/inference.py", line 142, in <module>
    main()
  File "/data/open-sora/inference.py", line 113, in main
    samples = scheduler.sample(
  File "/data/open-sora/opensora/schedulers/iddpm/__init__.py", line 67, in sample
    model_args = text_encoder.encode(prompts)
  File "/data/open-sora/opensora/models/text_encoder/t5.py", line 332, in encode
    caption_embs, emb_masks = self.t5.get_text_embeddings(text)
  File "/data/open-sora/opensora/models/text_encoder/t5.py", line 134, in get_text_embeddings
    text_encoder_embs = self.model_npu(input_ids.to(f"npu:{self.device_id}"),
RuntimeError: unknown format type:6
[ERROR] 2024-09-11-01:50:53 (PID:16275, Device:0, RankID:-1) ERR99999 UNKNOWN application exception

Ascend/modelzoo
暂停

内容风险标识

评论 (0)

Ascend/modelzoo暂停 .gitee-modal { width: 500px !important; }

内容风险标识

open-sora

评论 (0)

搜索帮助

Ascend/modelzoo
暂停