# MagicAnimate

**Repository Path**: s_c_k/magic-animate

## Basic Information

- **Project Name**: MagicAnimate
- **Description**: MagicAnimate：使用扩散模型的时间一致的人体图像动画
- **Primary Language**: Python
- **License**: BSD-3-Clause
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 3
- **Created**: 2024-03-08
- **Last Updated**: 2024-03-08

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

<!-- # magic-edit.github.io -->

<p align="center">

  <h2 align="center">用扩散模型进行的时间一致的人类形象动画</h2>
  <p align="center">
    <a href="https://scholar.google.com/citations?user=-4iADzMAAAAJ&hl=en"><strong>Zhongcong Xu</strong></a>
    ·
    <a href="http://jeff95.me/"><strong>Jianfeng Zhang</strong></a>
    ·
    <a href="https://scholar.google.com.sg/citations?user=8gm-CYYAAAAJ&hl=en"><strong>Jun Hao Liew</strong></a>
    ·
    <a href="https://hanshuyan.github.io/"><strong>Hanshu Yan</strong></a>
    ·
    <a href="https://scholar.google.com/citations?user=stQQf7wAAAAJ&hl=en"><strong>Jia-Wei Liu</strong></a>
    ·
    <a href="https://zhangchenxu528.github.io/"><strong>Chenxu Zhang</strong></a>
    ·
    <a href="https://sites.google.com/site/jshfeng/home"><strong>Jiashi Feng</strong></a>
    ·
    <a href="https://sites.google.com/view/showlab"><strong>Mike Zheng Shou</strong></a>
    <br>
    <br>
        <a href="https://arxiv.org/abs/2311.16498"><img src='https://img.shields.io/badge/arXiv-MagicAnimate-red' alt='Paper PDF'></a>
        <a href='https://showlab.github.io/magicanimate'><img src='https://img.shields.io/badge/Project_Page-MagicAnimate-green' alt='Project Page'></a>
        <a href='https://huggingface.co/spaces/zcxu-eric/magicanimate'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue'></a>
    <br>
    <b>National University of Singapore &nbsp; | &nbsp;  ByteDance</b>
  </p>
  
  <table align="center">
    <tr>
    <td>
      <img src="assets/teaser/t4.gif">
    </td>
    <td>
      <img src="assets/teaser/t2.gif">
    </td>
    </tr>
  </table>

## 📢 News
* **[2023.12.4] 发布推理代码和G无线电演示。我们正在努力改进魔术动画,继续收看!
* **[2023.11.23] 释放魔术动画纸和项目页。

## 🏃‍♂️  开始使用
请下载预训练模型[StableDiffusion V1.5](https://huggingface.co/runwayml/stable-diffusion-v1-5)和 [MSE-finetuned VAE](https://huggingface.co/stabilityai/sd-vae-ft-mse).

下载我们的 MagicAnimate [checkpoints](https://huggingface.co/zcxu-eric/MagicAnimate).

**按如下方式放置它们：**
```bash
magic-animate
|----pretrained_models
  |----MagicAnimate
    |----appearance_encoder
      |----diffusion_pytorch_model.safetensors
      |----config.json
    |----densepose_controlnet
      |----diffusion_pytorch_model.safetensors
      |----config.json
    |----temporal_attention
      |----temporal_attention.ckpt
  |----sd-vae-ft-mse
    |----...
  |----stable-diffusion-v1-5
    |----...
|----...
```

## ⚒️ 安装
先决条件： `python>=3.8`, `CUDA>=11.3`, and `ffmpeg`.

安装 `conda`: 
```bash
conda env create -f environment.yaml
conda activate manimate
```
或 `pip`:
```bash
pip3 install -r requirements.txt
```

## 💃 推理
在单个 GPU 上运行推理：
```bash
bash scripts/animate.sh
```
使用多个 GPU 运行推理：
```bash
bash scripts/animate_dist.sh
```

## 🎨 Gradio Demo 

#### 在线电台演示：
快速尝试我们的 [online gradio demo](https://huggingface.co/spaces/zcxu-eric/magicanimate) .

#### 本地电台演示：
在单 GPU 上启动本地 gradio 演示：
```bash
python3 -m demo.gradio_animate
```
如果您有多个 GPU，请启动本地 gradio 演示：
```bash
python3 -m demo.gradio_animate_dist
```
然后在本地浏览器中打开gradio demo。

## 🙏 致谢
我们要感谢 [AK(@_akhaliq)](https://twitter.com/_akhaliq?lang=en) 和huggingface 团队帮助设置oneline gradio演示

## 🎓 引文
如果您发现此代码库对您的研究有用，请使用以下条目。
```BibTeX
@inproceedings{xu2023magicanimate,
    author    = {Xu, Zhongcong and Zhang, Jianfeng and Liew, Jun Hao and Yan, Hanshu and Liu, Jia-Wei and Zhang, Chenxu and Feng, Jiashi and Shou, Mike Zheng},
    title     = {MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model},
    booktitle = {arXiv},
    year      = {2023}
}
```