# dh_live_hl

**Repository Path**: nextteam/dh_live_hl

## Basic Information

- **Project Name**: dh_live_hl
- **Description**: 基于DH_Live的开源数字人项目，此版本是基于B站刘悦的windows整合包修改为Linux的整合包。
- **Primary Language**: Unknown
- **License**: CC-BY-SA-4.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 2
- **Forks**: 1
- **Created**: 2024-08-14
- **Last Updated**: 2025-01-19

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Real-time Live Streaming Digital Human
# 实时直播数字人  [bilibili video](https://www.bilibili.com/video/BV1Ppv1eEEgj/?vd_source=53601feee498369e726af7dbc2dae349)
### Video Example


https://github.com/user-attachments/assets/7e0b5bc2-067b-4048-9f88-961afed12478


## Overview
This project is a real-time live streaming digital human powered by few-shot learning. It is designed to run smoothly on all 30 and 40 series graphics cards, ensuring a seamless and interactive live streaming experience.

### Key Features
- **Real-time Performance**: The digital human can interact in real-time with 25+ fps for common NVIDIA 30 and 40 series GPUs
- **Few-shot Learning**: The system is capable of learning from a few examples to generate realistic responses.
## Usage

### Unzip the Model File
First, navigate to the `checkpoint` directory and unzip the model file:
```bash
cd checkpoint
gzip -d -c render.pth.gz.001 > render.pth
```
### Prepare Your Video
Next, prepare your video using the data_preparation script. Replace YOUR_VIDEO_PATH with the path to your video:
```bash
python data_preparation YOUR_VIDEO_PATH
```
The result (video_info) will be stored in the ./video_data directory.
### Run with Audio File
Run the demo script with an audio file. Make sure the audio file is in .wav format with a sample rate of 16kHz and 16-bit single channel. Replace video_data/test with the path to your video_info file, video_data/audio0.wav with the path to your audio file, and 1.mp4 with the desired output video path:
```bash
python demo.py video_data/test video_data/audio0.wav 1.mp4
```
### Real-Time Run with Microphone
For real-time operation using a microphone, simply run the following command:
```bash
python demo_avatar.py
```

## Acknowledgements 
We would like to thank the contributors of [wavlip](https://github.com/Rudrabha/Wav2Lip), [dinet](https://github.com/MRzzm/DINet), [livespeechportrait](https://github.com/YuanxunLu/LiveSpeechPortraits) repositories, for their open research and contributions.

## License
This project is licensed under the MIT License.

## Contact
For any questions or suggestions, please contact us at [kleinlee1@outlook.com].


## QA
No matching distribution found for tb-nightly
```shell
#pip install torch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0
pip install tb_nightly==2.14.0a20230808 -i https://mirrors.aliyun.com/pypi/simple
pip install realesrgan
pip install moviepy
pip install transformers~=4.41.1
#pip install --upgrade torchvision
```
No module named 'torchvision.transforms.functional_tensor'
```shell
/home/mty/miniconda3/envs/dhlive311/lib/python3.11/site-packages/basicsr/data/degradations.py
修改：
torchvision.transforms.functional_tensor
为
torchvision.transforms._functional_tensor
```

```shell
#下载模型到conda中，需要指定模型路径，不要下载
Downloading: "https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.4.pth" to /home/mty/miniconda3/envs/dhlive311/lib/python3.11/site-packages/gfpgan/weights/GFPGANv1.4.pth
```