Loading [MathJax]/jax/output/HTML-CSS/jax.js
1 Star 0 Fork 1

qianxdong/DR2_Drgradation_Remover

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
MIT

DR2 + Enhancement

This is the implementation DR2: Diffusion-based Robust Degradation Remover for Blind face Restoration (CVPR 2023).

Overview

DR2E is a two-stage blind face restoration framework consists of the degradation remover DR2 and an enhance module which can be any existing blind face restoration model. In the first stage, DR2 utilizes the input image to control the diffusion sampling process and results in smooth, clean middle result in ˆx0 . In the second stage, the enhancement modules maps ˆx0 in to high-res and high-quality output.

overall_framework

Download pre-trained models

DR2

Our implementation of DR2 module is heavily based on ILVR_adm, and the pretrained model is trained on FFHQ dataset to generate 256×256 face images. You can download DR2 model we use for our paper from Baidu Cloud Drive , or try more pretrained weights at P2-weighting.

Enhancement Module

Variant blind face restoration models can be plugged in as the enhancement module. To use these model, you can

1. Training from Scratch

In our implementation, we choose SPARNetHD and train it from scratch. The training code and loss functions remain unchanged to the original paper, but we construct training set using DR2 augmentation (which is introduced in the paper section 3.4) as follows: y=DR2(x)kσ where x is the ground truth high-quality image, and y is the input image. This helps the enhancement module adapts faster to DR2 output. Other than this, no more degradation model is required. You can download SPARNetHD weights from Baidu Cloud Drive.

2. Use Pretrained Models

You can directly use pretrained blind face restoration methods like VQFR, CodeFormer, GPEN without further finetuning. Because these methods are trained on complex degradation model and can work fine on DR2 output.

Quick Start

First clone the repo and install dependent packages

# build dependency
pip install -r requirements.txt

Download pretrained weights for DR2 and SPARNetHD and put them in "./weights". And run demo

python demo.py

This will enhance the testing images is file "./test_images/input". Noting that for each subdirectory, we choose different controlling parameters (N,τ) (please refer to out paper). Feel free to change the controlling parameters and see how they affect the output. Noting N{1,2,4,8,16,...} and τ{0,1,2,3,..,T}. Since we inference every 10 steps for speed by default, T should be 100 rather than 1000.

# in demo.py, line 11-16
dir_para = [    
    # [data_dir, (N, \tau)]
    ["./test_images/input/01/", (4, 22)],
    ["./test_images/input/02/", (8, 35)],
    ["./test_images/input/03/", (16, 35)],
] 

The results of two stages are stored in "./test_images/output". If you installed other blind face restoration methods, you can run them on "./test_images/output/dr2" file.

Results

This the results reported in out paper and you can see our method is robust against heavy degradation and our framework enable VQFR to perform normally again.

comparison_on_celeba_test

Because the stochastic nature of diffusion sampling process, you may get different results on a single images (unless you're using very small N and τ). But we found it wouldn't affect the average quantitative score (like PSNR, SSIM, FID, LPIPS) if tested on a big testing set containing hundreds or thousands of images.

Contact

If you have any questions, please contact dedsec_z@sjtu.edu.cn

MIT License Copyright (c) 2023 ZXWang Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

简介

暂无描述 展开 收起
MIT
取消

发行版

暂无发行版

贡献者 (1)

全部

近期动态

不能加载更多了
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
1
https://gitee.com/qianxdong/DR2_Drgradation_Remover.git
git@gitee.com:qianxdong/DR2_Drgradation_Remover.git
qianxdong
DR2_Drgradation_Remover
DR2_Drgradation_Remover
main

搜索帮助

371d5123 14472233 46e8bd33 14472233