# DualAnoDiff **Repository Path**: scauzhijun/DualAnoDiff ## Basic Information - **Project Name**: DualAnoDiff - **Description**: 在工业制造中，异常检测任务（如检测、定位和分类）对保障产品质量起着关键作用。然而，现实中异常样本稀缺，这使得当前主流的异常检测方法在异常定位和分类任务上表现不佳。DualAnoDiff的提出，为解决这一难题提供了新的思路。该模型利用双相关扩散模型，同时生成整体图像和对应的异常部分，有效解决了现有异常生成方法中存在的生成异常多样性有限、难以与原始图像无缝融合以及生成掩码与异常不匹配等问题。 - **Primary Language**: Python - **License**: Not specified - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 1 - **Forks**: 0 - **Created**: 2025-08-07 - **Last Updated**: 2025-09-10 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README

Dual-Interrelated Diffusion Model for Few-Shot Anomaly Image Generation(CVPR2025)


Ying Jin, Jinlong Peng2#, Qingdong He2#, Teng Hu3, Hao Chen1, Haoxuan Wang1, Jiafu Wu2, wenbing zhu1, Mingmin Chi1*, Jun Liu2, Yabiao Wang2,4 (#Equal contribution,*Corresponding author) 1Fudan University, 2Youtu Lab, Tencent, 3Shanghai Jiao Tong University, 4Zhe Jiang University

arXiv
## Abstract The performance of anomaly inspection in industrial manufacturing is constrained by the scarcity of anomaly data. To overcome this challenge, researchers have started employing anomaly generation approaches to augment the anomaly dataset. However, existing anomaly generation methods suffer from limited diversity in the generated anomalies and struggle to achieve a seamless blending of this anomaly with the original image. Moreover, the generated mask is usually not aligned with the generated anomaly. In this paper, we overcome these challenges from a new perspective, simultaneously generating a pair of the overall image and the corresponding anomaly part. We propose **DualAnoDiff**, a novel diffusion-based few-shot anomaly image generation model, which can generate diverse and realistic anomaly images by using a dual-interrelated diffusion model, where one of them is employed to generate the whole image while the other one generates the anomaly part. Moreover, we extract background and shape information to mitigate the distortion and blurriness phenomenon in few-shot image generation. Extensive experiments demonstrate the superiority of our proposed model over state-of-the-art methods in terms of diversity, realism and the accuracy of mask. Overall, our approach significantly improves the performance of downstream anomaly inspection tasks, including anomaly detection, anomaly localization, and anomaly classification tasks. Code will be made available. # ✨Overview image # Getting Started Download [stable-diffusion-v1-5](https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5). ``` git clone https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5 ``` Environment Setup and Run. ``` pip -r requirments.txt cd dual-interrelated_diff # or bcm-dual-interrelated_diff sh run_mvtec_split.py # run_mvtec_split.py includes operations for training, inference, and mask generation. Since our method involves training the model for a single category, it is necessary to modify the name in the run_mvtec_split.py file, which represents the category to be generated from Mvtec. ``` To see the usage and instructions for U2-Net, please refer to: [U-2-Net](https://github.com/xuebinqin/U-2-Net) # Detail steps 0. Environment: ``` accelerate==0.24.1 clip Cython==0.29.35 matplotlib==3.8.0 numpy==1.24.3 open-clip-torch==2.23.0 opencv-python==4.7.0.72 opencv-python-headless==.7.0.72 pandas==2.0.3 Pillow==9.4.0 pytorch-lightning==1.5.0 PyYAML==6.0 scikit-image==0.22.0 scikit-learn==1.3.2 scipy==1.10.1 setuptools==65.6.3 tensorboard==.15.0 timm==0.4.12 torch==2.0.1+cu118 torchaudio==2.0.2+cu118 torchmetrics==0.6.0 torchvision==0.15.2+cu118 transformers==4.30.2 ``` 1. Download [stable-diffusion-v1-5](https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5). ``` git clone https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5 ``` 2. Train After running, the model weights will be saved in "all_generate". For different amounts of training data, you can choose the most appropriate steps by visualizing the generated results. ``` cd dual-interrelated_diff sh train.sh ``` 3. Infer You can modify the "guidance_scale" to observe the generated results that are closer in color to the training data. ``` python inference_mvtec_split.py hazelnut hole ``` 4. Get Mask Referencing [U-2-Net](https://github.com/xuebinqin/U-2-Net), use this model to segment the generated fg files. # Data and checkpoints | | url | |----------|----------| | Checkpoints for anomaly generation | [url](https://pan.quark.cn/s/19f5cbefc100) | | Checkpoints for anomaly generation with BCM | [url](https://pan.quark.cn/s/7ea8f1f13b11) | | 1-shot generate images | [url](https://pan.quark.cn/s/51d266cedef6) | # Result ![image](https://github.com/user-attachments/assets/7128b95d-3a35-4838-ad88-c2150afdee2d) ## Comparison in Anomaly Generation ### Anomaly Generation Quality ![image](https://github.com/user-attachments/assets/196d6147-f010-4c69-a5d5-89df94a80bb6) ### Anomaly Generation for Anomaly Detection and Localization ![image](https://github.com/user-attachments/assets/18e29fe2-b613-4fc2-98e3-1a5f2860b8a1) ## Comparison with Anomaly Detection Models ![image](https://github.com/user-attachments/assets/f793f984-e746-4d2d-bc1b-8d50144a0eb2) # Citation ``` @article{jin2024dualanodiff, title={DualAnoDiff: Dual-Interrelated Diffusion Model for Few-Shot Anomaly Image Generation}, author={Jin, Ying and Peng, Jinlong and He, Qingdong and Hu, Teng and Chen, Hao and Wu, Jiafu and Zhu, Wenbing and Chi, Mingmin and Liu, Jun and Wang, Yabiao and others}, journal={arXiv preprint arXiv:2408.13509}, year={2024} } ```