# UDRSIS
**Repository Path**: linweichiu/UDRSIS
## Basic Information
- **Project Name**: UDRSIS
- **Description**: [JSTARS 2025] Official Implementation for UDRSIS, Unsupervised Deep Image Stitching for Remote Sensing: Aligning Features Across Large-Scale Geometric Distortions
- **Primary Language**: Python
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No
## Statistics
- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-10-06
- **Last Updated**: 2025-10-06
## Categories & Tags
**Categories**: Uncategorized
**Tags**: None
## README
Unsupervised Deep Image Stitching for Remote Sensing: Aligning Features Across Large-Scale Geometric Distortions
Linwei Qiu1,2, Chang Liu3, Gongzhe Li4, Xiaomeng Dong5, Fengying Xie1,2, Zhenwei Shi2
1 Tianmushan Laboratory, Beihang
University, Hangzhou, China
2 Department of Aerospace Intelligence Science and Technology, School of Astronautics, Beihang University, Beijing, China
3 Aerospace Information Research Institute,
Chinese Academy of Sciences, Beijing, China.
4 School of Data Science, The Chinese University of Hong Kong (Shenzhen), Shenzhen, China.
5 Qian Xuesen Laboratory of Space Technology,
China Academy of Space Technology, Beijing, China
Fig
## Introduction
This is the PyTorch implementation of the paper [**Unsupervised Deep Image Stitching for Remote Sensing: Aligning Features Across Large-Scale Geometric Distortions**](https://ieeexplore.ieee.org/document/11164964) [JSTARS 2025].
## Requirements
- Packages
The code was tested with Anaconda and Python 3.10.13. The Anaconda environment is:
- pytorch = 2.1.1
- torchvision = 0.16.1
- cudatoolkit = 11.8
- tensorboard = 2.17.0
- tensorboardX = 2.6.2.2
- opencv-python = 4.9.0.80
- numpy = 1.26.4
- pillow = 10.3.0
Install dependencies:
- For PyTorch dependency, see [pytorch.org](https://pytorch.org/) for more details.
- For custom dependencies:
```bash
conda install tensorboard tensorboardx
pip install tqdm opencv-python thop scikit-image lpips scipy
```
- We implement this work with Ubuntu 18.04, NVIDIA Tesla V100, and CUDA11.8.
## Datasets
- Put data in `../dataset` folder or configure your dataset path in the `my_path` function of `dataloaders/__inint__.py`.
- These codes support the **UDIS-D** ([Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images](http://arxiv.org/abs/2106.12859) [TIP 2021]). You can download it at [Google Drive](https://drive.google.com/drive/folders/1kC7KAULd5mZsqaWnY3-rSbQLaZ7LujTY?usp=sharing) or [Baidu Cloud](https://pan.baidu.com/s/13KZ29e487datgtMgmb9laQ) (Extraction code: 1234).
- For UDIS, it is needed to download the [**WarpedCOCO**](https://pan.baidu.com/s/1MVn1VFs_6-9dNRVnG684og) (Baidu Cloud Extraction Code: 1234) dataset for pre-training.
- The details of the dataset **UDAIS-D** (Unsupervised Deep Stitching of Aerial Images Dataset) can be found in our paper ([Unsupervised Deep Image Stitching for Remote Sensing: Aligning Features Across Large-Scale Geometric Distortions](https://ieeexplore.ieee.org/document/11164964) [JSTARS 2025]). You can download it at [Baidu Cloud](https://pan.baidu.com/s/1U0Bw7DZGM5J8mAK8mwsxFw?pwd=1234) (Extraction code: 1234).
- The details of the dataset **UDAIS-D+** can be found in our paper ([Radiation-Tolerant Unsupervised Deep Image Stitching for Remote Sensing](https://ieeexplore.ieee.org/document/11146793) [TGRS 2025]). You can download it at [Baidu Cloud](https://pan.baidu.com/s/1cXSOosWZbIg9CSu_iRmSyw?pwd=1234) (Extraction code: 1234).
## Unsupervised Alignment
For input arguments: (see full input arguments via `python train_align.py --help` or `python test_align.py --help`):
### Model Training
Follow steps below to train your model:
**Step 0: SIFT-Guided Curriculum Learning**
Firtst, you need to prepare the SIFT-guided training dataset by running the following command:
```bash
python SIFT_prepare.py --feature_type SIFT --dataset UDAIS-D --save_path ../../dataset/UDIS-D-sup/training-SIFT/
```
Note:
1. You can change the feature type from SIFT to SURF by changing the `--feature_type` argument.
2. You can change the dataset from **UDIS-D** to **UDAIS-D** or **UDAIS-D+** by changing the `--dataset` argument.
3. `--save_path` is the path to save the SIFT-guided training dataset. The default path is `../../dataset/UDAIS-D-sup/training-SIFT/`. You need to keep the same path for the `--save_path` argument and the `my_sup_path` and `make_align_sup_data_loader` functions in the `dataloader/__init__.py` script.
4. You can custom your own dataset and set the input path by changing the `--input_path` argument.
Then, you can train your model with the following command:
```bash
CUDA_VISIBLE_DEVICES=0 python train_align_sup.py --dataset UDAIS-D --model UDRSIS --epochs 200 -b 4 --loss-type UDRSIS --freq_record 1000 --height 512 --width 512 --run_path ./run_align/ --lam_mse 1.0 --lam_sup 0.0 --lam_unsup 0.0
```
**Step 1: Unsupervised training on UDAIS-D**
You can train your model with the following command:
```bash
CUDA_VISIBLE_DEVICES=0 python train_align.py --dataset UDAIS-D --model UDRSIS --epochs 200 -b 4 --loss-type UDRSIS --freq_record 600 --height 512 --width 512 --run_path ./run_align/ --lam_lp1 3.0 --lam_lp2 1.0 --lam_grid 10 --lam_grad 0.5 --lr 1e-4 --resume ./run_align/UDAIS-D/UDRSIS/experiment_0/epoch200_model.pth --ft --freq_save 20
```
Note: You can change the dataset from **UDIS-D** to **UDAIS-D** or **UDAIS-D+** by changing the `--dataset` argument. `--resume` is the path to the trained model by **SIFT-Guided Curriculum Learning**.
### Model Testing
**Step 2: Evaluating the alignment results**
Using the trained model, you can evaluate the alignment results with the following command in the `test` mode:
```bash
CUDA_VISIBLE_DEVICES=0 python test_align.py --dataset UDAIS-D --model UDRSIS --height 512 --width 512 --mode test --model_path ./run_align/UDAIS-D/UDRSIS/experiment_1/epoch200_model.pth
```
**Step 3: Generate the warped images and corresponding masks for UDIS-D**
Using the trained model, you can generate the warped images and corresponding masks for training and testing sets with the following command in the `test_output` mode:
```bash
CUDA_VISIBLE_DEVICES=0 python test_align.py --dataset UDAIS-D --model UDRSIS --height 512 --width 512 --mode test_output --model_path ./run_align/UDAIS-D/UDRSIS/experiment_1/epoch200_model.pth --save_path ./Warp/UDAIS-D/UDRSIS/align/training/ --get_warp_path training
CUDA_VISIBLE_DEVICES=0 python test_align.py --dataset UDAIS-D --model UDRSIS --height 512 --width 512 --mode test_output --model_path ./run_align/UDAIS-D/UDRSIS/experiment_1/epoch200_model.pth --save_path ./Warp/UDAIS-D/UDRSIS/align/testing/ --get_warp_path testing
```
### Inference
**Step 4: Generate the warped images and corresponding masks for custom datasets**
Using the trained model, you can generate the warped images and corresponding masks for custom datasets with the following command in the `test_other` mode:
```bash
CUDA_VISIBLE_DEVICES=0 python test_align.py --dataset UDAIS-D --model UDRSIS --height 512 --width 512 --mode test_other --model_path ./run_align/UDAIS-D/UDRSIS/experiment_1/epoch200_model.pth --save_path {your_save_path} --input_path {your_custom_dataset_path}
```
Note:
1. You can finetune the UDRSIS model on the single image by adding `--ft` to the command.
2. The `--save_path` is the path to save the warped images and corresponding masks.
3. The `--input_path` is the path to the custom dataset.
4. Make sure to put your custom data files as the following structure:
```
input_path
├── input1
| ├── 001.png
│ ├── 002.png
│ ├── 003.png
│ ├── 004.png
│ ├── ...
|
├── input2
| ├── 001.png
│ ├── 002.png
│ ├── 003.png
│ ├── 004.png
| ├── ...
```
## Unsupervised Composition
For input arguments: (see full input arguments via `python train_compo.py --help` or `python test_compo.py --help`):
### Model Training
**Step 5: Unsupervised training on UDAIS-D after unsupervised alignment**
Using the aligned images, you can train the model with the following command:
```bash
CUDA_VISIBLE_DEVICES=0 python train_compo.py --dataset UDAIS-D --train_path ./Warp/UDAIS-D/UDRSIS/align/training --model UDRSIS --epochs 50 -b 4 --loss-type UDRSIS --freq_record 300 --height 512 --width 512 --run_path ./run_com/ --lam_bt 10000 --lam_st1 1000 --lam_st2 1000 --resize
```
Note: You can change the dataset from **UDIS-D** to **UDAIS-D** or **UDAIS-D+** by changing the `--dataset` argument.
### Model Testing
**Step 6: Generating the finally stitched results for UDAIS-D**
Using the trained model, you can generate the finally stitched results for training and testing sets with the following command in the `test_output` mode:
```bash
CUDA_VISIBLE_DEVICES=0 python test_compo.py --dataset UDAIS-D --model UDRSIS --height 512 --width 512 --mode test_output --test_path ./Warp/UDAIS-D/UDRSIS/align/testing --model_path ./run_com/UDAIS-D/UDRSIS/experiment_0/epoch50_model.pth --save_path ./Final/UDAIS-D/UDRSIS/
```
Note:
1. The `--test_path` is the path to put the warped images and corresponding masks.
2. The `--model_path` is the path to the trained model.
3. The `--save_path` is the path to save the finally stitched results.
4. You can change the dataset from **UDIS-D** to **UDAIS-D** or **UDAIS-D+** by changing the `--dataset` argument.
**Step 7: Evaluating the finally stitched results by LVLM-based metrics**
In our paper, we use the LVLM-based metrics to evaluate the finally stitched results: Single Remote Sensing Image Stitching Quality (RSISQ-S) and Comparative Remote Sensing Image Stitching Quality (RSISQ-C).
It includes the **seamline_visibility**, **geometric_accuracy**, **feature_preservation**, and **overall_quality**.
Here, we use the Qwen-VL-Chat model to evaluate the RSISQ-S and RSISQ-C. More information about Qwen-VL-Chat, please refer to [[Qwen-VL-Chat 中文教程]](https://modelscope.cn/models/qwen/Qwen-VL-Chat/summary) and [[Qwen-VL-Chat English Tutorial]](https://github.com/QwenLM/Qwen-VL). All codes about this step is in the `metrics/` folder.
#### Requirements
- python 3.8 and above
- pytorch 1.12 and above, 2.0 and above are recommended
- CUDA 11.4 and above are recommended (this is for GPU users)
- Before running the code, make sure you have setup the environment and installed the required packages. Make sure you meet the above requirements, and then install the dependent libraries.
```bash
pip install -r requirements.txt
```
#### Usage
To evaluate the finally stitched results, you can use the following command (make sure in the `./metrics/` folder):
```bash
% For RSISQ-S
CUDA_VISIBLE_DEVICES=0 python evaluate_stitches.py --input1 ../Final/UDAIS-D/UDRSIS/composition/ --model_save_path results/UDAIS-D/UDRSIS/ --total 5996
% For RSISQ-C
CUDA_VISIBLE_DEVICES=1 python evaluate_stitches.py --input1 ../Final/UDAIS-D/UDRSIS/composition/ --input2 ../Final/UDAIS-D/UDIS2/composition/ --model_save_path results/UDAIS-D/UDRSIS_UDIS2
```
Note:
1. For RSISQ-S, the `--input1` is the path to put the finally stitched results of one specific model.
2. For RSISQ-C, the `--input1` is the path to put the finally stitched results of one specific model. The `--input2` is the path to put the finally stitched results of the comparison model.
3. The `--model_save_path` is the path to save the results.
4. `--total` is the image number of the dataset.
5. All settings are in `config.yaml`.
## Citation
If our work is useful for your research, please consider citing and staring our work:
```tex
@article{qiu2025unsupervised,
author={Qiu, Linwei and Liu, Chang and Li, Gongzhe and Dong, Xiaomeng and Xie, Fengying and Shi, Zhenwei},
journal={IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
title={Unsupervised Deep Image Stitching for Remote Sensing: Aligning Features Across Large-Scale Geometric Distortions},
year={2025},
pages={1-19},
doi={10.1109/JSTARS.2025.3609808}
}
```
## Questions
Please contact [qiulinwei@buaa.edu.cn](mailto:qiulinwei@buaa.edu.cn).
## Acknowledgement
[UDIS](https://github.com/nie-lang/UnsupervisedDeepImageStitching)
[UDIS2](https://github.com/nie-lang/UDIS2)
[UnsupDIS-pytorch](https://github.com/kail8/UnsupDIS-pytorch)
[pytorch_deep_stitching](https://github.com/yyywxk/pytorch_deep_stitching)
[RT-UDRSIS](https://github.com/yyywxk/RT-UDRSIS)