# DTUeval-python

**Repository Path**: uho163/DTUeval-python

## Basic Information

- **Project Name**: DTUeval-python
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-09-28
- **Last Updated**: 2025-09-28

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# DTUeval-python

A python implementation of DTU MVS 2014 evaluation. It only takes 1min for each mesh evaluation. And the gap between the two implementations is negligible. 

## Setup and Usage

This script requires the following dependencies.

```
numpy open3d scikit-learn tqdm scipy multiprocessing argparse
```

Download the STL point clouds and Sample Set and prepare the ground truth folder as follows.

```
<dataset_dir>
- Points
    - stl
        - stlxxx_total.ply
- ObsMask
    - ObsMaskxxx_10.mat
    - Planexxx.mat
```

Run the evaluation script (e.g. scan24, mesh mode)
```
python eval.py --data <input> --scan 24 --mode mesh --dataset_dir <dataset_dir> --vis_out_dir <out_dir_for_visualization>
```

## Discussion on randomness
There is randomness in point cloud downsampling in both versions. It iterates through the points and delete the points with distance < 0.2. So the order of points matters. We randomly shuffle the points before downsampling. 

## Comparison with the official script
We evaluate a set of meshes from Colmap and compare the results. We run our script 10 times and take the average. 

|     | diff/official | official | py_avg   | py_std/official |
|-----|---------------|----------|----------|-----------------|
| 24  | 0.0184%       | 0.986317 | 0.986135 | 0.0108%         |
| 37  | 0.0001%       | 2.354124 | 2.354122 | 0.0091%         |
| 40  | 0.0038%       | 0.730464 | 0.730492 | 0.0104%         |
| 55  | 0.0436%       | 0.530899 | 0.531131 | 0.0104%         |
| 63  | 0.0127%       | 1.555828 | 1.556025 | 0.0118%         |
| 65  | 0.0409%       | 1.007686 | 1.008098 | 0.0080%         |
| 69  | 0.0082%       | 0.888434 | 0.888361 | 0.0125%         |
| 83  | 0.0207%       | 1.136882 | 1.137117 | 0.0096%         |
| 97  | 0.0314%       | 0.907528 | 0.907813 | 0.0089%         |
| 105 | 0.0129%       | 1.463337 | 1.463526 | 0.0118%         |
| 106 | 0.1424%       | 0.785527 | 0.786646 | 0.0151%         |
| 110 | 0.0592%       | 1.076125 | 1.075488 | 0.0132%         |
| 114 | 0.0049%       | 0.436169 | 0.436190 | 0.0074%         |
| 118 | 0.1123%       | 0.679574 | 0.680337 | 0.0099%         |
| 122 | 0.0347%       | 0.726771 | 0.726519 | 0.0178%         |
| avg | 0.0153%       | 1.017711 | 1.017867 |                 |


## Error visualization
`vis_xxx_d2s.ply` and `vis_xxx_s2d.ply` are error visualizations.
- Blue: Out of bounding box or ObsMask
- Green: Errors larger than threshold (20)
- White to Red: Errors counted in the reported statistics