# ISP_Pipeline **Repository Path**: abonaventure/isp_pipeline ## Basic Information - **Project Name**: ISP_Pipeline - **Description**: Algorithm of SuperNight which can be used in CellPhone. - **Primary Language**: C++ - **License**: Not specified - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2023-04-20 - **Last Updated**: 2026-06-30 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # Mat & HDR+ ISP Pipeline ## 1. Introduce ![debug](./Picture/debug.png) ![ref](./Picture/ref.png) **Matlib** is an open-source C/C++ image processing library with a complete HDR+ ISP pipeline, supporting **Windows**, **Linux** and **Android** platforms. All underlying codes adopt low-level optimization technologies including **ARM NEON**, **x86 SSE**, **OpenMP** multi-threading, **Linebuf** and **Mempool**. Its processing performance outperforms OpenCV. The whole codebase is only around **1MB** with no heavy third-party dependencies, which is easy to integrate into your own projects. NEON instructions can be simulated and executed on Windows platforms to lower development costs. The bundled ISP pipeline implements a full commercial-grade **RAW‑to‑BMP/YUV** workflow, covering multi‑frame HDR fusion, lens shading correction, color correction, multi‑stage noise reduction, local tone mapping, wide‑angle distortion correction and more. A full offline calibration toolchain for color matrix and lens distortion is also provided. --- ## 2. Frame Structure Here we briefly describe the core class of Mat basic image library (parent class `Mat.h`). Subclasses include: * `MultiIntImage` * `MultiShortImage` * `MultiUcharImage` * `MultiUshortImage` * `SingleUcharImage` * `Yuv420Image` They encapsulate various basic image processing algorithms and support multiple image formats including **RAW**, **RGB**, **GRAY** and **YUV**. For detailed code usage, please refer to the example files in the project. --- ## 3. Complete Project Directory Structure Based on the project source tree `isp_pipeline.txt`, the full root directory structure and function description are supplemented as follows: - ./ - ├── 3rd # Third-party SIMD & OpenGL ES emulator dependencies - ├── Mat # Low-level image container core library - ├── Layer # Separate implementation of all ISP algorithm modules - ├── ISPpipeline # Main program, CR2 decoder, unit tests & Windows VS projects - ├── build_linux # Output directory for CMake compiled binaries on Linux - ├── data # RAW samples, color checker assets, weight config & output images - ├── doc # ISP algorithm principle PDF documents - ├── Interface # External C++ API & Android JNI wrapper layer - ├── Picture # Flowcharts & comparison screenshots for README - ├── tools # Offline calibration tools (CCM color & wide-angle distortion) - ├── CMakeLists.txt # Global cross-platform CMake build config - ├── CMakePresets.json # Preset configurations for multi-arch CMake compilation - ├── sync_repo.sh # Repository synchronization shell script - ├── LICENSE # Open-source license file - └── README.md # This documentation ### 3.1 Key Subdirectory Description #### `3rd` * **ARM_NEON_2_x86_SSE**: Header files for NEON to SSE instruction translation, supporting Windows simulation of ARM vector instructions. * **Mali_OpenGL_ES_Emulator**: Mali GPU ES simulator for YUV/RGB preview and hardware rendering logic debugging. * **simde**: Full cross-platform SIMD compatibility library, adapting ARM, SSE, MSA, WASM and other multi-architecture instruction sets. #### `Mat` Low-level image data container, encapsulates multi-format image storage, matrix calculation, memory pool management and time‑consuming statistical tools. #### `Layer` Independent `.h/.cpp` implementation of each ISP algorithm module, supporting separate enable/disable and parameter tuning. #### `ISPpipeline` * **DecodeCR2**: Canon CR2 RAW format parsing tool. * **test*.cpp**: Independent unit tests for AWB, lens correction, noise reduction, wide‑angle correction and other single modules. * Built‑in `libraw` source code for RAW image decoding. #### `data` Stores test RAW materials, color checker calibration files, core weight configuration files and algorithm output images. #### `tools` Supporting offline calibration tools for laboratory and production lens & color parameter calibration. #### `Interface` External business call encapsulation layer, including PC native C++ interface and Android JNI interaction code. --- ## 4. Third-Party Dependencies Instruction All third-party dependencies are built into the `3rd` directory without additional system‑wide installation. The header and library paths are pre‑configured in build scripts. ### 1. **simde + ARM_NEON_2_x86_SSE** Core function: Realize cross‑platform consistency of SIMD accelerated instructions, simulate ARM NEON instructions on x86 Windows, and ensure unified high‑performance code for Windows/Linux/Android. ### 2. **Mali OpenGL ES Emulator** Core function: Simulate mobile Mali GPU rendering on Windows for real‑time YUV effect preview and wide‑angle ISP debugging. ### 3. **libraw** Built‑in in `ISPpipeline/libraw`, used for parsing CR2 raw data and shooting metadata. --- ## 5. Complete HDR+ ISP Pipeline Workflow ![ISPpipeline](./Picture/ISPpipeline.png) The project implements a complete commercial RAW image processing full‑link algorithm, and the standard workflow is as follows: RAW Input → Black Level Correction (BLC) → Defect Pixel Correction (DPC) → RAW Spatial Denoise → Multi-Frame Block Matching HDR Fusion → Demosaicing → Auto White Balance (AWB) → Lens Shading Correction (LDC) → CCM Color Matrix Correction → Chroma Noise Reduction → Local RAW Detail Enhancement → Local RGB Adaptive Curve → Global Contrast & Tone Mapping → Gamma Correction → Wide-Angle Distortion Correction → YUV Denoise & Sharpen → YUV/BMP Export ### 5.1 Core Module Classification * Basic correction module * HDR multi‑frame fusion module * Color calibration module * Multi‑stage noise reduction module * Detail enhancement module * Tone mapping module * Hardware ISP simulation module * Geometric distortion correction module ### 5.2 Global Configuration File (`weight.param`) ![weight.param](./Picture/weight.param.png) All algorithm switches, strength coefficients, calibration file paths and tuning parameters are uniformly controlled by `weight.param`. Modifications take effect immediately without recompilation. --- ## 6. How to Compile the Project ### 6.1 Windows Platform Compile with **VS2017**, open `ISPpipeline.sln`. Sub‑project segmentation: * `Mat.vcxproj` – Underlying image static library * `Layer.vcxproj` – ISP algorithm static library * `ISPpipeline.vcxproj` – Main executable program, decoding tool and test programs Compiled outputs are located in `ISPpipeline/x64/Release` and `ISPpipeline/x64/Debug`. ### 6.2 Linux Platform Support standard CMake compilation, all outputs are stored in `build_linux`: ```bash mkdir build_linux && cd build_linux cmake .. -DCMAKE_BUILD_TYPE=Release # or simply cmake .. make -j$(nproc) # or make -j8 ``` Outputs: bin/isp_pipeline – executable file lib/libmat.a & lib/liblayer.a – static libraries ### 6.3 Android Platform Compile with NDK, execute build scripts in: /Mat/Android/jni/build.cmd /Interface/jni/build.bat to generate .so dynamic libraries for Android APP invocation. --- ## 7. Offline Calibration Tools Introduction The project provides two core offline calibration tools in the tools directory for camera image tuning: ### 7.1 CCM Color Calibration Based on standard 24‑color checker RAW images, automatically calculate the optimal color correction matrix to fix color cast, and write the matrix parameters to weight.param. ### 7.2 Wide-Angle Distortion Calibration Generate lens distortion correction lookup table through chessboard calibration images, eliminate wide‑angle edge stretching and vignetting, support commercial lens tuning. --- ## 8. HDR+ Demo Running Example Demo test image download link: https://pan.baidu.com/s/1PdGphn8Z5f7zKtkReOgmhg Extraction Code: aaaa Description: DecodeCR2.exe decodes .CR2 raw files into .raw image data and generates .txt parameter files; ISPpipeline.exe implements the complete RAW‑to‑BMP ISP processing pipeline. ![run23detail](./Picture/run23detail.png) ### 8.1 Open Weight Configuration File Open weight.param and locate the three core module switches. ### 8.2 Disable Three Core ISP Modules Set the following parameters to 0: nBlockMatchFusionEnable=0; // Multi-frame noise reduction & HDR fusion nChromaDenoiseEnable=0; // Chromatic noise suppression nTonemappingEnable=0; // Dynamic range compression ### 8.3 Execute run23.bat Script Run the script to process the test image. ### 8.4 Check Basic Output Image The Normalize.bmp generated in the outbmp directory is the original unoptimized image: ![basicispss](./Picture/basicispss.png) ### 8.5 Enable Three Core ISP Modules ![fullweight](./Picture/fullweight.png) Delete outbmp/Normalize.bmp and re‑run run23.bat to get the fully optimized ISP processing result: ![myispss](./Picture/myispss.png) ### 8.6 Official HDR+ Reference Project Open‑source official HDR+ project link: https://github.com/timothybrooks/hdr-plus This open‑source project is maintained officially, supporting Nix Flakes & CMake compilation, providing standard HDR+ algorithm implementation and test cases. ![hdrplusziplianjie](./Picture/hdrplusziplianjie.png) ![hdrpluszip](./Picture/hdrpluszip.png) Official HDR+ test output reference image: ![hdrplusdemooutdir](./Picture/hdrplusdemooutdir.png) ### 8.7 Extract Reference Results Extract the 001 compressed package to get the standard reference image output23.png. ### 8.8 Overall Image Comparison ![fullcompare](./Picture/fullcompare.png) ### 8.9 Local Detail Comparison ![detailcompare](./Picture/detailcompare.png) --- ## 9. Unit Test & Secondary Development ### 9.1 Single Module Test All test*.cpp files in ISPpipeline are independent verification cases for AWB calibration, lens mapping generation, hardware noise reduction, wide‑angle correction and other single functions. ### 9.2 External Interface Development The Interface directory provides standard external calling interfaces: PC native C++ interface – Quickly integrate full ISP pipeline into third‑party projects. Android JNI interface – Support mobile APP to call image processing algorithms. Custom ISP algorithms can be expanded by adding new module files in the Layer directory and matching configuration parameters. --- ## 10. Troubleshooting & Performance Tuning ### 10.1 Compilation Problems Windows NEON instruction error: Check that simd and NEON‑SSE conversion header paths are correctly configured. Linux CMake compilation failure: Execute compilation commands in the project root directory. Android NDK build error: Match the correct NDK version and ABI configuration. ### 10.2 Image Quality Problems Color cast: Re‑calibrate CCM color matrix via the calibration tool. Lens distortion & vignetting: Regenerate lens correction mapping file. Excessive noise / over‑sharpening: Adjust parameter strength in weight.param. ### 10.3 Runtime Optimization Enable OpenMP multi‑threading and SIMD acceleration, adjust Mempool buffer size to improve processing speed and reduce memory overhead. --- ## 11. Supplementary Instructions doc folder: Stores ISP algorithm principle documents for theoretical reference. sync_repo.sh: Project multi‑module synchronization script. LICENSE: Open‑source usage specification and authorization description.