# VINS-Fusion-GPU-CV4

**Repository Path**: jerry-ironman/vins-fusion-gpu-cv4

## Basic Information

- **Project Name**: VINS-Fusion-GPU-CV4
- **Description**: vins-fusion GPU版本，对于OpenCV4版本的修改，运行平台为xavier nx
- **Primary Language**: C++
- **License**: GPL-3.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 5
- **Forks**: 3
- **Created**: 2023-04-23
- **Last Updated**: 2025-07-23

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# NVIDIA Jetson Xavier NX部署VINS-fusion-GPU
代码来源于 https://github.com/pjrambo/VINS-Fusion-gpu, 改变了一些代码以适应OpenCV4版本
## 环境配置（Ubuntu 20.04）
### 1、Cuda的安装
NVIDIA Jetson Xavier NX 安装完ubuntu20.04系统，自带cuda11.4</br>
在 .bashrc中配置环境变量。source之后，nvcc – version 即可查看cuda版本
```
export PATH=/usr/local/cuda-11.4/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64:$LD_LIBRARY_PATH
export CUDA_HOME=/usr/local/cuda
```
### 2、Eigen（用ROS自带的也可以）
```
sudo apt-get remove libeigen3-dev 

wget -O eigen.zip https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.zip #check version
unzip eigen.zip
mkdir eigen-build && cd eigen-build
cmake ../eigen-3.3.7/ && sudo make install
pkg-config --modversion eigen3 # Check Eigen Version
```
### 3、Ceres solver
使用 /Fast-Drone-250-master/3rd_party/ 下的ceres安装包也可以，它是2.0.0版本的
```
sudo apt-get install -y cmake libgoogle-glog-dev libatlas-base-dev libsuitesparse-dev
wget http://ceres-solver.org/ceres-solver-1.14.0.tar.gz
tar zxf ceres-solver-1.14.0.tar.gz
cd ceres-solver-1.14.0
mkdir build && cd build
cmake -DEXPORT_BUILD_DIR=ON \
      -DCMAKE_INSTALL_PREFIX=/usr/local \
      ../
make -j $(nproc) # number of cores
make test -j $(nproc)
sudo make install -j $(nproc)
```
### 4、OpenCV安装
关于Opencv 的安装需要特别谨慎，不建议频繁删除以前的库，因为多个项目使用的版本可能不同，/usr/下默认已有OpenCV，所以可以将常用的版本安装在/usr/local下，将其余版本安装在自己OpenCV的下载目录/build下。</br>
因为VINS-FUSION-GPU版本需要支持Cuda的OpenCV，这里选择4.5.3版本
```
wget -O opencv.zip https://github.com/opencv/opencv/archive/4.5.3.zip # check version
unzip opencv.zip
cd opencv-3.4.1/ && mkdir build && cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
        -D CMAKE_INSTALL_PREFIX=/usr/local \  #这里的/usr/local可以换成自己的安装路径，到时候在Cmakelist里设置自己设置的路径即可。
        -D WITH_CUDA=ON \
        -D CUDA_ARCH_BIN=7.2 \  #对于NX和AGX,选7.2，Nano，TX2选6.2 
        -D CUDA_ARCH_PTX="" \
        -D ENABLE_FAST_MATH=ON \
        -D CUDA_FAST_MATH=ON \
        -D WITH_CUBLAS=ON \
        -D WITH_LIBV4L=ON \
        -D WITH_GSTREAMER=ON \
        -D WITH_GSTREAMER_0_10=OFF \
        -D WITH_QT=ON \
        -D WITH_OPENGL=ON \
        -D CUDA_NVCC_FLAGS="--expt-relaxed-constexpr" \
        -D WITH_TBB=ON \
         ..
make  # running in single core is good to resolve the compilation issues         
sudo make install
cd ../../ && sudo rm -rf opencv-3.4.1 # optional (can save 10GB Disk Space)
pkg-config --modversion opencv # Check opencv Version
```
OpenCV 在NX上的安装比较慢，差不多需要1个小时。OpenCV安装好以后，需要进一步编译CV-Bridge(cv_bridge是在ROS图像消息和OpenCV图像之间进行转换的一个功能包)
### 5、CV-Bridge的编译
CV-Bridge的安装需要下载对应的版本，由于NX 是基于Ubuntu 20.04的ARM系统,因此下载noetic版本的CV-Bridge。首先，通过以下连接进入下载页面：
```
https://github.com/ros-perception/vision_opencv
```
选择对应的noetic分支下载即可</br>
进入 /vision_opencv-noetic/cv_bridge 目录，指定CMakeLists.txt文件中OpenCV的路径
```
gedit vision_opencv-noetic/cv_bridge/CMakeLists.txt
# Edit OpenCV PATHS in CMakeLists 
# 选择自己安装的OpenCV对应路径，若是安装到/usr/local下，则一般为/usr/local/share/OpenCV ；若是本地安装的，一般指定到OpenCV下载目录下的/build文件夹下即可
set(OpenCV_DIR /home/jerry/deps/opencv-4.6.0/build)
# 编译
mkdir build && cd build
cmake ..
make
```
## 二、Vins-Fusion GPU编译安装
### 1、下载源码
```
# 下载修改过的vins-fusion gpu版本
cd ~/catkin_ws/src && git clone https://gitee.com/jerry-ironman/vins-fusion-gpu-cv4.git 

sudo apt-get install ros-noetic-tf
sudo apt-get install ros-noetic-image-transport
sudo apt-get install ros-noetic-rviz
```
### 2、更改openCV和cv-bridge路径（Change the OpenCV and cv_bridge path in the CMakeLists）
The essential software environment is same as VINS-Fusion. Besides, it requires OpenCV cuda version.(Only test it on OpenCV 4.5.3 and 4.6.0 with CUDA/CUDNN), and it have to rebuild the cv_bridge after install the OpenCV4 so that the OutOfMemoryError will not happend.

for opencv, add the path to the **OpenCVConfig.cmake** file.
```
set(cv_bridge_DIR /home/jerry/deps/vision_opencv-noetic/cv_bridge/build/devel/share/cv_bridge/cmake)
```
for cv_bridge, add the path to the **cv_bridgeConfig.cmake** file.
```
set(OpenCV_DIR /home/jerry/deps/opencv-4.6.0/build)
```

- In /camera_models/CMakeLists.txt, change Line 16 to your path.
- In /loop_fusion/CMakeLists.txt, change Line 9 and 19 to your path.
- In /vins_estimator/CMakeLists.txt, change Line 9 and 20 to your path.

### 3、更改配置参数（Change the acceleration parameters as you need）
In the config file, there are two parameters for gpu acceleration(if there's no use_gpu and use_gpu_acc_flow parameters in the config file, just manual add). 
- use_gpu: 0 for off, 1 for on  
- use_gpu_acc_flow:  0 for off, 1 for on  

If your GPU resources is limitted or you want to use GPU for other computaion. You can set  
- use_gpu: 1  
- use_gpu_acc_flow: 0  

If your other application do not require much GPU resources, I recommanded you to set  
- use_gpu: 1  
- use_gpu_acc_flow: 1  

### 4、编译 VINS-Fusion
Clone the repository and catkin_make:
```
cd ~/catkin_ws/
catkin_make
source ~/catkin_ws/devel/setup.bash
```

### 5、启动VINS
#### 5.1、启动 D435i
需要修改一下D435i的发布频率，建议为infra设置为15HZ，这样NX上不会有较大的延迟。
```
<arg name="depth_fps" default="15"/>
<arg name="infra_fps" default="15"/>
<arg name="color_fps" default="30"/>
```
```
# 启动D435i
roslaunch realsense2_camera rs_camera.launch
```
#### 5.2、启动 飞控板IMU
```
roslaunch mavros px4.launch
roslaunch mavros px4_imu_200.launch #修改IMU频率为200hz
```
#### 5.3、启动 VINS
```
roslaunch vins fast_drone_250.launch
```