# OmniNxt
**Repository Path**: zhanghao132/OmniNxt
## Basic Information
- **Project Name**: OmniNxt
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: GPL-3.0
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No
## Statistics
- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-09-13
- **Last Updated**: 2025-09-13
## Categories & Tags
**Categories**: Uncategorized
**Tags**: None
## README
## 📢News
* **[30/06/2024]**: OmniNxt is accepted to IROS 2024.
* **[15/08/2024]** V0.1 Release
## Outline
[TOC]
Please cite our paper if you use this project in your research:
* [OmniNxt: A Fully Open-source and Compact Aerial Robot with Omnidirectional Visual Perception](https://ieeexplore.ieee.org/document/10802134), Peize Liu, Chen Feng, Yang Xu, Yan Ning, Hao Xu, and Shaojie Shen, 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
```
@inproceedings{liu2024omninxt,
title={Omninxt: A fully open-source and compact aerial robot with omnidirectional visual perception},
author={Liu, Peize and Feng, Chen and Xu, Yang and Ning, Yan and Xu, Hao and Shen, Shaojie},
booktitle={2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
pages={10605--10612},
year={2024},
organization={IEEE}
}
```
Please kindly star ⭐️ this project if it helps you. We take great efforts to develop and maintain it 😁.
## 🤖Build your own OmniNxt
> You should have basic knowledge of the standard quadrotor's electronic system.
#### BOM
| 🧰Component | 📏 Specification | 🔗 Purchase Link |
| ------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| ***Quad-Fisheye Camera Set*** | | |
| OAK-FFC-4P | Camera control board | [Official](https://shop.luxonis.com/products/oak-ffc-4p) / [Taobao](https://detail.tmall.com/item.htm?abbucket=2&id=797243380525&ns=1&priceTId=213e37fe17166914596117457e9698&skuId=5432965948084&spm=a21n57.1.item.4.2d47523c7FgKjv) |
| OAK-FFC-4P-new (Highly recommended! ) | Camera control board designed for OmniNxt | [Official_Email](contact@oakchina.cn) / [Taobao](https://item.taobao.com/item.htm?id=844758039668) |
| B0335 (OV9782) | Camera modules (Fisheye lens FOV larger than 210 degrees) | [Official](https://www.arducam.com/product/arducam-1mp-ov9782-global-shutter-color-mipi-camera-module-22pin-for-depthai-oak-dm1090ffc) / [Taobao](https://item.taobao.com/item.htm?abbucket=2&id=681463610876&ns=1&priceTId=213e376b17166916784225765e28ee&skuId=5076193835807&spm=a21n57.1.item.10.2d47523c7FgKjv) |
| Camera cables | Customized | Camera cable SCH |
| ***Onboard computer*** | | |
| Jetson Orin Nx | | |
| Jetson Orin Nx carrier board | | [DM](https://item.taobao.com/item.htm?id=719455999699&spm=a1z10.1-c-s.w4004-23557095002.16.165328f6HpXRcJ&skuId=5047593809727) |
| Jetson Orin Nx radiator | | |
| Jeston Orin Network card | | |
| 2230 SSD | | |
| Intel AX200 | | |
| ***Flight platforms*** (6S) | | |
| Nxt-FC | [Project Page](https://github.com/HKUST-Aerial-Robotics/Nxt-FC) | [MicoAir](https://micoair.com/flightcontroller_nxtpx4v2/) / [Taobao](https://item.taobao.com/item.htm?abbucket=2&id=720171355815&ns=1&priceTId=2150440c17167079494632690e1d2e&spm=a21n57.1.item.4.54dc523cLAwNaW) |
| Flight frame | Customized | [Fusion360](https://a360.co/3vK6dJd) |
| Protector | Customized / Oddity RC XI35 protector | [Fusion360](https://a360.co/3vK6dJd) / [Oddity RC](https://oddityrc.com/products/oddityrc-xi35-pro-1?variant=42274592981142) |
| Motors | 2204 1750KV/ 1804 2450KV | |
| 6S Battery | | |
Our latest design can be accessed by Fusion360.🔗[Link](https://a360.co/3vK6dJd ) Access code: **hkustUAV**
#### Multi-fisheye Camera Module
ROS wrap driver 🔗[oak_ffc_4p](https://github.com/D2SLAM-Fusion/oak_ffc_4p_ros), which contains hardware information, ROS1 driver and test tools.
Calibration tool 🔗 [quater-calib](https://github.com/D2SLAM-Fusion/tools-quarterKalibr?tab=readme-ov-file)
#### Jetson Orin
**Environment Info**
We suggest you follow these settings to avoid conflict (especially CUDA and TensorRT).
#### Nxt-FC
We open-source hardware designs at 🔗[NXT-FC](https://github.com/HKUST-Aerial-Robotics/Nxt-FC). Nxt-FC supports [PX4](https://github.com/PX4/PX4-Autopilot) and [Ardupilot](https://github.com/ArduPilot/ardupilot) firmware.
Follow the **instructions for setting up the Nxt-FC** part to configure your flight controller.
The PX4 parameter of OmniNxt can be downloaded from [Onedrive]([OmniNxt-03.params](https://hkustconnect-my.sharepoint.com/:u:/g/personal/pliuan_connect_ust_hk/ET4mtqjwqqNOmA8y2WK9rqoBrwlOHXtEKi6IFt4mSGJmqg?e=KJVn5Y)).
## 💾Runtime setup
Our Omni-VINS and Omni-Depth are developed from D2SLAM. Since building the Docker images on the local machine(Jetson Orin) usually takes a long time, we suggest you pull the images from the Docker Hub. If you want to build the Docker image on your local machine, follow the instructions [Build Docker Images On Local Machine](#Build Docker images in local machines).
the docker images structure is as follows:
#### Step 1 Clone the repository into your local machine
```shell
git clone --branch pr_fix_main https://github.com/HKUST-Aerial-Robotics/D2SLAM.git
```
#### Step 2 Configure your quad-cam parameters and VINS parameters following our template
```shell
cd ./D2SLAM/config/quadcam_drone_nxt_tmp
```
If you have already calibrated your quad fisheye camera set, you can simply replace the files with the same name, which includes:
1. **stereo_calib_n_m_240_320.yaml** (n and m is the camera number; used in Omni-Depth).
2. **fisheye_cams.yaml**. (used in Omni-VINS)
#### Step 3 Configure ./start_docker.sh script with your local environment.
**./start_docker.sh** is under the **./D2SLAM** directory.
Modify the following parameters with the absolute path under your local host environment.
1. DATA_SET (where the data set is) (Optional)
Then run with the following command under the D2SLAM directory (very important, this will map your D2SLAM dir into the container)：
```shell
./start_docker.sh 1
```
#### Step 4 Launch algorithm modules
***run all algorithm modules together (Omni-VINS & Omni-Depth)***
```shell
## you should under /root/swarm_ws/
source ./devel/setup.bash
roslaunch d2vins quadcam.laucnh
```
***Only launch Omni-VINS. Please remove the nodes in the red box***
```shell
source ./devel/setup.bash
roslaunch d2vins quadcam.laucnh
```
***Only launch Omni-Depth***
```shell
source ./devel/setup.bash
roslaunch quadcam_depth_est depth-node.launch
```
**[Notice]** If you are running Omni-VINS and Omni-Depth for the first time, the initialization process of these two modules would be a little bit long because of the building of the inference engine.
#### Almost done
If everything works well, you will see
which means Omni-VINS(D2VINS) initialized normally.
Omni-Depth
### 🖥️Build Docker images in local machines
#### Step 1 Build Jeston_base image
```shell
cd /D2SLA/docker
make jetson_orin_base
```
#### Step 2 Build Jeston Orin image
```shell
cd /D2SLA/docker
make jetson_orin
```
####
#### PX4 Controller
We highly suggest you refer to Fast-Lab's PX4 controller [PX4-Control](https://github.com/ZJU-FAST-Lab/Fast-Drone-250/tree/master/src/realflight_modules/px4ctrl)
We also provide our yaw-rotation-free version of Fast-Lab's PX4 controller [PX4-Control](https://github.com/D2SLAM-Fusion/controller-PX4Control).
Please cite ZJU-Fast-Lab if these modules are useful for your research and project.
#### Flight planer
We modified some of the strategies in the [ego-planner](https://github.com/ZJU-FAST-Lab/ego-planner.git). Our version is here [ego-planner-omni-modify](https://github.com/D2SLAM-Fusion/planner-EgoPlanner)
Please cite ZJU-Fast-Lab's [ego-planner](https://github.com/ZJU-FAST-Lab/ego-planner.git) if these modules are useful for your research and project.
## 🔧Troubleshooting
## 💯Acknowledgment 
