# AD_EventCamera **Repository Path**: abonaventure/AD_EventCamera ## Basic Information - **Project Name**: AD_EventCamera - **Description**: AD_EventCamera - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 1 - **Forks**: 0 - **Created**: 2025-04-22 - **Last Updated**: 2025-08-11 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README

MvHeat-DET

Official PyTorch implementation of "Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset"

arXiv GitHub Xiao Wang¹, Yu Jin¹, Wentao Wu², Wei Zhang³, Lin Zhu⁴, Bo Jiang¹, Yonghong Tian^3,5,6

--- ### Abstract Object detection in event streams has emerged as a cutting edge research area, demonstrating superior performance in low-light conditions, scenarios with motion blur, and rapid movements. Current detectors leverage spiking neural networks, Transformers, or convolutional neural networks as their core architectures, each with its own set of limitations including restricted performance, high computational overhead, or limited local receptive fields. This paper introduces a novel MoE (Mixture of Experts) heat conduction based object detection algorithm that strikingly balances accuracy and computational efficiency. Initially, we employ a stem network for event data embedding, followed by processing through our innovative MoE-HCO blocks. Each block integrates various expert modules to mimic heat conduction within event streams. Subsequently, an IoU-based query selection module is utilized for efficient token extraction, which is then channeled into a detection head for the final object detection process. Furthermore, we are pleased to introduce EvDET200K, a novel benchmark dataset for event-based object detection. Captured with a high-definition Prophesee EVK4-HD event camera, this dataset encompasses 10 distinct categories, 200,000 bounding boxes, and 10,054 samples, each spanning 2 to 5 seconds. We also provide comprehensive results from over 15 state-of-the-art detectors, offering a solid foundation for future research and comparison. ### Our Proposed Approach

### Experimental Results

### Dataset visualizations

--- ## Quick start ### Operation video [Click to download the operation video](https://gitee.com/abonaventure/AD_EventCamera/raw/main/operations.mp4) ### Install we use single RTX 4090 24G GPU for training and evaluation. ``` conda create -n mvheat python=3.8 conda activate mvheat pip install -r requirements.txt ``` ### Data Download the [EvDET200K (Baiduyun)](https://pan.baidu.com/s/1HfkDyVv_dV_lbJGX0cQEVg?pwd=ahue) dataset or from [[Dropbox](https://www.dropbox.com/scl/fo/2x3qf8bcwd6qb4f70fnda/AL2ULrSzZuVgpVlH8RTqhsY?rlkey=hh7k0lqg1tru4iisi0vo12y6x&st=nz4b3c13&dl=0)], and modify the dataset path in `configs\dataset\EvDET200K_detection.yml`. ### Train #### training on single-gpu ``` python tools/train.py -c configs/evheat/MvHeatDET.yml ``` #### training on multi-gpu ``` NCCL_P2P_DISABLE=1 CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node=2 tools/train.py -c configs/evheat/MvHeatDET.yml ``` ### Test #### testing on single-gpu ``` python tools/train.py -c configs/evheat/MvHeatDET.yml -r ckp/mvheatdet_input640_layers18_dim768.pth --test-only ``` #### testing on multi-gpu ``` NCCL_P2P_DISABLE=1 CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node=2 tools/train.py -c configs/evheat/MvHeatDET.yml -r ckp/mvheatdet_input640_layers18_dim768.pth --test-only ``` --- ## Checkpoint Download You can download the pretrained checkpoint on EvDET200K dataset from [BaiduYun](https://pan.baidu.com/s/1UZ3TH0VA31XvVaH-BpiTKA?pwd=ahue), [Dropbox](https://www.dropbox.com/scl/fi/r42vh0chc6l8k9m6qdmxi/mvheatdet_input640_layers18_dim768.pth?rlkey=e3ecowimydplm7l62ujndsoqj&st=xfc0i1qm&dl=0), with model config:

Dataset	Input Size	Block Num.	Channel
EvDET200K	640	(2,2,18,2)	(96,192,384,768)

--- ## Acknowledgments Our code is extended from the following repositories. We sincerely appreciate for their contributions. * [vHeat](https://github.com/MzeroMiko/vHeat) * [RT-DETR](https://github.com/lyuwenyu/RT-DETR) ## Citation If you find this work helps your research, please cite the following paper and give us a star. ``` @misc{wang2024EvDET200K, title={Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset}, author={Xiao Wang and Yu Jin and Wentao Wu and Wei Zhang and Lin Zhu and Bo Jiang and Yonghong Tian}, year={2024}, eprint={2412.06647}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2412.06647}, } ```