# DMBP-RL

**Repository Path**: Krasjet_Yu/dmbp-rl

## Basic Information

- **Project Name**: DMBP-RL
- **Description**: Diffusion Model-Based Predictor for UAV Trajectory Prediction.
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2024-04-12
- **Last Updated**: 2025-05-12

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# GCRL-min(AoI)
This is the code accompanying the paper:  "[AoI-minimal UAV Crowdsensing by Model-based
Graph Convolutional Reinforcement Learning](https://ieeexplore.ieee.org/document/9796732/)", published in IEEE INFOCOM 2022.

## :page_facing_up: Description
Mobile Crowdsensing (MCS) with smart devices has become an appealing paradigm for urban sensing.With the development of 5G-and-beyond technologies, unmanned aerial vehicles (UAVs) become possible for real-time applications, including wireless coverage, search and even disaster response. In this paper, we consider to use a group of UAVs as aerial base stations (BSs) to move around and collect data from multiple MCS users, forming a UAV crowdsensing campaign (UCS). Our goal is to maximize the collected data, geographical coverage whiling minimizing the age-of-information (AoI) of all mobile users simultaneously, with efficient use of constrained energy reserve. We propose a model-based deep reinforcement learning (DRL) framework called ”GCRL-min(AoI)”, which mainly consists of a novel model-based Monte Carlo tree search (MCTS) structure based on state-of-the- art approach MCTS (AlphaZero). We further improve it by adding a spatial UAV-user correlation extraction mechanism by a relational graph convolutional network (RGCN), and a next state prediction module to reduce the dependance of experience data. Extensive results and trajectory visualization on three real human mobility datasets in Purdue University, KAIST and NCSU show that GCRL-min(AoI) consistently outperforms five
baselines, when varying different number of UAVs and maximum coupling loss in terms of four metrics.

## :wrench: Installation
1. Clone repo
    ```bash
    git clone https://github.com/BIT-MCS/GCRL-min-AoI.git
    cd GCRL-min-AoI
    ```
2. Install dependent packages
    ```sh
    # system-env
    sudo apt-get install libgeos++-dev libproj-dev
    
    # python-env
    conda create -n mcs python==3.8
    conda activate mcs
    conda install pytorch cudatoolkit tensorboard future
    conda install --channel conda-forge cartopy
    pip install -r requirements.txt
    
    # Install movingpandas
    mkdir requirements && cd requirements
    git clone https://github.com/anitagraser/movingpandas.git
    python setup.py develop

    pip install torch==1.12.0+cu113 torchvision==0.13.0+cu113 torchaudio==0.12.0 --extra-index-url https://download.pytorch.org/whl/cu113
    ```


## :computer: Training

Train our solution
```bash
python train_our_policy.py --overwrite --output_dir logs/debug
```


Train our solution with trajectory visualization for debugs

```sh
python train_our_policy.py --overwrite --test_after_every_eval --vis_html --plot_loop --moving_line --output_dir logs/debug
```

## :checkered_flag: Testing

Test with the trained models 

```
python test_our_policy.py --vis_html --plot_loop --moving_line --model_dir logs/debug
```

Random test the env

```
python test_random.py --overwrite --vis_html --plot_loop --moving_line --output_dir logs/debug
```

## :clap: Reference
- https://github.com/vita-epfl/CrowdNav
- https://github.com/ChanganVR/RelationalGraphLearning


## :scroll: Acknowledgement

This work is supported by the National Natural Science Foundation of China (No. 62022017). 
<br>
Corresponding author: Chi Harold Liu.

## :e-mail: Contact

If you have any question, please email `3120215520@bit.edu.cn`.

## Paper
If you are interested in our work, please cite our paper as

```
@inproceedings{dai2022aoi,
  author    = {Dai, Zipeng and Liu, Chi Harold and Ye, Yuxiao and Han, Rui and Yuan, Ye and Wang, Guoren and Tang, Jian},
  title     = {AoI-minimal UAV Crowdsensing by Model-based Graph Convolutional Reinforcement Learning},
  booktitle = {IEEE International Conference on Computer Communications (INFOCOM)},
  year      = {2022},
}
```