# Spectrum-Power-Allocation **Repository Path**: fucgg/Spectrum-Power-Allocation ## Basic Information - **Project Name**: Spectrum-Power-Allocation - **Description**: No description available - **Primary Language**: Unknown - **License**: MIT - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2021-07-12 - **Last Updated**: 2021-11-03 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # Deep Reinforcement Learning for Joint Spectrum and Power Allocation in Cellular Networks ### Authors: Yasar Sinan Nasir and Dongning Guo #### Please email me for any questions or comments: snasir [at] u.northwestern [dot] edu > **Note:** Instructions are for Linux. > Please create a virtual environment and install the specified tensorflow and numpy versions: > > `python3.6 -m pip install --user virtualenv`
> > `virtualenv -p python3.6 venv`
> > `source $HOME/venv/bin/activate`
> > `pip install -r requirements.txt`
#### DEMO: The repository comes with data to reproduce immediate results, run:
##### Reproduce Fig. 3.a:
* > `python3 -i ./train_results.py --json-file "train_K5_N20_M2_shadow10_episode4-5000_travelIND_fd10" `
* **Expected Output:** Deployment: train_K5_N20_M2_shadow10_episode4-5000_travelIND_fd10; policy: ddpg200_100_50; K: 20; N: 5
Averages for last 200 episodes:
Sum rate per link - proposed: 2.54
Sum rate per link - joint learning: 2.47
Sum rate per link - FP: 2.42
Sum rate per link - FP Multi delayed: 2.22
Sum rate per link - random: 0.82
Sum rate per link - full: 0.82
Average FP iterations per run: 106.99
![](./fig/spectraleff_train_K5_N20_M2_shadow10_episode4-5000_travelIND_fd10_network_0.png) ##### Reproduce Fig. 3.b:
* > `python3 -i ./train_results.py --json-file "train_K5_N20_M4_shadow10_episode4-5000_travelIND_fd10" `
* **Expected Output:** Deployment: train_K5_N20_M4_shadow10_episode4-5000_travelIND_fd10; policy: ddpg200_100_50; K: 20; N: 5
Averages for last 200 episodes:
Sum rate per link - proposed: 4.14
Sum rate per link - joint learning: 3.96
Sum rate per link - FP: 3.76
Sum rate per link - FP Multi delayed: 3.53
Sum rate per link - random: 1.85
Sum rate per link - full: 1.85
Average FP iterations per run: 127.75
![](./fig/spectraleff_train_K5_N20_M4_shadow10_episode4-5000_travelIND_fd10_network_0.png)
#### Reproducing Table I Row 3: (Trained policy parameters are included as well for further testing) The repository contains the pre-trained policy parameters in ./simulations/sumrate/policy, so there is no need to run training to reproduce the results. For each row of Table I, you can execute: * > `python3 ./test_results.py --json-file "test_K5_N20_M4_shadow10_episode10-500_travel0_fd10" --json-file-train "train_K5_N20_M4_shadow10_episode4-5000_travelIND_fd10" --json-file-PA-train "ddpg200_100_50" --json-file-CS-train "dqn100_50_50"`
* > `python3 ./test_results.py --json-file "test_K5_N20_M4_shadow10_episode10-500_travel0_fd10" --json-file-train "train_K5_N20_M4_shadow10_episode4-5000_travelIND_fd10" --json-file-PA-train "dqn200_200_100" --json-file-CS-train "dqn200_200_100"`
**Expected Output:** Test: test_K5_N20_M4_shadow10_episode10-500_travel0_fd10; train train_K5_N20_M4_shadow10_episode4-5000_travelIND_fd10; policy: ddpg200_100_50
Sum rate per link for policy after 4 episodes: 4.57, percentage: 1.20
Sum rate per link - FP: 3.81, percentage: 1.00
Sum rate per link - FP w delay: 3.57, percentage: 0.94
Sum rate per link - random: 2.12, percentage: 0.56
Sum rate per link - full: 2.12, percentage: 0.56
Average FP iterations per run: 122.15
Test: test_K5_N20_M4_shadow10_episode10-500_travel0_fd10; train train_K5_N20_M4_shadow10_episode4-5000_travelIND_fd10; policy: dqn200_200_100
Sum rate per link for policy after 4 episodes: 4.38, percentage: 1.15
Sum rate per link - FP: 3.81, percentage: 1.00
Sum rate per link - FP w delay: 3.57, percentage: 0.94
Sum rate per link - random: 2.12, percentage: 0.56
Sum rate per link - full: 2.12, percentage: 0.56
Average FP iterations per run: 122.1
#### Reproducing Table I Row 7: (Trained policy parameters are included as well for further testing) The repository contains the pre-trained policy parameters in ./simulations/sumrate/policy, so there is no need to run training to reproduce the results. For each row of Table I, you can execute: * > `python3 ./test_results.py --json-file "test_K10_N50_M5_shadow10_episode10-500_travel0_fd10" --json-file-train "train_K10_N50_M5_shadow10_episode4-5000_travelIND_fd10" --json-file-PA-train "ddpg200_100_50" --json-file-CS-train "dqn100_50_50"`
* > `python3 ./test_results.py --json-file "test_K10_N50_M5_shadow10_episode10-500_travel0_fd10" --json-file-train "train_K10_N50_M5_shadow10_episode4-5000_travelIND_fd10" --json-file-PA-train "dqn200_200_100" --json-file-CS-train "dqn200_200_100"`
**Expected Output:** Test: test_K10_N50_M5_shadow10_episode10-500_travel0_fd10; train train_K10_N50_M5_shadow10_episode4-5000_travelIND_fd10; policy: ddpg200_100_50
Sum rate per link for policy after 4 episodes: 3.79, percentage: 1.19
Sum rate per link - FP: 3.18, percentage: 1.00
Sum rate per link - FP w delay: 2.94, percentage: 0.92
Sum rate per link - random: 1.64, percentage: 0.52
Sum rate per link - full: 1.64, percentage: 0.52
Average FP iterations per run: 206.38
Test: test_K10_N50_M5_shadow10_episode10-500_travel0_fd10; train train_K10_N50_M5_shadow10_episode4-5000_travelIND_fd10; policy: dqn200_200_100
Sum rate per link for policy after 4 episodes: 3.76, percentage: 1.18
Sum rate per link - FP: 3.18, percentage: 1.00
Sum rate per link - FP w delay: 2.94, percentage: 0.92
Sum rate per link - random: 1.64, percentage: 0.52
Sum rate per link - full: 1.64, percentage: 0.52
Average FP iterations per run: 206.38
#### To train a policy from scratch: We prepared a bash script file that runs all the python files required to run all simulations from scratch:
> ./scripts/train.sh #### Table I: It takes several hours to complete the above script, so you can just run test scenarios by using the provided pretrained parameters.
> `./scripts/testK5N20M1.sh`
> `./scripts/testK5N20M2.sh`
> `./scripts/testK5N20M4.sh`
> `./scripts/testK10N50M1.sh`
> `./scripts/testK10N50M2.sh`
> `./scripts/testK10N50M4.sh`
> `./scripts/testK10N50M5.sh`
> `./scripts/testK10N50M10.sh`
#### Configuration files: 1. Proposed learning DQN hyper-parameters are in ./config/policy/dqn100_50_50.json.
Proposed learning DDPG hyper-parameters are in ./config/policy/ddpg200_100_50.json.
Joint learning DQN hyper-parameters are in ./config/policy/dqn200_200_100.json.
> **Note:** If you set cuda to True, running python files in parallel as done in the scripts will not work.
> **Note:** neightresh is unused in the current version.
> **Note:** Scaling parameters are used to scale the channel gains in dB before feeding them into neural networks.
> **Note:** We provide a DQN implementation for reference. Its config file has num_actions option in addition to the ones listed above.
2. Deployment scenarios are in ./config/deployment/. There are two types of configuration files: train and test. Please examine both before using.