# GKCMN

**Repository Path**: bigaisesec/GKCMN

## Basic Information

- **Project Name**: GKCMN
- **Description**: A PyTorch Implementation of Gaussian Kernel-Based Cross Modal Network for Spatio-Temporal Video Grounding
- **Primary Language**: Python
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 0
- **Created**: 2023-02-14
- **Last Updated**: 2023-02-16

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# A PyTorch Implementation of Gaussian Kernel-Based Cross Modal Network for Spatio-Temporal Video Grounding

**The paper is available [here](https://arxiv.org/pdf/2207.00744).**

**The pretrained word vectors can be downloaded [here](https://nlp.stanford.edu/projects/glove/).**

**The VidSTG dataset can be downloaded [here](https://github.com/Guaranteer/VidSTG-Dataset).**

**The original Vidor dataset can be downloaded [here](https://xdshang.github.io/docs/vidor.html).**

+ Train:
```
python main.py \
--word2vec-path XXX/glove_model.bin \
--dataset VidSTG_bbox \
--feature-path XXX/2d_feature\
--train-data XXX/vidstg/annotations/train_annotations.json \
--val-data XXX \
--test-data XXX \
--max-num-epochs 20 \
--dropout 0.2 \
--warmup-updates 300 \
--warmup-init-lr 1e-06 \
--lr 8e-4 \
--num-heads 4 \
--num-attn-layers 2 \
--weight-decay 1e-7 \
--train \
--model-saved-path models_vidstgbbox \
--frame-dim 1024 \
--batch-size 16 \
--gpu-idx 0 \
--d-model 512 \
--bbox-path xxx/vidstg/vidor_json/bbox \
--heatmaps-path word/heatmaps_64.npy 
```
+ Test:
```
python main.py \
--word2vec-path word/glove_model.bin \
--dataset VidSTG_bbox \
--feature-path x/vidstg/2d_feature \
--train-data x \
--val-data x \
--test-data x \
--max-num-epochs 50 \
--dropout 0.2 \
--warmup-updates 300 \
--warmup-init-lr 1e-06 --lr 8e-3 --num-heads 4 \
--num-gcn-layers 2 \
--num-attn-layers 2 --weight-decay 1e-7 \
--evaluate \
--model-saved-path xxx \
--frame-dim 32 \
--batch-size 8  \
--gpu-idx 1 \
--d-model 512 \
--bbox-path xxx/vidstg/vidor_json/bbox \
--heatmaps-path word/heatmaps.npy \
--model-idx 1
```