# Joint Scenario Generation 

**Repository Path**: fgy-lab/joint-scenario-generation

## Basic Information

- **Project Name**: Joint Scenario Generation 
- **Description**: Joint Scenario Generation for Multiple Wind Farms via a Spatiotemporal Dual Expert Conditional Diffusion Model
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-07-30
- **Last Updated**: 2026-08-25

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# STDE-CDM

Official research package for:

> **Joint Scenario Generation for Multiple Wind Farms via a Spatiotemporal
> Dual Expert Conditional Diffusion Model**

STDE-CDM generates synchronized day-ahead trajectories for multiple wind
farms. A conditional distribution expert and a joint spatiotemporal expert
learn complementary residual distributions. Their samples are indexed by
shared Gaussian innovations and fused at trajectory level. The repository
also contains the jointly trained baselines, the locked-test evaluation, the
paper figures, and the downstream FICA dispatch study.

## Repository layout

```text
STDE-CDM/
├── src/
│   ├── stde_cdm/          # proposed model, joint baselines, data and FICA adapter
│   ├── cldm/              # copied CLDM implementation used by this project
│   └── models/            # UMNN normalizing-flow implementation
├── scripts/               # all training, evaluation, plotting and FICA entry points
├── tests/                 # model and data-interface tests
├── configs/               # frozen configurations for the paper experiments
├── data/                  # packaged GEFCom2014 multisite wind data
├── artifacts/checkpoints/ # frozen final weights for all ten paper seeds
├── results/               # locked metric records and compact FICA outputs
├── figures/results/       # reproducible plots and compact plotting data
├── paper/                 # manuscript and synchronized figure copies
├── notebooks/             # trace notebook retained for inspection
├── fica_dispatch_optimizer/ # FICA dispatch solver and upstream license
├── docs/                  # protocol, experiment log and plotting standard
├── environment.yml
└── pyproject.toml
```

All executable Python files are in `scripts/`; model code is in `src/`; all
paper assets are in `paper/`. No script needs the former parent workspace.

## Environment

The experiments on this machine use the existing Conda environment `dl`.

```bash
cd /path/to/STDE-CDM
conda env update -n dl -f environment.yml
conda activate dl
python -m pip install -e .
```

FICA requires a valid Gurobi installation and license. The scenario-generation
models and the main metric evaluation do not require Gurobi.

## Quick verification

```bash
conda run -n dl python -m compileall -q src scripts tests
conda run -n dl python -m pytest -q
```

The latest script-by-script audit is recorded in
`docs/SCRIPT_VERIFICATION.md`.

Compile the manuscript:

```bash
cd paper
latexmk -pdf -pdflatex="pdflatex %O %S" -interaction=nonstopmode -file-line-error manuscript.tex
```

## Training

Every model receives one joint target with shape `24 hours × K sites`. The
scripts use the same train, validation and locked-test split.

```bash
conda run -n dl python scripts/train_joint.py --seed 0
conda run -n dl python scripts/train_st_jcdm.py --seed 0
conda run -n dl python scripts/train_joint_wgan_gp.py --seed 0
conda run -n dl python scripts/train_joint_vae.py --seed 0
conda run -n dl python scripts/train_joint_umnn.py --seed 0
conda run -n dl python scripts/train_joint_ddpm.py --seed 0
```

Repeat with seeds `0` through `9` to retrain the full paper protocol. Training
writes new checkpoints under `outputs/checkpoints/`. The repository already
contains the frozen final checkpoints for all ten published training seeds in
`artifacts/checkpoints/`, so the locked-test metrics can be recomputed without
retraining.

## Locked-test evaluation

Evaluate the six paper metrics for all jointly trained model families:

```bash
conda run -n dl python scripts/evaluate_all_models_locked_test.py \
  --seeds 0 1 2 3 4 5 6 7 8 9 --scenarios 200

conda run -n dl python scripts/evaluate_dual_expert_locked_test.py \
  --seeds 0 1 2 3 4 5 6 7 8 9 --scenarios 200 --weight 0.4

conda run -n dl python scripts/build_paper_locked_test_result.py
```

The last command validates and combines the frozen source evaluations into
`results/metrics/paper_locked_test_10seed.json`, the single authoritative
record used by the manuscript and its comparison figure.

## Paper figures

The primary generated figures and compact plotting data are under
`figures/results/`. Each plotting script also synchronizes the corresponding
PDF and PNG files into `paper/figures/`, so the manuscript remains directly
compilable. Rebuild the quantitative figures with:

```bash
conda run -n dl python scripts/make_paper_visualizations.py
conda run -n dl python scripts/make_locked_test_dm_tests.py
conda run -n dl python scripts/make_fica_backtest_figure.py
```

Editable architecture concepts and Visio material are under
`paper/figure_sources/`.

## FICA dispatch validation

The formal downstream experiment uses each model's native deterministic
forecast and its own generated candidate distribution. Constraint-relevant
trajectories are selected under one common fixed scenario budget, after which
the optimized policy is evaluated against the locked observation. The paper
reports only this common constraint-aware protocol.

Run a dry check first:

```bash
conda run -n dl python scripts/run_fica_native_sthead_50day.py --dry-run
```

Run selected days or all 50 locked test days:

```bash
conda run -n dl python scripts/run_fica_native_sthead_50day.py --days 0 1
conda run -n dl python scripts/run_fica_native_sthead_50day.py
```

Each day is committed independently, so an interrupted run can be restarted
with the same command. Compact frozen summaries are in `results/fica/`; the
large regenerable per-day candidate pools are intentionally excluded from the
portable package.

## Data and artifact policy

- `data/wind_data_all_zone.csv` is the joint GEFCom2014 wind dataset used by
  the code. Review the original dataset terms before public redistribution.
- The 60 final ten-seed model checkpoints are packaged. Intermediate
  component checkpoints, optimizer states and per-day 6,000-scenario
  candidate pools remain regenerable and are not packaged.
- Use Git LFS or a release archive for the packaged checkpoint binaries.
- `PROJECT_MANIFEST.md` records what was copied and what was intentionally
  left out.

## Licensing

No project-wide license has been selected yet. Before making the repository
public, the authors should choose a license for the new STDE-CDM code and
review the GEFCom2014 data and retained third-party terms. The FICA optimizer
keeps its upstream GPL-3.0 notice under `fica_dispatch_optimizer/`, while
Gurobi is proprietary software requiring a separate license.