# wqi

**Repository Path**: limmin/wqi

## Basic Information

- **Project Name**: wqi
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-06-21
- **Last Updated**: 2026-09-08

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Water Quality WQI Prediction — Stacking with Hyperopt

Multi-optimizer MLP-lbfgs tuning + CV-RQE residual-constrained ensemble for
water quality index prediction, evaluated on four public WQI datasets under a
unified 80/20 holdout + 5-fold OOF protocol.

## Scope

- **Base model:** `MLP-lbfgs` (shared across all optimizers)
- **Optimizers:** `DE`, `SHADE`, `CMA-ES`, `NRBO`, `BOA`, `HHO` (60 function
  evaluations each, same 5-D search space)
- **Ensemble:** `CV-RQE` (Cross-Validated Residual-Quality Ensemble) — OOF error-covariance
  weighting with inverse-NRMSE quality prior and meta-CV λ selection
- **Datasets:** `Jajpur` (n=74), `Irish` (n=501), `AKH` (n=657), `Tiete` (n=7059)
- **Reporting baseline:** best single model chosen by **training CV**, never by
  test performance (see `AGENTS.md` hard constraint)

## Repository Layout

```
python_code/
├── _run_unified_holdout.py        # canonical entry point (80/20 + 5-fold OOF)
├── _preprocess_tiete_full.py      # rebuild Tiete .npz from raw CSV
├── common_codes/
│   ├── models/mlp_lbfgs.py
│   ├── optimizers/{DE,SHADE,CMAES,NRBO,BOA,HHO}.py
│   └── ensemble/{cv_rqe.py, holdout_evaluation.py}
├── experiments/
│   ├── _generate_paper_figures.py # regenerate manuscript figures
│   └── _verify_cv_rqe.py          # sanity-check weights/λ/OOF
├── analysis/
│   ├── _finalize_paper_results.py # rebuild unified_results_table.csv + bootstrap_ci_results.json
│   └── _analyze_difficult_samples.py
├── datasets/
│   ├── raw/                       # original CSV files
│   └── processed/                 # canonical *.npz
├── baseline/
│   ├── run_baseline_sklearn.py    # baseline comparison (5 seeds)
│   └── _generate_baseline_figures.py
└── results/                       # canonical result CSV/JSON + baseline/ (do not hand-edit)
论文/                              # main_filled.tex + correct_figures/ + compiled PDF
参考文献/                          # reference PDFs and bibliographies
AGENTS.md                          # canonical repository guidelines (read this first)
```

## Quick Start

```powershell
cd python_code
$env:PYTHONPATH="."

# 1. Run all four datasets (canonical rep 0, overwrites *_s42.* files)
python _run_unified_holdout.py --seed 42 --datasets Irish,AKH,Tiete,Jajpur --ea_seed_scheme B --ea_seed_base 1000 --overwrite

# 2. Finalize unified table + bootstrap CI
python analysis/_finalize_paper_results.py

# 3. Regenerate manuscript figures
python experiments/_generate_paper_figures.py
```

Environment note: NumPy 2.x breaks binary compatibility with the scikit-learn /
pandas wheels used here. Use the original conda environment with NumPy 1.x.

## Canonical Outputs

- `python_code/results/holdout_results_{Irish,AKH,Tiete,Jajpur}_s42.json`
- `python_code/results/unified_results_table.csv`
- `python_code/results/bootstrap_ci_results.json`
- `论文/correct_figures/*.pdf` (generated by `experiments/_generate_paper_figures.py` and `baseline/_generate_baseline_figures.py`)
- `论文/main_filled.pdf` (compiled manuscript)

## Experimental Invariants

See `AGENTS.md` for the full seed table, data-isolation rules, and coding
conventions. Key points:

- 80/20 split `random_state=42`; all inner CV / OOF / meta-CV use `random_state=1`
- EA seeds: scheme B, base 1000 (DE=1000 … HHO=1005)
- Test set never participates in EA optimization, weight fitting, or λ selection
- λ ∈ {0, .05, .1, .25, .5, .75, .9}, selected by nested meta-CV on OOF rows
- Best single model for reporting is chosen from training CV, not test R²