# CGSN

**Repository Path**: mselab_admin_1/cgsn

## Basic Information

- **Project Name**: CGSN
- **Description**: CGSN — Core Gene Synteny Network
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-10-20
- **Last Updated**: 2026-10-22

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# CGSN — Core Gene Synteny Network

`cgsn` is a command-line tool for building **core gene synteny networks** and computing **topological conservation metrics**, supporting the "evolutionary blueprint" research in pathogen evolutionary genomics.

> Status: core functionality (`build` / `metrics` / `validate` / `diagnose` / `viz`) is implemented and validated by reproducing the mSystems paper (modularity Q = 0.9851, bit-exact match). The shell/cloud layer extension with physical-coordinate interpolation (direction 3) is also implemented and validated in silico.
> See [docs/CGSN程序开发设计方案.md](docs/CGSN程序开发设计方案.md) for the full design, and [docs/mSystems复现与验证结果.md](docs/mSystems复现与验证结果.md) for reproduction and analysis results.

## Features

- Consumes CVNet orthogroup output (grouped format) to build core gene synteny networks;
- Supports **complete and draft genomes** (auto-detection + explicit `complete/draft` column + censoring correction);
- Retains strand orientation (Head-to-Tail / Head-to-Head / Tail-to-Tail), physical distance, oriC-ter circular coordinates, and replichore information;
- Computes topological conservation metrics: CECI / SLE / MSI / NAC / RAII / TBCI;
- Built-in in-silico fragmentation self-check (`validate`);
- **Shell/cloud layers** positioned by physical coordinates: core anchors for complete genomes, double-anchor interpolation / single-anchor extrapolation for draft genomes;
- **Concentric network diagram** (`viz --circular --include-shell --cloud`): core ring + shell ring + cloud density halo.

## Installation

**Requirements**: C++17 compiler (GCC ≥ 7 / Clang ≥ 5), CMake ≥ 3.16, Python ≥ 3.9. OpenMP is optional. GCC < 9 needs `libstdc++fs` for `std::filesystem` — the build system links it automatically.

```bash
# 1. Build the C++ core
cmake -S . -B build && cmake --build build

# 2. Python front-end
python3 -m venv .venv
.venv/bin/pip install -e .
```

Core dependencies are only `numpy`, `networkx`, `pyyaml`, `pillow`, `svgwrite` — all either pure-Python or with working wheels. Optional extras:

```bash
.venv/bin/pip install -e '.[plot]'   # matplotlib：spectrum PNG（缺省时 Pillow 回退）
.venv/bin/pip install -e '.[gmm]'    # scikit-learn：tools/split_multicopy_og.py --method gmm
```

> **macOS hardened runtime note**: `scipy` / `scikit-learn` / `matplotlib` PyPI wheels can fail to load under some hardened Python runtimes (code-signature mismatch). They are deliberately **not** core dependencies. If you need the GMM method, install scikit-learn in a conda env instead, e.g. `conda create -n cgsn python numpy scikit-learn` and run with that env's Python.

## Subcommands

| Subcommand | Status | Description |
|---|---|---|
| `cgsn build` | ✅ | Adjacency counting → edge table (wraps the C++ hot loop) |
| `cgsn metrics` | ✅ | Topological conservation metrics (CECI/SLE/MSI/NAC/RAII/TBCI) |
| `cgsn validate` | ✅ | In-silico fragmentation self-check |
| `cgsn diagnose` | ✅ | Direction-3 diagnostic plots (presence / W_cond / insertion / contig_len / orient) |
| `cgsn viz` | ✅ | Network export (GraphML/GEXF) + concentric diagram + conservation spectrum |
| `cgsn recomb` | placeholder | Smoking-gun recombination localization |

## Usage

### build — build the network

```bash
cgsn build --genomes genomes.txt --og-table Orthogroups.txt \
    --cgsn-build ./build/src/cpp/cgsn-build --out edges.tsv
```

- `genomes.txt`: one GFF path per line; an optional second column `complete`/`draft` (auto-detected when omitted);
- `Orthogroups.txt`: CVNet grouped format (one OG per line, tab-separated `gene_id`, e.g. `accession|ORF_num`);
- Optional `--config config.yaml`: YAML config file (see `examples/config.example.yaml`); CLI arguments override the config values;
- Key options:
  - `--core-freq` — core-gene frequency threshold (default 0.99);
  - `--shell-low` — shell/cloud presence boundary (default 0.15); use `0` to retain **all** genes (including cloud) in the edge table, required for the `viz --cloud` halo; can also be set to `auto` to estimate the threshold automatically (Bernoulli mixture model + EM + ICL, see `partition.estimate_shell_low`);
  - `--core-positions` — output core-gene consensus positions (for `viz --circular`);
  - `--non-core-positions` — output all non-core (shell+cloud) gene consensus positions (physical theta);
  - `--interpolated-positions` — output interpolated positions for non-core genes in draft genomes (higher coverage than `--non-core-positions`);
  - `--orient-out` — orientation output (forward/inverted per complete genome);
  - `--replicon-labels` — replicon soft-labels (chromosome/plasmid/unknown);
  - `--cache-dir` — binary cache for parsed genomes (skip GFF re-parsing);
  - `--id-separator` — gene-ID separator between accession and gene number (default `'|'`; use `'-'` for most species);
  - contig gates: `--min-contig-len` (5000), `--contig-count-max` (200), `--contig-n50-min` (50000), `--complete-gene-fraction` (0.7), `--complete-max-seqids` (100).

### metrics — metrics

```bash
cgsn metrics --edges edges.tsv --out metrics.json
```

### validate — fragmentation self-check

```bash
cgsn validate --genomes complete_list.txt --og-table Orthogroups.txt \
    --cgsn-build ./build/src/cpp/cgsn-build --n-cuts 90 --out report.json
```

- `--min-n-adj` — minimum raw `n_adj` count (edge-table column 2) for an edge to be counted as high-adjacency (default 10); this is a raw count, **not** the normalized `W_cond` used by `metrics --weight-threshold` / `viz --backbone-threshold`.

### diagnose — diagnostic plots

```bash
cgsn diagnose --edges edges.tsv --og-table Orthogroups.txt \
    --core-positions positions.tsv --orient orient.tsv --out-dir diag/
```

Produces presence / W_cond / insertion-site / contig-length / orientation diagnostic plots (PNG) + a text summary.

### viz — visualization

```bash
# Network export (for Gephi)
cgsn viz --edges edges.tsv --out network.graphml --with-community

# Conservation spectrum (multi-species ranking)
cgsn viz --metrics m1.json m2.json ... --out spectrum.tsv --plot spectrum.png
```

### Concentric diagram — core + shell + cloud (the biologically meaningful default)

Full pipeline (build with `--shell-low 0` so the edge table retains cloud genes, then render the three-layer concentric diagram):

```bash
# 1. Build, emitting core / non-core / interpolated positions
cgsn build --genomes genomes.txt --og-table Orthogroups.txt \
    --shell-low 0 \
    --core-positions positions.tsv \
    --non-core-positions non_core_positions.tsv \
    --interpolated-positions interpolated_positions.tsv \
    --out edges.tsv

# 2. Render the concentric network: red core ring + green shell ring + gray cloud density halo
cgsn viz --edges edges.tsv --core-positions positions.tsv \
    --include-shell --cloud \
    --non-core-positions non_core_positions.tsv \
    --interpolated-positions interpolated_positions.tsv \
    --og-table Orthogroups.txt --id-separator '|' \
    --circular --out circular_cloud.png
```

- The shell ring encodes each shell gene's mean shell-shell `W_cond` (large/dark = frozen, small/light = loose); blue arcs are shell-shell edges with `W_cond ≥ --min-shell-weight` (default 0.5);
- The cloud halo is a Gaussian-smoothed density of cloud-gene positions (presence `< --shell-low`, default 0.15).

## Tools (`tools/`)

Standalone helper scripts (run with the same venv Python):

| Script | Purpose | Extra deps |
|---|---|---|
| `find_dnaA_hmmer.py` | Locate oriC via dnaA HMM (`--mode hmmsearch` uses the external `hmmsearch` binary; `--mode pssm` is a pure-Python fallback) | HMMER3 (optional) |
| `split_multicopy_og.py` | Split multi-copy core OGs into K copy positions (`--method kde` peak-finding, default, numpy-only; `--method gmm` GMM/BIC) | scikit-learn (`gmm` only) |
| `split_multicopy_edges.py` | Re-count copy-level core-core edges (Phase 2 of the split) | numpy |

The multi-copy split feeds the concentric diagram via `viz --og-copies ... --og-copy-edges ...`; see [docs/多拷贝OG拆分与镜像伪影.md](docs/多拷贝OG拆分与镜像伪影.md).

## References

1. Lu Y-F, Zuo G-H, Zhi X-Y. A structural backbone with sequestered plasticity organizes the *Escherichia coli* pangenome. *mSystems*. 2026;11:e00207-26. <https://doi.org/10.1128/msystems.00207-26>

2. Hai X, Wang J-L, Chen Q, Lu Y-F, Li X-P, Li C-J, Yue C-F, Song J-M, Yang L-L, Chen M, Li H-W, Zhi X-Y. A central hub for convergence: multi-layer genomic networks reveal lineage-specific plasmid compatibility driving hypervirulence and carbapenem resistance in ST11 *Klebsiella pneumoniae*. *Frontiers in Microbiology*. 2026;17:1932987. <https://doi.org/10.3389/fmicb.2026.1932987>

## License

MIT