# longstraw **Repository Path**: cqjsc_admin/longstraw ## Basic Information - **Project Name**: longstraw - **Description**: No description available - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2026-07-23 - **Last Updated**: 2026-07-23 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # LongStraw **Long-context RL beyond 2M tokens under a fixed GPU budget** [Paper](https://arxiv.org/abs/2607.14952) | [Architecture](ARCHITECTURE.md) | [Reproduction levels](docs/reproduction.md) | [Validation status](STATUS.md) LongStraw is an objective-aware, architecture-aware resident-state virtualization, response-replay, and distributed-gradient execution system for million-token RL post-training. It evaluates a shared prompt once without autograd, retains only the model-specific state needed by later tokens, and replays short response branches one at a time under autograd. This trades replay time for a much smaller live training graph. The generic update contract is: ```text S_prompt = no_grad_capture(prompt) for response_i in group: restore(S_prompt) old/ref scoring # values only; no training graph policy response replay # one response graph backward(loss_i) pop response state # restore the prompt boundary finalize distributed gradients optimizer.step(); zero_grad() ``` After the step, the objective chooses between recapturing `S_prompt` at the new parameters and retaining the old state as `stale` for a measured reuse window. Resident state determines what persists, replay determines what is recomputed, and gradient closure determines how the update is composed across ranks. LongStraw belongs to a broader fixed-budget memory-systems lineage that includes ZeRO-style partitioning and offload, activation recomputation, and [OOMB](https://github.com/wenhaoli-xmu/OOMB) ([ICLR 2026 paper](https://openreview.net/forum?id=dSa3ImCQr7)). OOMB targets full-sequence causal-LM training with chunkwise recomputation and paged/offloaded KV state. LongStraw targets a different structure: several RL responses share one long prompt, so prompt-state lifetime, serial old/reference/policy replay, group accumulation, and cache refresh must be coordinated across model architectures. The Qwen prototype uses selected `chunkoptim` cache and paged-attention kernels as implementation components; LongStraw is not an algorithmic extension of OOMB. The [paper](https://arxiv.org/abs/2607.14952) describes implementations for Qwen3.6-27B and GLM-5.2. This repository currently contains the GLM-5.2 resident-prefix implementation, direct-Ray execution contracts, validation tools, and compact historical evidence. Paper-wide evidence is stronger than this public checkout's rerun status. Qwen completes exact-attention 2M `G=2/G=8` response-only steps and reuses one 4.25M prefix across eight `G=8` optimizer cycles (64 member replays). Response-only retains the direct response gradient while avoiding the prompt-side backward graph. Cross-step cache reuse is a separate efficiency trade-off: in a 1M fresh-versus-reused probe, loss differs by only `0.1236%` and `0.0378%` after steps 1 and 2, then drifts more at steps 4 and 8, motivating periodic refresh. Historical response-side native/blockwise operator checks report sub-1% mean relative gradient differences (0.7547% for full attention, 0.3402% for GDN, and 0.5667% for a 1-to-4-chunk check); these are implementation self-checks, not a full-sequence prompt-gradient comparison. The declared response-only receipt is complete. The separate CP8 question is whether page-owner dK/dV contributions are finalized into one coherent update for replicated K/V LoRA copies; that is a distributed optimizer-finalization boundary, not evidence that response replay lacks gradients. For GLM-5.2, a 32-rank deterministic 2M run completes two all-78-layer backwards and one optimizer step. Separate 32K and 64K canaries complete real external policy-sampler LoRA rollouts, DAPO ground-truth-rule rewards, two Tinker/Megatron response-only backwards, and one optimizer step. The remaining gates are a fresh 2M run with the current global-DSA/gradient-finalization tree, post-fix full gradient parity, and 2M online rollout. The public-status block below applies to the GLM artifacts in this repository. > [!IMPORTANT] > **Public status: `review_only_not_runnable`.** The candidate backend is > checked in and enabled, but the immutable CUDA 13 image, execution and > redistribution authorization, approved fixtures, and current-tree 32x H20 > evidence are not public release assets. This checkout status describes whether > an external reviewer can launch a fresh run; it does not erase the archived > **2M deterministic execution closure or the passed 32K/64K online GRPO > canaries**. A fresh current-tree 2M parity receipt is pending. See > [STATUS.md](STATUS.md) for the exact release gates. ## Why LongStraw Long-context inference can prefill a prompt, retain a cache, and discard the forward graph. GRPO post-training has a different lifetime problem: several responses share the prompt, each response must be scored under multiple model roles, and policy gradients must be accumulated before an optimizer step. Keeping the full prompt graph and every response graph live at once quickly exhausts device memory. LongStraw instead separates long-lived conditional state from short-lived response graphs and makes the ownership of that state explicit across context and expert parallelism. ## Execution model 1. Run the shared prompt once without autograd. 2. Place compact, architecture-specific prompt state in owned GPU or CPU pages. 3. Reconstruct one response branch at a time for old, reference, and policy scoring. 4. Backpropagate only through the policy response branch. 5. Accumulate response gradients, finalize distributed gradients, and invoke the optimizer once. The following diagram is the GLM-5.2 CP32/EP32 instantiation of that contract: ```text prompt tokens | v CP32 prefix forward -----------------> CPU-resident MLA/DSA pages | +---------------------+---------------------+ | | | response 0 response 1 response G-1 old/policy/ref old/policy/ref replay | | | +---------- serial LoRA backward ---------+ | finalize_model_grads | one optimizer step ``` Raw rollout services, model weights, datasets, and credentials are not distributed with this training repository. The checked-in fixtures exercise interfaces and validators; the archived online evidence below records the real external sampling and reward runs. ## What is implemented - **Resident MLA/DSA prefix state.** Paged rank-local storage supports explicit ownership, append/pop transactions, and versioned validity boundaries. - **Global context-parallel sparse composition.** Rank-local DSA candidates are gathered and merged deterministically across CP32. - **Serial response-only GRPO.** Group members reuse one prefix while LoRA gradients accumulate across response replays. - **Bounded GLM-5.2 memory path.** The implementation combines chunked LM head and cross entropy, activation-input offload, DSA holder release, MoE replay checkpointing, and bounded expert kernels. - **Distributed evidence surfaces.** Parity tooling exports trainable shards, routing metadata, layer traces, gradient metrics, and optimizer state. ## Qwen and GLM paths | Dimension | Qwen3.6-27B | GLM-5.2 | | --- | --- | --- | | Resident state | 48 GDN recurrent states plus 16 full-attention KV page sets | MLA latent KV pages, DSA index-key pages, global positions, and IndexShare metadata | | Attention contract | CP8 page ownership and global LSE/output merge | CP32 candidate exchange and deterministic global sparse selection in the current tree | | Response replay | Four-block reverse replay with lazy/pruned detached-prefix gradient pages | MLA/DSA layer replay plus native router, EP32 all-to-all, and top-8 MoE experts | | Parallel layout | 8 H20, CP8 | 32 H20, TP1/CP32/EP32/ETP1/PP1 | | Main memory lever | Compact physical pages and one response block's graph | CPU pages, whole-layer checkpointing, bounded expert kernels, and chunked LM head/loss | | Demonstrated scale | 2.097M single steps; 4.456M eight-step resident reuse | 2.097M deterministic training-side receipt; 32K/64K online canaries | Qwen virtualizes and replays dense/recurrent history. GLM additionally reconstructs sparse selection and native MoE communication over compressed history. The current GLM implementation includes the global-DSA and `finalize_model_grads` contracts; fresh 2M parity remains a separate gate. ## Reported evidence | Track | Configuration | Observed result | Evidence scope | | --- | --- | --- | --- | | Standard full sequence | 32K, 32x H20, TP4/CP8/EP32 | Passed; model creation 115s, forward/backward 35s, optimizer 5s | [Historical baseline](evidence/megatron_full_sequence_baseline.json) | | Standard full sequence | 2,097,152 prompt tokens, same hardware/topology | OOM at 139.74 GiB process memory before a 2.54 GiB allocation | [Historical baseline](evidence/megatron_full_sequence_baseline.json) | | LongStraw resident prefix | 2,097,152 prompt tokens, 32x H20, TP1/CP32/EP32, 78 layers, LoRA rank 8, G2 | Prefix capture, two policy backwards, one terminal optimizer invocation on every rank | [Archived 2M execution receipt](evidence/2m_execution_receipt.json) | | Global-CP forward gate | 32K, CP32 | Selected-logprob max abs `9.84669e-5`; rank consensus `0` | [Forward-only canary](evidence/forward_parity_32k.json) | | Online GRPO canary | 32K, 32x H20, G2 | Real external completions and old logprobs, mixed DAPO rewards, two policy backwards, one optimizer step | [32K online receipt](evidence/online_canary_32k.json) | | Online GRPO canary | 64K, 32x H20, G2 | Same rollout-to-Tinker/Megatron transaction completes with finite values and full traces | [64K online receipt](evidence/online_canary_64k.json) | The resident-prefix run reached 64x the verified full-sequence baseline length. It used three response tokens per group member, scored two tokens, returned 32 rank reports and 128 layer/rank traces, and recorded 145.15 GB peak device allocation. The two online JSON receipts are archived external-run summaries (`source_state=external_run_not_source_bound`); they document the rollout/training handoff and are not current-checkout provenance. Together, these GLM results establish deterministic 2M training-side execution and preliminary end-to-end online compatibility at 32K/64K. The retained 2M receipt predates the current global-CP and gradient-finalization fixes, while the online canaries are not 2M gradient-parity or policy-quality experiments. The remaining boundaries are documented in [STATUS.md](STATUS.md) and [evidence/README.md](evidence/README.md). ## Local review The CPU and release-contract surface is reviewable from a fresh clone. Release binding checks require the commit's parent, so convert a shallow clone first with `git fetch --unshallow`. ```bash python -m pip install -r requirements/test-cpu.txt python -m pip install -e . make check make payload-2m python -m glm52 doctor --scope checkout --root . ``` `make check` runs the local contract suite, and `make payload-2m` constructs the deterministic 2M payload description. In the current public state, checkout doctor is expected to exit `2` with `"status":"blocked"`: it reports the missing runtime, authorization, input, and validation gates without importing Ray, Torch, Megatron, or the model. The Python distribution and installed CLI retain the `mint-2m` and `mint2m` names for compatibility: ```bash python -m glm52 --help mint2m --help ``` ## Full cluster validation This repository is not a turnkey 32-GPU demo. A live run requires four nodes with eight H20 GPUs each, an operator-managed Ray 2.51.1 cluster, a digest-pinned runtime image, a pinned GLM-5.2 snapshot, approved hash-bound fixtures, and an external writable evidence directory. The candidate execution is enabled, but every external prerequisite remains fail-closed. Start from the two schema-compatible deployment templates: ```bash sudo mkdir -p /etc/mint2m sudo cp environment/deployment.example.json /etc/mint2m/validation.json sudo cp environment/deployment.training.example.json /etc/mint2m/training.json export VALIDATION_DEPLOYMENT_FILE=/etc/mint2m/validation.json export TRAINING_DEPLOYMENT_FILE=/etc/mint2m/training.json ``` The validation deployment uses the fixture-set contract; the training deployment uses `mint2m.training-fixture/v1`. Replace every placeholder, including the all-zero digest, before planning. Plan mode has no Ray side effects: ```bash export ATTEMPT=longstraw-review-0001 python -m glm52 plan \ --config configs/run/glm52_2m_grpo_cp32.json \ --deployment "$TRAINING_DEPLOYMENT_FILE" \ --run-id longstraw-2m \ --instance-nonce "$ATTEMPT" \ --output artifacts/direct-plan.json ``` Before a live run, follow the runbook to validate the image, model, fixtures, runtime identity, Ray placement, and output directory. Training and owned resource cleanup use the same content-addressed plan: ```bash python -m glm52 train \ --plan artifacts/direct-plan.json \ --ownership artifacts/ownership.json \ --deployment "$TRAINING_DEPLOYMENT_FILE" \ --root . python -m glm52 cleanup \ --plan artifacts/direct-plan.json \ --ownership artifacts/ownership.json ``` Accepted evidence must advance through one identity-bound chain: ```text doctor -> kernel -> NCCL -> actor -> 32K -> 64K -> 2M ``` Each stage must pass before the next starts. A process exit code or unbound JSON status is not sufficient evidence. A validation command returns exit 0 only when its complete identity-bound bundle passes; exit 2 means blocked or invalid input and stops the chain. Use the following documents for the full workflow: - [Runbook](docs/runbook.md): deployment files, direct-Ray planning, execution, interruption recovery, and cleanup. - [Reproduction levels](docs/reproduction.md): what can be checked locally and what requires the external cluster. - [Release signing](docs/release-signing.md): bundle verification and the offline trust model. The operator entry point is `scripts/verify_release_bundle.py`. - [Limitations](docs/limitations.md): runtime, licensing, input, and evidence boundaries. The unsigned `v0.1.0` tag is review-only. Do not infer authenticity or run readiness from a tag name or mutable OCI tag. ## Repository layout ```text glm52/ resident-prefix and response-replay implementation runtime_patches/ opt-in backend, Bridge, and verl worker hooks integrations/ model preparation and runtime integration configs/run/ 32K, 64K, and 2M execution profiles scripts/ release, runtime, validation, and cleanup tools fixtures/ non-claiming parser and harness examples tests/ CPU contracts and distributed-interface tests evidence/ historical receipts and evidence index environment/ deployment schemas and runtime provenance docs/ runbook, reproduction guide, and limitations ``` ## Use and redistribution This evaluation repository does not grant an open-source license or redistribution permission. Authorized collaborators may inspect it for review; model weights, datasets, credentials, and raw run artifacts are not included. See [NOTICE.md](NOTICE.md) before using or redistributing any material. ## Citation If LongStraw is useful in your research, please cite: ```bibtex @misc{zhou2026longstrawlongcontextrl2m, title = {LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget}, author = {Changhai Zhou and Kieran Liu and Yuhua Zhou and Qian Qiao and Jun Gao and Harry Zhang and Irvine Lu and Nolan Ho and Lucian Li and Andrew Lei and Cleon Cheng and Steven Chiang and Yihang Zeng and Di Zhang and Rio Yang and Kaijie Chen and Andrew Chen and Pony Ma and Weizhong Zhang and Cheng Jin}, year = {2026}, eprint = {2607.14952}, archivePrefix = {arXiv}, primaryClass = {cs.LG}, url = {https://arxiv.org/abs/2607.14952} } ```