# 织云 LoomNode

**Repository Path**: NorthGod_BFDG/LoomNode

## Basic Information

- **Project Name**: 织云 LoomNode
- **Description**: 内网异构显卡聚合推理集群 · OpenAI 兼容 · 生产级容错
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-06-22
- **Last Updated**: 2026-07-01

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

<div align="center">

# Loom (织云)

**Weave a roomful of mismatched GPUs into a cloud that runs large models.**

Aggregate the heterogeneous NVIDIA GPUs inside a single LAN (e.g. an internet café) into one cluster that serves an OpenAI-compatible API. Nodes can join and leave at any time — the service stays up.

[简体中文](./README.md) · [Engine](./docs/ENGINE.md) · [Architecture](./docs/ARCHITECTURE.md) · [Roadmap](./docs/ROADMAP.md) · [FAQ](./docs/FAQ.md)

> ⚠️ **Current stage: community warm-up.** This repository currently hosts public documentation only; the source code has not been released yet.
> We are finalizing the open-source boundary, license, and pre-release checklist. Star / Watch to follow along, and feel free to open an Issue to share ideas.

</div>

---

## ✨ Highlights

- **On par with vLLM** — head-to-head on one RTX 5090 (dense Qwen3-8B, bf16): we edge ahead at low-to-mid concurrency (N=1 `89` vs `80`, N=32 `2403` vs `2178`); vLLM wins at N=64.
- **Blackwell to Volta** — one engine runs across NVIDIA generations (RTX 5090 / Tesla V100), and can even **mix old and new cards in a single cross-machine pipeline**.
- **Big models via quantization** — multi-format quantization (int8 / int4 / fp8 + KV-cache quant) fits a **30B-class MoE** on a single consumer card, and **128K** context under int4.
- **Nodes drop, service doesn't** — ~6.7s **zero-loss** failover when the primary is killed; **lossless mid-stream resume** when a node dies during streaming.

> All numbers are real-machine measurements. Full capabilities and benchmarks (in Chinese): [Engine](./docs/ENGINE.md).

---

## What is it

**Loom** is an inference orchestration system built for **volatile, heterogeneous compute**.

The problem it targets is concrete: you have a fleet of **mixed-model consumer NVIDIA GPUs that may be occupied or go offline at any moment** (think 5060 / 5070 / 5080 cards in an internet café). No single card fits a large model, and together they are just a loose pile. Loom aggregates them — **within a local-area network** — into one unified inference cluster:

- **Outward**: a standard OpenAI-compatible API endpoint; existing apps connect with zero changes.
- **Inward**: the model is automatically split by layers and distributed across cards for collaborative inference (pipeline parallelism runs inside the LAN).
- **Fault-tolerant**: when a card gets claimed by a gamer, loses power, or crashes, the cluster routes around it and keeps serving — requests don't drop.

## Why "intranet"

Pipeline parallelism is extremely sensitive to inter-node round-trip latency. The tens-of-milliseconds RTT of a cross-region WAN gets amplified by pipeline depth, physically capping single-request latency — which is where comparable public-internet approaches tend to hit a wall.

Inside a **LAN, RTT is around 0.5 ms**, and interactive inference becomes viable. A café with 50 machines holds a substantial pool of idle VRAM — enough to run mainstream mid-sized models. Loom's mission is to awaken this **local compute that already exists but sits idle**.

## What it can do

- 🧩 **Heterogeneous aggregation** — no requirement for identical GPUs, or even the same generation; model layers are allocated by each card's real VRAM and compute.
- 🔀 **Mixed architectures** — from the latest Blackwell (RTX 5090) to the years-old Volta (Tesla V100), one engine adapts automatically, and old and new cards can even share one cross-machine pipeline.
- 🎛️ **Multi-format quantization** — weights in bf16 / int8 / int4 / fp8, KV cache in fp8 / int8 / int4; fit bigger models and longer context in less VRAM.
- 🔌 **OpenAI-compatible** — standard endpoints like `/v1/chat/completions`; drop-in for existing SDKs and tooling.
- 🛡️ **Volatility tolerance** — layer-granularity redundancy and failover designed for "nodes leave anytime."
- 🏗️ **Two-tier topology** — global gateway → per-cluster scheduler → multiple models per cluster; pipeline parallelism stays inside the cluster LAN, while the WAN only carries requests and generated tokens.
- 🖥️ **Desktop clients** — graphical desktop apps for both the node side and the cluster-controller side.
- 📊 **Production operations** — model rollout/rollback, maintenance mode, node trust management, built-in inference testing.

> Implementation details of quantization, long context, mixed architectures, and cross-machine scheduling will be documented alongside the code at release time; capabilities and real-machine numbers are on the [Engine](./docs/ENGINE.md) page.

## Architecture at a glance

Loom uses a layered "**Go control plane + minimal inference data plane**" design: each node runs just two processes — a Go daemon handling all networking and orchestration, and a Python process doing nothing but GPU forward passes. Requests carry their full route, keeping the scheduler off the request's critical path.

See [Architecture](./docs/ARCHITECTURE.md).

## Self-hosted vs commercial

Loom plans **two product lines**, with the boundary stated up front:

- **Self-hosted community edition (open source)**: node, cluster scheduler, cluster gateway, inference engine, desktop clients. You can stand up your own intranet cluster and run it yourself.
- **Commercial SaaS (closed source)**: the global access layer for end users (accounts, billing, cross-cluster scheduling, risk control) is not open-sourced and is the project's revenue source.

The open-source cluster side contains no billing/funds logic; connecting to the commercial layer is entirely optional.

## License

> **License pending.** The code has not been released under any open-source license yet. We lean toward a license that prevents cloud vendors from directly reselling a hosted service while staying open (e.g. AGPLv3 or BSL), and are confirming *license × patents × release timing* with legal counsel. Until a formal `LICENSE` file ships, the documents here are for reference only and grant no license.

## Brand & copyright

- **Company**: Shaanxi Qianling Technology Co., Ltd. (陕西千翎科技有限公司)
- **Brand narrative**: *A thousand plumes weave the cloud* (千羽织云)
- **Product**: Loom (织云)

"Loom / 织云" and related marks belong to Shaanxi Qianling Technology Co., Ltd.

© 2026 Shaanxi Qianling Technology Co., Ltd. · All rights reserved