# microduck-lab **Repository Path**: ncnynl/microduck-lab ## Basic Information - **Project Name**: microduck-lab - **Description**: https://github.com/jonathanhawkins/microduck-lab 备份并作中文化处理 - **Primary Language**: Unknown - **License**: Apache-2.0 - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2026-09-04 - **Last Updated**: 2026-09-04 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # Microduck Lab 🦆: RL experimentation on your Mac Train reinforcement-learning policies for the [Microduck](https://pollen-robotics.com/microduck), Pollen Robotics' open-source ~25 cm bipedal robot, **on an ordinary Apple Silicon Mac with no CUDA GPU**. Watch every policy walk, learn, and backflip live in your browser. ![The duck lab viewer: nine ducks running live, mid-backflip and mid-headstand](docs/media/viewer.png) | Running (locally trained on CPU, BAM actuator physics) | Backflip showcase: spotter-assisted launch, policy landing, stand handoff | |---|---| | ![running](docs/media/running.gif) | ![backflip](docs/media/backflip.gif) | The official [microduck_rl](https://github.com/pollen-robotics/microduck_rl) stack trains through MuJoCo Warp and needs a CUDA GPU. This project runs on the laptop you already have. It's a prototyping loop for reward design, curricula, and new tricks, built on the same MJCF robot model, the same 61-obs / 14-action deployment contract, and the same 50 Hz timing. A behavior you invent here ports straight to the official stack for the final sim2real run, and the ONNX you export is drop-in compatible with the official tooling. **Not affiliated with Pollen Robotics.** Two of their repos are used as side-by-side checkouts (see setup). ## What's in the box - **`microduck_local/`**: CPU-MuJoCo + Stable Baselines 3 PPO harness - `train-walk` / `train-behavior`: velocity-command walking and a library of teachable tricks, with mjlab-distilled rewards, symmetry augmentation, obs normalization, and a penalty-sign guard - Two actuator models: fast linearized XML servos, or the honest BAM XL330 voltage model (numba-fused) for maneuvers that saturate servos - `export-walk`: ONNX export with the obs normalizer baked in - `eval-walk`: headless eval battery (falls, command tracking) - `render-rollout`: mp4 for humans, **plus a captioned frame contact sheet an AI assistant can read**, carrying per-frame heights, angles, and contacts ([example](docs/media/contact-sheet.png)) - `bench-walk` / `bench-envs`: find the right worker count for *your* machine - `duck-lab`: the streaming backend that drives the browser viewer - **`duck-viewer/`**: Next.js + react-three-fiber viewer - Many ducks side by side, live over WebSocket at 25 Hz; drag policy chips onto ducks to hot-swap brains mid-stride - **🎓 Teach panel**: ask for one of nine built-in tricks ("stand on one leg") — keyword-matched, no LLM in the loop — see its reward recipe in plain English, watch the trainee improve every ~15 s as live snapshots hot-load, then drag the reward sliders and fine-tune. Reward shaping with no Python in the loop. - Staged curricula for hard tricks (the backflip is 5 chained stages), with the viewer narrating the chain - **🎬 Animate panel**: a keyframe pose editor with a game-style control rig. Author a motion clip in the browser, then "train this" makes RL learn to physically execute it - **🎥 Capture panel**: 📷 for a full-res PNG, 🎥 to have the camera frame a duck and film it. The lab converts the take to an mp4 and a GIF you can paste straight into a PR - **⤓ ONNX download** on any run (the baked export, normalizer included), and **⚙ settings** to connect your own Hugging Face token (stored and validated today; the GPU-job launcher is not wired up yet) ![Teaching a trick from the browser](docs/media/teach.png) ## Quick start Prereqs: macOS on Apple Silicon (Linux works too), [uv](https://docs.astral.sh/uv/), Node 20+, ~3 GB of disk for the checkouts and models. ```bash git clone https://github.com/jonathanhawkins/microduck-lab && cd microduck-lab git clone https://github.com/pollen-robotics/microduck # shipped policies, docs git clone https://github.com/pollen-robotics/microduck_rl # MJCF models, official stack cd microduck_local uv sync uv run --with pytest pytest tests/ # contract tests, should be all green # train your first walking policy (a few minutes on an M-series Mac) uv run train-walk --envs 32 --steps 3_000_000 --run-name first-gait uv run export-walk runs/first-gait uv run eval-walk runs/first-gait/policy.onnx # fire up the lab + viewer uv run duck-lab runs/first-gait ../microduck/policies/alpha_walking.onnx cd ../duck-viewer && npm install && npm run dev # open the printed URL ``` Then open the 🎓 teach panel and ask the duck to "stand on one leg". ## Performance (measured, Apple M-series) Every number below is reproduced in [microduck_local/README.md](microduck_local/README.md) with its methodology, including the experiments that got **rejected**. - **~16.5k env-steps/s** on the shipped 32-env recipe (about 1 min per 1M steps); ~27k steps/s peak in throughput configs - **One compiled MuJoCo model shared across all workers** (fork + copy-on-write): 64 envs dropped from **41 GB to 1.5 GB** of memory - Semaphore + shared-memory IPC instead of pipes/pickles per step - numba-fused BAM actuator kernels, bitwise-identical to the numpy reference - Optional MPS (Apple GPU) PPO updates, auto-enabled only where they measured faster - Two throughput "wins" (overlapped updates, big-batch) raised steps/s 25-40% but **halved learning per step** in seed-matched A/Bs, so they ship as opt-in flags rather than defaults. Reward per wall-second is the metric that matters. ## Teach the duck your own trick Tricks are plain-English reward recipes in [`behaviors/`](microduck_local/src/microduck_local/behaviors). A `Behavior` is a set of reward terms, chat keywords, and optionally a staged curriculum for the harder maneuvers. Add one, lock it with a test, and it shows up in the viewer's teach panel with live sliders. The full playbook is in [microduck_local/AGENTS.md](microduck_local/AGENTS.md): contract invariants, reward-design rules, and the verification discipline that keeps you from fooling yourself. The teach panel only offers tricks that exist in `behaviors/`: an unrecognized request returns the catalog rather than improvising a recipe. Adding a tenth is a Python change — see [Working with AI assistants](#working-with-ai-assistants) if you'd rather have a coding agent draft it. ## Animate: keyframe a motion, then make it real The 🎬 animate panel is a pose-and-timeline editor for authoring **reference motion clips** in the browser, and the bridge from animation to RL. ![Dragging the rig's squat handle](docs/media/animate-rig.gif) - **Pose the duck directly** (click a body part, drag) or through the **🎮 control rig**, a set of animator-style macro handles (`squat`, `lean`, leg swings, `sway`, `stance`, `twist`, `toes`, `look`). Each control is a direction in joint space chosen so feet stay planted and the controls are mutually orthogonal: squatting never disturbs the lean slider, and a stride keyed over a crouch keeps the crouch. A ⇕ handle parks on the duck itself, and dragging it down is what drives the squat in the clip above. - **Keyframe timeline** with auto-key, scrub/playback, and looping; the lab solves each pose server-side so the preview duck stays grounded. Clips save to [`microduck_local/clips/`](microduck_local/clips) as plain JSON (`run`, `sprint-cycle`, `backflip` ship as examples). - **⚡ Train this**: the clip becomes the reward. DeepMimic-style motion imitation ("be in the reference pose for right now") turns an open-ended search like *discover a backflip* into a tracking problem the policy can solve, without touching the 61-obs deployment contract. See [`motion.py`](microduck_local/src/microduck_local/motion.py). ## Capture: screenshots, video and GIFs, from the browser The capture panel sits at the top of the viewer and turns whatever the lab is doing right now into files you can drop into a PR or an issue. No screen recorder, no ffmpeg incantation. ![A finished take: the panel offers the mp4 and the gif](docs/media/capture.png) - **📷 shot** (always available) downloads a full-resolution PNG of the current view, named after the selected duck, or `duck-lab` for a crowd shot. The selection ring is hidden for the capture render, and the whole render → read → download runs inside the click's own user gesture, because Chrome silently drops a page's second "automatic" download. - **🎥 record** (appears once you click a duck) films that one duck for you: the camera glides to a ¾ front shot chosen from its heading, then holds it with a slow drift while MediaRecorder captures the WebGL canvas. Orbit and the camera keys pause for the take, and the footage comes out clean because the labels and panels are DOM, not canvas. - **■ stop** uploads the take to the lab ([`POST /captures`](microduck_local/src/microduck_local/viz_server.py)), whose bundled ffmpeg writes a full-resolution h264 **mp4** and a 480 px palette **gif** into `microduck_local/captures/`, and the panel offers both as ⬇ downloads. Takes cap at 60 s. One gotcha: frames are pushed per *rendered* frame (`captureStream(0)` + `requestFrame()`), because automatic capture rides the browser's compositor and records almost nothing in a throttled tab. Keep the tab visible while recording. A take where the scene never rendered gets refused with a message instead of saved as a 0.1 s "video". ## Working with AI assistants This repo is set up for agentic coding tools: - **`AGENTS.md`** (root and per-project): the workspace map and the training playbook, in the cross-tool convention used by Claude Code, Codex/ChatGPT, and most open-source agents. `CLAUDE.md` includes it for Claude Code. - **`.claude/skills/`**: three skills, all of which read as plain documentation for any agent, humans included. `render-rollout` teaches an agent to *look at* what a policy actually does (render the rollout, read the contact sheet) before believing reward curves; `watch-training` does the same for the run that is training right now; `restart-servers` brings the lab and viewer back up. ## Take the brain with you: ⤓ ONNX and 🤗 Hugging Face **⤓ Download the brain.** Hover a run in the 🧠 policies panel and a ⤓ appears next to it; one click saves that run's `.onnx`. You always get `policy.onnx`, the deployable export with the observation normalizer baked in, and never a raw checkpoint. A checkpoint handed over without its normalizer is quietly a different policy. While a run is still training the button falls back to its newest `live.onnx` snapshot, so you can pull a brain mid-run. On a staged trick, the chain's ⤓ gives you the **final** stage: every stage fine-tunes the same network, so the last one is the whole trick. **🤗 Connect Hugging Face (BYOK).** The ⚙ button in the duck-lab HUD opens settings, where you paste your own Hugging Face access token. Create one with write access at [hf.co/settings/tokens](https://huggingface.co/settings/tokens). ![The ⚙ settings pane: bring your own Hugging Face key](docs/media/settings.png) Bring your own key: your account, your billing, and the token never goes anywhere but huggingface.co. It gets validated with `whoami()` before anything is written, so a bad paste is rejected rather than stored. It lands in `microduck_local/hf-token.json`, mode `0600`, gitignored. The browser never sees it again: `GET /settings/hf` returns only your username and a mask like `hf_abcd…wxyz`. **disconnect** deletes the file. That key is for the one step a laptop can't do, retraining a behavior you prototyped here on real GPUs under your own account. Storing and validating the token is what ships today. The HF Jobs launcher isn't wired up yet. ## Sim2real, honestly This harness is for **prototyping**: minutes-long feedback loops on reward design, observations, and curricula. It runs a subset of the official stack's domain randomization, so don't ship its policies to a real robot. Once a behavior works here, port the env design to an mjlab cfg in `microduck_rl` and retrain on GPU (that repo's `AGENTS.md` is the sim2real recipe). Everything here keeps the deployment contract so that port is mechanical. ## License Apache-2.0 (same as the upstream Microduck repos). Not affiliated with or endorsed by Pollen Robotics; "Microduck" is their project.