The 1.46" round panel — each state has its own color, earcon, and elapsed-time counter.
### In action
Working and waiting states on the desk — the panel updates as the coding agent changes state.
## Install from a release (no build needed) The quickest path — grab the prebuilt assets from the [latest release](https://github.com/MapMindAI/ESP32-Monitor/releases/latest). You'll need [`esptool`](https://pypi.org/project/esptool/) (`pip install esptool`) and the board on USB-C. 1. **Flash the firmware.** Download `ESP32-Monitor-firmware.zip` and unzip it. It bundles the bootloader, app, partition table, and esp-sr voice models — no ESP-IDF toolchain required. From the unzipped folder (adjust the port): ```bash esptool.py --chip esp32s3 -p /dev/ttyACM0 -b 460800 \ --before default_reset --after hard_reset write_flash \ --flash_mode dio --flash_freq 80m --flash_size 16MB \ 0x0 bootloader/bootloader.bin \ 0x8000 partition_table/partition-table.bin \ 0x10000 ESP32-S3-Touch-LCD-1.46-Test.bin \ 0x394000 srmodels/srmodels.bin ``` (The same command is in `FLASH.md` inside the zip.) 2. **Run the PC app.** Download the bundle for your OS — `ESP32-Vibe-Monitor-windows.zip`, `ESP32-Vibe-Monitor-linux-ubuntu22.04.zip` (glibc ≥ 2.35), or `...-ubuntu24.04.zip` (glibc ≥ 2.39) — unzip and run `ESP32 Vibe Monitor` (the executable's filename; it opens as **VibeOrb**). No Python needed. 3. **Connect and wire up hooks** exactly as in steps 2–3 below: **Refresh Ports** → pick the ESP32 port → **Connect**, then **Install Claude Code Hooks** / **Install Codex Hooks**. ## Quick start (from source) ### 1. Flash the firmware Requires [ESP-IDF 5.x](https://docs.espressif.com/projects/esp-idf/en/stable/esp32s3/get-started/index.html) installed and sourced, and the board connected via USB-C. ```bash . $IDF_PATH/export.sh idf.py set-target esp32s3 idf.py build idf.py -p /dev/ttyACM0 flash monitor # adjust the port to match your device ``` First build fetches `lvgl` and `espressif/esp-sr` via the ESP-IDF Component Manager (declared in `main/idf_component.yml`) — no vendored copies are checked into this repo, so it needs network access once. ### 2. Launch the PC-side app Run it from source (for a prebuilt bundle, see [Install from a release](#install-from-a-release-no-build-needed) above): ```bash cd pc_app pip install -r requirements.txt python3 vibe_monitor_app.py ``` A small window opens: **Refresh Ports** → pick the ESP32 port (ESP32-S3 ports are marked `★` and shown first) → **Connect**. This app owns the serial link and keeps it open; leave it running. Install it as a clickable launcher with `python3 install_desktop_launcher.py`. ### 3. Wire it to Claude Code / Codex Click **Install Claude Code Hooks** in the app (or `python3 install_hooks.py`). It's idempotent and merges with whatever's already in your settings file (see [`pc_app/README.md`](pc_app/README.md) for `--project`/`--dry-run`/`--remove`). From then on Claude Code's hook events (`PreToolUse`, `Notification`, `Stop`, ...) forward to the running app, which drives the panel. For Codex, use **Install Codex Hooks** (or `python3 install_codex_hooks.py`) — same idea, writing to `~/.codex/hooks.json`. The hooks are a silent no-op whenever the app isn't running, so if the panel shows NO SIGNAL, check the app window is open and connected. ### 4. Try the voice command Say the wake word ("Hi ESP"), then "what is the status" — the panel flashes and re-plays the earcon for whatever it's currently showing. ## What's in this repo This started life as the vendor demo/test firmware for the board (LVGL demo widgets, a forked music-player UI, Wi-Fi/BLE scan readouts) and has been stripped down to one purpose: reporting AI coding status. See the plan/history in git log for the full before/after if you're curious what got removed. ``` ├── main/ ESP-IDF app component (device firmware) │ ├── main.c app_main(): driver init, then the LVGL + status loop │ ├── LVGL_Driver/ LVGL 9 display/touch driver glue for the QSPI panel │ ├── LVGL_UI/ AI_Status_UI - the one screen this device shows │ ├── Status_Link/ reads newline-JSON status updates over USB-serial │ ├── MIC_Driver/ esp-sr wake-word + voice command recognition │ ├── Audio_Driver/ I2S output + embedded earcon tones (no MP3/audio_player) │ ├── LCD_Driver/ SPD2010 QSPI panel driver │ ├── Touch_Driver/ capacitive touch driver │ ├── BAT_Driver/ battery voltage ADC │ ├── PCF85063/ RTC │ ├── QMI8658/ IMU │ ├── SD_Card/ SD/FAT filesystem │ ├── Wireless/ Wi-Fi/BLE scan (present but unused - no networking needed) │ ├── EXIO/, I2C_Driver/, PWR_Key/ board-support glue │ └── idf_component.yml pins lvgl (~9.5) and espressif/esp-sr (~1.9.4) ├── pc_app/ PC-side Python app │ ├── vibe_monitor_app.py PyQt5 GUI: owns the serial link + a local status server │ ├── theme.py dark Qt theme │ ├── install_hooks.py merges the hooks into ~/.claude/settings.json (or --project) │ ├── install_codex_hooks.py Codex CLI counterpart (~/.codex/hooks.json) │ ├── install_desktop_launcher.py installs a .desktop launcher for the GUI │ ├── requirements.txt pyserial + PyQt5 │ └── claude_esp32_monitor/ │ ├── protocol.py SessionTracker + status/device-payload logic + IPC host/port │ ├── transport.py async SerialTransport (port auto-detect + JSON writes) │ ├── ipc_client.py TCP client used by the hook + CLI │ ├── report_status.py Claude Code / Codex hook entry point (stdin -> IPC) │ └── __main__.py manual CLI for testing without live hooks ├── tools/ │ └── gen_earcons.py synthesizes the tones baked into main/Audio_Driver/earcons.c ├── partitions.csv flash layout (includes the esp-sr model partition) └── sdkconfig.defaults board/Kconfig defaults (PSRAM, speech recognition, fonts, ...) ``` Nothing under `managed_components/` or `build/` is checked in — both are fetched/generated by `idf.py build` and are gitignored. `dependencies.lock` *is* checked in so builds resolve the exact same component versions. ## How it fits together ``` Claude Code hooks --(stdin JSON)--> report_status.py --(TCP 127.0.0.1:47615)--> vibe_monitor_app.py --(serial JSON line)--> USB-C --> Status_Link --> AI_Status_UI ``` - **PC side**: `vibe_monitor_app.py` is a long-running GUI that owns the serial link and a local status server. Claude Code fires a hook on events like `PreToolUse` or `Stop`; `report_status.py` maps the event to one of `idle` / `working` / `waiting` / `error` and forwards it (with the session id + project name) to the app over a loopback socket. The app tracks all active sessions, picks the effective one, and re-sends its status every 2s as a keepalive so a long-running command doesn't look like "no signal" between hook events. When the app isn't running the hooks silently no-op (see `pc_app/README.md`). - **Device side**: `Status_Link` reads those lines off the same USB-serial console used for flashing/logging (no Wi-Fi/BLE), and hands the parsed status to `AI_Status_UI` once a second. If nothing arrives for 15s, the panel falls back to a local "no signal" state on its own - comfortably longer than the app's 2s keepalive interval. - **Voice**: `MIC_Driver` runs esp-sr's wake-word + command recognition independently of the serial link, and calls back into `AI_Status_UI` for backlight commands and the "what is the status" query. ### Statuses The panel shows one of five states (`main/LVGL_UI/AI_Status_UI.c`): | status | panel shows | color | when | |---|---|---|---| | `idle` | IDLE | grey `#60666d` | session start, Claude finished responding (`Stop`), session end | | `working` | WORKING (spinner) | blue `#2d7dd2` | prompt submitted, a tool is running, a subagent is active | | `waiting` | WAITING | amber `#e8a33d` | Claude is waiting on you (permission prompt, idle nudge) | | `error` | ERROR | red `#d64545` | not currently wired to a hook event - reserved for future use | | *(local only)* | NO SIGNAL | dark grey `#33363a` | no status line received in the last 15s (cable unplugged, hooks not installed, PC off) | Each transition plays a distinct earcon (`tools/gen_earcons.py`) and updates the secondary tool/detail line and the elapsed-time-in-state counter. ### Status wire format One JSON object per line, newline-terminated: ```json {"status": "working", "project": "ESP32-Monitor", "tool": "Edit", "detail": "editing main.c"} ``` `status` is one of `idle`, `working`, `waiting`, `error`. `project` (shown as a label at the top of the panel), `tool`, and `detail` (a secondary line) are all optional. ## Voice commands Wake word: **"Hi ESP"**, then say one of: | phrase | action | |---|---| | "Backlight is brightest" | backlight to 100% | | "Backlight is darkest" | backlight to 30% | | "Turn off the backlight" | backlight off | | "Turn on the backlight" | backlight to 100% | | "What is the status" | re-announce current status (earcon + flash) | New phrases are added as a Kconfig string in `sdkconfig.defaults` (`CONFIG_EN_SPEECH_COMMAND_ID*`) plus a matching `case` in the command handler (`mic_command_handler` in `AI_Status_UI.c`; recognition itself runs in `MIC_Speech.c`) - see esp-sr's `tool/multinet_g2p.py` for how the phonetic encoding works.