# hermes-plus **Repository Path**: timebox/hermes-plus ## Basic Information - **Project Name**: hermes-plus - **Description**: 基于 Nous Hermes Agent 构建的开源“电脑 + 手机双模智能体”桌面连接器。完整保留官方电脑端 Agent 能力，支持浪花 AI 手机毫秒级触控映射、VLM 视觉大模型截屏闭环、真机应用管理与 SSE 实时事件流监听。 - **Primary Language**: TypeScript - **License**: MIT - **Default Branch**: main - **Homepage**: https://bot.beikke.com/docs - **GVP Project**: No ## Statistics - **Stars**: 2 - **Forks**: 0 - **Created**: 2026-09-26 - **Last Updated**: 2026-10-20 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README

# Hermes Plus — AI Phone & Desktop Dual-Mode Agent

--- > 🚀 **Hermes Plus** is an open-source, dual-mode desktop client and developer boilerplate built on top of [fathah/hermes-desktop](https://github.com/fathah/hermes-desktop). > It preserves **100% of the native desktop agent experience** (powered by Nous Hermes Agent) while introducing a **physical AI Phone extension module** (`phone/`) for hardware device discovery, low-latency touch/gesture control, Vision-Language Model (VLM) screenshot perception, and Server-Sent Events (SSE) streaming. --- ## 🌟 Key Highlights ### 1. 💻 100% Native Desktop Agent Experience - **Full Upstream Parity**: All upstream features — multi-provider LLM chat, profiles, memory system, MCP tools, cron schedules, messaging gateways, and 3D office — remain completely functional and untouched. - **Zero Framework Bias**: Third-party developers can freely build custom agents with LangChain, CrewAI, AutoGen, or custom workflows without being locked into proprietary agent architectures. ### 2. 📱 Physical AI Phone Extension (`phone/`) - **Seamless Dual-Mode Switcher**: Switch between **[ 💻 Computer ]** and **[ 📱 Phone ]** modes with a single click, sharing the native macOS/Windows window frame and theme colors. - **Millisecond-Grade Atomic Actions**: Instant execution of physical taps, multi-point swipes, text input, and Android hardware keys (Back, Home, Recents). - **VLM Multimodal Loop**: Single-frame high-resolution screenshot retrieval (Base64 + real screen dimensions) pre-configured for Vision-Language Models (GPT-4o, Claude 3.5 Sonnet, Qwen-VL). - **System Privileges**: Remote screen unlock without root friction, notification bar pull-down, and direct photo album injection. - **Reactive Device Center**: Real-time device discovery, battery tracking, and online status synchronization via SSE. ### 3. 🔄 Low-Intrusion Architecture (Smooth Upstream Sync) - **Physical Code Isolation**: All mobile capabilities are strictly self-contained within the [`phone/`](./phone) directory. - **Minimal Diffs**: Only 3 hook points in the core upstream codebase (navigation mount in `Layout.tsx`, CSP header in `start.ts`, and path inclusion in `tsconfig.web.json`). - **Seamless Updates**: Smoothly pull and merge upstream releases via `git merge upstream/main` in minutes. --- ## 📸 Feature Showcase ### Dual-Mode & Device Management | Agent Starter Workspace (`01-new-work`) | Real Device Center (`02-my-phones`) | | :---: | :---: | |

| | Instant QR-Code Pairing (`03-binding-modal`) | Developer Gateway Settings (`04-gateway-settings`) | | :---: | :---: | |

| | System Privileges & Album Injection (`07-system-priv`) | VLM Multimodal Perception (`08-screenshot-vlm`) | | :---: | :---: | |

| | Real-Time SSE Event Stream (`09-event-stream`) | | :---: | |

| --- ## ⚡ Quickstart ### 1. Clone & Install Dependencies ```bash # Clone the repository git clone https://github.com/terrancehxx/hermes-plus.git cd hermes-plus # Install dependencies (requires Node.js 20+) npm install ``` ### 2. Launch Development Mode ```bash npm run dev ``` ### 3. Connect to Langhua AI Gateway 1. Open the app and switch to **[ 📱 Phone ]** mode at the top of the sidebar. 2. Click **Developer Settings** (系统与开发 -> 开发者设置). 3. The official production gateway is configured by default: ```text https://api6.beikke.com/aibot/v1 ``` 4. Enter your developer `AppKey` (obtainable for free at [Langhua AI Open Platform](https://bot.beikke.com/open/appkey)). 5. Click **Save & Verify Credential**. Once validated, the green **● Connected** badge will light up! --- ## 💻 TypeScript SDK Usage The client provides an ergonomic, strongly typed singleton client `phoneClient`: ```typescript import { phoneClient } from "./phone/sdk/phone-client"; // 1. Simulate Touch & Swipe await phoneClient.tap("DEVICE_ID", 540, 1200); await phoneClient.swipe("DEVICE_ID", 540, 1600, 540, 600, 300); // 2. Launch Target App & Input Text await phoneClient.launchApp("DEVICE_ID", "com.tencent.mm"); await phoneClient.inputText("DEVICE_ID", "Hello from Hermes Plus!"); // 3. Capture Screen for Vision-Language Models (VLM) const shot = await phoneClient.screenshot("DEVICE_ID", 50, 0.5); console.log(`Captured image: ${shot.width}x${shot.height}, base64 length: ${shot.image_base64.length}`); // 4. Feed Base64 into Multimodal LLM (GPT-4o / Claude 3.5 / Qwen-VL) const vlmResponse = await openai.chat.completions.create({ model: "gpt-4o", messages: [{ role: "user", content: [ { type: "text", text: "Find the coordinates of the 'Search' button: {x, y}" }, { type: "image_url", image_url: { url: `data:image/jpeg;base64,${shot.image_base64}` } } ] }], response_format: { type: "json_object" } }); ``` --- ## 🔄 Upstream Synchronization We maintain strict architectural boundaries documented in [`UPSTREAM_SYNC.md`](./UPSTREAM_SYNC.md). To merge upstream bug fixes and new features: ```bash # Add upstream remote (if not already added) git remote add upstream https://github.com/fathah/hermes-desktop.git # Fetch and merge git fetch upstream main git merge upstream/main # Build & test npm run build ``` --- ## 📄 License & Attribution - **License**: Released under the permissive [MIT License](./LICENSE). - **Attribution**: Based on the open-source project [`fathah/hermes-desktop`](https://github.com/fathah/hermes-desktop) by Fathah and the Nous Research community. - **Disclaimer**: Hermes Plus is an independently developed open-source community extension. It is not directly affiliated with or endorsed by Nous Research.