# agent-test-service

**Repository Path**: aiagentdemo/agent-test-service

## Basic Information

- **Project Name**: agent-test-service
- **Description**: 基于 Flask 构建一个 Agent 测试服务，通过逐步调用大模型解析测试步骤，驱动 agent-browser（Web）和 agent-device（App）执行手工测试用例，并提供任务状态查询和结果下载接口。
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-07-17
- **Last Updated**: 2026-07-17

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README



# Agent Test Service

[Chinese Version](#中文版)

## Overview

Agent Test Service is a Flask-based testing platform designed for AI agent evaluation and testing. It supports multiple testing modes including browser automation, mobile device testing (iOS), and provides integrations with LLM (Large Language Model) services and MCP (Model Context Protocol) servers.

## Features

- **Browser Testing**: Automated browser testing with screenshot capture and command execution
- **Device Testing**: iOS device testing with screen capture and command execution
- **Task Management**: Asynchronous task execution with status tracking
- **Device Locking**: Redis-based device locking mechanism for concurrent test management
- **MCP Server Integration**: Register and manage MCP servers for extended functionality
- **Skill Management**: Upload, manage, and execute test skills
- **Report Generation**: Generate JSON and HTML reports for test results

## Architecture

```
agent-test-service/
├── app.py                 # Flask application entry point
├── config.py              # Configuration module
├── routes/                # API route handlers
│   ├── browser_routes.py  # Browser testing endpoints
│   ├── device_routes.py   # Device testing endpoints
│   ├── mcp_routes.py      # MCP server management endpoints
│   └── skill_routes.py    # Skill management endpoints
├── runners/               # Test execution engines
│   ├── browser_runner.py  # Browser test execution
│   └── device_runner.py   # Device test execution
├── services/              # Core business logic
│   ├── task_store.py      # Task status management
│   ├── llm_service.py     # LLM integration service
│   ├── report_builder.py  # Report generation
│   ├── device_locker.py   # Device locking mechanism
│   ├── mcp_registry.py    # MCP server registry
│   └── skill_store.py     # Skill storage management
└── docs/                  # Documentation and planning
```

## Installation

1. Clone the repository:
```bash
git clone <repository-url>
cd agent-test-service
```

2. Install dependencies:
```bash
pip install -r requirements.txt
```

3. Configure environment variables:
```bash
cp .env.example .env
# Edit .env with your configuration
```

4. Start the service:
```bash
python app.py
```

## API Endpoints

### Browser Testing
- `POST /browser/run` - Start a browser test task
- `GET /browser/status/<task_id>` - Get task status
- `GET /browser/result/<task_id>` - Get task result

### Device Testing
- `POST /device/run` - Start a device test task
- `GET /device/status/<task_id>` - Get task status
- `GET /device/result/<task_id>` - Get task result
- `GET /device/locks` - List all locked devices
- `GET /device/locks/<serial>` - Get device lock info
- `DELETE /device/locks/<serial>` - Force release device lock

### MCP Server Management
- `GET /mcp` - List all MCP servers
- `GET /mcp/tools` - List all available tools
- `POST /mcp` - Register a new MCP server
- `DELETE /mcp/<name>` - Unregister an MCP server
- `POST /mcp/<name>/start` - Start an MCP server
- `POST /mcp/<name>/stop` - Stop an MCP server
- `GET /mcp/<name>/tools` - List tools from a specific server
- `POST /mcp/<name>/call` - Call a tool from a server

### Skill Management
- `GET /skills` - List all skills
- `GET /skills/<name>` - Get skill details
- `POST /skills` - Add a new skill
- `PUT /skills/<name>` - Update a skill
- `DELETE /skills/<name>` - Remove a skill
- `POST /skills/upload` - Upload a skill package

## Configuration

Configure the service using environment variables. Key configurations include:

- **Redis**: For device locking and task storage
- **OpenAI API**: For LLM integration
- **MCP Servers**: Path to MCP server configurations

## Requirements

See `requirements.txt` for a complete list of dependencies.

---

# 中文版

## 简介

Agent Test Service 是一个基于 Flask 的测试平台，专为 AI 代理（Agent）的评估和测试而设计。支持多种测试模式，包括浏览器自动化测试、移动设备测试（iOS），并提供 LLM（大型语言模型）服务和 MCP（模型上下文协议）服务器的集成。

## 功能特点

- **浏览器测试**：自动化浏览器测试，支持截图和命令执行
- **设备测试**：iOS 设备测试，支持屏幕截图和命令执行
- **任务管理**：异步任务执行，状态跟踪
- **设备锁定**：基于 Redis 的设备锁定机制，支持并发测试管理
- **MCP 服务器集成**：注册和管理 MCP 服务器以扩展功能
- **技能管理**：上传、管理和执行测试技能
- **报告生成**：生成 JSON 和 HTML 格式的测试报告

## 使用方法

1. 安装依赖：
```bash
pip install -r requirements.txt
```

2. 配置环境变量：
```bash
cp .env.example .env
# 根据需要编辑 .env 文件
```

3. 启动服务：
```bash
python app.py
```

## 许可证

本项目遵循项目仓库中指定的许可证。