# AI Agents hallucinate. Fix it faster.
**The open-source platform for shipping self-improving AI agents.** Evaluations, tracing, simulations, guardrails, gateway, optimization. Everything runs on one platform and one feedback loop, from first prototype to live deployment.
Try Cloud (Free) · Self-Host · Docs · Blog · Discord · Discussions
| ### All-in-one No more stitching Langfuse + Braintrust + Helicone + Guardrails AI + a custom simulator. One platform covers the lifecycle: **simulate → evaluate → protect → monitor → optimize**, with data flowing back as a loop. | ### Open & self-hostable Apache 2.0 core. Every evaluator, every prompt, every trace is inspectable — **no black-box scoring**. Self-host for data sovereignty or use our managed Cloud. Drop in your own stack at any layer via OTel / OpenAI-compatible HTTP. | ### Built for production Go-based gateway with **~9.9 ns weighted routing**, **~29 k req/s on t3.xlarge**, **P99 ≤ 21 ms with guardrails on**. OpenTelemetry-native traces. 50+ framework instrumentors. Every claim reproducible via the committed benchmark harness. |
| Cloud (fastest) | Self-host (Docker) |
|---|---|
| **No install. Free tier.** ```bash # Sign up free: # app.futureagi.com pip install ai-evaluation ``` SOC 2 Type II · HIPAA · data stays in your region. | **One command, full stack. Published images, no source build.** ```bash # macOS / Linux / WSL git clone https://github.com/future-agi/future-agi.git cd future-agi ./bin/install # Windows (PowerShell) git clone https://github.com/future-agi/future-agi.git cd future-agi .\bin\install.ps1 ``` Open [http://localhost:3000](http://localhost:3000). For production, use `./deploy/setup.sh` to generate required secrets and pin the image version. |
| **Python** ```python from fi_instrumentation import register from traceai_openai import OpenAIInstrumentor register(project_name="my-agent") OpenAIInstrumentor().instrument() # Your existing OpenAI code is now traced. response = client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": query}], ) ``` | **TypeScript** ```typescript import { register } from "@traceai/fi-core"; import { OpenAIInstrumentation } from "@traceai/openai"; register({ projectName: "my-agent" }); new OpenAIInstrumentation().instrument(); // Your existing OpenAI code is now traced. const response = await openai.chat.completions.create({ model: "gpt-4o", messages: [{ role: "user", content: query }], }); ``` |
| ### 🧪 Simulate Thousands of multi-turn conversations against realistic personas, adversarial inputs, and edge cases. Text **and voice** (LiveKit, VAPI, Retell, Pipecat). [Docs →](https://docs.futureagi.com/docs/simulation) | ### 📊 Evaluate 50+ metrics under one `evaluate()` call: groundedness, hallucination, tool-use correctness, PII, tone, custom rubrics. **LLM-as-judge + heuristic + ML.** [Docs →](https://docs.futureagi.com/docs/evaluation) | ### 🛡️ Protect 18 built-in scanners (PII, jailbreak, injection, …) + 15 vendor adapters (Lakera, Presidio, Llama Guard, …). Inline in gateway or standalone SDK. [Docs →](https://docs.futureagi.com/docs/protect) |
| ### 👁️ Monitor OpenTelemetry-native tracing across 50+ frameworks (LangChain, LlamaIndex, CrewAI, DSPy…). Span graphs, latency, token cost, live dashboards. Zero-config. [Docs →](https://docs.futureagi.com/docs/observe) | ### 🎛️ Agent Command Center OpenAI-compatible gateway. 100+ providers, 15 routing strategies, semantic caching, virtual keys, MCP, A2A. **~29k req/s, P99 ≤ 21ms with guardrails on.** [Docs →](https://docs.futureagi.com/docs/command-center) · [Benchmarks →](./agentcc-gateway/README.md#-benchmarks) | ### 🔁 Optimize Six prompt-optimization algorithms (GEPA, PromptWizard, ProTeGi, Bayesian, Meta-Prompt, Random). Production traces feed back as training data. [Docs →](https://docs.futureagi.com/docs/optimization) |
| Future AGI | Langfuse | Phoenix | Braintrust | Helicone | |
|---|---|---|---|---|---|
| Open source | ✅ Apache 2.0 | ✅ MIT | ✅ Elastic v2 | ❌ | ✅ Apache 2.0 |
| Self-host | ✅ | ✅ | ✅ | ❌ | ✅ |
| LLM tracing (OpenTelemetry) | ✅ | ✅ | ✅ | ✅ | ⚠️ via OpenLLMetry |
| Evaluation suites | ✅ 50+ metrics | ✅ | ✅ | ✅ | ⚠️ Limited |
| Agent simulation | ✅ | ❌ | ❌ | ❌ | ❌ |
| Voice agent eval | ✅ | ❌ | ⚠️ Cookbook | ❌ | ❌ |
| LLM gateway built in | ✅ 100+ providers | ❌ | ❌ | ✅ | ✅ |
| Guardrails built in | ✅ 18 + 15 adapters | ❌ | ❌ | ❌ | ❌ |
| Prompt optimization | ✅ 6 algorithms | ❌ | ❌ | ❌ | ❌ |
| Prompt management | ✅ | ✅ | ✅ | ✅ | ✅ |
| Datasets & experiments | ✅ | ✅ | ✅ | ✅ | ✅ |
| No-code eval builder | ✅ | ⚠️ | ⚠️ | ⚠️ | ⚠️ |
| Recently shipped | In progress | Coming up | Exploring |
|---|---|---|---|
| - [x] Prompt optimization engine - [x] Taxonomy-based Feed Clustering - [x] Agent Runs in Dataset Experiments - [x] Simulate from Production Calls - [x] LiveKit Configuration via UI - [x] System Metric Filtering for Voice - [x] Agent Playground - [x] Dashboards - [x] Access platform via MCP - [x] Annotation Queues - [x] Command Center - [x] Open source Future AGI stack - [x] Eval Explanation Output Size Control | - [ ] Agent Changelog & Diff View - [ ] Smart Queue Assignment - [ ] Essential Node Library for Agent Builder - [ ] Full Execution Tracing for Agents - [ ] Multi-modal Support for Agents | - [ ] Agent Changelog & Diff View - [ ] Smart Queue Assignment | - [ ] Import agents to Agent Playground - [ ] Simulating CUA agents - [ ] Simulating Coding agents - [ ] Scheduled Simulations |