# agent-service
**Repository Path**: hi_tom/agent-service
## Basic Information
- **Project Name**: agent-service
- **Description**: 基于 AI Agent Service 开源项目，添加了 LangGraph 的断点续执行功能
- **Primary Language**: Python
- **License**: MIT
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No
## Statistics
- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-02-28
- **Last Updated**: 2025-03-20
## Categories & Tags
**Categories**: Uncategorized
**Tags**: None
## README
# 🧰 AI Agent Service Toolkit
[](https://github.com/JoshuaC215/agent-service-toolkit/actions/workflows/test.yml) [](https://codecov.io/github/JoshuaC215/agent-service-toolkit) [](https://github.com/JoshuaC215/agent-service-toolkit/blob/main/pyproject.toml)
[](https://github.com/JoshuaC215/agent-service-toolkit/blob/main/LICENSE) [](https://agent-service-toolkit.streamlit.app/)
**add:基于 AI Agent Service 开源项目，添加了 LangGraph 的断点续执行功能**
A full toolkit for running an AI agent service built with LangGraph, FastAPI and Streamlit.
It includes a [LangGraph](https://langchain-ai.github.io/langgraph/) agent, a [FastAPI](https://fastapi.tiangolo.com/) service to serve it, a client to interact with the service, and a [Streamlit](https://streamlit.io/) app that uses the client to provide a chat interface. Data structures and settings are built with [Pydantic](https://github.com/pydantic/pydantic).
This project offers a template for you to easily build and run your own agents using the LangGraph framework. It demonstrates a complete setup from agent definition to user interface, making it easier to get started with LangGraph-based projects by providing a full, robust toolkit.
**[🎥 Watch a video walkthrough of the repo and app](https://www.youtube.com/watch?v=pdYVHw_YCNY)**
## Overview
### [Try the app!](https://agent-service-toolkit.streamlit.app/)
### Quickstart
Run directly in python
```sh
# At least one LLM API key is required
echo 'OPENAI_API_KEY=your_openai_api_key' >> .env
# uv is recommended but "pip install ." also works
pip install uv
uv sync --frozen
# "uv sync" creates .venv automatically
source .venv/bin/activate
python src/run_service.py
# In another shell
source .venv/bin/activate
streamlit run src/streamlit_app.py
```
Run with docker
```sh
echo 'OPENAI_API_KEY=your_openai_api_key' >> .env
docker compose watch
```
### Architecture Diagram
### Key Features
1. **LangGraph Agent**: A customizable agent built using the LangGraph framework.
1. **FastAPI Service**: Serves the agent with both streaming and non-streaming endpoints.
1. **Advanced Streaming**: A novel approach to support both token-based and message-based streaming.
1. **Content Moderation**: Implements LlamaGuard for content moderation (requires Groq API key).
1. **Streamlit Interface**: Provides a user-friendly chat interface for interacting with the agent.
1. **Multiple Agent Support**: Run multiple agents in the service and call by URL path
1. **Asynchronous Design**: Utilizes async/await for efficient handling of concurrent requests.
1. **Feedback Mechanism**: Includes a star-based feedback system integrated with LangSmith.
1. **Dynamic Metadata**: `/info` endpoint provides dynamically configured metadata about the service and available agents and models.
1. **Docker Support**: Includes Dockerfiles and a docker compose file for easy development and deployment.
1. **Testing**: Includes robust unit and integration tests for the full repo.
### Key Files
The repository is structured as follows:
- `src/agents/`: Defines several agents with different capabilities
- `src/schema/`: Defines the protocol schema
- `src/core/`: Core modules including LLM definition and settings
- `src/service/service.py`: FastAPI service to serve the agents
- `src/client/client.py`: Client to interact with the agent service
- `src/streamlit_app.py`: Streamlit app providing a chat interface
- `tests/`: Unit and integration tests
## Setup and Usage
1. Clone the repository:
```sh
git clone https://github.com/JoshuaC215/agent-service-toolkit.git
cd agent-service-toolkit
```
2. Set up environment variables:
Create a `.env` file in the root directory. At least one LLM API key or configuration is required. See the [`.env.example` file](./.env.example) for a full list of available environment variables, including a variety of model provider API keys, header-based authentication, LangSmith tracing, testing and development modes, and OpenWeatherMap API key.
3. You can now run the agent service and the Streamlit app locally, either with Docker or just using Python. The Docker setup is recommended for simpler environment setup and immediate reloading of the services when you make changes to your code.
### Building or customizing your own agent
To customize the agent for your own use case:
1. Add your new agent to the `src/agents` directory. You can copy `research_assistant.py` or `chatbot.py` and modify it to change the agent's behavior and tools.
1. Import and add your new agent to the `agents` dictionary in `src/agents/agents.py`. Your agent can be called by `//invoke` or `//stream`.
1. Adjust the Streamlit interface in `src/streamlit_app.py` to match your agent's capabilities.
### Docker Setup
This project includes a Docker setup for easy development and deployment. The `compose.yaml` file defines two services: `agent_service` and `streamlit_app`. The `Dockerfile` for each is in their respective directories.
For local development, we recommend using [docker compose watch](https://docs.docker.com/compose/file-watch/). This feature allows for a smoother development experience by automatically updating your containers when changes are detected in your source code.
1. Make sure you have Docker and Docker Compose (>=[2.23.0](https://docs.docker.com/compose/release-notes/#2230)) installed on your system.
2. Build and launch the services in watch mode:
```sh
docker compose watch
```
3. The services will now automatically update when you make changes to your code:
- Changes in the relevant python files and directories will trigger updates for the relevantservices.
- NOTE: If you make changes to the `pyproject.toml` or `uv.lock` files, you will need to rebuild the services by running `docker compose up --build`.
4. Access the Streamlit app by navigating to `http://localhost:8501` in your web browser.
5. The agent service API will be available at `http://0.0.0.0:8080`. You can also use the OpenAPI docs at `http://0.0.0.0:8080/redoc`.
6. Use `docker compose down` to stop the services.
This setup allows you to develop and test your changes in real-time without manually restarting the services.
### Building other apps on the AgentClient
The repo includes a generic `src/client/client.AgentClient` that can be used to interact with the agent service. This client is designed to be flexible and can be used to build other apps on top of the agent. It supports both synchronous and asynchronous invocations, and streaming and non-streaming requests.
See the `src/run_client.py` file for full examples of how to use the `AgentClient`. A quick example:
```python
from client import AgentClient
client = AgentClient()
response = client.invoke("Tell me a brief joke?")
response.pretty_print()
# ================================== Ai Message ==================================
#
# A man walked into a library and asked the librarian, "Do you have any books on Pavlov's dogs and Schrödinger's cat?"
# The librarian replied, "It rings a bell, but I'm not sure if it's here or not."
```
### Development with LangGraph Studio
The agent supports [LangGraph Studio](https://github.com/langchain-ai/langgraph-studio), a new IDE for developing agents in LangGraph.
You can simply install LangGraph Studio, add your `.env` file to the root directory as described above, and then launch LangGraph studio pointed at the root directory. Customize `langgraph.json` as needed.
### Using Ollama
⚠️ _**Note:** Ollama support in agent-service-toolkit is experimental and may not work as expected. The instructions below have been tested using Docker Desktop on a MacBook Pro. Please file an issue for any challenges you encounter._
You can also use [Ollama](https://ollama.com) to run the LLM powering the agent service.
1. Install Ollama using instructions from https://github.com/ollama/ollama
1. Install any model you want to use, e.g. `ollama pull llama3.2` and set the `OLLAMA_MODEL` environment variable to the model you want to use, e.g. `OLLAMA_MODEL=llama3.2`
If you are running the service locally (e.g. `python src/run_service.py`), you should be all set!
If you are running the service in Docker, you will also need to:
1. [Configure the Ollama server as described here](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-configure-ollama-server), e.g. by running `launchctl setenv OLLAMA_HOST "0.0.0.0"` on MacOS and restart Ollama.
1. Set the `OLLAMA_BASE_URL` environment variable to the base URL of the Ollama server, e.g. `OLLAMA_BASE_URL=http://host.docker.internal:11434`
1. Alternatively, you can run `ollama/ollama` image in Docker and use a similar configuration (however it may be slower in some cases).
### Local development without Docker
You can also run the agent service and the Streamlit app locally without Docker, just using a Python virtual environment.
1. Create a virtual environment and install dependencies:
```sh
pip install uv
uv sync --frozen
source .venv/bin/activate
```
2. Run the FastAPI server:
```sh
python src/run_service.py
```
3. In a separate terminal, run the Streamlit app:
```sh
streamlit run src/streamlit_app.py
```
4. Open your browser and navigate to the URL provided by Streamlit (usually `http://localhost:8501`).
## Projects built with or inspired by agent-service-toolkit
The following are a few of the public projects that drew code or inspiration from this repo.
- **[raushan-in/dapa](https://github.com/raushan-in/dapa)** - Digital Arrest Protection App (DAPA) enables users to report financial scams and frauds efficiently via a user-friendly platform.
**Please create a pull request editing the README or open a discussion with any new ones to be added!** Would love to include more projects.
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request. Currently the tests need to be run using the local development without Docker setup. To run the tests for the agent service:
1. Ensure you're in the project root directory and have activated your virtual environment.
2. Install the development dependencies and pre-commit hooks:
```sh
pip install uv
uv sync --frozen
pre-commit install
```
3. Run the tests using pytest:
```sh
pytest
```
## License
This project is licensed under the MIT License - see the LICENSE file for details.