# MCP

**Repository Path**: mirrors_mitre/MCP

## Basic Information

- **Project Name**: MCP
- **Description**: An AI-powered plugin for Caldera that orchestrates long-running LLM workflows to automatically create adversary emulation abilities and plan operations. Optionally enriches workflows with Retrieval-Augmented Generation (RAG) using Cyber Threat Intelligence (CTI) from STIX JSON files. 
- **Primary Language**: Unknown
- **License**: Apache-2.0
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-11-16
- **Last Updated**: 2026-01-18

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Caldera MCP Plugin

An AI-powered plugin for Caldera that orchestrates long-running LLM workflows to automatically create adversary emulation abilities and plan operations. Optionally enriches workflows with Retrieval-Augmented Generation (RAG) using Cyber Threat Intelligence (CTI) from STIX JSON files. All executions are tracked via MLflow for full observability into LLM reasoning and tool usage.

## Features

- **LLM Ability Factory**: Generate custom Caldera abilities from natural language descriptions
- **LLM Operation Planner**: Create and execute complete adversary emulation operations
- **CTI Integration**: Enhance abilities with real-world threat intelligence from STIX bundles
- **MLflow Tracking**: Full observability of LLM reasoning, tool calls, and execution trajectory
- **Flexible Model Support**: Works with most LLM providers (OpenAI, Anthropic, etc.)
- **Run History**: Browse and search all historical executions with full details

## Quick Start

### 1. Start Caldera

From the Caldera root directory:

```bash
python3 server.py --insecure
```

The MCP plugin automatically starts MLflow on port 5000 during Caldera initialization.

### 2. Access the MCP Interface

Navigate to the Caldera web interface and select the **MCP** plugin from the sidebar.

### 3. Configure Your LLM

In the Global Model Configuration panel:
- Enter your **API key** (required)
- Select your **model** (default: gpt-4o)
- Adjust **temperature** and **max_tokens** as needed
- Set **max tool calls** for ReAct iterations (default: 5)

### 4. Choose Your Workflow

**LLM Ability Factory**: Create specific abilities
- Example: "Create a Windows ability that dumps credentials using PowerShell"

**LLM Operation Planner**: Plan and execute operations
- Example: "Execute a ransomware simulation on Windows agents"

### 5. Run Your Task

1. Enter your prompt in natural language
2. (Optional) Select STIX CTI files to enhance with threat intelligence
3. Click **Execute**
4. Watch real-time progress via MLflow stages and reasoning
5. View results and created abilities/operations

## Architecture

### Core Components

**Frontend (Vue.js)**
- `mcp.vue`: Main landing page with navigation
- `local_mcp_ability_factory.vue`: Ability creation interface
- `public_mcp_ability_factory.vue`: Public ability interface
- `mcp_history.vue`: Historical run browser
- `mcp_extension_guide.vue`: Developer extension guide

**Backend (Python)**
- `mcp_api.py`: aiohttp API routes
- `mcp_svc.py`: Service orchestration layer
- `mcp_factory_client.py`: Ability factory DSPy client
- `mcp_planner_client.py`: Operation planner DSPy client
- `mcp_server.py`: MCP tool server exposing Caldera API
- `factory.py`: Command generation DSPy module
- `rag.py`: STIX CTI retrieval service

**Integration**
- `hook.py`: Plugin initialization and MLflow startup

## Configuration

### Default Settings

Edit `conf/default.yml` to set default LLM configuration:

```yaml
llm:
  model: gpt-4o
  api_key: YOUR_API_KEY
  offline: true
  use_mock: false
factory:
  model: gpt-4o
  api_key: YOUR_API_KEY
  temperature: 0.4
```

**Note**: Frontend model configuration overrides these defaults per run.

### RAG Configuration

When using CTI enhancement:
- **Embedding Model**: Default `openai/text-embedding-3-small`
- **Top-K Retrieval**: Default 5 objects (configurable via UI)
- **STIX File Location**: Upload files via UI, stored in `data/` directory

## Using CTI Integration

### Upload STIX Files

1. Navigate to **MCP → Ability Factory** or **Planner**
2. In the RAG Configuration section, click **Upload STIX File**
3. Select your STIX JSON bundle(s)
4. Files are stored in `plugins/mcp/data/`

### Enable CTI for a Run

1. In the RAG Configuration panel, select which STIX files to use
2. (Optional) Adjust embedding model and top-K retrieval
3. Execute your task normally

The LLM will receive relevant CTI context based on your prompt, including:
- Attack patterns and techniques
- Malware and tool descriptions
- Threat actor TTPs
- Campaign information

### CTI Data Flow

```
User Prompt → RAG Search (Semantic) → Top-K CTI Objects Retrieved
                                    ↓
                        Detailed Context for Top 3 Objects
                                    ↓
                        Formatted CTI Context String
                                    ↓
                        LLM Receives Task + CTI Context
                                    ↓
                        Creates CTI-Informed Abilities/Operations
```

## MLflow Tracking

### Access MLflow UI

Open your browser to: **http://localhost:5000**

Navigate to **Experiments → Traces** to view:
- Run status and stages
- LLM chain of thought (`thought_0`, `thought_1`, etc.)
- Tool calls and arguments (`tool_name_N`, `tool_args_N`)
- RAG retrieval steps (when CTI is used)
- Final results and reasoning

### Understanding Run Tags

**Status Tags**:
- `status`: running, complete, failed
- `stage`: Current execution phase
- `reasoning`: LLM's final reasoning summary
- `process_result`: Summary of what was created

**RAG Tags** (when CTI is enabled):
- `rag_retrieval_step_N`: RAG retrieval process
- `rag_retrieved_object_N`: Names of CTI objects retrieved
- `cti_context_preview`: First 1000 chars of CTI sent to LLM
- `cti_context_length`: Total CTI context size

**LLM Trajectory**:
- `thought_N`: LLM reasoning at each step
- `observation_N`: Tool execution results
- `tool_name_N`: Tool that was called
- `tool_args_N`: Arguments passed to tool

## Extending the Framework

See the **Extend & Customize** guide in the UI for detailed instructions on:
- Creating custom DSPy clients
- Adding new MCP tools
- Building custom workflows
- Integrating with the service layer

Example use cases:
- Threat Hunter: Analyze adversary profiles and generate detection rules
- Operation Optimizer: Review completed operations and suggest improvements
- Campaign Builder: Create multi-stage campaigns from threat actor profiles

## Development

### Testing Individual Components

```bash
# Test factory client (requires running Caldera)
cd app
python mcp_factory_client.py

# Test planner client
python mcp_planner_client.py

# Test MCP server tools
python mcp_server.py
```

### Project Structure

```
plugins/mcp/
├── app/                    # Python backend
│   ├── mcp_api.py         # API routes
│   ├── mcp_svc.py         # Service layer
│   ├── mcp_factory_client.py
│   ├── mcp_planner_client.py
│   ├── mcp_server.py      # MCP tool server
│   ├── factory.py         # Command generation
│   └── rag.py             # CTI retrieval
├── gui/views/             # Vue frontend
│   ├── mcp.vue
│   ├── local_mcp_ability_factory.vue
│   ├── public_mcp_ability_factory.vue
│   ├── mcp_history.vue
│   └── mcp_extension_guide.vue
├── conf/default.yml       # Default configuration
├── data/                  # STIX JSON files
├── hook.py                # Plugin initialization
└── README.md
```

## Dependencies

- **dspy**: LLM orchestration framework with ReAct pattern
- **mcp**: Model Context Protocol SDK for tool servers
- **mlflow**: Experiment tracking and tracing
- **aiohttp**: Async web framework
- **psutil**: Process management for MLflow server
- **requests**: HTTP client for Caldera API

## Troubleshooting

### API Key Errors

**Error**: "API key is required but not provided"

**Solution**: Enter your API key in the Global Model Configuration panel before executing.

### MLflow Not Starting

**Error**: Cannot access http://localhost:5000

**Solution**: Check Caldera logs for MLflow startup messages. The plugin automatically kills processes on port 5000 and starts MLflow during initialization.

### RAG Retrieval Failures

**Error**: "RAG service not initialized" or embedding errors

**Solution**:
- Verify STIX files are valid JSON
- Ensure API key has access to embedding models
- Check MLflow logs for detailed error messages

### Tool Execution Failures

**Error**: Tools fail to initialize or execute

**Solution**:
- Verify Caldera API is accessible at `http://localhost:8888/api/v2/`
- Check MCP server subprocess logs for environment issues
- Ensure PYTHONPATH includes venv packages

### Viewing Detailed Logs

Check MLflow UI for:
1. Full trajectory of tool calls
2. Error messages in `error` param
3. Traceback in `traceback` param
4. Stage where failure occurred

## Support

For bugs and feature requests:
- Check MLflow traces for detailed execution information
- Review Caldera logs with `[MCP]` prefix
- Consult the in-app Extension Guide for development questions

## License

Part of the Caldera project. See main Caldera repository for license information.