# ReflectBench

**Repository Path**: sdu-aes-lab/reflectbench

## Basic Information

- **Project Name**: ReflectBench
- **Description**: An Agentic Framework for Generating System-Level Design Testbench via Consensus and Reflection
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-04-01
- **Last Updated**: 2026-04-02

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# ReflectBench: An Agentic Framework for Generating System-Level Design Testbench via Consensus and Reflection

## Overview


## Architecture
```
project/
├── config/                 # Configuration files
│   ├── config.yaml        # Main configuration
│   ├── llm_config.py      # LLM settings
│   └── tool_config.py     # Tool settings
├── src/
│   ├── agents/            # AI agents with memory
│   ├── llm/              # LLM client and prompts
│   ├── tools/            # VCS, Iverilog wrappers
│   ├── rag/              # RAG system (optional)
│   ├── models/           # Data models
│   ├── utils/            # Utilities
│   ├── workflow/         # Pipeline orchestration
│   └── main.py           # Entry point
├── bench/                # Input modules
├── rag/                  # RAG knowledge base
├── logs/                 # Output logs
├── temp/                 # Temporary workspace
└── scripts/              # Helper scripts
```


## Installation

### Prerequisites

- Python 3.12 or higher
- VCS (Synopsys) or Iverilog
- URG (for coverage reports)
- OpenAI-compatible API access


### Manual Installation

```bash
# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt
```

## Configuration

### API Configuration

Edit `config/config.yaml`:

```yaml
llm:
  api:
    api_key: ""  
```

### Tool Configuration

Configure simulation tools in `config/config.yaml`:

```yaml
tools:
  vcs:
    enabled: true
    executable: "vcs"
  iverilog:
    enabled: true
    executable: "iverilog"
```

## Usage

### Basic Usage

```bash
# Run with default configuration
python -m src.main

./scripts/run_pipeline.py 
```

### Advanced Usage

```bash
# Dry-run to preview
./scripts/run_pipeline.py --dry-run

# Run specific modules
./scripts/run_pipeline.py --modules adder counter

# Enable RAG for error correction
./scripts/run_pipeline.py --enable-rag

# Verbose output with report
./scripts/run_pipeline.py --verbose --output results.json

# Custom configuration
./scripts/run_pipeline.py --config my_config.yaml
```

### Command-Line Options

```
--config PATH          Configuration file path
--modules NAME [...]   Specific modules to process
--enable-rag          Enable RAG-based error correction
--dry-run             Preview without executing
--verbose, -v         Debug logging
--quiet, -q           Minimal logging
--output, -o FILE     Save report to JSON
--batch               Non-interactive mode
```

## Input Directory Structure

```
bench/
└── modulename1/
    ├── modulename1.v      # Golden module
    ├── testbench.sv       # Golden testbench
    ├── specification.md   # Module specification
    ├── prompt_IR.txt      # Module prompt
    └── mutant/            # Mutant modules
        ├── mutant1.v
        └── mutant2.v
```

## Output Structure

```
logs/
└── modulename1/
    ├── coverage_file/     # Coverage HTML reports
    ├── module_code/       # Generated modules (code1.v - code5.v)
    ├── simulation_info/   # Analysis and summaries
    ├── stimulus_gen/      # Generated stimuli
    └── testbench/         # Generated testbenches
```

## RAG (Optional)

Enable Retrieval-Augmented Generation for improved error correction:

### Setup RAG Knowledge Base

Create `rag/rag.json`:

```json
[
  {
    "error_description": "Syntax error in loop declaration",
    "guidance": "Use integer declaration before loop..."
  }
]
```

### Enable RAG

```bash
# Install dependencies
pip install faiss-cpu sentence-transformers

# Run with RAG
./scripts/run_pipeline.py --enable-rag
```

## Development

### Adding New Agents

1. Inherit from `BaseAgent`
2. Implement `execute()` method
3. Use memory management APIs
4. Register in workflow

### Extending Prompts

Edit `src/llm/prompt_templates.py` to customize prompts for different tasks.


## License

MIT license