# kreuzberg

**Repository Path**: mirrors_trending/kreuzberg

## Basic Information

- **Project Name**: kreuzberg
- **Description**: Document intelligence framework for Python - Extract text, metadata, and structured data   from PDFs, images, Office documents, and more. Built on Pandoc, PDFium, and Tesseract.
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-02-19
- **Last Updated**: 2026-09-26

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

<!-- This file is auto-generated by alef — DO NOT EDIT. -->
<!-- alef:hash:eb699b4d0faa2ba51fc64c7ac0826abd2fad5065e5f78a12e1cc9c49a18246d0 -->
<!-- To regenerate: alef readme -->
<!-- To verify freshness: alef verify -->

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://cdn.jsdelivr.net/gh/xberg-io/assets@v1/banner/readme-banner-dark.svg">
    <img alt="Xberg" width="420" src="https://cdn.jsdelivr.net/gh/xberg-io/assets@v1/banner/readme-banner-light.svg">
  </picture>
</p>

# Xberg

<div align="center" style="display: flex; flex-wrap: wrap; gap: 8px; justify-content: center; margin: 20px 0;">
  <a href="https://github.com/xberg-io/xberg/actions/workflows/ci-rust.yaml">
    <img src="https://github.com/xberg-io/xberg/actions/workflows/ci-rust.yaml/badge.svg?branch=main" alt="Rust CI">
  </a>
  <a href="https://codecov.io/gh/xberg-io/xberg">
    <img src="https://codecov.io/gh/xberg-io/xberg/branch/main/graph/badge.svg" alt="Coverage">
  </a>
  <a href="https://github.com/xberg-io/alef">
    <img src="https://img.shields.io/badge/built%20with-alef%20%D7%90-007ec6" alt="Built with alef">
  </a>
  <!-- Language Bindings -->
  <a href="https://crates.io/crates/xberg">
    <img src="https://img.shields.io/crates/v/xberg?label=Rust&color=007ec6" alt="Rust">
  </a>
  <a href="https://pypi.org/project/xberg/">
    <img src="https://img.shields.io/pypi/v/xberg?label=Python&color=007ec6" alt="Python">
  </a>
  <a href="https://www.npmjs.com/package/@xberg-io/xberg">
    <img src="https://img.shields.io/npm/v/@xberg-io/xberg?label=Node.js&color=007ec6" alt="Node.js">
  </a>
  <a href="https://www.npmjs.com/package/@xberg-io/xberg-wasm">
    <img src="https://img.shields.io/npm/v/@xberg-io/xberg-wasm?label=WASM&color=007ec6" alt="WASM">
  </a>
  <a href="https://central.sonatype.com/artifact/io.xberg/xberg">
    <img src="https://img.shields.io/maven-central/v/io.xberg/xberg?label=Java&color=007ec6" alt="Java">
  </a>
  <a href="https://github.com/xberg-io/xberg/tree/main/packages/go">
    <img src="https://img.shields.io/github/v/tag/xberg-io/xberg?label=Go&color=007ec6&filter=v1*" alt="Go">
  </a>
  <a href="https://www.nuget.org/packages/XbergIo.Xberg/">
    <img src="https://img.shields.io/nuget/v/XbergIo.Xberg?label=C%23&color=007ec6" alt="C#">
  </a>
  <a href="https://packagist.org/packages/xberg-io/xberg">
    <img src="https://img.shields.io/packagist/v/xberg-io/xberg?label=PHP&color=007ec6" alt="PHP">
  </a>
  <a href="https://rubygems.org/gems/xberg">
    <img src="https://img.shields.io/gem/v/xberg?label=Ruby&color=007ec6" alt="Ruby">
  </a>
  <a href="https://hex.pm/packages/xberg">
    <img src="https://img.shields.io/hexpm/v/xberg?label=Elixir&color=007ec6" alt="Elixir">
  </a>
  <a href="https://pub.dev/packages/xberg">
    <img src="https://img.shields.io/pub/v/xberg?label=Dart&color=007ec6" alt="Dart">
  </a>
  <a href="https://central.sonatype.com/artifact/io.xberg/xberg-android">
    <img src="https://img.shields.io/maven-central/v/io.xberg/xberg-android?label=Kotlin&color=007ec6" alt="Kotlin">
  </a>
  <a href="https://github.com/xberg-io/xberg/tree/main/packages/swift">
    <img src="https://img.shields.io/badge/Swift-SPM-007ec6" alt="Swift">
  </a>
  <a href="https://github.com/xberg-io/xberg/tree/main/packages/zig">
    <img src="https://img.shields.io/badge/Zig-package-007ec6" alt="Zig">
  </a>
  <a href="https://github.com/xberg-io/xberg/releases">
    <img src="https://img.shields.io/badge/C-FFI-007ec6" alt="C FFI">
  </a>
  <a href="https://github.com/xberg-io/xberg/pkgs/container/xberg">
    <img src="https://img.shields.io/badge/Docker-ghcr.io-007ec6?logo=docker&logoColor=white" alt="Docker">
  </a>
  <a href="https://docs.xberg.io/guides/kubernetes/">
    <img src="https://img.shields.io/badge/Helm-chart-007ec6?logo=helm&logoColor=white" alt="Helm chart">
  </a>
  <!-- Project Info -->
  <a href="https://github.com/xberg-io/xberg/blob/main/LICENSE">
    <img src="https://img.shields.io/badge/License-MIT-007ec6" alt="License">
  </a>
  <a href="https://docs.xberg.io">
    <img src="https://img.shields.io/badge/Docs-xberg-007ec6" alt="Documentation">
  </a>
  <a href="https://huggingface.co/xberg-io">
    <img src="https://img.shields.io/badge/Hugging%20Face-Xberg-007ec6" alt="Hugging Face">
  </a>
</div>

<div align="center" style="display: flex; flex-wrap: wrap; gap: 12px; justify-content: center; margin: 28px 0 24px;">
  <a href="https://discord.gg/xt9WY3GnKR">
    <img height="22" src="https://img.shields.io/badge/Discord-Chat-007ec6?logo=discord&logoColor=white" alt="Join Discord">
  </a>
  <a href="https://docs.xberg.io/demo.html">
    <img height="22" src="https://img.shields.io/badge/Live%20Demo-Open-007ec6?logo=webassembly&logoColor=white" alt="Live Demo">
  </a>
  <a href="https://github.com/xberg-io/xberg/stargazers">
    <img height="22" src="https://img.shields.io/github/stars/xberg-io/xberg?style=social" alt="GitHub Stars">
  </a>
</div>

<div align="center">

**The fast, precise document-intelligence engine — for every language.**

Point Xberg at anything — a PDF, a scanned image, a spreadsheet, an audio file, a URL, a whole archive, or a source tree — and get back clean text, tables, metadata, and structured data. One engine handles format detection, reading, OCR, and extraction, so you never stitch a pipeline together from a dozen libraries.

**107 formats · 141 file extensions** · **371 code languages** · **15 language bindings** · **6 output formats** · OCR · transcription · embeddings

The fastest, most precise open-source document and PDF-to-Markdown engine — [see the benchmarks](https://xberg.io/benchmarks).

[Install](#installation) · [What you get](#what-you-get) · [Capabilities](#capabilities) · [CLI](#cli-reference) · [Docs](https://docs.xberg.io)

</div>

> **Xberg is the next iteration of [Kreuzberg](https://github.com/kreuzberg-dev/kreuzberg-v4-lts).** Same document-intelligence engine, rebuilt and rebranded under a fresh v1 line.

---

## What you get

Point Xberg at anything — a PDF, a spreadsheet, a scanned image, an audio file, a URL, an archive, a source tree — and get back clean, structured content you can use right away. One core does the format detection, reading, and extraction, so you don't assemble a pipeline yourself. Call it from Rust, Python, Node.js, Go, Java, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WASM, Kotlin, or C FFI, and run it as a library, CLI tool, REST API, or MCP server.

| Capability | What you get |
|---|---|
| **107 document formats** | PDFs, Office, images, HTML, email, e-books, scientific publications, and structured data across 141 file extensions, with intelligent MIME detection and bounded extraction controls. |
| **URLs & the web** | Point Xberg at an `http(s)` URL — it fetches and extracts a single document, or crawls and follows links (Auto / Document / Crawl modes via the [crawlberg](https://github.com/xberg-io/crawlberg) engine). *Requires the `url-ingestion` feature.* |
| **Audio & video transcription** | Speech-to-text from MP3, M4A, WAV, WebM, and MP4 tracks via Whisper ONNX (tiny → large-v3). *Requires the `transcription` feature.* |
| **Archives, traversed** | List and **recursively** extract nested `.zip`, `.tar`, `.gz`, `.7z` — documents inside documents — guarded by zip-bomb, compression-ratio, and nesting-depth limits. |
| **OCR on demand** | Tesseract, PaddleOCR, Candle, or VLM backends — fallback chains, confidence scores, language auto-detection, extensible via plugins. |
| **Layout & tables** | ML layout models (PP-DocLayout-V3, RT-DETR) and table structure (TATR, SLANet) reconstruct reading order and cell grids for clean Markdown. |
| **Code intelligence** | Functions, classes, imports, symbols, docstrings from 371 programming languages. Syntax-aware chunking for RAG pipelines. |
| **Embeddings & search** | Local (ONNX) or provider-hosted embeddings (165 providers via liter-llm), sparse and late-interaction, cross-encoder reranking. |
| **Enrichment** | NER, keyword extraction (YAKE/RAKE), summarization, translation, redaction, page classification, QR detection, language detection, token reduction (TOON). |
| **Structured extraction** | Schema-driven JSON straight from any document via local (Ollama, LM Studio, vLLM) or hosted LLMs — no prompt engineering. |
| **6 output formats** | Plain text, Markdown, Djot, HTML, JSON tree, or Docling DocTags, plus registered custom renderers. |
| **Runs anywhere** | Library, CLI (14 commands), REST API (`xberg serve`), MCP server, Docker, Helm — CPU by default, no GPU required. Content-hash caching, parallel batch, per-file timeouts. |

> Capabilities marked *requires a feature* are Cargo feature flags on the core crate (`url-ingestion`, `transcription`, `reranker`, layout/ORT). Prebuilt language packages and the Docker image bundle the common set; a from-source build enables only what you select.

---

## Installation

### Language Packages

<details open>
<summary><strong>Python</strong></summary>

```sh
pip install xberg
```

See [Python README](https://github.com/xberg-io/xberg/tree/main/packages/python) for full documentation.

</details>

<details>
<summary><strong>Node.js / TypeScript</strong></summary>

```sh
npm install @xberg-io/xberg
```

See [Node.js README](https://github.com/xberg-io/xberg/tree/main/crates/xberg-node) for full documentation.

</details>

<details>
<summary><strong>Rust</strong></summary>

```sh
cargo add xberg
```

See [Rust README](https://github.com/xberg-io/xberg/tree/main/crates/xberg) for full documentation.

</details>

<details>
<summary><strong>Go</strong></summary>

```sh
go get github.com/xberg-io/xberg/packages/go@latest
```

> ⚠️ The repository root is not a Go module — `go get github.com/xberg-io/xberg` will fail.
> Always target the `/packages/go` subdirectory as shown above.

See [Go README](https://github.com/xberg-io/xberg/tree/main/packages/go) for full documentation.

</details>

<details>
<summary><strong>Java</strong></summary>

Available on Maven Central as `io.xberg:xberg`. See [Java README](https://github.com/xberg-io/xberg/tree/main/packages/java) for the dependency snippet.

</details>

<details>
<summary><strong>C#</strong></summary>

```sh
dotnet add package XbergIo.Xberg
```

See [C# README](https://github.com/xberg-io/xberg/tree/main/packages/csharp) for full documentation.

</details>

<details>
<summary><strong>Ruby</strong></summary>

```sh
gem install xberg
```

See [Ruby README](https://github.com/xberg-io/xberg/tree/main/packages/ruby) for full documentation.

</details>

<details>
<summary><strong>PHP</strong></summary>

```sh
composer require xberg-io/xberg
```

See [PHP README](https://github.com/xberg-io/xberg/tree/main/packages/php) for full documentation.

</details>

<details>
<summary><strong>Elixir</strong></summary>

Add `{:xberg, "~> 1.0"}` to your `mix.exs` dependencies. See [Elixir README](https://github.com/xberg-io/xberg/tree/main/packages/elixir) for full documentation.

</details>

<details>
<summary><strong>WebAssembly</strong></summary>

```sh
npm install @xberg-io/xberg-wasm
```

See [WebAssembly README](https://github.com/xberg-io/xberg/tree/main/crates/xberg-wasm) for full documentation.

</details>

<details>
<summary><strong>Kotlin (Android)</strong></summary>

Available on Maven Central as `io.xberg:xberg-android`. See [Kotlin README](https://github.com/xberg-io/xberg/tree/main/packages/kotlin-android) for the dependency snippet.

</details>

<details>
<summary><strong>Swift</strong></summary>

Add via Swift Package Manager. See [Swift README](https://github.com/xberg-io/xberg/tree/main/packages/swift) for full documentation.

</details>

<details>
<summary><strong>Dart / Flutter</strong></summary>

```sh
dart pub add xberg
```

See [Dart README](https://github.com/xberg-io/xberg/tree/main/packages/dart) for full documentation.

</details>

<details>
<summary><strong>Zig</strong></summary>

Add via `zig fetch`. See [Zig README](https://github.com/xberg-io/xberg/tree/main/packages/zig) for full documentation.

</details>

<details>
<summary><strong>C/C++ (FFI)</strong></summary>

Build from source as part of this workspace. See [C (FFI) README](https://github.com/xberg-io/xberg/tree/main/crates/xberg-ffi) for full documentation.

</details>

### CLI & Deployment

<details>
<summary><strong>CLI Tool</strong></summary>

```sh
brew install xberg-io/tap/xberg
```

Windows users can install the same binary through [Scoop](https://scoop.sh):

```powershell
scoop bucket add xberg https://github.com/xberg-io/scoop-bucket
scoop install xberg
```

14 commands: `extract`, `batch`, `detect`, `formats`, `version`, `cache`, `tree-sitter`, `doctor`, `serve`, `mcp`, `api`, `embed`, `chunk`, and `completions`.

See [CLI usage guide](https://docs.xberg.io/cli/usage/) for detailed documentation.

</details>

<details>
<summary><strong>Docker</strong></summary>

```sh
docker pull ghcr.io/xberg-io/xberg:latest
```

Run in API, CLI, or MCP modes. See [Docker guide](https://docs.xberg.io/guides/docker/) for examples.

</details>

<details>
<summary><strong>REST API Server</strong></summary>

```sh
xberg serve --host 0.0.0.0 --port 8000
```

One POST endpoint handles all formats. Returns JSON or Markdown. Stream large files. See [API server guide](https://docs.xberg.io/guides/api-server/).

</details>

<details>
<summary><strong>MCP Server</strong></summary>

```sh
xberg mcp --transport stdio
```

9 tools (extract, extract_batch, detect_mime_type, cache_stats, list_formats, cache_clear, get_version, cache_manifest, cache_warm). 3 prompts (extract_document, extract_with_ocr, semantic_search). 4 resources (formats, models, OCR languages, embedding presets).

Add to Claude Desktop or Cursor:

```json
{
  "mcpServers": {
    "xberg": { "command": "xberg", "args": ["mcp"] }
  }
}
```

See [MCP integration guide](https://docs.xberg.io/guides/mcp-integration/).

</details>

### AI Coding Assistants

Install the Xberg plugin from [`xberg-io/xberg`](https://github.com/xberg-io/xberg). Ships extraction APIs, OCR backends, configuration, and language conventions.

<details open>
<summary><strong>Claude Code</strong></summary>

```text
/plugin marketplace add xberg-io/xberg
/plugin install xberg@xberg
```

</details>

<details>
<summary><strong>Codex CLI</strong></summary>

```text
/plugins add https://github.com/xberg-io/xberg
```

Search for `xberg` and select **Install Plugin**.

</details>

<details>
<summary><strong>Cursor</strong></summary>

Settings → Plugins → Add from URL → `https://github.com/xberg-io/xberg`, then select **xberg**.

</details>

<details>
<summary><strong>Gemini CLI</strong></summary>

```text
gemini extensions install https://github.com/xberg-io/xberg
```

</details>

<details>
<summary><strong>Factory Droid</strong></summary>

```text
droid plugin marketplace add https://github.com/xberg-io/xberg
droid plugin install xberg@xberg
```

</details>

<details>
<summary><strong>GitHub Copilot CLI</strong></summary>

```text
copilot plugin marketplace add https://github.com/xberg-io/xberg
copilot plugin install xberg@xberg
```

</details>

<details>
<summary><strong>opencode</strong></summary>

Add to `opencode.json`:

```json
{
  "$schema": "https://opencode.ai/config.json",
  "plugin": ["@xberg-io/opencode-xberg"]
}
```

</details>

---

## Quick Start

Extract text from a document:

```rust
use xberg::{extract, ExtractInput, ExtractionConfig};

#[tokio::main]
async fn main() -> xberg::Result<()> {
    let config = ExtractionConfig::default();
    let output = extract(
        ExtractInput::from_uri("document.pdf"),
        &config
    ).await?;

    println!("{}", output.results[0].content);
    Ok(())
}
```

Common use cases — see [Quick start guide](https://docs.xberg.io/getting-started/quickstart/) for language-specific examples, OCR, batch processing, and API configuration.

---

## Capabilities

<details>
<summary><strong>Full feature list</strong></summary>

### Supported File Formats (107 formats · 141 file extensions · 56 MIME aliases)

107 formats across 140 unique file extensions, with 56 compatibility MIME aliases, intelligent format detection, and comprehensive metadata extraction.

#### Office Documents

| Category | Formats | Capabilities |
|----------|---------|--------------|
| **Word Processing** | `.docx`, `.docm`, `.doc`, `.dotx`, `.dotm`, `.dot`, `.odt`, `.pages`, `.wpd`, `.wp`, `.wp5`, `.wp6` | Full text, tables, images, metadata, styles |
| **Spreadsheets** | `.xlsx`, `.xlsm`, `.xlsb`, `.xls`, `.xla`, `.xlam`, `.xltm`, `.xltx`, `.xlt`, `.ods`, `.numbers` | Sheet data, formulas, cell metadata, charts |
| **Presentations** | `.pptx`, `.pptm`, `.ppt`, `.pps`, `.ppsx`, `.potx`, `.potm`, `.pot`, `.odp`, `.key` | Slides, speaker notes, images, metadata |
| **PDF** | `.pdf` | Text, tables, images, metadata, OCR support |
| **eBooks** | `.epub`, `.fb2` | Chapters, metadata, embedded resources |
| **Database** | `.dbf`, `.sqlite`, `.sqlite3`, `.db`, `.gpkg`, `.gpkx` | Bounded table extraction, schema metadata, GeoPackage detection |
| **Hangul** | `.hwp`, `.hwpx` | Korean document format, text extraction |

#### Images (OCR-Enabled)

| Category | Formats | Features |
|----------|---------|----------|
| **Raster** | `.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`, `.bmp`, `.tiff`, `.tif` | OCR, table detection, EXIF metadata, dimensions, color space |
| **Advanced** | `.jp2`, `.jpg2`, `.j2c`, `.j2k`, `.jpc`, `.jbig2`, `.jb2`, `.pnm`, `.pbm`, `.pgm`, `.ppm` | OCR via pure-Rust JPEG2000 decoder, JBIG2 support, table detection |
| **HEIC family** | `.heic`, `.heics`, `.heif`, `.heifs`, `.hif`, `.avif`, `.avcs` | EXIF metadata, optional pixel decoding |
| **Vector** | `.svg` | DOM parsing, embedded text, graphics metadata |

#### Audio & Video

| Category | Formats | Features |
|----------|---------|----------|
| **Audio** | `.mp3`, `.mpga`, `.m4a`, `.wav`, `.webm` | Whisper transcription |
| **MP4 audio track** | `.mp4`, `.mpg4`, `.mp4v`, `.m4v` | Audio-track transcription only |
| **MPEG audio track** | `.mpeg`, `.mpg`, `.mpe`, `.m1v`, `.m2v` | Audio-track transcription only |
| **WebM audio track** | `.webm` | Audio-track transcription only |

#### Web & Data

| Category | Formats | Features |
|----------|---------|----------|
| **Markup** | `.html`, `.htm`, `.xhtml`, `.xht`, `.xml`, `.kml`, `.svg` | DOM parsing, metadata (Open Graph, Twitter Card), link extraction |
| **Structured Data** | `.json`, `.geojson`, `.jsonl`, `.ndjson`, `.yaml`, `.yml`, `.toml`, `.csv`, `.tsv` | Schema detection, nested structures, validation |
| **Text & Markdown** | `.txt`, `.adoc`, `.asciidoc`, `.vtt`, `.md`, `.markdown`, `.commonmark`, `.qmd`, `.rmd`, `.djot`, `.dj`, `.mdx`, `.doctags`, `.rst`, `.org`, `.rtf` | AsciiDoc, CommonMark, MyST Markdown, Quarto, R Markdown, Djot, MDX, DocTags, reStructuredText, Org Mode |

#### Email & Archives

| Category | Formats | Features |
|----------|---------|----------|
| **Email** | `.eml`, `.msg`, `.pst` | Headers, body (HTML/plain), attachments, threading |
| **Archives** | `.zip`, `.tar`, `.tgz`, `.gz`, `.7z` | File listing, nested archives, metadata, recursive extraction |

#### Academic & Scientific

| Category | Formats | Features |
|----------|---------|----------|
| **Citations** | `.bib`, `.ris`, `.nbib`, `.enw` | Structured parsing: RIS, PubMed/MEDLINE, EndNote XML, BibTeX/BibLaTeX |
| **Scientific** | `.tex`, `.latex`, `.typ`, `.typst`, `.jats`, `.nxml` | LaTeX, Typst, PubMed JATS |
| **Text notebooks** | `.ipynb`, `.md`, `.py`, `.R`, `.jl` | Jupyter, MyST-NB, Jupytext percent/light, saved outputs, cell visibility tags |
| **Publishing** | `.fb2`, `.docbook`, `.dbk`, `.docbook4`, `.docbook5`, `.opml` | FictionBook, DocBook XML, OPML outlines |

### Code Intelligence (371 Languages)

Extract structure from 371 programming languages via tree-sitter:

| Feature | Description |
|---------|-------------|
| **Structure Extraction** | Functions, classes, methods, structs, interfaces, enums |
| **Import/Export Analysis** | Module dependencies, re-exports, wildcard imports |
| **Symbol Extraction** | Variables, constants, type aliases, properties |
| **Docstring Parsing** | Google, NumPy, Sphinx, JSDoc, RustDoc, and 10+ formats |
| **Syntax-Aware Chunking** | Split code by semantic boundaries for RAG pipelines |
| **Diagnostics** | Parse errors with line/column positions |

Powered by [tree-sitter-language-pack](https://github.com/xberg-io/tree-sitter-language-pack).

### Output Formats (6)

| Format | Use case | Example |
|--------|----------|---------|
| **Plain** | Raw text, no markup | `"Chapter 1\nIntroduction"` |
| **Markdown** | Readable, structured, RAG-friendly | `"# Chapter 1\n## Introduction"` |
| **Djot** | Modern lightweight markup | Similar to Markdown but stricter |
| **HTML** | Styled, browser-ready | `<h1>Chapter 1</h1>` |
| **JSON** | Machine-readable tree structure | Hierarchical sections with heading levels |
| **DocTags** | Docling-compatible tag stream for document elements and tables | `<text>Chapter 1</text>` |

### Deployment Modes

| Mode | Command | Transport | Use case |
|------|---------|-----------|----------|
| **Library** | `xberg::extract()` | Async functions | Embed in your application |
| **CLI** | `xberg extract document.pdf` | 14 commands | Scripts, batch jobs, CI/CD |
| **REST API** | `xberg serve` | HTTP POST | Microservice, serverless deployment |
| **MCP Server** | `xberg mcp` | stdio or HTTP | Claude, Cursor, IDE agents |
| **Docker** | `docker run ghcr.io/xberg-io/xberg` | All modes | Container deployment |

### OCR Backends

- **Tesseract** — Native C FFI (Linux/macOS/Windows) and WASM (browser)
- **PaddleOCR** — ONNX Runtime, mobile-optimized models
- **Candle** — Pure Rust, CPU-only, lightweight
- **VLM** — GPT-4 Vision, Claude Vision, Gemini Vision, or 165 providers via liter-llm

Fallback chains. Extensible via plugin system.

### Embeddings

**Local (ONNX Runtime):**

- Preset models: fast, balanced (default), quality, multilingual
- Dimensions: 384, 768, 1024

**Provider-hosted:**

- OpenAI, Anthropic, Google, Hugging Face, Mistral, Cohere, and 165 providers total
- Via [liter-llm](https://github.com/xberg-io/liter-llm) integration

**Reranking:**

- Local ONNX rerankers (cross-encoder models)
- Provider-hosted: Cohere Rerank, others

### Structured LLM Extraction

Local engines: Ollama, LM Studio, vLLM

Remote: OpenAI, Anthropic, Google, Mistral, Cohere, and 165 providers via liter-llm

Schema validation. Temperature, top-p, frequency penalty tuning.

### Enrichment

- **NER** — GLiNER or LLM-based entity recognition
- **Redaction** — Mask PII (phone, email, SSN, credit card, addresses)
- **Summarization** — Document and section summaries via LLM
- **Translation** — Multi-language via LLM
- **Page Classification** — Tag document pages (cover, toc, content, etc.)
- **QR Code Detection** — Extract and decode QR codes from images
- **Keyword Extraction** — YAKE or RAKE algorithms
- **Language Detection** — Detect document language
- **Layout Detection** — RT-DETR + TATR models for document structure
- **Table Extraction** — Cell-level structure and content
- **Token Reduction** — TOON wire format (~30–50% fewer tokens than JSON)

</details>

---

## CLI Reference

<details>
<summary><strong>All 14 commands</strong></summary>

| Command | Subcommands | Purpose |
|---------|-------------|---------|
| `extract` | — | Extract text from a single document (path, URL, or stdin) |
| `batch` | — | Extract from multiple documents in parallel |
| `detect` | — | Identify MIME type of a file |
| `formats` | — | List all supported formats and MIME types |
| `version` | — | Show Xberg version |
| `cache` | `stats`, `clear`, `manifest`, `warm` | Manage extraction cache and models |
| `tree-sitter` | `download`, `list`, `cache-dir`, `clean` | Manage code-intelligence grammars |
| `doctor` | — | Diagnose the local installation and runtime dependencies |
| `serve` | — | Start REST API server (default: <http://127.0.0.1:8000>) |
| `mcp` | — | Start MCP server (stdio or HTTP transport) |
| `api` | `schema` | Output OpenAPI 3.1 specification |
| `embed` | — | Generate embeddings for text (local or provider-hosted) |
| `chunk` | — | Split text into chunks (text, markdown, YAML, or semantic) |
| `completions` | — | Generate shell completion scripts |

Run `xberg --help` or `xberg <command> --help` for detailed options.

</details>

---

## Documentation

Full guides, API references for every binding, format reference, and configuration docs live at **[xberg.io](https://docs.xberg.io/)**.

- [Getting Started](https://docs.xberg.io/getting-started/installation/)
- [Quick Start](https://docs.xberg.io/getting-started/quickstart/)
- [Guides](https://docs.xberg.io/guides/extraction/)
- [API Reference (Rust core)](https://docs.xberg.io/reference/api-rust/) — every binding has its own page under `/reference/`
- [Format Reference](https://docs.xberg.io/reference/formats/)
- [Live Demo](https://docs.xberg.io/demo.html) (browser, WASM)

---

## Built with Xberg

Projects that declare Xberg as a dependency. Xberg was previously published as `kreuzberg`, and most of these projects declare the package under that name.

| Project | What it is | Stars |
|---|---|---|
| **[basemind](https://github.com/Goldziher/basemind)** | AI context and content layer for coding agents over one MCP server: code map, document RAG, shared memory and web crawl | ![Stars](https://img.shields.io/github/stars/Goldziher/basemind?color=007ec6) |
| **[delulu](https://github.com/mratsim/delulu)** | A suite of MCP servers and CLI tools that give your LLM better search and fewer hallucinations | ![Stars](https://img.shields.io/github/stars/mratsim/delulu?color=007ec6) |
| **[docs-mcp-server](https://github.com/arabold/docs-mcp-server)** | Grounded documentation MCP server, an open-source alternative to Context7, Nia and Ref.Tools | ![Stars](https://img.shields.io/github/stars/arabold/docs-mcp-server?color=007ec6) |
| **[erato](https://github.com/EratoLab/erato)** | The open-source AI platform | ![Stars](https://img.shields.io/github/stars/EratoLab/erato?color=007ec6) |
| **[fastmail-cli](https://github.com/radiosilence/fastmail-cli)** | CLI and MCP server for Fastmail: email, contacts, masked email, attachments and text extraction | ![Stars](https://img.shields.io/github/stars/radiosilence/fastmail-cli?color=007ec6) |
| **[ghfdb-portal](https://github.com/ihfc-iugg/ghfdb-portal)** | Web portal for the Global Heat Flow Database | ![Stars](https://img.shields.io/github/stars/ihfc-iugg/ghfdb-portal?color=007ec6) |
| **[hawki-toolkit-file-converter](https://github.com/hawk-digital-environments/hawki-toolkit-file-converter)** | Prepares and converts PDF files for the HAWKI toolkit | ![Stars](https://img.shields.io/github/stars/hawk-digital-environments/hawki-toolkit-file-converter?color=007ec6) |
| **[haystack-core-integrations](https://github.com/deepset-ai/haystack-core-integrations)** | Integrations that extend Haystack with extra components and document stores | ![Stars](https://img.shields.io/github/stars/deepset-ai/haystack-core-integrations?color=007ec6) |
| **[kreuzakt](https://github.com/anaisbetts/kreuzakt)** | A search engine for humans and computers, aimed at your most boring documents | ![Stars](https://img.shields.io/github/stars/anaisbetts/kreuzakt?color=007ec6) |
| **[lilbee](https://github.com/tobocop2/lilbee)** | The whole local AI stack in one executable, with conversational search and cited answers over your files, code and the web | ![Stars](https://img.shields.io/github/stars/tobocop2/lilbee?color=007ec6) |
| **[llm-workflow-engine](https://github.com/llm-workflow-engine/llm-workflow-engine)** | Power CLI and workflow manager for LLMs | ![Stars](https://img.shields.io/github/stars/llm-workflow-engine/llm-workflow-engine?color=007ec6) |
| **[MANSPIDER](https://github.com/blacklanternsecurity/MANSPIDER)** | Spiders entire networks for files sitting on SMB shares, searching filenames or contents with regex | ![Stars](https://img.shields.io/github/stars/blacklanternsecurity/MANSPIDER?color=007ec6) |
| **[otoroshi-llm-extension](https://github.com/cloud-apim/otoroshi-llm-extension)** | Connect, secure and manage LLM models behind one OpenAI-compatible API | ![Stars](https://img.shields.io/github/stars/cloud-apim/otoroshi-llm-extension?color=007ec6) |
| **[sift-kg](https://github.com/juanceresa/sift-kg)** | Turns a collection of documents into a knowledge graph, extracting entities and relationships with an LLM | ![Stars](https://img.shields.io/github/stars/juanceresa/sift-kg?color=007ec6) |
| **[sirchmunk](https://github.com/modelscope/sirchmunk)** | Turns raw data into a self-evolving, real-time search and intelligence layer | ![Stars](https://img.shields.io/github/stars/modelscope/sirchmunk?color=007ec6) |
| **[support-chatbot](https://github.com/rodekruis/support-chatbot)** | Level-1 support chatbot for the Netherlands Red Cross 510 team | ![Stars](https://img.shields.io/github/stars/rodekruis/support-chatbot?color=007ec6) |

Using Xberg in your project? Open a PR adding it to this list.

---

## Contributing

Contributions are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

Join our [Discord community](https://discord.gg/xt9WY3GnKR) for questions and discussion.

---

## Part of Xberg.io

- [Xberg](https://github.com/xberg-io/xberg) — the open-source content-intelligence engine: text, tables, and metadata from 107 formats (141 file extensions), with OCR, transcription, and code intelligence. MIT.
- [Xberg Pro](https://xberg.io) — a complete self-hosted content-intelligence backend in a single container. Commercial.
- [Xberg Enterprise](https://xberg.io) — the distributed, governed content-intelligence platform, scaled on Kubernetes with team governance and support. Commercial.
- [crawlberg](https://github.com/xberg-io/crawlberg) — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
- [html-to-markdown](https://github.com/xberg-io/html-to-markdown) — fast, lossless HTML→Markdown engine.
- [liter-llm](https://github.com/xberg-io/liter-llm) — universal LLM API client with native bindings for 14 languages and 165 providers.
- [tree-sitter-language-pack](https://github.com/xberg-io/tree-sitter-language-pack) — tree-sitter grammars and code-intelligence primitives.
- [alef](https://github.com/xberg-io/alef) — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.

---

## License

MIT License (MIT) — see [LICENSE](LICENSE) for details.