# kirana-saathi

**Repository Path**: soon14/kirana-saathi

## Basic Information

- **Project Name**: kirana-saathi
- **Description**: https://hf-mirror.com/spaces/build-small-hackathon/kirana-saathi
- **Primary Language**: Python
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-08-05
- **Last Updated**: 2026-08-05

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

---
title: Kirana Saathi
emoji: 🏪
colorFrom: yellow
colorTo: red
sdk: docker
app_port: 7860
pinned: false
short_description: Hindi voice + photo inventory & udhaar ledger for kiranas
tags:
  - track:backyard
  - sponsor:modal
  - sponsor:openbmb
  - achievement:offbrand
  - achievement:llama
  - achievement:sharing
  - achievement:fieldnotes
---

# Kirana Saathi

A Hindi-first assistant for a small kirana (corner) shop. The owner just talks, or shows a photo of a bill, and the app keeps the two things that used to live on paper: the stock, and the *udhaar*, the running credit customers pay back later. It also watches the shelf, flags what is about to expire or is not selling, suggests a discount to clear slow stock, and reminds the owner to restock before a festival.

Built for the **Build Small Hackathon**, Backyard AI track. Everything runs on open-weight models, each well under 32B. Understanding and vision come from **MiniCPM-V-4.6** (~5-6B) by OpenBMB. No proprietary frontier API touches the loop. For this demo the model is reached through OpenBMB's hosted MiniCPM API, while speech runs on Modal and the bill OCR pre-pass runs on llama.cpp (see Hosting).

- Live Space: https://huggingface.co/spaces/build-small-hackathon/kirana-saathi
- Blog (Field Notes): https://build-small-hackathon-kirana-saathi.hf.space/blog
- Demo video: https://www.loom.com/share/7a675a1918bf4233bd44c8e049f84c97
- Social post: https://x.com/ps_1506/status/2066625864482349310
- Code: https://github.com/PrathamSingla15/kirana-saathi

## Architecture

![Kirana Saathi turns a spoken line or a bill photo into a finished ledger entry, in five steps](figures/kirana_final.png)

One shopkeeper turn runs through five steps:

1. **Hear and read.** faster-whisper turns Hindi speech into text. For a bill or *khata* photo, Surya OCR does a first pass over the page, then MiniCPM-V-4.6 reads it.
2. **Decide.** MiniCPM-V-4.6 (~5-6B, with vision) runs as a guided-JSON tool router. Instead of native tool-calling, the model is constrained to emit a small JSON object that names the tool and its arguments, so the choice is always well-formed.
3. **Act.** Tools split in two: writes (add stock, record a sale, note credit, record a payment) and reads (the day's dashboard, a customer's dues, why an item is not moving).
4. **Confirm.** Anything that would write is read back as a yes/no question. Nothing is saved until the owner says yes.
5. **Reply.** The answer shows up in the Bahi-Khata ledger screen. When the owner taps the speaker, Veena (with a SNAC decoder) reads it back in one steady Hindi voice.

The router keeps the two guarantees that matter: confirm-before-write, and a visible tool-call trace under every reply.

See [`design.md`](design.md) for the full design.

## What it does

- **Voice credit book.** *"Sharma ji ne 200 ka udhaar liya"* stages an entry; *"kiska kitna baaki hai?"* returns a ranked list of who owes what.
- **Receive stock by photo.** Hold up the supplier bill. The lines come into a table you can fix before anything is saved, with an estimated expiry when the bill prints none.
- **Expiry and FEFO.** Sells the oldest stock first and warns before items go off.
- **Festival nudges.** Restock reminders before demand jumps, not after.
- **"Why isn't X selling?"** Reasons over the sales trend, the stock, and the price, then gives a plain answer and a fix, such as a small clearance discount.
- **Polite reminders.** Drafts a Hindi collection message for overdue credit, and never sends it on its own.
- **Money at a glance.** Cost, price, and margin per item, the value of everything on the shelf, and the day's takings.

Two rails keep it safe: it never writes to the books without a yes, and it never sells stock the shop does not have.

## Stack (open-weight models, ≤32B)

| Layer | Choice |
|---|---|
| LLM + vision | **MiniCPM-V-4.6** (~5-6B) by **OpenBMB**, via OpenBMB's OpenAI-compatible API |
| Agent | guided-JSON tool routing (`guided_json` / `guided_choice`) over MiniCPM |
| Bill OCR pre-pass | **Surya**, via **llama.cpp** |
| Speech to text | **faster-whisper** large-v3 (Hindi) |
| Text to speech | **Veena** (Hindi / Hinglish) with a **SNAC** decoder |
| Database | two **SQLite** files, `inventory.db` + `transactions.db`, read together via `ATTACH` |
| Frontend | custom **Gradio** "Bahi-Khata" single-screen app |

## Hosting (open-weight, no proprietary frontier API)

Every model here is open-weight. MiniCPM-V-4.6 is small enough to self-host on a single GPU, and the speech models are smaller still, so nothing forces a hosted dependency. To be clear about what the live demo actually does:

- **LLM + vision** is served through **OpenBMB's hosted MiniCPM API** (OpenAI-compatible). The Space calls it over `KIRANA_LLM_BASE_URL` with `KIRANA_LLM_API_KEY` and `KIRANA_LLM_MODEL`.
- **Speech** runs on **Modal**: faster-whisper for STT and Veena + SNAC for TTS, on a warm L4 GPU, at `/stt` and `/tts`.
- **Bill OCR pre-pass** runs **Surya on llama.cpp**.

The Hugging Face Space runs the Gradio UI on CPU and points at those backends through the secrets `KIRANA_LLM_BASE_URL` / `KIRANA_LLM_API_KEY` / `KIRANA_LLM_MODEL` and `KIRANA_STT_BASE_URL` / `KIRANA_TTS_BASE_URL`, plus `HF_TOKEN` for the gated Veena weights. The Modal app stays warm with `min_containers=1`. Deploy it with `MODAL_PROFILE=projects-ps MIN_CONTAINERS=1 PYTHONPATH="$PWD" modal deploy scripts/modal_app.py`.

There is no proprietary cloud AI in the true sense: all of these are open-weight models, none are proprietary frontier APIs. The demo does not claim to run entirely offline or on the owner's own hardware. Because MiniCPM-V-4.6 also ships an official GGUF, the LLM + vision can be moved to local llama.cpp if you want a fully self-hosted setup.

## Tracks and badges

| Badge | Evidence |
|---|---|
| 🏡 **Backyard AI** | A real kirana owner's daily problem: voice *udhaar*, bill OCR, expiry/FEFO, festival nudges, run on his own books (demo video). |
| 🟢 **Modal** | Speech is hosted on Modal: faster-whisper STT and Veena + SNAC TTS on a warm L4 GPU (`scripts/modal_app.py`). |
| 🤖 **Best Agent** | Guided-JSON tool router over MiniCPM, 10 read/write/vision tools, confirm-before-write, a visible tool-call trace under every reply. |
| 🎨 **Off-Brand** | Custom "Bahi-Khata" HTML/CSS/JS ledger UI with instant EN/हिं. |
| 🟣 **OpenBMB** | MiniCPM-V-4.6 powers all understanding and vision: tool routing, bill reading, and the slow-mover reasoning. |
| 🦙 **Llama Champion** | Surya OCR runs on `llama.cpp`, and MiniCPM-V-4.6 ships an official GGUF that runs on `llama.cpp` too. |
| 📓 **Field Notes** | Build write-up, served at `/blog` and linked above. |
| 📡 **Sharing-is-Caring** | Exported agent trace (`scripts/export_trace.py`). |
| 🎬 **Best Demo** | Real-owner Hindi demo (video linked above). |

## Evaluation

- **Headless suite:** `uv run pytest -q` gives 56 pass / 2 skip (LLM, STT, and vision mocked). It covers cross-DB integrity, FEFO lots, balances, the oversell guard, the tools, onboarding, and festival intent.
- **Real-model end-to-end:** `scripts/e2e_full.py` drives about 24 real MiniCPM turns over the seeded shop across 38 checks: tool routing (read vs write), ground-truth lookups, confirm-before-write, a FEFO sale, a quantity-merge restock, the oversell block, a vision bill receive, Hindi STT, TTS, the morning briefing, and the festival calendar.

## Team

- Pratham Singla ([@yobro4619](https://huggingface.co/yobro4619))
- Adesh Gupta ([@aadex](https://huggingface.co/aadex))
- Shivank Garg ([@shivank21](https://huggingface.co/shivank21))

## Quickstart (local dev)

```bash
uv sync
# Point the LLM + vision at the OpenBMB MiniCPM API
export KIRANA_LLM_BASE_URL=...   # OpenBMB OpenAI-compatible endpoint
export KIRANA_LLM_API_KEY=...
export KIRANA_LLM_MODEL=...       # the MiniCPM-V-4.6 model name
# Whisper, Veena + SNAC. Veena is gated: run `hf auth login` first.
bash scripts/download_models.sh
# build and seed the two demo databases
uv run python -m kirana.db --reset
# UI at :7860, blog at :7860/blog
uv run python -m kirana.app
```

Prefer a fully local stack. MiniCPM-V-4.6 ships an official GGUF, so you can self-host the LLM + vision on **llama.cpp** (`llama-server`, OpenAI-compatible `/v1`) and point `KIRANA_LLM_BASE_URL` at it instead. For the rest of the hosted setup, see Hosting above.