# P-ID_tag_extraction **Repository Path**: dlml2/P-ID_tag_extraction ## Basic Information - **Project Name**: P-ID_tag_extraction - **Description**: No description available - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2026-09-09 - **Last Updated**: 2026-09-09 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # PipeSight AI Automated instrument tag and symbol extraction for P&ID (Piping & Instrumentation Diagram) engineering drawings — turning unstructured PDF drawings into a structured, auditable, human-verified database. Developed as part of a Schneider Electric internship project. ## What this is Engineering drawings encode critical plant information — instrument tags, equipment identifiers, symbol relationships — that today is transcribed by hand. PipeSight AI reads P&ID PDFs directly, detects symbols and reads instrument tags off the page, and produces a structured database with full audit traceability, so every extracted fact can be traced back to the exact page, coordinates, and confidence it came from. The pipeline is verification-first by design: nothing is silently accepted. Every extracted tag is either auto-confirmed against a calibrated confidence threshold or routed to a human reviewer, and every review action — accept, correct, dismiss, promote — is logged. This is a prototype architecture, under active validation — some components (see Known Limitations) are still being tested rather than finalized. ## Architecture The pipeline runs in five stages: 1. **Input processing** — PDF pages are rendered to high-resolution images (300 DPI, via PyMuPDF). 2. **Symbol detection** — a YOLO-family object detector locates instrument and equipment symbols. Because P&ID pages run far larger than a standard model's input size, detection runs as a dual-scale tiled strategy (small tiles for individual symbols, large tiles for equipment context), merged and de-duplicated across both passes. Major equipment (pumps, compressors, vessels, tanks, heat exchangers) is routed into OCR alongside instrument symbols, since it frequently carries its own equipment tag — not excluded as purely structural. 3. **OCR and tag extraction** — instrument-class (and tagged-equipment) detections are read via OCR in a layered, cost-proportional strategy (cheap zone-level reads first, escalating to whole-page OCR and a rescue pass only where needed), matched to their nearest symbol using multiple factors together (OCR confidence, spatial position, and ISA pattern validity — not confidence alone), then normalized into ISA tag format: - A deterministic regex/ISA-pattern engine is the mandatory, always-active baseline for every tag. - An optional local LLM cross-check (Qwen2.5-7B-Instruct, run on-premises via llama-cpp-python, no cloud dependency) can additionally re-read the same OCR text and independently propose a tag, selectable via a Configurator (regex / llm / hybrid backend). This layer is experimental — see Known Limitations. - Every tag receives a calibrated confidence score. 4. **Database and human review** — extracted data is written to a relational database with full audit history. A live review interface lets an engineer accept, correct, dismiss, or promote any tag or candidate; every action re-generates the affected page's structured export immediately. 5. **Structured output** — an Excel workbook and an annotated overlay PDF, both generated from the same canonical per-page JSON export that the dashboard and review interface also read from — a single source of truth, not three independent exports. ## Repository structure ``` tag_extractor_v2.py Core OCR + tag-normalization pipeline (Phase 3) inference_tiled_v2_batch.py Symbol detection — dual-scale tiled YOLO inference (Phase 2) matching_engine.py Spatial tag-symbol matching, instrument-type inference, noise filtering configurator.py Runtime configuration: processing mode (supervisory/non_supervisory) and normalization backend (regex/llm/hybrid) llm_normalizer.py Optional local LLM (Qwen2.5-7B-Instruct) tag-normalization cross-check — experimental, never the sole source of truth for a tag compare_backends.py Diffs regex vs. hybrid backend output across pages, for evaluating the experimental LLM layer's accuracy/latency tradeoff rescue_to_db.py Rescue pass — recovers symbols missed at production confidence rescue_low_conf.py Low-confidence review-queue rescue logic db_builder.py Database schema and connection/insert helpers (SQLite) page_json.py Canonical per-page JSON export (single source of truth) page_summary.py Human-readable companion export, grouped by ISA loop review_server.py Live review API/backend (Flask) templates/review.html Review interface frontend human_review_export.py Parallel spreadsheet-based review workflow (writes to the same DB) dashboard.py Operational dashboard — status, metrics, and embedded review (Streamlit) knowledge_graph.py Builds the knowledge graph (loops, signal lines, equipment hierarchy, functional roles) from reviewed tags (Phase 5) export_db_to_kg_csv.py Exports the reviewed database state to CSV, ahead of a KG rebuild rebuild_kg_preserving_manual.py Rebuilds kg.json via knowledge_graph.py without losing manually-added signal-line edges from the previous kg.json verdict_store.py Stores reviewer verdicts (real/false/needs-recheck) on signal-line edges, enforced on every graph rebuild evaluate_signal_lines.py Scores detected signal-line edges against a ground-truth file make_overlay.py Renders the signal-line overlay image for a page build_demo_graph.py Builds a demo/presentation copy of the knowledge graph run_pipeline.py One-command orchestrator — runs the full pipeline end to end (see "Running the pipeline" below) pdf_excel_export.py Excel + annotated-PDF export pdf_to_image.py Single-PDF to image conversion (Phase 1) batch_pdf_to_image.py Batch wrapper — converts every PDF in a folder merge_supplemental_dataset.py Merges additional labeled source data into the unified 61-class training dataset schema train_yolo26_gpu.py YOLO26 transfer-learning training script for a local GPU machine kaggle_train_yolo26.py Cell-based YOLO26 training script for Kaggle's free GPU sessions finetune.py Short fine-tuning pass on top of a retrained YOLO26 checkpoint, using corrected labels from human review exclude_regions.json Per-page pixel regions excluded from detection/tiling (title blocks, revision history, notes) config.toml Streamlit configuration ``` ## Requirements - Python 3.10+ (see `requirements.txt` for the exact pinned version this project was developed against) - A YOLO-family model checkpoint trained on the project's instrument/equipment symbol classes (not included in this repository — see below) - The full pinned Python package list is in `requirements.txt`. Packages that matter most for compatibility: - torch - torchvision - ultralytics - paddleocr - paddlepaddle - llama-cpp-python (only required to use the optional llm/hybrid normalization backend) - opencv-python - PyMuPDF - PyYAML - GPU is optional. The pipeline runs CPU-only by default; no GPU configuration is required. If running on a machine with an NVIDIA GPU (e.g. for YOLO26 training, or to speed up the LLM normalization layer), CUDA-enabled builds of torch, llama-cpp-python, and paddlepaddle-gpu can be installed instead — see the setup guide for details. This GPU path has not yet been benchmarked by the development team. ## What you need to provide This repository ships code only — no data, no trained weights, no PDFs. To run the pipeline, you'll need to supply: - Source P&ID PDFs, placed under a local `data/` folder (git-ignored; organize as one subfolder per drawing set — `batch_pdf_to_image.py` expects this layout). - A trained YOLO model checkpoint for symbol detection, placed under `models/`. This repo doesn't include one. - A class-name file (`dataset.yaml`) listing the model's symbol classes in order, matching your checkpoint. Point the detection script at your copy via the `PIPESIGHT_DATASET_YAML` environment variable, or edit its path directly. - PaddleOCR's model weights, which PaddleOCR downloads automatically on first run. - A Qwen2.5-7B-Instruct GGUF file, placed under `models/` — only required if using the `llm` or `hybrid` normalization backend. Not needed for the default regex path. ## Running the pipeline ### One command (recommended) `run_pipeline.py` runs the entire pipeline — detection through the dashboard — as a single supervised command. Unlike chaining the individual scripts with `&&` on Windows, it checks each step's real exit code and stops immediately with a clear message naming the failed step, instead of continuing on top of half-updated data (the manual chain's `copy` step, for example, can print "0 file(s) copied" and let `&&` continue as if nothing went wrong). Pick exactly one scope for the run: ``` python run_pipeline.py --page "SET 2_page_1" # a single page python run_pipeline.py --set "SET 4" # every page in one drawing set ``` **Step order:** 1. **Symbol detection** (`inference_tiled_v2_batch.py`) — skipped by default; see "Cached vs. fresh detection" below. 2. **Tag extraction** (`tag_extractor_v2.py`) — OCR + ISA normalization for the requested scope. 3. **Tag review** (`review_server.py`) — opens at `http://127.0.0.1:5000`. Review/confirm/correct the extracted tags, then click "Done reviewing" on the page — the server no longer exits on its own. 4. **Export the reviewed database to CSV** (`export_db_to_kg_csv.py`) — always exports *every* page regardless of this run's scope, so other pages' tags aren't lost from the rebuilt graph. 5. **Rebuild the knowledge graph** (`rebuild_kg_preserving_manual.py`) — wraps `knowledge_graph.py` so manually-added signal-line edges from the previous `kg.json` survive the rebuild. 6. **Edge review** (`review_server.py`, a second time) — reviews the signal-line connections the rebuild's fresh computer-vision pass just detected. This is genuinely different from step 3 (edges, not tags) — check the Edge Review tab / `/api/edge_queue`, then click "Done reviewing" again. 7. **Copy the reviewed CSV** into the knowledge-graph output folder. 8. **Build the demo graph** (`build_demo_graph.py`). 9. **Render the signal-line overlay** (`make_overlay.py`) — once per page in scope. 10. **Evaluate signal lines against ground truth** (`evaluate_signal_lines.py`) — once per page in scope; a page with no matching ground-truth file is skipped rather than scored incorrectly. 11. **Launch the dashboard** (`streamlit run dashboard.py`) — left running in the foreground; Ctrl+C to stop. ### Cached vs. fresh detection Step 1 is the slowest step, so by default it's skipped and the pipeline reuses whatever's already in `detections_all_sets.json`: ``` python run_pipeline.py --page "SET 2_page_1" ``` Force a fresh detection pass — after retraining the model, for example, or if you don't trust the cached result: ``` python run_pipeline.py --page "SET 2_page_1" --run-inference ``` With `--page`, this is scoped to just that page's set, not the whole corpus — the set name is derived automatically from the page name (`"SET 2_page_1"` → set `"SET 2"`); pass `--inference-set` if your page naming doesn't follow the `_page_` convention. With `--set`, the whole folder is redetected; with `--all-pages`, every folder is. A genuinely new page or set with zero cached detections is auto-bootstrapped: the pipeline notices there's nothing cached for it and runs inference once automatically, even without `--run-inference`. Pass `--skip-inference-bootstrap` to turn this off and get the old behavior instead (tag extraction hard-stopping with "no pages matched"). ### Other cache flags - `--recompute-zone-ocr` — force-refresh the tag extractor's zone-OCR cache - `--recompute-ocr` — force-refresh the full-page OCR cache used by the resize-OCR escalation pass (independent of the flag above) - `--normalization-backend {llm,regex,hybrid}` — which tag-normalization path to use (default: `llm`) - `--require-llm` — hard-stop before step 1 if the LLM backend (llama-cpp-python + the GGUF weights) isn't available, instead of the default warn-and-fall-back-to-regex ### Partial runs - `--skip-dashboard` — run everything through evaluation, skip the final dashboard launch Every input/output path has a default matching this project's existing folder layout and can be overridden individually — run `python run_pipeline.py --help` for the complete flag list. ### Running a stage by hand Each script above still runs standalone for debugging a single stage in isolation — `tag_extractor_v2.py`, `review_server.py`, `export_db_to_kg_csv.py`, and so on all accept the same `--page`/`--set` scoping directly. `run_pipeline.py` is what supervises all of them together as one run; it doesn't replace running any of them individually when that's what you actually need. Deliverables (Excel + annotated PDF) can be re-generated at any time against current database state: ``` python pdf_excel_export.py ``` ## Training YOLO26 (optional) Scripts are provided for retraining the symbol detector on YOLO26 as a candidate successor to the current YOLO baseline: - `train_yolo26_gpu.py` — run directly on any local machine with an NVIDIA GPU (`python train_yolo26_gpu.py`). Reads `data\unified_dataset\dataset.yaml` from this project's own folder structure and writes the resulting weights straight into `models\`. - `kaggle_train_yolo26.py` — a cell-based variant for training on Kaggle's free GPU sessions instead. Copy each `# %%`-delimited block into its own Kaggle notebook cell, in order (or upload the file directly if your Kaggle setup accepts script kernels). Requires the training dataset zipped and uploaded as a private Kaggle Dataset first — see the comments at the top of the file for the full setup steps (GPU accelerator, internet access, session time-limit handling). - `finetune.py` — a short fine-tuning pass (15 epochs) on top of an already-retrained YOLO26 checkpoint, using labels drawn from tags corrected through human review. This is the step that produced the production checkpoint currently in use, not a full from-scratch retrain. YOLO26 cannot warm-start from the current detection checkpoint — it uses an architecturally different, NMS-free detection head — so both the full retrain and the fine-tuning pass are transfer-learning runs from a COCO-pretrained checkpoint, not a continuation of the old model's weights. ## Known limitations - Symbol detection has a documented recall gap on dense equipment clusters — some correctly-OCR'd tags currently have no detected symbol nearby. Actively being investigated, not treated as a fixed ceiling. - YOLO26 is not yet a validated replacement for the current detector — no per-class precision/recall/mAP comparison has been completed, and training scripts exist but no full validated run has finished yet. - The confidence-scoring formula's stated weighting (0.4 × detection confidence + 0.6 × OCR confidence) is accurate on paper but collapses in practice to being driven almost entirely by detection confidence, since OCR confidence saturates near its maximum on this document type regardless of read quality. The pipeline's actual acceptance threshold accounts for this; documentation describing the formula in isolation can be misleading without that context. - ISA tag normalization: the regex/ISA engine is the mandatory, always-active baseline. The optional local LLM cross-check (llm/hybrid backend) is experimental — testing so far (regex vs. hybrid across a full 3-page set) found zero tag disagreements, with hybrid mode taking roughly 2x the runtime of regex alone. Whether the LLM should be gated to only fire on ambiguous/low-confidence cases (rather than every tag) is an open question; no GPU benchmark for this layer has been run yet. - No ground-truth validation against a manually verified reference set, and no revision-alignment check across drawing versions, has yet been completed — both are required before any stage of this pipeline is considered production-ready. ## Roadmap - **Detector upgrade**: complete YOLO26 per-class validation against the current baseline (training scripts now in place — see "Training YOLO26" above), targeting the dense-cluster recall gap alongside general accuracy and CPU inference speed. - **LLM normalization tuning**: decide whether the experimental LLM cross-check should gate on ambiguous/low-trust cases only, rather than running unconditionally on every tag, given no accuracy benefit has been observed yet against its latency cost. Benchmark the GPU path once available. - **Ground-truth validation & revision alignment**: establish a validation gate — extracted tags checked against a manually verified reference set, plus a revision-alignment check across drawing versions — before any stage of the pipeline is described as production-ready. --- Internship project — Schneider Electric.