# PID-Digitization-using-Neo4j

**Repository Path**: dlml2/PID-Digitization-using-Neo4j

## Basic Information

- **Project Name**: PID-Digitization-using-Neo4j
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-09-09
- **Last Updated**: 2026-09-09

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# PlantGraph AI: P&ID Diagram Digitization using Neo4j and Computer Vision

PlantGraph AI is a full-stack intelligent system that processes Piping and Instrumentation Diagrams (P&IDs) using Computer Vision, Graph Databases, and Generative AI.
It extracts symbols, text, arrows, and lines, constructs a graph representation, and stores it in Neo4j Aura Cloud, enabling AI-powered querying through a chatbot interface.

### 🚀 Key Features

 1. AI Image Processing Pipeline — Multi-stage pipeline integrating YOLOv11, Roboflow, OCR, and custom line/arrow detection models.
 2. Graph Construction Engine — Automatically builds a graph (nodes + edges) from extracted entities and uploads to Neo4j Aura.
 3. Interactive Visualization — Explore the resulting P&ID graph interactively via PyVis and React D3.
 4. AI-Powered Chatbot — Conversational agent using OpenAI + RAG to convert natural language queries into Cypher and fetch results from Neo4j.
 5. Modular Full-Stack Design — Clean separation between pipeline modules, graph handlers, and frontend.
 6. Cloud Integration — Uses Neo4j Aura via Bolt/HTTP API, OpenAI API, and Roboflow API for scalability.

### Tools & Technologies Used

| Layer             | Tools / Libraries                                           | Purpose                                           |
| ----------------- | ----------------------------------------------------------- | ------------------------------------------------- |
| **Frontend**      | React, Vite, TypeScript, Tailwind CSS                       | UI, file uploads, graph rendering, chat interface |
| **Backend**       | Python, Flask/Streamlit, FastAPI (optional)                 | Pipeline orchestration and API                    |
| **ML/AI Models**  | YOLOv11, Roboflow API, EasyOCR, OpenCV, PyTesseract         | Symbol detection, text extraction, line detection |
| **Database**      | Neo4j Aura Cloud (via Bolt API)                             | Graph storage and query execution                 |
| **AI Chat Layer** | OpenAI GPT API + RAG                                        | Cypher generation, result interpretation          |
| **Visualization** | PyVis, D3.js                                                | Interactive graph visualization                   |
| **Deployment**    | Vercel (frontend), Streamlit Cloud / Local server (backend) | Scalable hosting                                  |

### 🧩 Architecture Overview

The application is divided into backend and frontend components:

#### 🧠 Backend (Python)

Handles AI model inference, graph creation, and Neo4j integration.

#### Core Modules:

1. predict_yolo_symbols.py — Detects P&ID symbols using YOLOv11.
2. predict_robo_symbols.py — Uses Roboflow API for additional symbol verification.
3. predict_text.py — Performs OCR using EasyOCR, chosen for its multilingual robustness and edge-device compatibility.
4. predict_arrows.py — Custom model for detecting arrows and flow directions.skeletonize.py / line_boxes.py — Extracts and connects lines via OpenCV skeletonization + contour analysis.
5. graph_nodes.py / graph_edges.py — Generates node-edge structure from detected objects.
6. neo4j_io.py — Pushes graph data to Neo4j Aura using the official Neo4j Python driver (neo4j protocol).
7. openai_utils.py — Converts natural-language queries to Cypher queries via GPT, executes them, and formats responses.
8. graph_preview.py — Creates PyVis HTML graph previews.

#### 💻 Frontend (React + TypeScript)

1. FileUploader.tsx — Uploads diagrams to backend.
2. ProcessingGate.tsx — Displays pipeline stages and progress.
3. GraphView.tsx — Renders interactive graph.
4. ChatUI.tsx — Provides conversational interface for querying the P&ID.
5. App.tsx — Integrates all components into the main page.

Built with Tailwind CSS for responsive styling and Vite for lightning-fast development.

### ☁️ Neo4j Aura Integration

PlantGraph AI connects to Neo4j Aura Cloud Database via the Neo4j Python Driver (neo4j package):
```
  from neo4j import GraphDatabase
  
  driver = GraphDatabase.driver(
      uri=os.getenv("NEO4J_URI"),
      auth=(os.getenv("NEO4J_USER"), os.getenv("NEO4J_PWD"))
  )
  
  with driver.session(database="neo4j") as session:
      session.run("CREATE CONSTRAINT IF NOT EXISTS FOR (n:Symbol) REQUIRE n.id IS UNIQUE")

 ```

#### How It Works:

After processing, each detected element (symbol, text, arrow, line) is represented as a node. Spatial or semantic relationships (e.g., “connects_to”, “flows_to”) are stored as edges. The full graph is pushed to Neo4j Aura where users can query relationships using Cypher, or visualize via Neo4j Browser.

#### What I Learned from Neo4j:

1. Graph modeling is far more expressive for engineering diagrams than relational schemas.
2. Learned to use Cypher efficiently for subgraph retrievals.
3. Implemented constraints and indexing for faster lookups.4.
4. Understood how to visualize and traverse connected components for flow-path tracing.

### 🔍 OCR Method and Rationale

PlantGraph AI uses EasyOCR (instead of Tesseract) because:

1. It supports non-standard engineering fonts and diagram text.
2. Pre-trained models handle rotated and multi-language text better.
3. Faster inference and better accuracy for small-text recognition in diagrams.

### 💬 Chatbot & AI Query Engine

The chatbot connects OpenAI GPT with Neo4j to allow natural queries like:
 ```
  “Show all instruments connected to valve FV-101.”
  “Which pipelines feed into tank T-201?”
 ```

#### Flow:

1. User query → GPT model → Cypher query generation
2. Cypher → Neo4j Aura → JSON result
3. GPT formats answer → shown in chat UI
 ```
  User: "What is connected to Pump P-101?"
  GPT → "MATCH (p:Symbol {name:'P-101'})-[r:connects_to]->(n) RETURN n.name"
  Neo4j → Result → GPT formats → Chat response
 ```
This RAG-style retrieval allows combining natural language reasoning with structured graph data.

### 📊 Analytics Feature

Implemented an analytics system to monitor OpenAI API usage across the P&ID pipeline. It logs every API call to a local SQLite database, tracking tokens, costs, and model usage. A Streamlit dashboard (via view_api_analytics.py) visualizes total calls, costs, token percentiles (p50/p90), and usage trends.
Built with sqlite3, pandas, and numpy, with configurable pricing via environment variables. Integrated seamlessly through log_openai_usage() in analytics.py for automatic tracking after each API call.

### 🧭 Flowchart: Architecture Pipeline

 ```

          ┌───────────────────────────────────────────────────────┐
          │                    Upload Image                       │
          │               (React Frontend - FileUploader)          │
          └───────────────┬───────────────────────────────────────┘
                          │
                          ▼
               ┌──────────────────────┐
               │   Streamlit Backend   │
               └──────────────────────┘
                          │
        ┌─────────────────┼──────────────────┐
        ▼                 ▼                  ▼
  YOLO Detection     OCR Extraction     Arrow & Line Detection
 (Symbols)           (EasyOCR)          (Skeleton + OpenCV)
        └─────────────┬───────────────┬─────────────┘
                      ▼
             ┌───────────────────┐
             │ Graph Builder     │
             │ (Nodes + Edges)   │
             └───────────────────┘
                      │
                      ▼
             ┌───────────────────┐
             │ Neo4j Aura Cloud  │
             │  (Graph Storage)  │
             └───────────────────┘
                      │
                      ▼
             ┌───────────────────┐
             │  Chatbot + RAG    │
             │ (OpenAI + Neo4j)  │
             └───────────────────┘
                      │
                      ▼
             ┌───────────────────┐
             │ Graph & Chat UI   │
             │ (React Frontend)  │
             └───────────────────┘
 ```

#### 🧠 Future Enhancements

1. Add 3D P&ID visualization via Three.js or Plotly.
2. Implement graph auto-layout optimization.
3. Support multi-image batch processing.
4. Integrate context-aware RAG with LangGraph for richer chat queries.

#### References:

1. Microsoft PID Digitization - (Link: https://devblogs.microsoft.com/ise/engineering-document-pid-digitization/)

#### Demos:

![Main Page](images/Img1.png)
![Graph](images/Img2.png)
![ChatBot](images/Img3.png)
![Neo4j Query Visualization](images/visualisation.png)