📘 Retrieval-Augmented Generation (RAG) pipeline
offline indexing · online query & grounded inference
%%{init: {
'theme': 'base',
'themeVariables': {
'background': '#ffffff',
'primaryColor': '#eef4ff',
'primaryBorderColor': '#3b82f6',
'primaryTextColor': '#1e293b',
'lineColor': '#5b6e8c',
'secondaryColor': '#fff7ed',
'tertiaryColor': '#f0fdf4',
'clusterBkg': '#fafcff',
'clusterBorder': '#dce5ef',
'nodeBorder': '#cbd5e1',
'edgeLabelBackground':'#ffffff'
},
'flowchart': {
'nodeBorderRadius': 16,
'clusterBorderRadius': 20,
'curve': 'basis',
'padding': 20
}
}}%%
flowchart TB
%% ------------------------------------------------------------------
%% OFFLINE DATA PREPARATION PIPELINE (Indexing side)
%% ------------------------------------------------------------------
subgraph PREP [📂 1 · OFFLINE INDEXING 🔧]
direction TB
A["✨ 1. Select & Clean
📎 data sources · filtering · normalization"] B["✂️ 2. Intelligent Chunking
🔹 semantic splitting · overlap control"] C["🔢 3. Embedding & Vectorization
🧠 dense vectors · embedding model"] D["🗄️ 4. Vector Database
⚡ indexed storage · metadata"] A --> B --> C --> D end %% ------------------------------------------------------------------ %% QUERY & RETRIEVAL SUBGRAPH (Online / inference) %% ------------------------------------------------------------------ subgraph QUERY ["💡 2 · ONLINE QUERY & RETRIEVAL"] direction TB E["❓ User Query
🔍 natural language input"] F["🔄 5. Query Vectorization
📐 same embedding model"] G["📚 6. Retrieve Relevant Context
🎯 similarity search · top‑k"] E --> F --> G end %% ------------------------------------------------------------------ %% FINAL GROUNDED ANSWER (Output) %% ------------------------------------------------------------------ H["💎 7. Grounded Generation
🤖 LLM + context → faithful answer"] %% ------------------------------------------------------------------ %% CROSS-CONNECTIONS: vector DB feeds into retrieval %% ------------------------------------------------------------------ D -->|"🔁 indexed vectors"| G G -->|"🧩 context passages"| H %% ========== ELEGANT STYLING (enhanced readability & aesthetics) ========== classDef prepNode fill:#EFF6FF, stroke:#3b82f6, stroke-width:2px, color:#0c4a6e, font-weight:500, rx:12, ry:12; classDef queryNode fill:#FFF7ED, stroke:#f97316, stroke-width:2px, color:#7c2d12, font-weight:500, rx:12, ry:12; classDef outputNode fill:#F0FDF4, stroke:#16a34a, stroke-width:2px, color:#14532d, font-weight:600, rx:14, ry:14, font-size:1.05em; class A,B,C,D prepNode; class E,F,G queryNode; class H outputNode; %% additional subtle style for subgraph titles style PREP fill:#f1f5f9, stroke:#b9d0f0, stroke-width:1.5px, rx:16 style QUERY fill:#fffaf5, stroke:#ffdec2, stroke-width:1.5px, rx:16
📎 data sources · filtering · normalization"] B["✂️ 2. Intelligent Chunking
🔹 semantic splitting · overlap control"] C["🔢 3. Embedding & Vectorization
🧠 dense vectors · embedding model"] D["🗄️ 4. Vector Database
⚡ indexed storage · metadata"] A --> B --> C --> D end %% ------------------------------------------------------------------ %% QUERY & RETRIEVAL SUBGRAPH (Online / inference) %% ------------------------------------------------------------------ subgraph QUERY ["💡 2 · ONLINE QUERY & RETRIEVAL"] direction TB E["❓ User Query
🔍 natural language input"] F["🔄 5. Query Vectorization
📐 same embedding model"] G["📚 6. Retrieve Relevant Context
🎯 similarity search · top‑k"] E --> F --> G end %% ------------------------------------------------------------------ %% FINAL GROUNDED ANSWER (Output) %% ------------------------------------------------------------------ H["💎 7. Grounded Generation
🤖 LLM + context → faithful answer"] %% ------------------------------------------------------------------ %% CROSS-CONNECTIONS: vector DB feeds into retrieval %% ------------------------------------------------------------------ D -->|"🔁 indexed vectors"| G G -->|"🧩 context passages"| H %% ========== ELEGANT STYLING (enhanced readability & aesthetics) ========== classDef prepNode fill:#EFF6FF, stroke:#3b82f6, stroke-width:2px, color:#0c4a6e, font-weight:500, rx:12, ry:12; classDef queryNode fill:#FFF7ED, stroke:#f97316, stroke-width:2px, color:#7c2d12, font-weight:500, rx:12, ry:12; classDef outputNode fill:#F0FDF4, stroke:#16a34a, stroke-width:2px, color:#14532d, font-weight:600, rx:14, ry:14, font-size:1.05em; class A,B,C,D prepNode; class E,F,G queryNode; class H outputNode; %% additional subtle style for subgraph titles style PREP fill:#f1f5f9, stroke:#b9d0f0, stroke-width:1.5px, rx:16 style QUERY fill:#fffaf5, stroke:#ffdec2, stroke-width:1.5px, rx:16
⚡ end‑to‑end RAG flow: grounding with vector store
🎨 refined structure · semantic grouping
Comments
Post a Comment