Data Ingestion & Parsing · Week 05

RAG Architecture

Retrieval-Augmented Generation grounds LLM answers in your own data. The system has two distinct paths: an offline indexing pipeline (ingest once) and an online query pipeline (run per user question). Week 05 notebooks cover the ingestion phase — loaders, Documents, and splitters — with no API keys required.

Step: — / 12
1 · Document Ingestion Phase
📂 Raw Data Sources
Files and structured records from this week's six notebooks.
.txt PDF Word CSV / Excel JSON SQLite
LangChain loaders in langchain_community.document_loaders — e.g. TextLoader, PyPDFLoader, Docx2txtLoader, CSVLoader, JSONLoader, SQLDatabase — each returns Document objects with metadata.
↓
✂️ Document Splitter
Large documents are broken into smaller, embeddable chunks.
Chunk 1 Chunk 2 Chunk 3
from langchain_text_splitters import RecursiveCharacterTextSplitter — chunk_size≈200–1000 and overlap≈10–20% (see notebook 1). Metadata (source, page) is copied to every child chunk.
↓
🧮 Embedding Model
Each chunk's text is converted to a dense vector.
[0.23, -0.45, 0.67, 0.12, -0.89, …]
Models like text-embedding-3-small, nomic-embed-text, or BGE map semantically similar text to nearby points in vector space.
↓
🗄️ Vector Database
Vectors + metadata stored for fast similarity lookup.
FAISS Pinecone Chroma pgvector
Indexed offline. At query time the DB returns top-k nearest neighbors without re-reading original files. Choosing FAISS vs Pinecone? Vector store vs database →
2 · Query Processing Phase
💬 User Query
A natural-language question triggers the retrieval path.
Unlike fine-tuning, no model weights change — the query is processed at runtime against the indexed knowledge base.
↓
🔢 Query → Embedding
The same embedding model encodes the question into a vector.
[0.31, -0.22, 0.85, 0.44, -0.15, …]
Critical: use the same embedding model for indexing and querying, or similarity scores are meaningless.
↓
🔍 Similarity Search
Find the k most relevant chunks via cosine similarity or approximate nearest neighbors.
Optional metadata filters narrow results (e.g. doc_type == "policy"). Typical k = 3–8 chunks.
↓
📋 Retrieved Chunks
Top-k chunks ranked by relevance score.
Chunk A · 0.92 Chunk B · 0.87 Chunk C · 0.81
↓
📎 Augmented Context
Query + retrieved chunks are assembled into a single prompt.
Given the following context: [Chunk A: "RAG retrieves relevant docs…"] [Chunk B: "Embeddings enable semantic search…"] Answer: What is RAG?
Prompt engineering here matters — cite sources, set tone, and instruct the model to say "I don't know" when context is insufficient.
3 · Generation Phase
👤
End User
↑
✅ Generated Response
LLM Output
Click "Walk Through Pipeline" or type a query to see a simulated answer…
Responses should include citations from metadata (source file, page) for trust and auditability in production systems.
↑
Large Language Model
GPT-4o · Claude · Llama 3 · Gemini
Processes augmented context — does not search the vector DB itself; it only reads what retrieval provides.
Temperature, max tokens, and system prompts control creativity vs. faithfulness to retrieved context.
🔄

Current Information

Update the knowledge base without retraining the model — just re-index new documents.

🎯

Reduced Hallucinations

Answers are grounded in retrieved evidence instead of parametric memory alone.

🏢

Domain Knowledge

Private docs, policies, and proprietary data become queryable via natural language.

📎

Traceable Citations

Metadata enables source attribution — critical for legal, medical, and enterprise use cases.

Deep Dive

Why RAG vs. Fine-Tuning Alone?

RAG is not a replacement for fine-tuning — they solve different problems. Most production systems combine both.

❌ LLM without retrieval

  • Knowledge frozen at training cutoff date
  • No access to private or internal documents
  • Higher risk of confident but wrong answers
  • Cannot cite specific source documents

✓ LLM + RAG

  • Fresh data by re-indexing documents
  • Queryable private knowledge bases
  • Context injected at inference time
  • Source metadata for citations and filtering

Implementation

Typical RAG Stack (LangChain)

Each architecture component maps to a concrete library in a modern Python RAG app.

Phase Component LangChain / Ecosystem
Ingestion Load & parse files langchain_community.document_loaders · notebooks 1–6
Ingestion Split into chunks langchain_text_splitters · notebooks 1–2
Ingestion Embed text langchain_openai.OpenAIEmbeddings (next module)
Ingestion Store vectors Chroma, PineconeVectorStore, PGVector
Query Retrieve top-k vectorstore.as_retriever(search_kwargs={"k": 4})
Query Build prompt ChatPromptTemplate + context injection
Generation Generate answer ChatOpenAI, ChatAnthropic, etc.
All-in-one Chain orchestration create_retrieval_chain() or LCEL | pipe syntax
# Minimal RAG chain (LangChain LCEL) — covered in a later module
# Week 05 stops at: loaders → Documents → splitters → chunks
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain

retrieval_chain = create_retrieval_chain(retriever, document_chain)
result = retrieval_chain.invoke({"input": "What is RAG?"})
print(result["answer"]) # grounded in retrieved chunks

Week 05 Lab

What This Module Implements

The interactive diagram above shows the full RAG system. Your notebooks implement steps 1–2 (load + split). Steps 3–12 (embed, store, retrieve, generate) come in later weeks. Each notebook includes a Google Colab %pip install cell — no .env file needed.

Step in diagramNotebookOutput
Raw Data → Loaders1-dataingestion … 6-databaseparsingList[Document]
Document Splitter1-dataingestion, 2-dataparsingpdfChunked Documents with metadata
Embedding → Vector DBfuture moduleRequires API key / local embed model
Query → LLMfuture modulecreate_retrieval_chain

Deep dive on the Document model: LangChain Document Components → · Storage layer: Vector Store vs Database → · System design: RAG & Agents (Part 2) →