Retriever Guide

RAG pipeline & what a retriever is

In a RAG system the retriever sits after the vector store and before the LLM. It finds evidence; it does not write the final answer.

“A retriever takes a user query and returns the most relevant documents or chunks from a knowledge source. It normally does not generate the final answer itself.” — Retriever Guide PDF

Simple example

Database has 1,000 chunks. User asks: What is semantic chunking? The retriever does not send all 1,000 chunks. It returns only relevant ones (definition, example, advantages) for the LLM.

Analogies

LangChain as_retriever knobs

From a vector store you typically build a retriever with a search type and kwargs, then invoke a query.

retriever = vector_store.as_retriever(
    search_type="mmr",
    search_kwargs={
        "k": 4,          # final number of documents to return
        "fetch_k": 20    # candidates considered by MMR
    }
)
documents = retriever.invoke("What is a vector database?")
for document in documents:
    print(document.page_content)
    print(document.metadata)

search_type

Similarity metrics

Vector DBs commonly expose these three families:

Metadata filtering

Do not rely on semantic similarity alone. Metadata constrains search to department, year, document type, source, role, or tenant.

metadata = {
    "department": "HR",
    "year": 2026,
    "document_type": "policy",
    "access_role": "manager"
}
# User: "Show the HR leave policy for 2026."
filter = {"department": "HR", "year": 2026}

With that filter, the retriever ignores other departments/years even if content is semantically similar.

Common filter types

Pre-filtering vs post-filtering

Pre-filter flow

retriever = vector_store.as_retriever(
    search_type="similarity",
    search_kwargs={
        "k": 4,
        "filter": {"department": "HR", "year": 2026}
    }
)
documents = retriever.invoke("Show the HR leave policy for 2026.")

smaller search space fewer irrelevant hits better tenant isolation can improve speed

Post-filter flow

Top-5 semantic hits might include Finance / Legal / wrong year; after filtering for HR + 2026 only one remains — even if better HR docs never made the initial five.

documents = vector_store.similarity_search(
    "Show the HR leave policy for 2026.", k=5
)
filtered = [
    d for d in documents
    if d.metadata.get("department") == "HR"
    and d.metadata.get("year") == 2026
]

Sparse, dense, hybrid

Query transformation

Improve the user’s query before retrieval so the system can find more relevant information. Rewrite, expand, decompose — or use HyDE / multi-query (next sections).

Hypothetical Document Embeddings (HyDE)

HyDE = generate a hypothetical answer first, then use its embedding to retrieve real documents.

Multi-Query Retriever

Creates multiple versions of the same user query, runs retrieval for each, then merges and deduplicates.

Example variations for “How does Llama 2 improve safety?”:

1. What safety techniques are used in Llama 2?
2. How was Llama 2 safety fine-tuned?
3. How does Llama 2 reduce unsafe responses?

Parent Document vs Sentence Window

Parent Document flow

Large doc → child chunks → embed children → query hits child → return its parent section. Small chunks retrieve accurately; parents give the LLM enough context.

Sentence Window example

Query: How is Llama 2 aligned using human feedback? may match Sentence 10 (“RLHF is used…”). Return Sentences 8–12 so the LLM sees SFT → preference data → RLHF → reward model → PPO.

Multi-Hop Retrieval

Retrieve → use the result to retrieve again → combine evidence. Used when one retrieval pass cannot answer the question.

Hybrid merge strategies & Result Fusion

Hybrid retrieval = use multiple retrievers. It does not by itself define how to merge outputs. Common merge options:

Maximal Marginal Relevance (MMR)

MMR first takes the most relevant document, then picks remaining docs by balancing query relevance against diversity — avoiding near-duplicates of what is already selected.

Worked example (from PDF)

Query: How does Llama 2 improve safety? · fetch_k=5 · k=2 · λ = 0.5

YouTube trick: without MMR you get “Python Tutorial Part 1–4”; with MMR you get Tutorial + Project + Interview + Best Practices.

retriever = vector_store.as_retriever(
    search_type="mmr",
    search_kwargs={"k": 4, "fetch_k": 20, "lambda_mult": 0.5}
)

Reranking

Reranking takes the initial candidate set, scores each document more carefully against the query, and reorders so the best evidence reaches the LLM.

better order drop weak candidates less irrelevant context / tokens works on sparse, dense, or hybrid pools

Bi-encoder vs cross-encoder

Production cascade

1. User submits query → apply metadata + security filters
2. Retrieve a broad candidate set (BM25 / vector / hybrid)
3. Rerank → optional minimum-score threshold
4. Send best documents to the LLM

Example scale: 1,000,000 chunks → retrieve 50 → rerank → pass top 5. Elasticsearch exposes the window as rank_window_size.

Ranking families

Contextual Compression

First retrieve relevant documents, then remove parts that are not useful for the current query.

Example: a 1,000-token HR chunk covers leave, dress code, WFH, travel, holidays. Query “How many annual leave days?” keeps only the leave sentences.

Compressor types

from langchain_classic.retrievers.contextual_compression import (
    ContextualCompressionRetriever,
)
from langchain_classic.retrievers.document_compressors import LLMChainExtractor

compressor = LLMChainExtractor.from_llm(llm)
compression_retriever = ContextualCompressionRetriever(
    base_retriever=dense_retriever,
    base_compressor=compressor,
)

Multimodal retriever

Retrieves across modalities (text ↔ image) via multimodal embeddings in a shared vector space — text-to-image, image-to-text, and image-to-image.

Concept → LangChain class

From the updated PDF (langchain_classic / community imports — check your package versions).

# Hybrid via EnsembleRetriever (weighted RRF internally)
from langchain_community.retrievers import BM25Retriever
from langchain_classic.retrievers import EnsembleRetriever

bm25 = BM25Retriever.from_documents(chunks); bm25.k = 5
dense = vector_store.as_retriever(search_kwargs={"k": 5})
hybrid = EnsembleRetriever(retrievers=[bm25, dense], weights=[0.5, 0.5])
docs = hybrid.invoke("Llama 2 grouped query attention")

# Multi-query
from langchain_classic.retrievers.multi_query import MultiQueryRetriever
mq = MultiQueryRetriever.from_llm(
    retriever=vector_store.as_retriever(), llm=llm, include_original=True
)

Practice checklist

1. Define a retriever in one sentence; mark where it sits in the RAG pipeline.
2. Compare similarity vs mmr vs score-threshold (use the demo above). Walk the MMR Doc A / Doc E example.
3. Explain k, fetch_k, lambda_mult, and when pre-filter beats post-filter.
4. Contrast sparse / dense / hybrid; sketch simple merge vs weighted fusion vs RRF vs rerank-after-merge.
5. Contrast HyDE vs Multi-Query; give one rewrite / expansion / decomposition example.
6. Explain Parent Document vs Sentence Window; when you’d use multi-hop.
7. State why a cross-encoder is second-stage only; how compression differs from reranking.
8. Map three concepts to LangChain classes from the table above.

Analogy Handbook RAG Architecture RRF vs Reranker Vector Store vs Database CRAG Decision Guide Agentic RAG All learning pages

Source: retriever-updated.pdf · 41-Day-08Aug26 · Full-Stack GenAI Bootcamp