RAG pipeline cheat sheet

The eight stages, what each is for, and the debug checklist that maps a bad answer to one failing stage.

The pipeline

INGEST -> CHUNK -> EMBED -> INDEX          (offline, per document change)
QUERY  -> RETRIEVE -> RERANK -> GENERATE   (online, per question)

Stage	Job	Skipped or broken, you see
Ingest	Pull sources into clean text + metadata; track updates and deletes	Missing or stale content: correct answers are impossible for the generator
Chunk	Split into retrievable fact-bundles with context kept	Fragments that match but cannot answer; answers cut mid-fact
Embed	Turn each chunk into a vector; same model for queries	Paraphrased questions retrieve nothing relevant at any k
Index	Store vectors + keywords + metadata for fast filtered search	Slow queries, or filters silently dropping valid rows
Query	Rewrite, expand or route the raw question; extract filters	Exact constraints ("under 400k") treated as vague similarity
Retrieve	Top-k candidates via dense, sparse or hybrid + filters	Gold chunk absent from top k: recall@k is the ceiling
Rerank	Rescore candidates reading query + chunk together; keep top n	Right chunk retrieved at rank 18, ignored by the generator
Generate	Answer only from provided chunks, with citations, or say "not found"	Fluent inventions; correct chunks present but contradicted

Run top to bottom; stop at the first "no". Adapted from the seven failure points of Barnett et al.^[1]

Is the fact in the corpus at all? No: ingest gap (missing content). Fix ingestion, not prompts.
Is it in one chunk, with enough context to stand alone? No: chunking. Re-chunk or add chunk context.
Do the filters you extracted actually match the row's metadata? No: query/index. Fix filter extraction or metadata.
Is the gold chunk in the retrieved top k? No: retrieval. Try hybrid, raise k, rewrite the query (HyDE).
Is it in the top n after reranking, near the start or end of the prompt? No: rerank/ordering ("lost in the middle").
Does the answer use it, faithfully, with a citation? No: generation. Tighten the grounding instructions; require "not found" over guessing.
Is the answer complete and correctly formatted? No: generation instructions or context precision (too much noise retrieved).

Rule of thumb: measure retrieval before touching prompts. If recall@k is low, no prompt can save the answer.

Knob	Sane start	Move when
Chunk size	200-500 tokens, plus chunk context line	Recall low and chunks look fragmentary
Retrieve k	20-150 candidates	Reranker present: raise; no reranker: keep small
Rerank keep n	5-20 chunks	Faithfulness drops as n grows (noise)
RRF constant k	60	Rarely; it is deliberately insensitive