What makes a RAG system production-ready

A RAG demo takes an afternoon: chunk the PDFs, embed, retrieve top-k, stuff the prompt. It will look great on the three questions you test it with.

Then someone uploads a 400-page contract with tables, asks a question whose answer spans two sections, and the demo quietly returns something confident and wrong. The distance between that demo and a production system is where the actual engineering lives. Here's what closed the gap for me, building a document intelligence platform.

Contextual retrieval beats clever chunking

Plain chunks lose their context. A paragraph that says "this fee does not apply" is useless if the chunk boundary cut off which fee. The fix that mattered most: prepend each chunk with a short, LLM-generated summary of where it sits in the document before embedding. Anthropic calls this contextual retrieval; in my evals it cut retrieval failures more than any embedding model swap.

Hybrid search is the second non-negotiable. Semantic search misses exact identifiers like invoice numbers, clause references, and product codes, which keyword search catches trivially. Run both, fuse the results.

Citations are a feature, not a garnish

Every answer should carry references to the exact source passages. Two reasons:

1. Users stop trusting an uncited system the first time it's wrong. With citations, a wrong answer is a verifiable mistake; without them, it's an invisible one.
2. Citations make failures debuggable. When an answer is bad, the citation tells you instantly whether retrieval failed (wrong passages) or generation failed (right passages, wrong reading).

If the retrieved context doesn't contain the answer, the correct behavior is to say so. A grounded refusal is a feature.

If you don't measure retrieval, it's broken

The uncomfortable truth about RAG: quality degrades silently. New document types, drifting query patterns, a model upgrade: any of these can quietly tank retrieval, and no one notices until a user complains.

So the system needs evals as infrastructure, not as a one-time report. I use Ragas metrics (faithfulness, answer relevancy, context precision) over a curated question set, run on every meaningful change, with results on a dashboard. The dashboard is what turns "I think retrieval got worse" into "context precision dropped 12% after Tuesday's ingestion change."

The production checklist

What I now consider the minimum bar:

• Contextual chunks + hybrid retrieval, not just cosine similarity
• Citations on every answer, refusals when context is insufficient
• A versioned eval set with automated Ragas runs
• Quality dashboard: retrieval scores, latency, cost per query
• Tracing on every request, so a bad answer can be replayed
• Ingestion that handles tables, scans, and malformed PDFs without silent drops

None of this is glamorous. All of it is the difference between a demo that impresses for five minutes and a system people rely on.