RAG Production Pitfalls and Checklist

Common Failures

• Stale vectors - Updated documents still return old chunks.
• Slow reranking - Reranking every query adds latency.
• Cost creep - Wide retrieval inflates prompt tokens.
• Silent drift - Embedding or data changes degrade quality unnoticed.

Production Checklist

• Hash chunk ids so re-indexing is idempotent.
• Automate rebuilds when source documents change.
• Cache identical questions and answers.
• Cap retrieval width; rerank only when latency allows.
• Monitor retrieval hit-rate and hallucination reports.
• Pin embedding model versions in config.

Auditability

Log every prompt, the chunks used and the final answer. Debugging and trust improve immediately when each answer remembers its sources.

Key Points

• Keep indexes in sync with the source of truth.
• Slow operations need caching and cheaper paths.
• Monitor quality drift, not just uptime.
• Record the chunks behind every answer.