What is RAG (retrieval-augmented generation)?

In practice

A RAG system has three moving parts: an ingestion step that splits documents into passages and indexes them, a retrieval step that finds the closest passages to the question, and a generation step that answers using only those passages. The quality of the answer is usually decided by retrieval, not by which model you chose. Most disappointing RAG projects are retrieval problems wearing a model costume.

Where it fits in a build

RAG is the right pattern whenever the correct answer already exists somewhere in your files and the problem is that nobody can find it: policy manuals, contracts, product documentation, historic tenders, support history. It is the wrong pattern when the answer has to be calculated from live data — that is a database query, and a model should be asked to write it, not to guess it.

Common mistakes

• Splitting documents into fixed-size chunks that cut sentences and tables in half, then blaming the model for the answers.
• Skipping citations. Without a link back to the source passage, nobody can check the answer, so nobody trusts the system.
• Indexing everything, including drafts and superseded policies, so the assistant confidently quotes a document that was replaced two years ago.

Related terms

• Embedding
• Vector database
• Hallucination

Working on something that involves this?

ASTACKRA designs and builds AI systems, automation and custom software for businesses that need technology shaped around their own workflow. If this term turned up in a proposal and you want a straight answer about whether it applies to your situation, ask us — no obligation, and we will tell you if the answer is no.