Retrieval-Augmented Generation (RAG)

An LLM knows what it was trained on. It has never seen your pricing, your contracts, or last quarter's board pack — and if you ask about them, it will often invent a confident, plausible, wrong answer.

RAG fixes this by changing the order of operations. Instead of asking the model to answer from memory, you search your documents first, hand the model the relevant passages, and ask it to answer using only those. Closed book becomes open book.

That's the whole trick: retrieve, then generate.

Why RAG rather than training your own model

The instinct is to train a model on your data. RAG is almost always the better answer, for four reasons:

• It updates instantly. Change a document and the next answer reflects it. A fine-tuned model needs retraining.
• It's cheaper by orders of magnitude. No training run, no ML team.
• It can show its work. RAG can cite which document a claim came from — a fine-tuned model can't tell you why it said something. For anything regulated, this alone decides it.
• Access control survives. You can filter what gets retrieved per user. Once something is baked into model weights, it's baked in for everyone.

That last point deserves weight. Fine-tuning your HR files into a model means anyone who can query the model can potentially extract them. RAG keeps the documents where your permissions already live.

Where it goes wrong

RAG's weak point is the retrieval, not the generation — and this is consistently what teams get wrong. If the search hands back the wrong three paragraphs, the model writes a fluent answer grounded in irrelevant material. It'll sound just as confident as a correct one.

So the quality of a RAG system is mostly the quality of its search. Messy, duplicated, contradictory or out-of-date documents produce messy answers. RAG doesn't fix your knowledge base — it exposes it. Most disappointing RAG projects are a document problem wearing an AI costume.

Why it matters for your business

RAG is what turns "we have a wiki nobody reads" into something you can ask a question. The common shapes: an internal assistant over policies and process docs, customer support grounded in your real product documentation, or search across contracts and reports that understands meaning rather than matching keywords.

How we use RAG

For Kandor, we built AI-powered hybrid search blending semantic and keyword matching, with conversational AI and recommendations on top — the retrieval layer is what makes the answers trustworthy rather than plausible. Crumble summarises and analyses information across the web for students and people with learning disabilities, processing over 100,000 documents and websites within two months of launch.