RAG System Development
Retrieval augmented generation built the way it actually works: as a search problem with a language model attached, measured on retrieval quality rather than on how fluent the answer sounds.
Most disappointing RAG systems are not suffering from a model problem. They are suffering from a search problem that nobody measured. The generation step can only work with what retrieval hands it, so when a system answers confidently and wrongly, the fault is almost always three paragraphs upstream.
RAG system development is the engineering of a retrieval augmented generation pipeline: ingesting and segmenting your content, indexing it for both keyword and semantic search, retrieving and reranking the passages most likely to answer a given question, and grounding a language model in that evidence so its answers can be traced back to a source.
Why RAG development is a retrieval problem first
A language model given the right three paragraphs will produce a good answer with a mediocre prompt. The same model given the wrong three paragraphs will produce a fluent, confident and useless answer no matter how carefully the prompt is written. That asymmetry is the whole discipline.
So the first thing we build is not the chatbot. It is a measurement harness: a set of real questions, the passages that genuinely answer them, and a score for how often retrieval surfaces those passages in the top results. Everything after that is optimisation against a number instead of argument about a demo.
The most common cause is chunking, not embeddings
Teams reach for a different embedding model when recall is poor. In our experience the larger win is almost always in segmentation: splitting documents on structure rather than character count, keeping tables and headings intact, and carrying section context into each chunk so a fragment still makes sense on its own.
What we build into a RAG system
Ingestion that respects document structure
PDFs, wikis, ticket histories, contracts and spreadsheets each need different handling. Tables are preserved rather than flattened into prose, headings become retrievable context, scanned documents are put through document understanding before indexing, and every chunk keeps a durable pointer back to its source location so citations are exact.
Hybrid search and reranking
Semantic search alone misses exact identifiers, product codes and rare terms; keyword search alone misses paraphrase. We run both and fuse the results, then rerank the candidates with a cross-encoder so that precision at the top of the list is high enough for a small context window to be sufficient.
Access-aware retrieval
Retrieval filters on the permissions of the person asking, evaluated at query time. Without this, a knowledge assistant becomes an efficient way to leak documents to people who were never granted them, which is the fastest way to have a project stopped by security.
Grounded generation with citations
Answers cite the passages they were built from, and the system is designed to say it does not know when retrieval returns nothing relevant. A visible refusal is worth more than a plausible invention, particularly in regulated environments where an unsourced claim has consequences.
Freshness and reindexing
Content changes. The pipeline handles incremental updates, deletions and versioning so that a document retired last week stops being quoted this week, and the index does not need a full rebuild every time a page is edited.
How RAG quality is measured
We report retrieval and generation separately, because they fail for different reasons and are fixed by different work:
| Metric | What it tells you | Typical lever |
|---|---|---|
| Recall at k | Whether the answer was available to the model at all | Chunking, hybrid search, index coverage |
| Precision at k | How much noise the model had to read past | Reranking, query rewriting |
| Groundedness | Whether the answer is supported by the retrieved text | Prompting, refusal behaviour, citation enforcement |
| Answer correctness | Whether a human expert agrees with the result | Everything above, scored end to end |
| Latency and cost | Whether it is affordable at real volume | Caching, model routing, context size |
Separating these is what turns a stalled RAG project into a tractable one. A system with 62% recall does not need a better prompt; it needs a better index. The LLM evaluation engagement exists to build exactly this measurement layer where a team wants it independently of a build.
How the engagement runs
Retrieval is measured before generation is tuned, so effort goes where the score says it should.
Corpus audit and question set
We inventory the content, assess how much of it is actually answerable, and build a question set with the passages that should be retrieved for each.
Ingestion and baseline retrieval
Parsing, chunking and indexing are built and a baseline recall score is recorded, so every later change can be compared against it.
Hybrid search, reranking and generation
Search fusion, reranking, query rewriting and grounded generation with citations, each change scored rather than assumed.
Permissions, freshness and hardening
Access-aware filters, incremental reindexing, injection resistance through retrieved content, and cost and latency tuning.
Handover
Pipeline, index configuration, evaluation set and scores handed over with a working session for the owning team.
What you receive
The pipeline, the numbers behind it, and the means to keep improving it after we leave.
Ingestion pipeline
Parsing, chunking and indexing code for every source system, with incremental update and deletion handling.
Retrieval configuration
Hybrid search, reranking and query rewriting, tuned against your question set with the results recorded.
Retrieval scorecard
Recall and precision at k, groundedness and correctness, with a baseline and the effect of each change.
Grounded generation layer
Prompting, citation enforcement and refusal behaviour, versioned and tested.
Access control design
How permissions are enforced at query time, and how that is tested when roles change.
Improvement backlog
The ranked list of what would raise the score next, with effort estimates, so the work continues without us.
Is this the right engagement?
Worth being direct. RAG System Development is the wrong spend in some situations, and those are listed rather than buried.
Good fit if
- You have a body of internal content that people cannot find answers in quickly enough.
- An existing RAG prototype answers well in demos and poorly on real questions.
- Answers must cite sources, and unsourced answers are unacceptable.
- Retrieval has to respect existing document permissions.
- You need retrieval quality reported as a number rather than described as a feeling.
Choose something else if
- The content itself is contradictory or out of date. Fix the corpus first; retrieval cannot repair it.
- The task needs the model to act in your systems rather than answer from documents.
- The knowledge is entirely in a structured database. A query interface beats retrieval for that.
- You want an internal assistant end to end rather than the retrieval layer. See enterprise knowledge assistant.
Frequently asked questions
Marked up with FAQPage schema so these answers can surface directly in search results and inside AI assistant responses.
What is RAG system development in practice?
It is the work of turning a document collection into something a language model can answer from reliably: parsing and chunking the content, indexing it for keyword and semantic search, retrieving and reranking passages per question, and grounding generation in that evidence with citations. In practice most of the effort is in the retrieval half.
Do we need a vector database?
Usually, but it is the least consequential decision in the project. A vector index inside a database you already run is often enough at the volumes most organisations have, and the choice rarely moves the score. Chunking, hybrid search and reranking move it.
How accurate can a RAG system be?
It depends almost entirely on whether the answer exists in your content, clearly stated, in one place. Where it does, correctness in the nineties is normal. Where the answer has to be inferred across contradictory documents, no retrieval strategy rescues it, and we would rather establish that in week two than month four.
How is RAG different from fine-tuning?
Retrieval gives the model facts at request time and is the right tool for knowledge that changes. Fine-tuning changes how the model behaves and is the right tool for format, tone and task specialisation. They solve different problems and are often used together, as set out under LLM fine-tuning.
Can RAG be made resistant to prompt injection?
It can be made substantially more resistant, and it has to be, because retrieved content is untrusted input. We test with adversarial documents planted in the corpus, constrain what retrieved text is allowed to influence, and keep tool permissions separate from retrieval. No system is immune, and we document the residual risk rather than claiming otherwise.
Often paired with this
Most clients combine two or three engagements from the Generative AI & LLM Engineering pillar. These are the ones that most often run immediately before or after.
Custom LLM Application Development
A production LLM application built on your data, with evaluation, guardrails and cost control designed in rather than added later.
Read more →Enterprise Knowledge Assistant
Internal search that answers with citations, honours existing permissions and says when it does not know.
Read more →LLM Evaluation and Benchmarking
Golden sets, calibrated judges and a regression suite in CI, so quality is a number that moves rather than an opinion.
Read more →Is this the right engagement?
Tell us what you are trying to build. If a different service fits better, or if you do not need us at all, we will say so.