Semantic and Vector Search
Search that finds the right document when the user did not use the same words as the author, measured against your real query log rather than against how impressive the demo felt.
Keyword search fails when the user and the author chose different words for the same idea. Vector search fails when the user typed an exact product code. The systems that work in production use both, and the interesting engineering is in how they are combined and measured.
Semantic search retrieves content by meaning rather than by exact term match, using vector embeddings to compare a query with documents. In production it is almost always hybrid, combining semantic and keyword retrieval, with reranking and filtering, and its quality is measured against human relevance judgements on real queries.
Hybrid, because both halves fail differently
| Query type | Keyword search | Vector search | Hybrid |
|---|---|---|---|
| Exact code or identifier | Excellent | Poor | Excellent |
| Rare technical term | Excellent | Variable | Excellent |
| Paraphrase of a concept | Poor | Excellent | Excellent |
| Long natural question | Poor | Good | Very good |
| Misspelling | Poor | Good | Good |
| Acronym used internally | Good if indexed | Poor without context | Good |
Replacing keyword search with vector search is a common and expensive mistake: it fixes the paraphrase problem and breaks exact lookup, which is what your power users do all day. Fusing both and reranking the combined candidates is the design that holds up.
How we build search that measurably improves
Start from your query log
Real queries, weighted by frequency, including the ones that currently return nothing. Failed and zero-result queries are the most valuable input available, and they usually reveal that a fifth of searches are for a handful of things nobody realised were hard to find.
Build relevance judgements
For a sample of queries, your experts mark which results are relevant. This is the ground truth against which every change is scored, and without it search tuning is a sequence of opinions. It takes a few days of expert time and it is what makes the rest of the project measurable.
Chunk for retrieval, not for storage
How documents are split determines what can be found. Structure-aware chunking that keeps headings, tables and context with the text outperforms fixed-size splitting substantially, and it is the change we most often make on existing systems.
Rerank the candidates
A cross-encoder reranker applied to the top candidates from both retrievers is usually the single largest quality improvement available. It is more expensive per query and it is applied to a shortlist, which keeps the cost manageable.
Filters and permissions that actually work
Date, type, product, region and access rights applied during retrieval rather than after, so filtered searches return a full page of results and restricted content never appears in the first place.
Report the metrics that predict satisfaction
Recall at k, precision at k, mean reciprocal rank on your judgement set, plus behavioural measures from production: zero-result rate, click position, query reformulation rate and abandonment. The behavioural ones are the ones that move when search genuinely improves.
Search is the foundation for retrieval-augmented generation
If you are planning RAG or an enterprise knowledge assistant, this is the layer underneath it and the one that caps its quality. Building search properly first is rarely wasted, and answering questions on top of weak retrieval always is.
Practical decisions we make with you
- Embedding model and dimensionality, chosen on your content and measured, with cost and latency at your corpus size.
- Where the index lives, which is usually the least consequential decision: a vector index in a database you already run is often sufficient.
- Update frequency, including incremental indexing and deletion so retired content stops appearing.
- Multilingual handling, where users and content are in different languages.
- Personalisation, if results should differ by role, and the evaluation complications that introduces.
How the engagement runs
Relevance judgements come first, so every change after them is measured rather than argued.
Query analysis and judgements
Real queries sampled and weighted, zero-result queries identified, expert relevance judgements collected, current search measured as the baseline.
Indexing and hybrid retrieval
Structure-aware chunking, keyword and vector indexes, fusion tuned against the judgement set.
Reranking and filters
Cross-encoder reranking, filters and permission enforcement applied at retrieval, latency measured.
Interface and behavioural metrics
Result presentation, facets, and instrumentation for zero-result, click position and reformulation rate.
Rollout and handover
Staged release against the old search with behavioural comparison, then handover of judgements and tuning runbook.
What you receive
Search measured against your own queries, with the judgement set that keeps it improving.
Hybrid search system
Keyword and vector retrieval with fusion and reranking, deployed as infrastructure as code.
Relevance judgement set
Expert-marked results for a weighted sample of real queries, reusable for every future change.
Baseline comparison
Old search against new on the judgement set and on behavioural metrics.
Chunking and indexing pipeline
Structure-aware, with incremental update and deletion handling.
Filters and permissions
Applied at retrieval, tested across roles.
Search analytics
Zero-result rate, click position, reformulation and abandonment, with alerting on regression.
Is this the right engagement?
Worth being direct. Semantic and Vector Search is the wrong spend in some situations, and those are listed rather than buried.
Good fit if
- Users cannot find content that exists and complain about search.
- Query logs are available, or can be captured before the project starts.
- Experts can spend a few days marking relevance for a query sample.
- Content has structure that chunking can respect.
- You are planning retrieval-augmented generation and want the foundation right.
Choose something else if
- The content itself is missing or out of date; search cannot find what is not there.
- The corpus is small enough that browsing is faster than searching.
- Nobody can judge which results are relevant, so nothing can be measured.
- The requirement is answering questions rather than finding documents. See RAG system development.
Frequently asked questions
Marked up with FAQPage schema so these answers can surface directly in search results and inside AI assistant responses.
Should we replace keyword search with vector search?
No. Vector search fixes paraphrase and breaks exact lookup, which is what power users rely on. Hybrid retrieval fusing both, followed by reranking, is the design that performs well across the full range of real queries.
Do we need a dedicated vector database?
Usually not at the volumes most organisations have. A vector index inside a database you already run is often sufficient, and the choice rarely moves relevance. Chunking, hybrid fusion and reranking move it.
How do you measure whether search improved?
Against expert relevance judgements on a weighted sample of real queries, and against behavioural metrics from production: zero-result rate, click position, reformulation and abandonment. The behavioural measures are the ones that move when users genuinely find things faster.
What makes the biggest difference to search quality?
In our experience, structure-aware chunking and adding a reranker, in that order. Teams tend to reach for a different embedding model first, which usually moves the numbers least.
Is this the same as RAG?
It is the retrieval half of it. RAG adds a generation step that answers from the retrieved passages. Building search properly first is rarely wasted effort, because answer quality is capped by retrieval quality.
Often paired with this
Most clients combine two or three engagements from the Natural Language Processing pillar. These are the ones that most often run immediately before or after.
Knowledge Graph Construction
A graph built around the questions you cannot answer today, with entity resolution and provenance per fact.
Read more →Document Summarisation
Summaries written for a named reader and decision, checked for faithfulness against the source.
Read more →Text Classification and Entity Extraction
Labels and entities extracted from text at volume, with a scheme your experts agree on and per-class accuracy.
Read more →Is this the right engagement?
Tell us what you are trying to build. If a different service fits better, or if you do not need us at all, we will say so.