Text Classification and Entity Extraction
Turning free text into consistent labels and structured fields at volume, with a label scheme your own experts agree on and accuracy reported per class rather than as a comfortable average.
Classification looks like the easiest thing in this pillar and fails for the least technical reason: two experts given the same document disagree about which label applies. Until that is resolved, every accuracy figure is measuring confusion rather than capability.
Text classification assigns one or more predefined labels to a piece of text; named entity extraction identifies and pulls out specific items within it such as organisations, products, amounts, dates and clause types. Both are supervised tasks whose ceiling is set by the consistency of the labelled examples they learn from.
The label scheme is the deliverable that lasts
Before any model is trained we build the scheme with your experts and test whether they can apply it consistently. The exercise is uncomfortable and always worth it:
- Overlapping categories. Two labels that could both apply, with no rule for which wins.
- Categories nobody uses. Inherited from an old taxonomy and applied to three documents a year.
- Missing categories. A meaningful share of documents forced into 'other', which is where the useful signal has been hiding.
- Different granularity for different purposes. Operations want four labels; reporting wants forty. Both are legitimate and they need a hierarchy, not an argument.
We measure inter-annotator agreement before volume labelling starts. Where it is low, the fix is the scheme rather than more training data, and a model can only ever be as consistent as the examples it learned from.
How we build classification and extraction
Choose the technique per task, not per fashion
A fine-tuned small encoder model is frequently the right answer for high-volume classification: fast, cheap, deployable anywhere, and consistently accurate on a stable label set. A large language model is better where labels are numerous, examples are scarce or the task changes often. We benchmark both on your data rather than assuming.
Extract with position and confidence
Every extracted entity keeps its character offsets, so a value can be traced back to its exact place in the source, and carries a confidence score so low-confidence extractions can be routed for review rather than written silently into a system of record.
Handle multi-label and hierarchy properly
Most real classification is multi-label, and forcing a single label per document loses information. Where a hierarchy exists, we predict at the level the data supports and report accuracy at each level, because coarse accuracy usually looks far better than the fine-grained figure people will actually use.
Report per class, weighted by consequence
An average across forty classes conceals the three that matter and the six that never work. We report per class, and weight evaluation by business consequence rather than by frequency, because the rare label is often the expensive one.
Design the human loop for the tail
New categories appear, language changes, and edge cases accumulate. Low-confidence and unmatched cases route to a reviewer whose corrections feed back as training data, which is what keeps a classifier useful in year two rather than merely accurate at launch.
Cost at volume decides the architecture
Classifying ten million documents a month is a different engineering problem from classifying ten thousand. Model size, batching and whether inference runs locally or through an API all follow from volume, and we cost it before choosing rather than after the first invoice.
Where this sits against LLM engineering
LLM applications generate and reason; this pillar produces deterministic, measurable structure from text at volume. Many builds use a language model inside, and the difference is that the output is a label or a field with a confidence score, evaluated per class, rather than a paragraph.
What it is used for
| Text | Labels or entities | What it enables |
|---|---|---|
| Support tickets and emails | Intent, product area, urgency, sentiment | Routing, prioritisation, trend reporting |
| Contracts and agreements | Clause types, parties, dates, obligations | Obligation tracking; see contract analysis |
| Clinical and case notes | Conditions, medications, findings | Coding, cohort identification, audit |
| News and market text | Companies, events, products, risk signals | Monitoring and alerting |
| Product feedback | Feature area, issue type, severity | Roadmap evidence and defect trending |
| Regulatory filings | Entities, obligations, dates, amounts | Compliance tracking and change detection |
How the engagement runs
Annotator agreement is settled before volume labelling, because the model cannot exceed it.
Label scheme and agreement
Scheme built with your experts, agreement measured on a shared sample, definitions revised until it holds.
Dataset and baseline
Volume annotation with audit sampling, a simple keyword or rule baseline measured as the bar.
Model development
Encoder and language model approaches benchmarked on your data, per-class accuracy and confidence calibration.
Deployment and review loop
Inference at your volume, confidence thresholds set, review queue for low-confidence and unmatched cases.
Handover
Annotation guide, dataset, model and retraining runbook handed to your team.
What you receive
A working classifier and, more durably, a label scheme your organisation actually agrees on.
Label scheme and annotation guide
Definitions, precedence rules and worked examples, with agreement measurements.
Annotated dataset
Your text with labels and entity spans, yours to reuse for every future model.
Deployed model
Classification and extraction at your volume with confidence scores and character offsets.
Per-class performance report
Precision and recall by class, weighted by business consequence, with the confusions named.
Review queue
Low-confidence and unmatched routing with corrections captured as training data.
Cost model
Inference cost per million documents with the levers that change it.
Is this the right engagement?
Worth being direct. Text Classification and Entity Extraction is the wrong spend in some situations, and those are listed rather than buried.
Good fit if
- Text arrives at a volume people cannot read and label consistently.
- Experts can agree a label scheme and commit annotation review time.
- A downstream system will act on the labels or the extracted fields.
- You can accept a review queue for the uncertain cases.
- Volume is high enough that per-document cost matters.
Choose something else if
- Nobody can articulate what the categories mean, even after a workshop.
- Volume is low enough that people reading the text is cheaper.
- The requirement is open-ended reasoning rather than structured output.
- The text is in documents that need extraction first. See intelligent document processing.
Frequently asked questions
Marked up with FAQPage schema so these answers can surface directly in search results and inside AI assistant responses.
How much labelled text do we need?
Typically a few hundred examples per class for fine-tuning a modern model, with more for rare classes and subtle distinctions. Consistency matters more than volume: five hundred examples labelled to an agreed scheme beat five thousand labelled by people who disagreed with each other.
Should we use a small model or a large language model?
For a stable label set at high volume, a fine-tuned small encoder is usually faster, cheaper and equally accurate. For many labels, scarce examples or a task that changes often, a language model earns its cost. We benchmark both on your data rather than defaulting to either.
What is named entity extraction used for?
Pulling specific items out of free text — organisations, amounts, dates, product names, clause types — so they can populate a database or trigger a process. The engineering difference from classification is that every extraction keeps its position in the source, so any value can be verified against the original.
What happens to text that does not fit any category?
It routes to a review queue rather than being forced into the nearest label. Those cases are usually the most informative output of the system, because a growing unmatched pile is how you find the category your scheme is missing.
How do you stop accuracy degrading over time?
By capturing reviewer corrections as training data and monitoring the distribution of predictions and confidence. Language changes, new products appear and new issue types emerge, and a classifier that is never retrained becomes an expert in last year's text.
Often paired with this
Most clients combine two or three engagements from the Natural Language Processing pillar. These are the ones that most often run immediately before or after.
Sentiment and Intent Analysis
Aspect-level sentiment and intent tuned to your domain, calibrated against your own reviewers.
Read more →Contract and Legal Document Analysis
Clause extraction, deviation flagging and obligation tracking, cited back to the page for fast verification.
Read more →Semantic and Vector Search
Search that finds the right thing, measured on your real queries with hybrid retrieval and reranking.
Read more →Is this the right engagement?
Tell us what you are trying to build. If a different service fits better, or if you do not need us at all, we will say so.