LLM Fine-Tuning and Customisation
Fine-tuning a language model on your own data, undertaken only after prompting and retrieval have been shown to fall short, and measured against that baseline rather than against a hope.
Fine-tuning is the most requested and least frequently necessary service in this pillar. It is also genuinely the right answer more often than the sceptics claim. The difference between those two statements is a baseline measurement, which is where every engagement here starts.
LLM fine-tuning is the process of continuing a pretrained model's training on curated examples of your own so that it adopts a specific behaviour, format or domain vocabulary. It changes how the model responds, not what facts it can access, and it is worth doing when prompting and retrieval have been measured and found insufficient.
When LLM fine-tuning is the right answer
Fine-tuning earns its cost in a narrow set of situations, and recognising them early saves a great deal of money:
- Format and structure. The model must produce a rigid output shape every time, and prompting gets you to ninety-something per cent when you need consistency at scale.
- Domain language. Your field uses terminology, abbreviations or conventions that general models handle awkwardly, and examples teach faster than instructions.
- Task specialisation. A narrow, high-volume classification or extraction task where a smaller fine-tuned model matches a much larger prompted one at a fraction of the cost.
- Latency and unit cost. A tuned small model that runs in-house is dramatically cheaper per call at volume than a frontier model behind an API.
- Tone and house style. Output that must sound like your organisation consistently across thousands of generations, not approximately.
And the situations where it is the wrong answer, which are more common:
- The model needs to know something new. Facts belong in retrieval, not in weights, because facts change and retraining is slow. See RAG system development.
- The prompt has not been engineered properly yet. Most reported failures are prompt and context failures, addressed under prompt and context engineering.
- There are fewer than a few hundred quality examples and no realistic path to more.
- Nobody can articulate what 'better' means, which means no one can tell whether the tuning worked.
How we approach fine-tuning
Establish the baseline first
Before any training runs, we build the evaluation set and measure how far a well-engineered prompt with good retrieval already gets. That number is the bar. If tuning cannot clear it by a margin that justifies the ongoing cost of owning a custom model, we say so and stop, and you keep the baseline system and the evaluation set.
Curate the data properly
Fine-tuning quality is dominated by example quality, and a few hundred carefully curated examples routinely beat tens of thousands of scraped ones. We review for label consistency, deduplicate near-identical cases, strip personal data, hold out a genuine test split that never touches training, and look hard for the shortcuts a model could learn instead of the task. Where examples are scarce, synthetic data generation can extend the set, with contamination checks.
Choose the lightest method that works
Parameter-efficient methods such as LoRA are the default: cheaper to train, faster to iterate, trivial to version, and easy to unload if a newer base model arrives. Full fine-tunes are reserved for cases where the adapter genuinely cannot get there. Preference tuning is used where the requirement is about judgement rather than format.
Test for regression, not just improvement
A tuned model that is better at the target task and worse at everything adjacent is a liability. We test general capability alongside the specialised task, because catastrophic forgetting is real and usually only shows up in the cases nobody thought to check.
Plan for the base model to be replaced
A fine-tune is a commitment to a base model at a moment in time, and that moment lasts about six months. We keep the training pipeline reproducible so retuning on a newer base is an afternoon rather than a project, and we tell you the expected cadence before you commit.
What owning a fine-tuned model actually costs
The training run is rarely the expensive part. The ongoing costs are worth seeing before you decide:
- Re-tuning when the base model is superseded, typically once or twice a year.
- Maintaining the evaluation set as the task drifts, which is the same discipline any serious LLM system needs.
- Hosting, if the model is served yourself rather than through a provider's tuning API. Serving economics are covered under private LLM deployment and inference optimisation.
- Data curation as new edge cases appear and are folded back into training.
We put numbers against each of these before training begins, so the decision is made on total cost of ownership rather than on the appeal of having a model of your own.
How the engagement runs
The engagement is designed so the go or no-go decision happens early, while it is still cheap.
Baseline and feasibility
Evaluation set built, prompting and retrieval baseline measured, data volume and quality assessed, and a written recommendation on whether to proceed at all.
Data curation
Examples reviewed, cleaned, deduplicated and split. Personal data removed. Where volume is short, synthetic augmentation with contamination checks.
Training and iteration
Parameter-efficient tuning runs with hyperparameter sweeps, each candidate scored against the held-out set and the prompting baseline.
Regression and safety testing
General capability, refusal behaviour, bias and format compliance tested alongside the target task.
Deployment and handover
Serving path, versioning, rollback and the reproducible training pipeline handed over with the weights.
What you receive
The model, the evidence that it is better, and the pipeline to do it again on a newer base.
Tuned model weights
Adapters or full weights, versioned, with the exact training configuration that produced them.
Curated dataset
The cleaned, split and documented training and evaluation data, yours to reuse.
Baseline comparison
Scores for the prompted baseline against the tuned model on the held-out set, with the margin stated plainly.
Regression report
General capability, safety and format testing, including anything that got worse.
Reproducible training pipeline
Code and configuration to retune on a new base model without reconstructing the work.
Serving and rollback plan
How the model is deployed, versioned and reverted, with cost per thousand requests at your volume.
Is this the right engagement?
Worth being direct. LLM Fine-Tuning and Customisation is the wrong spend in some situations, and those are listed rather than buried.
Good fit if
- A well-engineered prompt with good retrieval has been measured and still misses the bar.
- You have, or can assemble, several hundred high quality examples of the task done correctly.
- Output format or house style must be consistent across very high volume.
- A smaller tuned model would cut unit cost or latency materially at your request volume.
- The task is narrow, stable and worth owning rather than renting.
Choose something else if
- The goal is for the model to know your documents. That is retrieval, not tuning.
- No baseline exists yet, so no one can tell whether tuning helped.
- Training examples are few, inconsistent, or nobody can adjudicate the correct answer.
- The requirement changes monthly, which makes a tuned model a maintenance burden rather than an asset.
Frequently asked questions
Marked up with FAQPage schema so these answers can surface directly in search results and inside AI assistant responses.
How much data do we need for LLM fine-tuning?
For format, tone and narrow task specialisation, several hundred well-curated examples are often enough, and quality dominates quantity beyond that. If a subject expert cannot agree on the correct output for a case, that case will not help the model learn, so curation effort is usually better spent than collection effort.
Is fine-tuning better than RAG?
They answer different questions. Retrieval supplies facts at request time and is right for knowledge that changes; fine-tuning changes behaviour, format and style. A large share of real systems use both, and choosing between them without measuring a prompting baseline is how budgets get spent on the wrong one.
Do we own the fine-tuned model?
You own the weights we produce and the curated dataset, transferred on final payment, along with the training pipeline. The licence of the underlying base model still governs what you can do with it, and we make that explicit in writing before training starts.
Will fine-tuning make the model worse at other things?
It can, and this is under-tested in most projects. We measure general capability before and after and report any regression rather than only the headline improvement, because a model that is excellent at one task and unreliable near it is difficult to deploy safely.
How often will we have to retrain?
Plan for once or twice a year, driven by base model releases rather than by your data changing. Because the pipeline is reproducible and the evaluation set is maintained, retuning is a short exercise rather than a repeat of the original engagement.
Often paired with this
Most clients combine two or three engagements from the Generative AI & LLM Engineering pillar. These are the ones that most often run immediately before or after.
Prompt and Context Engineering
Prompts treated as versioned, tested software: context budgets, regression suites and measured improvement rather than trial and error.
Read more →LLM Evaluation and Benchmarking
Golden sets, calibrated judges and a regression suite in CI, so quality is a number that moves rather than an opinion.
Read more →Synthetic Data Generation
Generated datasets for training and testing, with fidelity checks, privacy leakage testing and honest limits.
Read more →Is this the right engagement?
Tell us what you are trying to build. If a different service fits better, or if you do not need us at all, we will say so.