Legacy Data Migration and Modernisation
Moving data off systems that are older than most of the people using them, with reconciliation you can put in front of an auditor and a rollback that has actually been tested rather than merely documented.
Migrations fail on the parts nobody documented: the field that means something different after 2016, the batch job that quietly corrects records overnight, the report that reads a table directly rather than through the interface everyone believed was the only access path.
Legacy data migration is the movement of data from ageing systems to a modern platform, including profiling, transformation and mapping, reconciliation, and a cutover strategy. Modernisation extends it to restructuring the data so it serves current requirements rather than reproducing the constraints of the system it came from.
What legacy systems actually contain
Profiling a legacy source reliably contradicts its documentation, and the discrepancies determine the shape of the migration:
- Fields repurposed over time. A code that meant one thing until a system change and something else afterwards, with no flag to distinguish the eras.
- Free text in structured fields. Notes, exceptions and workarounds typed into whatever field had room.
- Undocumented business rules. Implemented in stored procedures, overnight jobs or a spreadsheet maintained by one person.
- Referential integrity that was never enforced. Orphan records, broken keys and duplicates that the application layer quietly tolerates.
- Encoding and date archaeology. Mixed character sets, two-digit years, ambiguous formats and placeholder dates standing in for null.
- Access paths nobody listed. Reports and integrations reading tables directly, which surface the morning after cutover.
Each of these is a decision for the business rather than for us, and every one of them is recorded with the reasoning, because in twelve months somebody will ask why a value looks the way it does.
How we run a migration
Profile before mapping
Actual values, actual distributions, actual integrity — measured, not read from a data dictionary. The gap between documentation and reality is the true scope of the project and it is better discovered in week two than during a cutover weekend.
Map with business sign-off
Every field mapped source to target with transformation rules, default handling and explicit decisions on ambiguous cases, signed off by the people who own the data. Unsigned mappings become disputes at reconciliation, when there is no time left to resolve them.
Decide what does not come across
Not all history is worth migrating. Retention obligations, actual usage and cost decide what moves, what is archived in a queryable form and what is retired. Migrating twenty years of data that nobody has queried since 2013 is a common and expensive default.
Reconcile at several levels
Row counts, control totals, distributions and sampled record-level comparison, with every discrepancy explained rather than tolerated. A reconciliation pack that a business owner and an auditor can both follow is the deliverable that makes a migration defensible.
Rehearse the cutover repeatedly
Full dress rehearsals against production-scale data, timed, until the runbook is boring. The rehearsal is where you discover that the load takes eleven hours rather than four, and that is a much better place to discover it than a cutover weekend.
Run in parallel, then decommission deliberately
Both systems running with outputs compared for an agreed period, so differences are found in daylight. Decommissioning happens only after parallel running is clean and every access path has been accounted for.
Migrate the defect or fix it?
A migration is the one moment when fixing data is cheap, because everything is being touched anyway. We identify quality defects during profiling and propose which to fix in flight and which to carry across unchanged, since fixing everything expands scope without limit. See data quality.
Cutover strategies
| Strategy | How it works | Suits |
|---|---|---|
| Big bang | Single cutover, legacy retired immediately | Smaller datasets, tolerable downtime window |
| Phased by domain | One domain or entity at a time | Large estates where the domains are genuinely separable |
| Parallel run | Both systems live, outputs compared | Regulated or financially sensitive data |
| Trickle with sync | Continuous replication then switch | Minimal downtime requirement, higher complexity |
The choice follows your tolerance for downtime and for risk, not our preference. We recommend one, cost the alternatives, and make the rollback point explicit in each case — including the last moment at which rolling back is still possible.
How the engagement runs
Cutover is rehearsed at production scale until the runbook is uneventful.
Profiling and discovery
Real state of the legacy data measured; undocumented rules and access paths found and catalogued.
Mapping and scope
Field-level mapping with business sign-off; retention decisions on what migrates, archives or retires.
Build and rehearse
Migration built and run repeatedly at production scale, with reconciliation tightened each pass.
Cutover
Executed to the rehearsed runbook, with reconciliation signed off and rollback available throughout.
Parallel and decommission
Outputs compared through the agreed period, then legacy retired with access paths accounted for.
What you receive
Data in the new system, and a reconciliation pack that stands up to scrutiny.
Profiling and discovery report
Actual state of the legacy data, undocumented rules and every access path found.
Signed mapping specification
Field-level mapping, transformation rules and ambiguous-case decisions with business sign-off.
Migration tooling
Repeatable, rerunnable migration code, yours to keep and to reuse.
Reconciliation pack
Counts, control totals, distributions and sampled comparisons, with discrepancies explained.
Cutover runbook
Rehearsed, timed sequence with the rollback procedure and its last valid moment.
Decommissioning checklist
Access paths, integrations and reports accounted for before anything is switched off.
Is this the right engagement?
Worth being direct. Legacy Data Migration and Modernisation is the wrong spend in some situations, and those are listed rather than buried.
Good fit if
- A legacy platform is being retired or is out of support.
- Data must move to a modern platform without losing history or auditability.
- The migration must be defensible to auditors or a regulator.
- Undocumented business logic is known to exist and nobody has mapped it.
- A previous migration attempt stalled or was rolled back.
Choose something else if
- The target platform is not chosen. Start with data strategy and architecture.
- A single table export and import genuinely covers it.
- The business cannot commit anyone to sign off the mapping, which makes reconciliation undecidable.
- The real need is quality improvement in place rather than a move. See data quality.
Frequently asked questions
Marked up with FAQPage schema so these answers can surface directly in search results and inside AI assistant responses.
How long does a legacy data migration take?
Typically ten to twenty weeks for a substantial system, with the variance driven by how much of the business logic is undocumented rather than by data volume. Profiling in the first three weeks usually settles the estimate, because that is when the difference between the documentation and the reality becomes visible.
Should we migrate all our history?
Usually not. Retention obligations, actual query patterns and cost should decide. A common outcome is migrating recent years into the live system and placing older data in cheaper queryable archive storage, which satisfies both the auditors and the finance team.
How do you prove the migration was correct?
With a reconciliation pack at several levels: row counts, control totals on financial and quantity fields, distribution comparisons and sampled record-level checks, with every discrepancy explained rather than waved through. It is written to be read by a business owner and an auditor, not only by engineers.
What if something goes wrong during cutover?
You roll back, using a procedure that has been tested rather than written. The runbook states the last moment at which rollback remains possible and what the decision criteria are, and the rehearsals are what make that a real option instead of a paragraph in a plan.
Can you fix data quality during the migration?
Yes, and it is the cheapest moment to do it since every record is being handled anyway. We identify defects during profiling and propose which to fix in flight and which to carry across unchanged, because an unbounded quality effort inside a migration is how migrations lose their end date.
Often paired with this
Most clients combine two or three engagements from the Data Engineering and AI Readiness pillar. These are the ones that most often run immediately before or after.
Data Cleaning, Quality and Enrichment
Profiling, deduplication, standardisation and enrichment, with automated checks that stop quality decaying again.
Read more →Data Lakehouse and Warehouse Build
A governed lakehouse or warehouse with a modelled core layer, access control and predictable cost.
Read more →Data Strategy and Architecture
A target data architecture and a sequenced roadmap, derived from the decisions you need to make rather than from a vendor shortlist.
Read more →Is this the right engagement?
Tell us what you are trying to build. If a different service fits better, or if you do not need us at all, we will say so.