Data Quality

Data quality describes how fit your information is for the use you want to put it to. The important word is fit, because quality cannot be assessed in the abstract.

The same customer list can be entirely adequate for sending statements and entirely inadequate for analysing churn. Neither assessment is wrong. They are answers to different questions.

The wrong target

The instinct is to aim for clean data. That is a project with no end, and chasing it is how businesses spend two years on a data programme and never ship anything.

The right target is clean enough for this specific decision. Which means you name the decision first, then assess the data against it, and fix only what that decision needs.

The six things to check

Completeness. What proportion of records have the fields you need? Not the record count, the field-level fill rate. A CRM with 12,000 contacts and industry populated on 900 cannot answer anything by industry.

Consistency. Is the same thing recorded the same way? "Ltd", "Limited", "LTD". Dates stored as text. A status field with nineteen variations of "closed". This fails most often and is the most tedious to fix.

Accuracy. Does it match reality? Hard to test in bulk. The practical method is to take thirty records and verify them by hand against something authoritative. Thirty is enough to tell a 2 per cent problem from a 30 per cent one.

Timeliness. How current is it? A contact database where a third of people have left their companies is not wrong exactly. It is expired.

Uniqueness. Duplicates inflate counts, split histories and corrupt any per-customer analysis. Every business has more than it thinks.

Lineage. Do you know where each field came from and when? This matters most when two systems disagree and somebody has to decide which to believe.

What good enough looks like

For a directional answer, such as which service line is underpriced: around 70 per cent fill rate on required fields, consistent categories after mapping, duplicates under 5 per cent.

For an operational answer feeding a system that acts: 95 per cent or better, enforced consistency, duplicates resolved, and a defined rule for what happens when data is missing.

For anything affecting individuals, such as pricing, credit or hiring: the above plus documented lineage, a human review step, and the ability to explain a specific decision afterwards.

The part that gets skipped

Cleaning an archive is a snapshot. If nothing changes about how records are captured going forward, the same problem returns within a couple of years.

The fix is usually small: constrain a free-text field to a dropdown, make one field mandatory, add a single field to something people already complete. It costs very little and it is forgotten in most projects, which is why businesses clean the same data twice.