Bad Data Blocking AI? Use AI to Fix the Data First

Every AI strategy deck has the same slide: "our data isn't ready." It's the most expensive sentence in the enterprise right now — and almost everyone is drawing the wrong conclusion from it.

The standard advice is to fix the data first, then do AI. That sequencing made sense in 2020. It doesn't anymore, because the tools that fix data have changed: the fastest way to build an AI-ready data foundation is now AI itself. If bad data is blocking your AI ambitions, your first AI project shouldn't be a chatbot — it should be pointed at the data.

The Problem: "Fix the Data First" Is a Sentence, Not a Plan

Here's the loop we see at nearly every mid-market company. Leadership wants AI. A pilot gets scoped. Someone discovers the customer table has duplicates, the product hierarchy exists in four conflicting versions, and the pipeline feeding the warehouse was written by someone who left in 2021 and documented nothing. The verdict comes back: "we need to sort our data out first."

Then nothing happens, for three predictable reasons:

The remediation estimate is unfundable. Done the traditional way — analysts profiling tables manually, workshops to agree definitions, engineers reverse-engineering pipelines — a foundation programme for a typical mid-market estate is 12–24 months and a seven-figure sum, with no visible business outcome until the end. No board funds that against a backdrop of "but other companies are shipping AI now."

The work is hostage to tribal knowledge. The only person who knows why order_status = 7 means "cancelled but refund pending" is busy. Documentation projects die waiting for interviews that never get scheduled.

Manual data quality doesn't scale. A human can write good validation rules for 20 tables. Nobody writes them for 2,000. So coverage stays thin, trust stays low, and every new AI proposal re-triggers the same objection.

The result is strategic paralysis: too much data debt to start, no credible plan to clear it. Meanwhile the gap between you and competitors who solved this compounds quarterly.

The Solution: Point the AI at the Data, Not Just at the Customers

Large language models turn out to be exceptionally good at exactly the work that made "fix the data first" a two-year sentence. Not as magic — as labour. The playbook:

1. AI-assisted profiling and documentation. An LLM given schema DDL, sample rows and query logs can draft table and column descriptions, flag likely PII, and infer business meaning from naming patterns and values — in hours, across an entire estate. Humans review and correct rather than author from scratch. That's the difference between a data catalogue that takes 12 months and one that takes six weeks. Tools like dbt's documentation workflow or an LLM pass over INFORMATION_SCHEMA get you 80% of the way; the remaining 20% of review is where your experts' time actually belongs.

2. Lineage reconstruction from code. Feed the model your SQL, stored procedures and orchestration configs and it can map which tables feed which, where transformations happen, and which pipelines are dead. This is tedious, mechanical reading — precisely what LLMs do cheaply and people do badly. (AWS Glue Data Quality and similar services now generate rule recommendations from profiling automatically — the pattern is going mainstream across every major platform.)

3. Generated data quality tests at scale. Instead of hand-writing validation rules for your top 20 tables, generate candidate rules (nullability, ranges, referential integrity, freshness) for all 2,000 from profiling output, then have engineers approve or amend. Coverage goes from thin-and-manual to broad-and-reviewed. dbt tests or Great Expectations suites are a natural target format.

4. Entity resolution with embeddings. Duplicate customers — "ACME Ltd", "Acme Limited", "ACME (UK)" — resist exact-match logic forever. Embedding-based matching plus an LLM adjudication pass resolves the fuzzy middle that rules can't, with humans confirming only the genuinely ambiguous cases.

One non-negotiable trade-off: AI drafts, humans decide. Never let a model silently mutate production data or auto-merge records. The pattern that works is generate → review → apply — the model produces candidates and evidence, a person approves, an auditable pipeline executes. This keeps hallucination risk out of your golden records while still capturing the 10x speed gain. This is the same governance-first approach we describe in our data platform assessment methodology.

Real-World Example: 14 Months of Foundation Work in One Quarter

A retail client came to us wanting a customer-facing AI assistant. Discovery showed the classic blocker: ~1.2 million customer records with an estimated 18% duplication rate, 400+ undocumented tables, and no agreed definition of "active customer" across three business units.

The traditional remediation quote they'd received elsewhere: 14 months. Instead, we ran the AI-on-data playbook for one quarter:

Weeks 1–3: LLM pass over schemas and query logs produced draft documentation for all 400 tables; domain experts corrected ~22% of the drafts in review sessions — about 15 hours of expert time in total, versus the hundreds of interview hours the manual plan assumed.

Weeks 4–8: embedding-based entity resolution collapsed 214,000 duplicate customer records into 96,000 golden records, with human review only on the 8% of matches below the confidence threshold. Generated dbt test suites took quality-rule coverage from 40 tables to 1,100.

Weeks 9–12: with a documented estate and a deduplicated customer spine, the original AI assistant pilot started — on foundations that hadn't existed 90 days earlier.

Total cost: roughly a fifth of the manual remediation quote. And the client's team learned the pattern, so the next domain (products) took them six weeks on their own.

Actionable Takeaways

• Reframe the sequencing. "Fix data first, then AI" and "use AI first" are not opposites — using AI on the data is doing both at once. Pitch it internally as your first AI project, not a data project; it funds differently.
• Start with documentation and profiling — it's the highest-value, lowest-risk application: wrong drafts get corrected in review, and nothing touches production data.
• Generate tests, don't write them. Broad reviewed coverage beats deep manual coverage. Aim for candidate rules on everything, human approval on all of them.
• Hold the line on generate → review → apply. No silent auto-fixes, no unaudited merges. The speed comes from drafting, not from removing humans.
• Measure in expert-hours saved, not just duration — 15 hours of expert review replacing hundreds of interview hours is the number that convinces a sceptical CFO.

The foundation you build this way is also what determines your platform costs later — see our guide to a modern data platform architecture and our Snowflake cost optimisation guide.