Is Your Data Actually Ready? A Practical Audit

Part 3 of 5 — AI for Operations & Supply Chain

Lesson 2 argued that you probably have enough data to start using AI. Now let us put that claim to the test.

Instead of guessing whether your data is good enough, run a structured audit. Five questions. Ten minutes of honest reflection. And a decision that saves you months of wasted effort.

The five-question audit

These are the same questions practitioners ask before committing to a project. They are not academic. Each one filters out a category of failure mode. Work through them in order — if a question fails, the next ones do not matter yet.

Question 1: Do you have historical records?

Not future projections, not plans, not what the ERP says should happen. What actually happened. Real dates. Real quantities. Real outcomes.

Look at your purchase order data. Do you have a list of POs from the past year with actual delivery dates? Or does the system only store the promised date and the actual arrives in an email or a phone call? If the actual never makes it back into the system, you do not have historical records. You have a plan that runs alongside reality.

Look at your production data. Do you have job-level output and yield from the past 12 months? Or is production tracked in aggregate and individual job results are lost? You need the grain of data to match the grain of the prediction. If you want to predict yield by job, you need job-level history.

If you answered no, the fix is straightforward: start capturing what actually happens. You cannot train a model on data you do not have. But the good news is that most ops teams have at least one historical dataset that passes this test.

Question 2: Do you capture the outcome?

This is the filter that eliminates most teams. It is easy to track what you decided. It is much harder to track what happened as a result of that decision.

You chose Supplier A for PO-88921. Did it arrive on time? Was the quality acceptable? What was the total landed cost including freight and duties? If those outcomes are never recorded against the original decision, the system has no answer key to learn from.

Here is how to spot the gap. Open your most-used tracking spreadsheet. Look at the columns. Do they describe the action (PO placed, job scheduled, supplier selected)? Or do they include the result (delivered on time, yield of 94%, zero defects)? If the columns stop at the action, you have an outcome gap.

Closing it is the highest-leverage improvement you can make. It does not require new technology. It requires adding a few fields to your existing process. When a PO arrives, log the actual delivery date and any variance. When a production job completes, log the actual yield and rework hours. When a supplier issue resolves, log the resolution time.

Most teams underestimate how much value lives in outcome data. Decisions are cheap. Outcomes are the teacher.

Question 3: Is the data consistent?

Consistency does not mean perfect. It means that the same field uses the same format, recorded the same way, every time. If one person logs supplier names as "Matsui Heavy Industries," another as "M. Matsui Co," and a third as "Matsui-HI," the system sees three different suppliers.

Check your data for three inconsistency patterns:

Format drift: Dates in different formats (MM/DD/YYYY versus YYYY-MM-DD). Units that change (pounds versus kilograms). Status codes that evolve (one person uses "Held," another uses "On Hold," another uses "Hold").

Manual entry variation: Free-text fields filled by different people. Description fields that capture different levels of detail depending on who types them. Reason codes that are sometimes chosen from a dropdown and sometimes typed by hand.

System fragmentation: Two departments tracking the same information in different systems. Procurement has supplier data in the ERP. Quality has supplier performance in a spreadsheet. Maintenance has equipment data in a log. None of these talk to each other.

Inconsistency is solvable. It usually takes a week of data cleaning and a commitment to standardize going forward. But it needs to happen before you train a model, because the model will treat inconsistent data as noise.

Question 4: Do you have enough volume?

"Enough" depends on what you are trying to predict. Simple binary outcomes (will this delivery be late?) need fewer examples than complex ones (what will demand be across 2,000 SKUs next month?).

Here is a practical rule of thumb. For a yes/no prediction with clear history, a few hundred examples often work. For a prediction that involves many variables or multiple categories, you need a few thousand. For a continuous prediction (exact demand quantity, exact time-to-repair), you need even more.

Do not panic if you are on the low side. AI systems are not magic, but they are also not as data-hungry as their reputation suggests. If you have 500 POs from the past year with delivery outcomes, that is enough to start learning delay patterns. If you have 2,000 production jobs with yield data, that is enough to start learning quality patterns.

Question 5: Is the data recent enough?

Patterns change. If your processes, suppliers, products, or customers changed significantly in the past 12 to 18 months, data from before that change is less relevant. Not useless, but less weighted.

Think about it this way. If you switched from domestic suppliers to global suppliers two years ago, delay patterns from before the switch do not apply to your current operations. The data exists. It is clean. But it describes a different world.

If your data is mostly old, you have two options: wait until you accumulate enough current data, or use the old data to build a first model while accepting that it will be less accurate until fresh data replaces it. The second option is usually better, because a working model on old data teaches you what to capture going forward.

Scoring your data honestly

Once you run through the five questions, you need a sense of whether the answer is "go" or "not yet." A scorecard helps.

Rate each dimension from 1 to 5. Be honest, not aspirational. The point is not to justify a decision you already made. The point is to avoid building on data that will not support it.

If your average score is 3 or above across all five dimensions, you are ready to experiment. Pick a project, build a model, see what happens.

If your average is around 2, pick one dimension to clean up first. Usually it is outcome coverage, because it is the easiest to improve and the most impactful. Add outcome fields to your tracking process. Six months of clean outcome data is enough to cross the threshold.

If your average is 1, do not start with AI. Start with data capture. Define what you want to track, standardize how it is recorded, and build the habit of closing the outcome loop. AI is waiting. It will still be there when your data is ready.

Three common data traps

Even experienced teams fall into patterns that look productive but are not. Here are the three most common.

The volume trap. "We generate millions of data points a day from our production equipment." That sounds impressive, until you realize that none of those data points are linked to quality outcomes. The system records temperature, pressure, and cycle time every second. It does not record whether the batch that was produced during that time passed inspection. Millions of data points with no answer key is not a learning resource. It is a log file.

The freshness trap. "Our historical dataset is complete and accurate for the past five years." That sounds great, until you discover that your supplier base changed dramatically 18 months ago, your product mix shifted two years ago, and your quality standards were tightened a year ago. Four years of data describe a different operation than the one you run today. The patterns the system learns from stale data will not match current reality.

The silo trap. "Procurement has good data. Quality has good data. Maintenance has good data." Individual data sources might be solid. But the insights that matter live at the intersection. Delivery delays that depend on both supplier performance and production schedule. Quality failures that depend on both raw material source and machine condition. If your data sources do not talk to each other, the system learns from half the picture.

Prioritizing your first project

Once you know your data is ready, the next question is: where to start? Most operations teams have multiple ideas. Not all of them are equally good starting points.

Evaluate each candidate project on two axes: business value and data readiness.

Business value is the impact if the project works. How much cost, time, or risk does it address? Is this a problem the team deals with weekly or yearly?

Data readiness is how well your audit scored for the specific data needed by this project. Not your overall data quality. The data relevant to this specific prediction.

Start here (high value, strong data): These are the obvious candidates. Delivery delay prediction using PO history with outcomes. Inventory optimization using order history and demand data. Production yield improvement using job-level quality records. If you have a project that lands here, begin with it.

Invest first in data (high value, weak data): These are important problems that need better data before AI can help. Supplier financial risk prediction requires supplier financial data most teams do not currently track. Customer demand forecasting for new products has no historical outcome to learn from. These projects are worth pursuing, but the first step is data collection, not model building.

Nice to have (low value, strong data): These are interesting but not urgent. Alternative route scoring when your logistics are already running well. Cosmetic data enrichment. Use these only if you have capacity after the high-value projects.

Deprioritize (low value, weak data): These are not worth the investment. They address problems that do not move the needle with data that would take effort to prepare.

Recommended first projects

If you are still deciding, here are proven starting points by function. Each one uses data you likely already have, solves a real problem, and produces value measurable within months.

The common thread across all of these: they are predictions about things that happen repeatedly, with historical outcomes recorded. That is the recipe for a first AI project that works.

Key takeaways

• Run the five-question audit before committing to any AI project: historical records, outcome capture, consistency, volume, and recency. Each question filters out a specific failure mode.
• The most common data gap is outcome coverage. Teams track decisions beautifully but rarely record what happened as a result. Closing this loop is the highest-leverage improvement.
• Rate your data honestly on a 1-to-5 scale. Average 3 or above means ready to experiment. Average 2 means clean one dimension first. Average 1 means focus on data capture, not AI.
• Beware three traps: volume without outcomes, data that is stale relative to current processes, and siloed systems that prevent cross-functional insight.
• Prioritize projects on two axes: business value and data readiness. Start where both are high. Invest in data collection where value is high but data is weak.
• The best first project predicts something that happens repeatedly with historical outcomes already recorded.

Next lesson: Choosing Your First AI Project.