Choosing Your First AI Project

Part 4 of 5 — AI for Operations & Supply Chain

Lessons 1 through 3 built a foundation. You understand the difference between rules and AI. You know how systems learn from data. You have audited your data and identified gaps.

Now comes the decision that determines whether your first AI project delivers value or becomes another expensive experiment: choosing the right problem.

This is harder than it sounds. Most teams approach it backward. They start with the technology and try to find a problem that fits. That produces solutions looking for a reason to exist.

Start with the problem instead. Narrow it until it fits a clear prediction. Set success criteria before building. And scope it small enough that you can prove value in weeks, not quarters.

Three kinds of AI projects

Before choosing a problem, recognize that every AI project in operations falls into one of three categories. They share the same underlying technology but produce very different outputs, require different data, and demand different ways of working with the people who use them.

Predict: estimate something before it happens. This is the most common starting point. Predict whether a supplier delivery will be late. Predict the yield of a production run. Predict how long a maintenance repair will take. Predict which customer will place an order next month.

Prediction projects are the most straightforward. The historical data has clear outcomes. You can evaluate accuracy by comparing predictions to what actually happened. And the output is a number or a probability that fits naturally into existing decision-making.

Optimize: find the best arrangement among alternatives. Allocate inventory across 12 warehouses to minimize stockouts and carrying cost. Schedule production jobs across 8 machines to balance throughput, setup time, and energy use. Route deliveries across 80 stops with time windows and vehicle constraints.

Optimization projects require historical data, but they also require a clear objective function. What is "best"? On-time delivery? Lowest cost? Highest throughput? And what are the constraints? Machine capacity? Labor availability? Contract obligations? The system can only optimize what you tell it to optimize.

Detect: find unusual patterns in current data. Flag a production run that shows quality characteristics different from the norm. Detect equipment degradation from sensor readings before failure occurs. Identify supplier financial distress from patterns in financial filings and news mentions. Find fraudulent invoices from anomalies in payment patterns.

Detection projects are powerful because they surface problems you did not know existed. They are also the hardest to evaluate, because you do not know how many anomalies you missed. A quiet month might mean everything is fine, or it might mean the system is blind to a new pattern.

For a first project, prediction is usually the safest choice. It is easy to evaluate, easy to explain, and easy to connect to business value. Start there unless you have a strong reason not to.

The problem statement template

Once you know which category your project falls into, write a problem statement. This is not corporate bureaucracy. It is a thinking tool. Teams that skip it build systems that are clever, well-engineered, and useless.

Every good problem statement fills in four blanks:

Every [time period], the system will evaluate each [item]. This defines the frequency and scope. Does the system run once a week on open POs? Once per shift on active production jobs? Daily on all supplier records? Frequency matters because it determines how often you need fresh data and how much attention the output demands.

To predict, the system will estimate [outcome]. This defines the prediction. Will it estimate whether delivery is late? What the yield will be? How long repair will take? The outcome must be something you can measure and something that has been measured historically. If you have never recorded it, you cannot predict it.

Using data, the system will consider [inputs]. This defines the features. Supplier history, order size, time of year, material type, machine condition, operator shift. List the inputs explicitly because the list reveals gaps you did not notice. If the most important predictor is not in your data, the problem statement surfaces that fact before you waste time building.

So that, [business outcome]. This defines the value. Why does this prediction matter? Because the planner can arrange coverage before the line stops. Because the buyer can negotiate from a position of information. Because the maintenance team can order parts before the machine goes down.

If you cannot fill in all four blanks, you do not have a project yet. You have a direction. That is fine. Fill in the blanks before you spend money.

What makes a good first problem

Not every problem is equally suited to be a first project. Some are inherently easier to solve, faster to validate, and more likely to produce visible value. The difference between a good first problem and a bad one usually comes down to five characteristics.

Narrow scope. The problem statement describes one prediction, not a family of related predictions. Predict delivery delays for purchased parts. Not predict delivery delays, quality failures, cost overruns, and schedule disruptions all at once. Start narrow. Prove it works. Then expand.

Repeated events. The prediction applies to something that happens frequently enough to generate meaningful history. If you place 15 POs per week, you have hundreds of examples per year. If you run a special production batch once per quarter, you have three data points. The system cannot learn from three examples. It can learn from three hundred.

Recorded outcomes. For past instances of this event, you know what happened. Not "we think so." Not "someone probably knows." The outcome is recorded in a system you can query. If the outcome is missing, you cannot train a model. Lesson 3 covered how to spot and close this gap.

High impact. The prediction matters. Not in an academic sense. In a "if this works, it saves money, prevents downtime, or reduces risk that someone cares about" sense. If the answer to "so what?" is "we will know sooner," that is not enough. Sooner than what, and what will you do with that extra time?

Stable process. The process has not changed dramatically during the period your data covers. If you restructured the supplier base, changed quality standards, or switched ERP systems in the middle of your data, the early records describe a different process. The system will learn from both and produce a confused model.

Scope creep: the most common reason first projects fail

This is worth its own section because it is the single most common reason that first AI projects deliver nothing. Teams start with a good problem and make it worse by adding scope.

The pattern always plays out the same way. The team starts with "predict delivery delays for POs." Then someone says, "We should also predict quality failures." Then, "And while we are at it, we should predict which suppliers will raise prices." And before the project is three weeks old, it is trying to predict everything about the supply chain.

The scope expanded because everyone wants value. But the expanded scope makes the project harder to build, harder to evaluate, and slower to deliver. The original narrow problem was solvable in months. The expanded problem is a year away, if it is solvable at all.

The rule for first projects is ruthless: one prediction, one outcome, one success metric. Everything else goes in the backlog for after the first version works.

And here is how to handle the pressure to expand scope, because someone will ask you to include more. Have a plan for it. Show them the three-phase roadmap. Phase 1 proves the concept on a narrow slice. Phase 2 adds related predictions. Phase 3 scales. The ambition does not shrink the scope. It sequences it.

Setting success criteria

Before building the model, define what success looks like. Not vague goals like "improve accuracy" or "reduce delays." Specific, measurable thresholds that the team agrees to before the project starts.

The accuracy threshold. How accurate does the prediction need to be for someone to actually use it? If the system flags POs that will be late, and it is right 60% of the time, the planner cannot trust it. Every false flag erodes confidence. Most ops teams need 80% to 90% accuracy before they change behavior based on a prediction. Set the threshold at the point where the output is actionable, not at the point where the model looks good on paper.

The lead time. How far in advance does the prediction need to be available? If the system correctly predicts a delivery delay two hours before the delivery arrives, the prediction is accurate but useless. You need the prediction three days before, so the planner has time to arrange coverage. Define the minimum lead time that enables action. Predictions that arrive too late are analysis, not foresight.

The actionability rate. Of the problems the system flags, what percentage have an available response? If the system flags 20 POs at risk but only 2 have alternative suppliers available, the other 18 flags create anxiety without options. Over time, unactionable alerts become background noise. Set a target for how many flagged items must have a viable response.

These three criteria together define a project that is accurate enough to trust, early enough to act, and useful enough to matter. If a candidate problem cannot meet all three, reconsider whether it is the right first project.

Who needs to be involved

AI projects are not technical projects. They are organizational projects that happen to use technology. The model is the easiest part. Getting the people aligned is the hard part.

The sponsor. This is the operations manager or director who owns the problem. They define what the problem is, decide what success looks like, and remove organizational barriers. Without a sponsor who has authority, the project drifts. It becomes a nice-to-have that loses priority whenever something urgent happens. And something urgent always happens.

The data source. These are the people who own the historical records. Procurement for PO data. Quality for inspection data. Maintenance for work order data. Production for output data. They provide the raw material the model learns from. They also help interpret the output. If the model predicts a delay for a supplier the procurement team has worked with for years, their judgment about whether that prediction makes sense is invaluable feedback.

The builder. This can be an internal data analyst, an external partner, or an AI tool you configure. They prepare the data, train the model, evaluate accuracy, and maintain the system. The builder does not need to be an AI expert. They need to be methodical, honest about limitations, and comfortable working with non-perfect data.

The user. This is the person who will act on the prediction. The planner who arranges coverage. The buyer who negotiates with the supplier. The production supervisor who adjusts the schedule. The user is the most important person in the project and the least involved in planning it. Involve them early. Ask what they need the output to look like. Ask what would make them trust it. Ask what would make them ignore it.

If any of these four roles is unfilled, the project is at risk. A model without a sponsor has no authority. Data without a source is unreliable. A builder without a user builds something nobody uses.

A practical timeline

Twelve weeks is enough to take a first AI project from problem statement to real-world pilot. Not a polished product. A pilot that produces predictions, gets used by real people, and generates evidence of value.

Weeks 1 to 4: Frame. Pick the problem. Fill in the problem statement template. Score your data quality against the five dimensions from lesson 3. Identify the four roles. Agree on success metrics. This phase takes longer than most teams expect and shorter than most teams give it. Do not rush it. A poorly framed problem cannot be fixed by building faster.

Weeks 5 to 8: Build. Prepare the historical data. Clean inconsistencies. Link inputs to outcomes. Train the first model. Test it against known outcomes. Review the results with the user. Adjust based on feedback. The first model will not be perfect. It will be a starting point. The goal is not to build a model that is ready for production. The goal is to build a model that is ready for evaluation.

Weeks 9 to 12: Run. Deploy the model in pilot mode. Generate real predictions on real data. Let the user act on them. Track performance against the success criteria defined in the framing phase. Document results. Decide whether to scale, refine, or stop.

At the end of twelve weeks, you will know whether the project is worth continuing. That is a better outcome than spending a year building something that nobody uses.

Key takeaways

• Every AI project in operations falls into one of three categories: predict an outcome, optimize a process, or detect an anomaly. Prediction is usually the safest first project.
• Write a problem statement before building. Define frequency, outcome, inputs, and business value. If you cannot fill in the template, you are not ready.
• A good first problem is narrow, repeated, has recorded outcomes, has high impact, and operates in a stable process. If it does not meet these criteria, it is not a bad problem. It is not a good first one.
• Scope creep is the most common reason first projects fail. One prediction, one outcome, one success metric. Everything else waits for version two.
• Set three success criteria before starting: accuracy threshold (actionable, not impressive), lead time (early enough to act), and actionability rate (flags that come with options).
• Four roles are required: sponsor, data source, builder, user. Missing any one puts the project at risk.
• Twelve weeks is enough to go from problem statement to pilot. Frame for four weeks, build for four, run for four.

Next lesson: Building and Testing Your First Model.