Most enterprise AI pilots die quietly. MIT’s NANDA study looked at 300 public deployments and concluded that the divide between the ones that worked and the ones that did not is not about model quality or regulation. It is about approach.
RAND reached a similar place from a different direction. After interviewing 65 experienced AI engineers, it concluded that failure is mostly a leadership problem: the wrong problem, the wrong metric, priorities that changed mid-build.
Both findings point at the same missing step. Almost nobody checks whether the thing they are about to automate is automatable before they start building. The decision is made in a meeting, the pilot is funded, and the diagnosis happens in month four, when the demo will not survive contact with production.
This article is that missing step, compressed into fifteen minutes and six questions.
95%
MIT NANDA
of enterprise generative AI pilots produce little or no measurable impact on profit and loss. Across 300 public deployments, the gap was a learning gap, not a model gap: generic tools do not adapt to the workflow they are dropped into.
80%+
RAND
of AI projects fail, roughly twice the rate of IT projects that do not involve AI. Of the 65 practitioners interviewed, 84% named leadership decisions, not technology, as the reason projects fail.
Readiness is usually assessed at the wrong level
Search for “AI readiness assessment” and you will find a dozen credible frameworks. Look closely at what they measure and they are almost all the same shape: strategy, data, technology, talent, governance, culture, scored for the organisation. That is the right instrument for a board deciding whether to commit to AI as a programme.
It is the wrong instrument for the decision in front of you, which is narrower and more concrete: should this invoice-matching process, this ticket triage queue, this month-end consolidation, be automated now?
An organisation can score well on every enterprise framework and still contain a process that will destroy any automation built on top of it. That is not a contradiction. It is the difference between climate and weather. The enterprise assessment tells you the climate. It cannot tell you whether it will rain on the specific thing you are building this quarter.
Harvard Business School’s AI Institute describes the mechanism precisely: when AI projects stall, “the primary obstacle to progress is rarely model quality or data availability, but rather the ‘last mile’ of transformation where technical capability must meet organizational design.” Their term for what accumulates over years of undocumented workarounds is process debt: fragmented and inconsistent workflows that AI then exposes rather than fixes.
So the six checks below are deliberately process-level. Each one maps to a dimension of the organisational model, so you can zoom out when you need to. But each one is answerable about a single workflow, by the people who run it, without a consultant in the room.
The mapping, if you want it up front. The same six dimensions the enterprise frameworks use, asked at the level where the decision actually gets made.
Process stability → Process & Operating Model
Data reachability → Data Readiness
Ownership → Strategy & Leadership
Connectivity → Technology & Infrastructure
Security and residency → Governance, Risk & Ethics
Change capacity → Talent & Culture
Check 1: Does the process behave the same way twice?
Automation does not create variance. It multiplies whatever variance already exists, then removes the human who used to absorb it.
Start with the exception list. It is the cheapest piece of work in this article and the one most often skipped.
Check 2: Can the data be reached, and is it right?
These are two separate questions and most teams answer only the first.
Reachable means the data lives in a system with an API or a supported export, not in an inbox, a shared drive, or one person’s spreadsheet. Right means the source system agrees with itself and with the number finance uses.
This is where RAND’s interviews land hardest. Thirty of fifty industry practitioners raised persistent data quality problems, and one of them put it bluntly:
“80 percent of AI is the dirty work of data engineering. You need good people doing the dirty work, otherwise their mistakes poison the algorithms.”
The failure is rarely dramatic. It is a field that is 94% populated, a currency that was stored as text in one system, or an account mapping that changed in April and nobody updated the report.
Check 3: Who owns it in month seven?
This is the check most often skipped and the most predictive of the ones on this list. Automations rarely die of technical failure. They die of ownership drift: the person who commissioned it changed role, the vendor considers launch to be the end of the engagement, IT considers it a business system, and the business considers it an IT system.
The pattern is consistent enough to be a rule: the project with two named owners survives, and the project with one enthusiastic sponsor does not.
Check 4: Can the systems actually talk?
Integration problems cause more automation failures than model problems, and they are boring enough that nobody puts them in the business case.
If you are already on a cloud platform, the vendor’s own readiness guidance for AI workloads is a useful technical checklist for the integration half. Treat it as an input, not as an answer to whether your process is ready.
If this is the constraint, it is worth knowing before the pilot rather than after. An IT system audit is usually faster and cheaper than discovering the same fact through a failed build.
Check 5: Where does the data go, and who can see it?
Every automation makes a data decision, whether or not anyone makes it consciously: which system processes the data, what is retained, who can read the logs, and which country the compute sits in.
The NIST AI Risk Management Framework is a useful structure for this, and it is not tied to one jurisdiction. It splits the work into four functions: govern, map, measure, manage. Governance is the one the other three depend on, because none of them are enforceable without it.
One distinction is worth settling before procurement rather than after. Regulation in most markets classifies AI by what a system is used for, not by the technology underneath it. Most internal workflow automation sits outside the high-risk categories. Some uses do not, and the difference changes your vendor shortlist rather than your business case. If you sell into a regulated market, that question belongs in week one, not in legal review after the pilot.
Check 6: Will the team actually use it?
Adoption is designed, not announced. The failure mode is consistent: the system works, the people it was built for keep doing it the old way, and the licence renews anyway.
We have seen both versions of this. The projects that hold are the ones where the person doing the work was in the room when the output format was decided, not the ones with the best launch email. Incentives matter more than training: if the new process does not make someone’s week measurably easier, it becomes a second way of working that sits next to the first one.
The technology is the easy half.
Score it: twelve points, four outcomes
Score each check 0, 1 or 2 using the table below. Be unkind to yourself; the number is only useful if it is true.
| Check | 0 points | 1 point | 2 points |
|---|---|---|---|
| 1. Process stability | Exceptions are the norm; no two runs match | Repeatable happy path, exceptions undocumented | Documented steps, named exception types, under ~20% |
| 2. Data reachability | Lives in inboxes and personal spreadsheets | In systems, but not reconcilable by hand | API or export, and one record reconciles end to end |
| 3. Ownership | No owner; “IT will handle it” | A team owns it, no named person | Named process owner and named technical owner, both internal |
| 4. Connectivity | Core system has no API and no supported export | Read-only access, or a manual step in the middle | Authenticated read and write, with a usable test environment |
| 5. Security & residency | Nobody can say where the data is processed | Architecture is known, paperwork is not | Retention, subprocessors and residency answered in writing |
| 6. Change capacity | Team hears about it at the demo | Team is informed and trained | Team helped define the output; one role gets measurably easier |
10-12
Ready
Pick the workflow and start small. One process, not a programme.
7-9
Ready after one fix
The lowest score tells you which fix, and it is almost always documentation or ownership. Neither requires a budget.
4-6
Foundations first
Automating now produces a working demo and a stalled pilot. Fix the two lowest checks, then rescore in a quarter.
0-3
Do not build yet
This is not a failure. It is the cheapest result in the article, because you found it for free.
One rule on top of the total
A zero anywhere caps the outcome. Eleven points with a zero on ownership is not a readiness score of eleven. It is a project that will be abandoned by someone who has left the company. The total describes the average state; the zero describes the failure mode.
The score above describes one process. If two or more checks came back low, the constraint is probably organisational rather than procedural, and that is a different and wider diagnosis. The six checks map one to one onto the six dimensions of our AI Readiness Assessment, which scores the whole organisation in about fifteen minutes.
The three disqualifiers
Three conditions override the score entirely. Any one of them and the project is not an AI project yet. It is a documentation project wearing an AI budget.
01
The process is undocumented
Not “in someone’s head” but genuinely not written down anywhere, including the exception handling. You cannot specify what you cannot describe, and no vendor can either, whatever the proposal says.
02
The knowledge is single-person
One person knows why the rule bends in March, why that customer is exempt, why the report runs on the 4th. Automate around them and you automate the documented half while the real logic walks out at retirement.
03
There is no named process owner
If the process belongs to a department rather than a person, nobody will notice when the output drifts, and nobody has the authority to change the rules when it does.
Each of these takes weeks to fix and costs almost nothing but attention. That is an unusually good trade, and it is why they belong before the build rather than during it.
Assistive and agentic are two different bars
The six checks describe readiness to automate a process. They do not describe readiness to hand a decision to a system, which is the harder question, and the one most current pitches are actually asking.
Assistive
AI drafts, summarises, extracts or suggests; a human decides and acts.
Needs checks 1, 2 and 4 to be solid. The rest matter, but a weak governance score does not stop a drafting tool.
Agentic
AI decides, routes and acts.
Needs all six, plus one condition the table cannot capture: reversibility. If the action cannot be undone cheaply, a human gate stays in the loop no matter how good the model is.
Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027 on cost, unclear value or inadequate risk controls, and notes that “many use cases positioned as agentic today don’t require agentic implementations.”
Its term for vendors who rebrand assistants, RPA and chatbots without the underlying capability is agent washing. A four-step process with a stable rule set is a workflow, not an agent. Calling it one adds risk without adding value.
What to fix first, in order
If the score came back low, the sequence matters more than the list.
Ownership
Cheapest to fix, highest leverage. One conversation, one name in a document.
Documentation
Write the happy path and the exception list. Two days with the person who does the work, not a consultant.
Data
Reconcile one record end to end. If it cannot be done, that is the project, not the automation.
Connectivity
Find out now whether the ERP can be written to, because the answer changes the scope rather than the timeline.
Security and residency
A gate rather than a step. It decides how you build, so it should not delay the diagnosis, and it should not be left until procurement.
Change capacity
Runs in parallel from day one. It is not a phase at the end, which is the mistake most projects make when they treat training as a deliverable.
When the right answer is “not yet”
Readiness work has an unglamorous property that makes it easy to skip: the reward for doing it is often deciding not to spend money.
A process with 40% exceptions, an ERP nobody can write to, and no named owner will not be fixed by a better model. It will be fixed by two weeks of documentation and one conversation about accountability. Both are cheaper than the pilot, and both are things you can start on Monday without a budget or a vendor.
The most expensive automation project is not the one that fails technically. It is the one that succeeds technically, runs quietly, and changes nothing, while the licence renews and the team works around it. That outcome is decided long before the build, in exactly the six places above.
Where this fits in the wider picture: these six checks are the procedural layer under our six-dimension framework for measuring digital maturity. A low score here usually tells you which of the four digital maturity stages the company is genuinely in, rather than the one it has been describing itself in. If the constraint turns out to be organisational rather than procedural, the AI Readiness Assessment scores the same six dimensions at company level.
Already know which workflow you want to automate?
That is where our AI automation and optimization engagements start: testing the scope against these six checks before anything gets built.
The first conversation is a scope test, not a pitch. If the answer is “not yet”, you will hear it in week one.
