The pilot worked. A year later nothing is in production. The failure is almost never model quality. It is data nobody owns, permissions nobody granted, and a workflow nobody changed.
The pattern is consistent enough to be predictable. A pilot is scoped, built and demonstrated. It works. Everyone in the room agrees it should go live. A year later it has not, and nobody can quite say why.
When we ask what went wrong, the answer offered is almost always about the technology. It rarely is.
In the failures worth studying, one or more of these is present, and none of them is a model problem.
A demo is optimised to remove friction, which means it is optimised to remove exactly the four things above. It runs on a fixed dataset, under a superuser account, in front of an audience that does not include the people whose work would change, sponsored by someone who will not be operating it.
Every one of those simplifications is reasonable for a demo. Together they guarantee the demo tells you nothing about whether the thing can ship.
Whose job changes on the day this works?
If there is no answer, the project has no destination. If there is an answer and that person is not in the room, the project has a destination and no route to it.
Asking it before a pilot starts costs nothing and cancels a meaningful fraction of pilots that were going to fail anyway, which is a good outcome for everyone except the vendor.
We treat the end of a project as the point at which an ordinary week of real work runs through the system, with the client’s people using it and nobody keeping the old spreadsheet open beside it. Not the point at which the software is delivered.
That definition is inconvenient. It puts the four causes above squarely inside the engagement rather than beyond its edge, which means they have to be solved rather than escalated. It is also the only definition under which the words "the project succeeded" mean anything.
A narrow system that a department actually runs on beats a broad platform that is still in evaluation, by an enormous margin, and the gap compounds. The narrow system generates real usage data, real objections and real trust, all of which make the second phase cheaper.
This is why we scope to a first release that changes one workflow completely rather than five workflows partially. Partial change is reversible, and reversible change reverts.
Every operation has documentation that stopped being true at some point. This is a method for finding that point: extracting entities from records, building a graph of what should be, and reading the divergences against what is.
A seaweed supply operation running from Korean farms and factories to U.S. warehouses kept its truth in four disconnected spreadsheets. Discovery found 25 places the sources disagreed and 34 questions nobody had answered.
A distributor rebuilt its product list by hand every morning because the same item was named differently on every supplier site. The fix was less about the crawler than about deciding when two names mean one product.