Skip to content
Applied AIOctober 28, 2026 · 6 min read

Why AI pilots stall before production

Most pilots do not fail on model quality. They fail on the four operational questions nobody asks until week ten.

The demo is the easy part

A prototype that answers a question convincingly takes a week. The work that decides whether it reaches a customer takes a quarter: retrieval that respects document versions, evaluation built from real cases, a refusal policy, and a review path for the answers the system should not give.

Teams that skip those steps do not discover the problem during the pilot. They discover it when a real user asks the one question the demo was never tested against, and there is no route to correct the answer.

Question one: what does correct mean, and who decides?

Before a single prompt is written, someone with domain authority has to define a correct answer for a sample of real queries. Without that, evaluation becomes opinion, and every review meeting re-argues the same cases.

In practice this means a set of 300 to 600 historical cases, graded by two people who disagree openly about the edge cases. The disagreements are the valuable output — they become the policy.

Question two: what happens when the source is missing?

Assistants fail quietly by improvising. The fix is structural rather than verbal: retrieval must return a source or the system must decline, and the decline must be a designed experience, not an error state.

A refusal rate of 8 to 12 percent is usually healthy on a first release. It means the boundary is real.

Question three: what does a wrong answer cost, per day?

Model the cost of an error at target volume before launch, not after. In support workflows an incorrect answer costs a repeat contact. In regulated advice it costs a compliance incident. Those two numbers justify very different architectures.

This is where most cost estimates go wrong: they price the tokens and ignore the human review queue behind them.

Question four: who owns it in month six?

A system without an owner decays. Name the person who reads the quality dashboard, the cadence of that review, and the condition under which the feature is rolled back.

If that name does not exist before launch, the pilot is not finished. It is a demonstration that has been left running.