Insight · Guide
How to Move an AI Pilot Into Production
The short answer
Moving a pilot into production means giving the workflow an owner, a real integration path, scoped permissions, a review point matched to consequence, logging, a trained second operator, and a measurement against baseline. The model rarely changes. What changes is everything around it. Most pilots stall not because the capability failed but because nobody owned the crossing.
57-word direct answer
Key takeaways
- The capability is usually already sufficient. The crossing is an organizational problem wearing a technical costume.
- If output still needs a person to copy it somewhere, you have not crossed. You have moved the integration onto a human.
- The single most common blocker is permissions, and it is almost always discovered late.
- Capture the baseline before you cross. Afterwards it is gone, and every claim you make becomes unfalsifiable.
Seven things that change at the crossing
A pilot and a production workflow can use the identical model and prompt. Everything in this table is what actually differs.
| Dimension | In the pilot | In production |
|---|---|---|
| Who runs it | The person who built it, by hand, when they remember | Anyone whose job includes the process, including people who were not consulted |
| Identity and access | The builder's own credentials, with their full visibility | A scoped service account with least-privilege permissions, revocable by the client |
| Where output goes | A chat window, a document, a spreadsheet | A named field on a specific record in the system of record |
| Input variety | Curated examples that demonstrate the capability | The incomplete record, the forwarded thread, the unusual request, the one in another language |
| Oversight | The builder eyeballs it because they are curious | A named reviewer at a defined point, sized to what happens if it is wrong |
| Failure handling | Notice it, shrug, rerun | Alerting, retry that cannot duplicate a write, and a rollback to the prior process |
| Evidence | It looked impressive in the demo | An instrumented result against a baseline captured beforehand |
Read the right column as the actual scope of work. Teams routinely estimate the crossing as “a bit of tidying up” because the left column already works.
Principle
A pilot proves a capability can produce the output. Production proves the organization can operate, oversee, and depend on it when the person who built it is on leave.
The sequence
Roughly in order, though steps two and three interleave in practice. Steps one and six are the ones most often skipped, and they are the ones that determine whether this survives.
Assign an owner, and capture the baseline
Before touching anything technical. The owner is accountable for the process outcome, not for AI, and is usually not the person who built the pilot. Then measure the current state: cycle time, volume, error or rework rate, cost per unit. This takes hours and cannot be recovered later.
Resolve identity and permissions
Decide whose identity the workflow acts under, and get a scoped service account created. Start this immediately; it is the step with an external dependency and the one that most often adds weeks. Insist on client-owned accounts with revocable, role-limited access.
Build the write path
Connect output to the specific field on the specific record where the work actually happens. Make writes idempotent so a retry cannot create a duplicate. This is the step that distinguishes production from a demo, and where “we'll paste it in for now” quietly becomes permanent.
Assemble an evaluation set from real cases
Thirty to fifty real historical cases with known good outcomes, deliberately including the awkward ones: the incomplete record, the unusual request, the case a person got wrong. Run the workflow against them. This set becomes your regression check for every later change.
Set the review point and the risk tier
Write down who reviews what, and at which point output may take effect. Match it to consequence: internal drafting needs a spot check; anything client-facing needs a named reviewer; anything affecting money, eligibility, or a person's standing needs a human deciding with the system advisory only.
Instrument, alert, and add a rollback
Log enough to reconstruct any single decision: input, retrieved context, model and version, output, who approved it, when. Alert a named person on failure and on silent stops. Document how to disable the workflow and resume the prior process without data loss.
Train a second operator, then hand over
Train on their own cases, not generic examples. The exit condition is that someone other than the builder can run it, explain it, and knows what to do when it is wrong. If that is not true, you have a dependency rather than a workflow.
Report against the baseline, and be willing to retire it
Compare to step one honestly, including where it underperformed. Then decide: extend the pattern to an adjacent workflow, or retire this one with the reason recorded. A documented retirement is a legitimate success. An abandoned pilot teaches nobody anything.
The eight things that block the crossing
In rough order of how often they are the actual cause. Each is followed by what tends to unblock it.
Permissions are unresolved
The pilot ran on one person's access; extending it needs a decision about who may see what, which turns out to involve confidentiality obligations. Unblock: raise it in week one, name the decision-maker, and scope to the minimum the workflow needs rather than requesting broad access.
No owner after the builder
An enthusiast built it on their own time and no role includes running it. Unblock: assign the process owner before building anything further. If nobody will own it, that is a real answer; stop.
The last mile is still manual
Output is good but lands in a chat window, and someone copies it across. Unblock: treat the write path as in scope, not as polish. Where no API exists, an intermediate export beats a person.
No baseline exists
Nobody measured before, so the benefit cannot be demonstrated and funding goes elsewhere. Unblock: measure now if the workflow has not changed yet; if it has, say plainly that no baseline exists rather than reconstructing one.
The review point was never decided
Staff do not know whether they may release output, so they either over-check everything or under-check it until an incident. Unblock: assign a risk tier and name a reviewer. Ambiguity is more expensive than a strict rule.
Real inputs break it
Curated pilot examples hid the forwarded thread, the scanned attachment, the record with missing fields. Unblock: the evaluation set in step four. Handle the awkward cases explicitly rather than letting them fail silently.
Cost surprises at volume
Per-call pricing was fine in testing and is not at production volume, particularly with retrieval over large documents. Unblock: model cost at realistic volume before launch, not after the first invoice.
Trust, presented as a training problem
Adoption is low despite good output, and the response is another training session. Usually the real issue is unstated: people are unsure whether they are accountable for the system's errors, or whether this is about headcount. Unblock: answer the unasked question directly. No amount of training addresses it.
First-hand experience
What the crossing actually looks like from inside
The pattern that recurs: a team is confident they are two weeks from production because the hard part, getting good output, is finished. The remaining work is characterized as plumbing.
Then week one of the crossing produces a permissions question nobody can answer, because the records involved carry obligations that were never surfaced during the pilot. That question sits with someone who was not in the room for the demo and has no particular stake in the timeline. Two weeks becomes six, and the delay is attributed to bureaucracy rather than to the pilot having been scoped without those constraints.
The teams that cross quickly are almost always the ones that raised access and oversight in the first week rather than the last. Nothing else correlates as strongly: not technical sophistication, not budget, not how impressive the pilot was.
Interpretation
Why “pilot” may be the wrong frame
A reading rather than an observation: the word pilot imports an assumption from software procurement, that the question is whether the thing works, and that a yes leads naturally to rollout.
With AI the capability question is usually answered before you start, because the same capability is publicly demonstrable. What is genuinely uncertain is whether your organization can operate it: whether the data is adequate, whether permissions can be granted, whether the people involved will trust it, whether anyone will own it.
Framed that way, a useful pilot is not a capability test. It is a rehearsal of the organizational conditions, which argues for making the first attempt small and end-to-end rather than large and partial. A workflow that reaches the system of record for ten cases teaches you more than one that produces excellent output for a thousand and stops short of the record.
When not to cross
Some pilots should be retired, and doing so deliberately is a better outcome than an indefinite half-life. Stop if no role will own the workflow, if the data cannot lawfully be used the way the workflow requires, if the failure mode is not recoverable and oversight cannot be made adequate, or if the workflow does not actually repeat often enough to be worth the maintenance.
Record the reason. A documented retirement stops the same idea being proposed again in a year with the same blocker unaddressed.
How long the crossing takes
For a single workflow with system access already available, the crossing is commonly four to eight weeks. That is not build time; build is often days. It is the elapsed time of permission decisions, evaluation-set assembly, and training around people's existing jobs.
Where access has to be provisioned through procurement or a security review, add whatever that process takes at your organization; that number is knowable in advance and worth asking for in week one.
A crossing that has taken more than a quarter without reaching the system of record usually has an unresolved blocker from the list above that has not been named. It is worth stopping to identify which one, rather than continuing to build.
Limitations of this article
This describes the crossing for internal operational workflows in organizations with existing business systems and no dedicated platform-engineering team. It does not cover shipping an AI feature in a product you sell, where the constraints are release engineering and customer-facing reliability rather than internal adoption.
It also assumes a workflow a person could perform given enough time. Where AI is doing something no person could, the sequence still applies but the risk analysis here is insufficient.
The eight blockers are drawn from practitioner observation, not a study. The ordering reflects how often each has been the actual cause in the engagements behind this article; treat it as a useful prior rather than a measured distribution. If your blocker is not on the list, the list is incomplete rather than your situation unusual.
Nothing here is legal or regulatory advice. Where a named compliance regime governs the workflow, that governs. See responsible AI governance.
External references
Frequently asked questions
Our pilot works well. Why would it not go into production?
Because “works well” describes the capability, and production depends on conditions the pilot did not test: whether permissions can be granted, whether output can reach the system of record, whether a role will own it, and whether real inputs behave like your examples.
A pilot that produced good output has answered a question you probably already knew the answer to. The eight blockers above are the questions it did not answer.
Should we run a second, larger pilot first?
Usually not. A larger pilot answers the capability question again, at more expense, while leaving the organizational questions untouched.
A smaller end-to-end attempt is more informative: take ten real cases all the way to the system of record with a review point and logging. That tests the things that actually block you.
What if we cannot get the permissions we need?
First check whether you asked for more than the workflow requires. Broad requests attract scrutiny that narrow ones do not, and least-privilege scoping often unblocks a stalled request.
If the access genuinely cannot be granted, that is a real constraint and the honest response is to pick a different first workflow rather than wait indefinitely. Record why, so the same candidate is not re-proposed later with the same blocker.
How do we know it is actually working after we cross?
Four things. Volume, so you notice when it silently stops. Failure and retry rates. Override rate: how often a reviewer materially changes output, and whether that is trending down. And cost per unit against what you modeled.
Plus adoption: what proportion of the people meant to use it do, four to six weeks after training. Low adoption alongside good output almost always signals an unresolved trust or accountability question rather than a training gap.