A familiar pattern in enterprise AI: a dozen pilots, each with a sponsor and a demo, and no agreed way to say yes to production. In Escaping Pilot Purgatory we described what scaling takes. This piece is about the process of making the decision itself: how to assess when a use case is ready to move towards production.
The answer is not heavier governance or a faster pilot. It is a stage-gate where rigor rises with the stage: light controls in Pilot, hardened controls in Staging, and proven controls with evidence in Production. Apply production controls to a pilot and it stalls. Skip them before production and the cost and risk arrive later, with interest.
We frame the climb as an ascent. Every use case moves from Pilot to Staging to Production, and five dimensions travel with it like strands of one rope: business-tech alignment, technical architecture, talent strategy, the operating model, and ecosystem and third-party risk. Each has one accountable executive, and the rope is only as strong as its weakest strand. At each gate, five must-pass checks, one per dimension, decide whether a use case moves up.
Select a step to see its must-pass items.
Baseline and success metric agreed; spend capped; cost per outcome tracked
Agents run in an isolated sandbox with scoped data access; mechanism in place to integrate the model with assets
Named supervisors review agent output; role impact understood
A named human owns every agent action and output
Vendor, implementation partner, model and data terms reviewed; exit path noted
- Must-pass items at each gate, one per dimension. Where each gate sits, and what counts as proof, depends on your risk appetite, industry and operating model.
- An outcome is one completed business process run, such as an invoice paid or a claim settled.
Here is the same climb on one use case: an agent that processes supplier invoices. The numbers are illustrative. Proof takes as long as the business cycle, so for a monthly process like this, expect four to six months from build to production.
A worked example: an invoice agent
Reads invoices for one supplier group in an isolated sandbox and proposes payments. A person approves every one. About 200 invoices.
Proof asked for
Baseline set: the human cost per invoice. Spend capped. The accounts payable manager is the named owner.
Gate decision
Go with conditions. Supervisor training is not done yet; it is due before Staging.
Decided by: The process owner and the CIO.
A full month of invoices for the same suppliers. A person still approves every payment, so autonomy and data are unchanged.
Proof asked for
Finance measures cost per outcome across a month-end close. Security tests containment. The process owner checks accuracy.
Gate decision
Not yet. Cost per outcome is still above the human baseline. Fix the exceptions and re-gate after the next month-end. Second attempt: go.
Decided by: The sponsor, with all five executives confirming their checks.
Pays small invoices without approval and refers the rest to a person. Same suppliers, same volume.
Proof asked for
Two month-ends below the human baseline, errors within range, verified by finance and internal audit.
Gate decision
Go. The Board sees one page: value against baseline, incidents, cost per outcome. Wider suppliers and data come next, one risk at a time.
Decided by: The sponsor, with CEO sign-off.
The sponsor could be the CFO in this use case. The CISO and General Counsel can stop a “go” on containment or legal grounds.
Three rules sit behind that example.
The three rules behind the example
- 1
Raise one risk at a time. Autonomy, data sensitivity or volume: raise only one between gates, so any failure has one explanation. In the example, Staging raised volume and Production raised autonomy. Keep a person on actions that cannot be undone, such as payments.
- 2
Proof is observed, repeatable and independent. Real volume, a full business cycle, and verification by someone outside the build team. A demo is not proof.
- 3
Stop when it cannot beat the baseline. If cost per outcome stays above the human baseline, or the use case misses its own success measure at two gates running, stop it. Stopping is a result, not a failure.
Pick one use case, gather the accountable executives (in a smaller company one person may hold several of these roles), and have them rate their check green, amber or red. If two or more come back amber, it is worth comparing notes.
Want to discuss how this applies to your portfolio or deal?
Request a Meeting