Use this checklist before moving an AI pilot into a broader rollout. It is meant to slow the decision down just enough to make the evidence visible and the operating risk explicit.
How To Use This Checklist
For each section, mark the gate as pass, partial, or not ready. Partial is not failure. It is a sign that the team should narrow the rollout, add controls, or collect better evidence first.
Use it with the Governance and Rollout Decisions lesson. The lesson explains the leadership questions; this worksheet turns those questions into a go/no-go record.
- Pass: evidence exists, owners are named, and the next stage is bounded.
- Partial: the idea may continue, but only with a narrower audience, extra review, or missing evidence task.
- Not ready: do not expand until the blocker is resolved.
Fit And Scope
This gate checks whether the pilot is still a specific workflow or has become a vague transformation promise.
- The workflow has a clear start, input, output, and definition of done.
- The task is narrow enough for a pilot, not a broad promise to transform a department.
- The AI role is named: drafter, reviewer, extractor, organizer, sparring partner, or bounded agent.
- The expected output can be checked by a person or system of record.
- The team understands what a wrong answer could affect.
Decision note:
This pilot is ready to continue because
[scope reason], or it should be narrowed to[smaller scope].
Data And Security
This gate checks whether the system knows what it may use, what it must avoid, and what could be exposed if the rollout expands.
- Approved source material is identified.
- Sensitive data boundaries are documented.
- Tool and record access are limited to what the workflow needs.
- The prompt or workflow says what the AI system must not invent, expose, or act on.
- The team has considered prompt injection, insecure output handling, and excessive agency risks.
Reference prompts:
- NIST AI RMF: use risk mapping and measurement to decide what must be governed before expansion.
- OWASP LLM Top 10: use the list as a security review prompt for AI-specific failure modes.
Governance And Ownership
This gate checks whether accountability exists after the demo is over.
- A responsible owner is named.
- Review and approval points are explicit.
- Prohibited actions are documented.
- Escalation criteria are documented.
- Exceptions have a review path.
- Changes to prompts, tools, models, or source material require retesting.
Decision note:
The rollout owner is
[owner or role]. They are responsible for[support, review, monitoring, change approval, escalation].
Pilot Design
This gate checks whether the pilot produced evidence, not just enthusiasm.
- Success measures were defined before the pilot started.
- The pilot used realistic examples.
- A ground-truth source, reviewer, or acceptance rule was available.
- The pilot had a stop condition.
- The team recorded failures, reviewer corrections, and unclear cases.
If the pilot did not record failures, the evidence is incomplete. A pilot with no observed failures may mean the workflow is strong. It may also mean the test was too easy.
Evidence Gate
This gate checks whether the team can explain why expansion is justified and what uncertainty remains.
Before rollout, the team can show:
- measured value from the pilot
- examples where the system performed well
- examples where the system failed or required review
- what changed after those failures
- which risks remain
- why the remaining risks are acceptable for the next rollout stage
Decision line:
Rollout is
[approved / approved with limits / not ready]because[evidence].
Rollout And Operate
This gate checks whether the workflow can be supported, paused, changed, and improved after launch.
- Expansion is staged.
- Monitoring continues after launch.
- Review burden is understood.
- Users know what the AI system can and cannot do.
- A rollback path exists.
- The team knows who can pause the workflow.
- The review cadence is scheduled.
Scoring And Interpreting Results
Use the six gates above as a decision record.
- All gates pass: proceed to the next bounded rollout stage.
- One or more partial gates: continue only with explicit limits, owners, and follow-up actions.
- Any not-ready gate in Data And Security or Governance And Ownership: block expansion.
- Any not-ready gate in Fit And Scope: narrow the workflow before another pilot.
- Any not-ready gate in Rollout And Operate: do not expand to a wider audience yet.
Mostly partial results usually mean the pilot taught the team something useful, but the next step is a smaller rollout or a better-designed second pilot, not broad launch.
Signals The Pilot Is Not Ready
Slow down when you see these patterns:
- success is described by vibe instead of measured examples
- no failures, reviewer corrections, or unclear cases were recorded
- the owner is “the team” rather than a role or named accountable person
- the pilot quietly expanded beyond the original workflow
- the review burden is higher than expected and no one owns it
- users are not told where the AI system is unreliable
- the rollback plan is “we will figure it out if needed”
Any one of these may be fixable. More than one usually means the operating model is still immature.
Worked Example
Scenario: a team piloted an AI assistant that summarizes internal policy updates for managers before they brief their teams.
- Fit And Scope: pass. The workflow is limited to first-draft summaries of approved policy updates.
- Data And Security: partial. Approved source documents are identified, but access rules for older archived policies need review.
- Governance And Ownership: pass. The policy operations lead owns review, change approval, and escalation.
- Pilot Design: pass. The pilot used ten realistic policy updates and compared drafts to a reviewer checklist.
- Evidence Gate: partial. Summaries saved time, but two drafts overgeneralized exceptions.
- Rollout And Operate: partial. A small manager group can use it, but support and monitoring are not ready for all departments.
Decision line:
Rollout is approved with limits because the workflow is bounded and useful, but expansion should stay with one manager group until archive access, exception handling, and support ownership are resolved.
When To Re-Run This Checklist
Run the checklist again when the rollout changes shape:
- the audience expands
- the source material changes
- a new data class is introduced
- prompts, tools, models, or retrieval sources change
- review rules or approval owners change
- the workflow moves from draft support to action or decision support
Small AI changes can create large operating changes. Treat retesting as normal maintenance, not a sign that the original pilot failed.
Reference Links
These references are for deeper review. The checklist above is original LIW training guidance.
- NIST AI Risk Management Framework - useful for framing AI risk as something leaders govern, map, measure, and manage.
- OWASP Top 10 for LLM Applications - useful for AI-specific security and misuse review.
- Microsoft AI Red Team resources - useful for thinking about adversarial testing and unexpected behavior before expansion.