How to Run a Safe AI Opportunity Interview Experiment

The interview is not the deliverable. The deliverable is a decision you can inspect later: one workflow, one narrow AI role, one safer comparison, and a clear reason to continue or stop.

The sourceable atom here is a simple advancement rule: a frontline opportunity does not advance while its card is missing a baseline, boundary, reviewer, falsifier, stop condition, or rollback path. That rule turns enthusiasm into a testable experiment.

Turn the interview into one opportunity card

Write one card per workflow candidate, and do not let a general complaint become an automation plan. The card must describe the work as it happens now, the smallest proposed AI step, and the decision the experiment is allowed to support.

Field	Record this	Why it matters
Job	The outcome the operator is trying to complete	Ties the test to work, not a model feature
Workaround	The current manual path, handoffs, and rework	Gives you a real comparison
Narrow AI role	One assistive step such as extraction, grouping, or draft preparation	Prevents a vague agent promise
Baseline	The current measure and simpler manual alternative	Makes success falsifiable
Cases	Normal, ambiguous, exception, and abstention cases	Exposes where the idea breaks
Data boundary	Approved inputs, excluded fields, retention, and access	Limits exposure before tooling exists
Action boundary	Read-only, reversible, or approval-gated behavior	Prevents hidden side effects
Reviewer	A role with authority to reject, edit, or stop	Makes oversight operational
Falsifier	A result that would disconfirm the hypothesis	Gives the team permission to stop
Stop and rollback	A trigger and the path back to current work	Makes failure recoverable

This structure extends the task-analysis logic in the Frontiers interview study. That study used semi-structured interviews to examine process steps, challenges, judgments, and decisions. Use the method to learn how work is done, not to collect feature requests.

The exception is a workflow with no stable owner or agreed outcome. Park the candidate and resolve ownership first.

Ask for a recent incident, not opinions

Ask the operator to reconstruct one recent case step by step. Opinions about where AI might help are useful prompts, but the case reveals inputs, decisions, exceptions, and hidden rework.

Use three sweeps:

1. Ask the operator to break the case into five to seven steps.
2. For every step, ask what they saw, changed, checked, waited for, or handed off.
3. Ask where an anomaly appeared, what cue they used, what would make them reject assistance, and what information a helper would need.

The Frontiers method adapts the critical incident technique to task analysis. It focuses on actions, decisions, and changes the participant would make, rather than asking only whether the project succeeded.

Worker participation must change the experiment, not decorate it. MIT's AI Work Redesign Playbook describes codesigned, implemented, and evaluated pilots, and frames expected benefits as hypotheses. Treat the operator's account as workflow evidence and a source of constraints, not automatic proof that an AI feature is valuable.

The exception is sensitive work. Collect only what is needed to understand the task. Record an anonymized description and the consent boundary. Do not paste a private case into a model to make the interview easier.

Compare the AI step with a manual baseline

Define the smallest comparison before selecting a model. If the proposed AI step cannot complement the current manual path on an agreed measure without unacceptable review work, it has not earned a pilot.

Part	Manual or non-AI baseline	Proposed AI step
Input	The same approved case set	The same approved case set
Work	The current operator procedure	One assistive transformation or suggestion
Decision	The operator's recorded outcome	The reviewer accepts, edits, or rejects
Measure	Time, correction effort, completeness, or another job measure	The same measure plus review burden and unsafe outputs
Decision rule	Keep, narrow, or improve the manual path	Advance only if threshold and safety conditions hold

Do not invent a universal accuracy target. Extraction, recommendation, drafting, and prioritization fail in different ways. The operator and workflow owner choose the measure, and they must count correction and escalation work when those costs matter.

Digital.gov.au's co-design guidance connects participation with measurable outcomes, safeguards, feedback, capability, and monitoring. Use it as a comparison checklist, not a promise that a pilot improves productivity.

The exception is a task where the outcome cannot be observed in the timebox. Use an earlier proxy only when the operator agrees that it relates to the real job. Otherwise, keep the candidate in discovery.

Lock the data and action boundaries

Start with the least sensitive input and least powerful output that can answer the hypothesis. A safe first experiment can read a redacted case and prepare a suggestion. It does not need permission to change a system of record.

Write the boundary plainly:

• Allowed inputs: exact fields and documents the experiment may receive.
• Excluded inputs: personal or confidential fields that are not needed.
• Retention: where prompts, outputs, and review notes live, and when they are deleted.
• Access: who can see the source case, output, and decision record.
• Allowed action: read-only output or a reversible draft.
• Forbidden action: sending, approving, deleting, changing status, or updating a record without an explicit human action.

NIST's AI RMF Playbook groups suggested actions under Govern, Map, Measure, and Manage. It also says the Playbook is voluntary and not a complete checklist. Copy the fields that fit the workflow, then add the controls its risks require.

The exception is a use case involving regulated decisions or meaningful power imbalance. Pause for the relevant privacy, employment, safety, or legal review. The European Commission's AI Act guidance is a reminder that context changes safeguards. It is not advice about a particular deployment.

Run a read-only rehearsal before live action

The first run should produce a suggestion for a human to inspect, not an action for a system to execute. A manual or concierge rehearsal can test the packet, reviewer path, and baseline before integration work begins.

1. Select approved representative cases, including an ambiguity or abstention case.
2. Apply the narrow AI step without sending or changing anything downstream.
3. Record output, reviewer decision, correction, time spent, and rejection reason.
4. Compare the record with the manual baseline using the agreed measure.
5. Apply the falsifier and stop rule before discussing expansion.

The worked decision record below is a control case, not a claimed customer or frontline result. It shows what the packet returns when advancement evidence is incomplete.

{
  "decision": "HOLD",
  "reason": "The opportunity card is incomplete",
  "missing": ["baseline", "humanReviewer", "falsifier", "rollback"],
  "allowedNextStep": "return to the interview and complete the card",
  "actionBoundary": "read-only"
}

The local dry-run verified that the missing fields produce HOLD rather than a default approval. That is the observed output of the artifact check. It is not evidence that any frontline team achieved a result.

The exception is a task where even read-only output can harm someone or expose protected information. Use an approved, redacted case set or stop the candidate until the owner defines a safer boundary.

Stop, roll back, and record the decision

Make stopping part of the protocol, not a dramatic response after failure. A stop condition should be observable, owned, and connected to a recovery action.

Good stop conditions include:

• the output crosses the approved data boundary;
• the reviewer cannot explain or verify a suggestion;
• a high-severity case is routed or drafted without required context;
• correction or review work exceeds the manual baseline;
• the system produces an unsafe action or attempts a forbidden side effect;
• the reviewer or operator withdraws consent for the test.

When a stop fires, disable the AI step, return to the manual baseline, preserve the decision record, and tell the workflow owner what happened. Do not silently edit the test set until the cause is understood.

The record needs four lines: what was tested, what the reviewer saw, why the decision was made, and what happens next. The decision can be GO, NARROW, HOLD, or REJECT. “The model seemed promising” is not a decision record.

This is where the artifact changes the reader's state. A complaint can become a bounded hypothesis, but only the workflow owner and reviewer can authorize movement beyond read-only rehearsal. If the card is incomplete, HOLD is a useful result.

For a broad list of ideas, use how to prioritize AI use cases in a small business. For process readiness, compare how to tell if a business process is ready for AI automation. The canonical parent for this packet is AI opportunity interviews with frontline staff.

Sources

• Opportunities for human factors in machine learning
• The AI Work Redesign Playbook
• Digital.gov.au co-design and transparency guidance
• NIST AI RMF Playbook
• European Commission: Navigating the AI Act