An AI proposal can look convincing because its assumptions agree with one another. The forecast assumes clean data, quick employee adoption, little review effort and a smooth connection to existing systems. Multiply those assumptions together and the result can look like a business case.
The leadership question is different: which assumption would change our decision if it proved wrong? That is where I would start before approving a substantial commitment.
My VORTA material connects the initial value hypothesis with operational readiness, an assumptions log, tests and financial sensitivity. The purpose is to make uncertainty visible while there is still room to change direction. A business case should explain what must become true, not just describe what success could look like.
Separate what you know from what you need to believe
Consider an illustrative proposal for AI-assisted drafting of routine service responses. The team expects faster turnaround. That expectation might depend on several different claims:
- The incoming requests contain enough reliable information to draft a useful answer.
- Employees can check the draft faster than they can write an answer themselves.
- Exceptions can be recognized and routed without creating a second queue of work.
- The freed capacity can improve a service outcome the organization values.
- The approved tool can connect to the required workflow at an acceptable operating cost.
These are not interchangeable. A successful writing demonstration provides little evidence about system integration. An employee liking the draft does not establish that the whole process is quicker. A potential time saving does not prove that staffing costs will fall.
Label each important claim as observed, estimated or untested. For an observation, record where it came from and what it covers. For an estimate, name its basis. For an untested claim, state the uncertainty plainly. This avoids a common planning mistake: giving a spreadsheet input the appearance of a fact merely because it has a number.
Find the assumption that can reverse the decision
Do not commission an elaborate test for every unknown. Ask two questions of each assumption:
- Decision impact: if this is wrong, would we stop, change the scope or choose another solution?
- Evidence strength: have we observed it in relevant conditions, or are we relying on a demonstration or expectation?
Start with assumptions that have a large effect on the decision and weak supporting evidence. Some are hard constraints: permission to use necessary information, an acceptable safety boundary or the ability to integrate with a critical system. Others affect the economics gradually, such as review time or the proportion of requests suitable for assistance.
A hard constraint should not disappear inside an average score. Strong potential benefits do not compensate for an unresolved requirement that makes the intended use unacceptable. Equally, uncertainty is not automatically a reason to reject the proposal. It can justify a smaller, explicitly bounded step to obtain the missing evidence.
Turn the claim into a question a test can answer
“AI will make the team more productive” is too broad to test usefully. A sharper question is: “For this category of service request, does a checked AI-assisted response take less total staff time than the current method, while meeting the same quality requirements?”
Define the comparison before collecting results. Include the time spent preparing inputs, checking drafts, correcting errors, handling exceptions and transferring the result into the working system. Compare similar tasks and account for differences in complexity and staff experience. Keep a simple non-AI alternative in the comparison when it could solve the same problem.
The NIST AI RMF Playbook’s Map guidance calls for documenting intended use, assumptions and limitations, and considering non-AI alternatives. That supports a practical discipline: evaluate whether AI is a suitable route to the outcome, rather than treating AI adoption itself as the outcome.
Use representative cases, including difficult and unusual ones. A small exploratory test can expose a weak assumption; it cannot establish that rare failures are acceptably controlled. The evidence required should reflect the consequences of getting the decision wrong.
An example: review time changes the answer
The following numbers are illustrative, not client results or performance benchmarks.
Suppose a checked response currently takes 12 minutes. An AI-assisted method takes 2 minutes to prepare the input, 1 minute of staff time to generate and transfer the draft, and 4 minutes to review and correct it. The total is 7 minutes, suggesting 5 minutes of capacity released per eligible request.
Now change only the review assumption. If review and correction take 9 minutes, the total becomes 12 minutes. The apparent time benefit disappears before considering training, system support or other operating costs.
That does not prove the idea has no value. It might still improve consistency or help inexperienced staff, if those benefits are demonstrated and matter to the business. It does show that “the tool drafts quickly” is not enough to approve a time-saving business case.
The next useful action is to measure review effort on relevant work and investigate why correction is needed. Perhaps the input is incomplete, the task is too broad or the current process needs redesign. The appropriate response may be a narrower scope rather than a larger software commitment.
For the distinction between released capacity and realized business value, see AI value measurement beyond time saved.
Write a decision rule before seeing the result
A test becomes less useful when its success criteria change to fit whatever happened. Before it starts, agree what evidence would support continuing, changing the approach or stopping.
A concise decision record can contain:
- Assumption: the specific claim that matters to the business case.
- Consequence: what changes if the claim is false.
- Current evidence: what is known and the limits of that evidence.
- Test: the cases, comparison, measurement and responsible person.
- Decision rule: the quality and operating conditions needed to proceed.
- Next decision: who will decide, when, and what commitment that decision permits.
Choose thresholds for the actual context. A draft used internally and checked by an experienced employee needs a different assessment from a system that directly affects customers or makes consequential decisions. Do not borrow a generic pass mark simply because it makes the approval form easier to complete.
Also distinguish a failed assumption from an inconclusive test. If the cases were unrepresentative or the baseline was unreliable, the result may support neither approval nor rejection. Record what remains unknown and decide whether another bounded test is worth its cost.
Approve the next commitment, not an undefined future
A limited investigation, a supervised pilot and an organization-wide rollout are different commitments. The first can be justified by the value of learning; the last needs stronger evidence about the operating model and consequences.
State the scope and resources approved now, the unresolved assumptions, and the evidence required for the next decision. When the task, data or system changes materially, earlier evidence may no longer apply. My AI change-control guide explains when to reopen an approval.
The useful leadership habit is simple: ask the team to name the assumption that could overturn the recommendation, then ask what evidence would settle it. That conversation can turn an attractive forecast into a decision the organization can explain and revisit.
If you need to challenge the assumptions behind an AI initiative before committing further, you can schedule a consultation.