An AI assistant reports that a service team completed 50% more requests this month. The calculation is correct: 60 of the 100 requests received in the first month were completed by its reporting cutoff, compared with 90 of the 200 received in the second. With the same follow-up period and eligibility rules, completed volume rose while the completion share fell from 60% to 45%.
These are illustrative figures, not results from a real organization. They show why checking arithmetic is only part of reviewing an AI-generated business report. A number can be correct and still support the wrong conclusion.
In my AI-for-business workshop material, strategy development connects AI opportunities with implementation, KPIs and employee input. Applied to reporting, that means keeping the business question in control of the calculation. The checks below are a practical extension of that principle, not a prescribed statistical framework.
Separate the count from the conclusion
In the example, “completed requests increased” is supported. “Service performance improved” needs more evidence. The team may have handled more complex requests, reduced an old backlog, changed staffing or missed a service deadline. The two counts alone cannot settle those questions.
A different report could contain an additional trap: its completed total might include work received in earlier months. Dividing that total by the current month's incoming requests produces a throughput-to-intake ratio, not the percentage of this month's requests that were completed. To calculate the latter, follow the same group of requests from receipt to the chosen cutoff, as in the opening example.
Before accepting a headline, ask: what decision is this number meant to support, and does the calculation actually measure it? If the decision concerns service timeliness, request age and completion within the agreed service period may matter more than a total count.
Give each important metric a short definition
Ask for a compact measurement note beside any figure that could change a decision:
- Unit: what is counted, such as a unique request rather than a status update.
- Population: which records qualify and which are excluded.
- Calculation: the numerator and denominator, or the rule for a total or average.
- Period: the event date used, the reporting window and its timezone.
- Evidence: the source, extraction cutoff and known coverage gaps.
- Owner: who can confirm that the definition matches the business process.
This is not paperwork for every sentence. Prioritize the few figures used to justify a budget, staffing change or operating decision. A label such as “completion rate” is not a substitute for the definition.
The UK Government Analysis Function's guidance on statistical quality distinguishes comparability across periods or groups from coherence between sources. It also emphasizes understanding the data journey and checking missing values and arithmetic relationships. Those principles are useful for management reporting; they do not make an AI-written report an official statistic or a certified analysis.
Reconcile sources before combining them
Two systems may record different stages of the same work. One might log a request when it arrives; another records each task needed to fulfil it. Adding both totals would not measure twice as much activity. It would mix requests with tasks and potentially count the same work repeatedly.
Use a stable identifier where available and appropriate. Check whether each row represents a unique item, an update, a line item or a recurring occurrence. Agree how cancellations, reopened items and duplicate records are handled. Keep the raw source intact and record any exclusions separately so the calculation can be reproduced.
If records cannot be matched reliably, show the two totals separately with their definitions. Do not ask the AI to invent a reconciliation merely to produce a single impressive number. A visible unresolved difference is more useful than false precision.
Share only the fields needed for the analysis in an approved environment. A reporting check rarely needs names, full message contents or unrelated personal details. The data-boundary checks before sharing information with AI apply here too.
Keep unavailable, zero and provisional results separate
“No completed requests” and “completion status was not supplied” are different findings. Replacing missing values with zero can create a false deterioration. Excluding all incomplete records without explaining it can create a falsely positive picture.
Report the usable records and the missing coverage. For example: “Completion status is available for 80 of 100 eligible requests; no conclusion is made about the remaining 20.” This is another illustrative statement, not a recommended completeness threshold. Whether that coverage is sufficient depends on the decision and on which records are missing.
Also distinguish the end of the reporting period from the extraction time. A month may have ended while status updates are still arriving. Label the result provisional where appropriate and state when it will be refreshed. Compare periods with compatible definitions and a similar opportunity for records to become complete, or explain why the comparison is limited.
Require a checkable calculation, not just another confident answer
Asking the same assistant “Are you sure?” is not a substitute for checking the result. Request the underlying counts, filters and calculation, then reproduce the important figures in a spreadsheet or another appropriate analytical tool. Review a small, purposeful selection of source records, including exclusions and unusual cases, without claiming that a spot check proves the entire dataset correct.
Have the process owner confirm the meaning of the fields. A technically valid formula cannot resolve an undocumented business rule. If “closed” includes cancelled work, the report should not quietly rename it “successfully completed.”
Keep the review proportionate. A minor internal update and a consequential resource-allocation decision need different levels of assurance. For the latter, a second reviewer should be able to retrace the key result from the permitted source data rather than only read the narrative.
End with a bounded decision
A useful reporting conclusion separates three things: what the evidence establishes, what remains uncertain, and what action is justified now.
For the opening example, that might be: “Completed volume rose, but intake rose faster. We need request age, timeliness and complexity evidence before explaining the change in service performance. Review overdue work before changing staffing.” It does not claim that AI, employee effort or any single intervention caused the result.
This discipline complements measuring AI's business value. Before debating whether the outcome is good, establish that the number describes the outcome you intended to measure.
If you are introducing AI-assisted management reporting, schedule a consultation to connect the reporting workflow, evidence checks and decision responsibilities. The aim is not a more polished report. It is a decision whose reasoning remains defensible when someone asks what was actually counted.