A team completes an AI course, receives certificates and starts using a new tool. What can its manager now safely assume about the work that team is ready to do?
Attendance confirms exposure to training. It does not, on its own, show that somebody can recognize an unreliable answer, protect sensitive information or handle an exception in a live workflow. For leaders, the useful question is more specific: what should this person be able to decide and demonstrate in this role?
Role-based AI training connects learning to those decisions. It gives an employee a realistic task, a clear boundary and a way to demonstrate competent performance. This is a practical extension of the skills matrix, training-needs analysis and adoption planning in my VORTA material: capability development should help people carry out their responsibilities, rather than simply accumulate course completions.
Start with the work, then choose the training
“Learn generative AI” is too broad to guide an assessment. “Prepare a customer-response draft using approved information, verify its claims and route unresolved cases to the right person” describes work that can be observed.
Before selecting a course, write down four things for each role:
- The task: what the person will actually use AI to do.
- The boundary: what data, decisions or actions are outside that permission.
- The evidence: what acceptable performance looks like.
- The escalation: what the person should do when the task cannot be completed reliably.
These questions can reveal a problem that training cannot fix. If staff do not have an approved source, a usable process or an escalation owner, another prompt-writing session will not supply those missing conditions. Resolve the operational gap alongside the learning need.
Give different roles different practice
A shared introduction can establish common language. The practical exercises should then reflect the responsibilities people hold.
A user preparing an AI-assisted output needs to work with permitted inputs, check important claims against appropriate sources and recognize when the output is unsuitable. The exercise should include an output that looks plausible but contains a meaningful defect, not only a perfect demonstration.
A manager reviewing the work needs to judge whether the evidence supports release. Ask the manager to distinguish an acceptable draft from one that needs correction or escalation, and to explain the reason. Familiarity with a tool interface is not enough for this responsibility.
A workflow owner needs to understand where the AI output goes next, how errors are discovered and who can pause the process. Practice should include a failed handoff or an exception that the standard procedure does not resolve.
An executive sponsoring adoption needs to challenge the assumptions behind a proposal. Give the executive a short business case with incomplete evidence and ask what must be clarified before funding or expanding it. The objective is a defensible decision, not technical fluency for its own sake.
This role-specific approach is consistent with NIST AI RMF 1.0, GOVERN 2.2, which connects AI risk-management training to the duties and responsibilities people perform. The practical exercises below are a suggested implementation approach, not a prescribed NIST test or certification requirement.
Use a realistic task and an explicit assessment rubric
Consider this illustrative exercise for a team that drafts responses to business enquiries. It is a fictional training scenario, not a client result.
Provide an approved product-information sheet, a sample enquiry and an AI-generated response. Include one unsupported delivery promise in the draft and leave one customer question unanswered in the source material. Ask the learner to prepare a response that can be reviewed for release.
The assessor checks whether the learner:
- Uses only information permitted for this exercise and the approved tool.
- Finds and removes the unsupported promise rather than making it sound more convincing.
- Distinguishes a missing fact from a fact that can be verified.
- Routes the unanswered question through the stated escalation path.
- Leaves a clear record of what was checked and what still needs a decision.
A fluent response that retains the unsupported promise should not pass merely because it reads well. Equally, a learner who correctly declines to invent the missing answer may have demonstrated better judgment than one who completes every field.
Agree the criteria before the exercise. Separate errors that create unacceptable exposure from points that can be improved through coaching. An average score should not hide a critical failure such as disclosing restricted information. The threshold must fit the role and consequences; a generic percentage is not a universal readiness standard.
Record the limits of competence
A useful assessment record is short: role, task, approved environment, assessment date, observed evidence, remaining gaps, reviewer and next action. Record whether the person can perform the task independently, with supervision or only after further practice.
Keep that conclusion narrow. Passing a drafting exercise does not establish readiness to approve payments, publish externally or use a different dataset. Training evidence informs an access or deployment decision; it does not replace the organization’s authorization process.
For the same reason, the training record should not become an unnecessary collection of personal or customer information. Use synthetic or appropriately approved examples, store only the evidence needed and follow the organization’s access and retention rules.
Check what transfers into daily work
The next review belongs in the workflow. A line manager can inspect an appropriate sample of completed work, look for recurring corrections and ask employees which exceptions they still cannot handle. Combine those observations with incident reports and employee feedback, rather than treating course satisfaction as proof of safe practice.
Compare like-for-like tasks and account for changes in workload, tools and review support. If quality improves after training, that is useful evidence, but it does not by itself prove that training caused the improvement. Keep the claim proportionate to the comparison you can support.
Choose follow-up timing to match the frequency and risk of the task. A rarely performed, consequential task may need a rehearsal before use; a routine task may provide enough observations during normal supervision. Avoid setting a review date that arrives before anyone has had a reasonable opportunity to apply the learning.
Refresh the assessment when the work changes
Capability evidence can become outdated. A new model, changed policy, unfamiliar data source or expanded permission may change what the person must know and demonstrate. Review the affected task and decide whether a briefing, supervised practice or reassessment is appropriate.
This connects training to AI change control: a change in the work should trigger a proportionate check of the people expected to perform it. It also gives human oversight a stronger foundation by checking the reviewer’s practical capability, alongside their authority to intervene.
A better question for the next training review
Instead of asking only how many people completed the program, ask: which work can they now perform to the required standard, under which conditions, and where do they still need support?
Start with one role and one consequential task. Define the boundary, build an exercise, observe the decisions and close the gaps. Then connect that evidence to your business-value review, so capability development is assessed through its contribution to better work.
If you are translating AI training into role-specific responsibilities and adoption plans, schedule a consultation to discuss the work your team needs to perform.