Academy learning group 4 · Academy module (not an ISO clause) M07 · Chapter 7 of 8
Monitor performance and control AI changes
About 5 minutes · ISO/IEC 42001:2023
Session only. Enable saving or download a backup from this course’s hub.
Completion is self-reported and does not certify understanding.
See it · Try it · Recall it
Picture the topic
Remember this: Watch outcomes and context, not just whether the endpoint is online.
Explore the anchors
Monitor relevant performance incidents and feedback.
Concept matching activity
Match this task or description to its concept:
Interpret changes against intended use and limits.
Choose a concept before checking the explanation.
Detection laboratory
Positive means the detector raised an alert. True/false compares that decision with known reality.
False negative — An attack exists. No alert was raised.
| Detector | Attack present | No attack |
|---|---|---|
| Alert | True positive | False positive |
| No alert | False negative | True negative |
Change the threshold
Eight fictional events have fixed known truth. An alert fires when the synthetic score is at least the threshold. The score is not a probability of attack.
TP 2 · FP 1 · FN 2 · TN 3. Precision 67%. Recall 50%.
Precision = TP ÷ (TP + FP). Recall = TP ÷ (TP + FN). Lower thresholds can increase detection and false alarms; this tiny dataset does not establish real-world performance.
A quick way to study this topic
- Hide the explanation and redraw these three anchors from memory.
- Explain the difference between the anchors in your own words.
- Apply the distinction to the worked example below. Check the full lesson for assumptions and exceptions.
- Recall it again tomorrow, then after a few days. A memory hint is a retrieval cue, not a replacement for understanding.
Original academy illustrations and fictional exercises. These are learning simulations, not a real virtual machine, network assessment or measure of job readiness. Activities reset when you leave or reload; they do not change quiz scores.
What this chapter explains
- Concepts and distinctions
- Apply the process and collect evidence
- Findings, documentation quality and closing the audit
Concepts and distinctions
AI-system monitoring should reflect intended outcomes, relevant groups, input conditions and operating limits. Technical availability is one signal; a reachable endpoint can still produce unsuitable results. Changes in input distribution, feedback processes or model service behavior may justify investigation even when average system metrics look stable.
Distinguish security incidents, performance problems and other adverse impacts while maintaining routes for coordinated response. One observation may support several hypotheses and should not be forced prematurely into a single category. Protect relevant evidence, apply appropriate privacy handling and involve the people who own data, models, operations and the affected business use.
Apply the process and collect evidence
Define thresholds and review responsibilities before launch. Record changes to models, data, configuration and purpose, then determine what revalidation or approval is needed. Use feedback to identify patterns without treating every complaint as proof of a specific defect. Escalate consequential concerns and validate the outcome of corrective actions before concluding the system is back within acceptable conditions.
Findings, documentation quality and closing the audit
A nonconformity report should connect an applicable requirement, identified evidence and a precise statement of the gap. For example, if the organization requires impact reassessment before a material use change, document the changed use, sampled approval records and missing assessment. Do not invent an ISO clause number from a training slide. Distinguish confirmed facts, reasonable inferences and matters needing further investigation.
Quality review checks that conclusions follow the workpapers, classifications follow agreed rules and limitations remain visible. Control the report’s versions, recipients, archival access and retention. At closing, review objectives achieved, findings, unresolved disagreements, action expectations and next steps. Closure communicates supported results and agreed next steps; it does not conceal unresolved limitations.
| Audit reasoning step | Worked example |
|---|---|
| Criterion | The organization requires impact review before a material use change. |
| Evidence | A sampled release changed intended use without recorded reassessment. |
| Finding | The stated reassessment process was not followed for that release. |
| Follow-up | Check cause, action owner and evidence of effectiveness. |
Worked example
A support assistant remains online but begins giving poorer responses to a newly introduced product line. Operations reports the system healthy because uptime is unchanged. Investigate performance for the changed use and relevant data rather than relying only on uptime. Identify version and context changes, apply interim review or restrictions if needed, and authorize appropriate revalidation before expanding use further.
Check your understanding
Why is endpoint uptime insufficient for AI monitoring?
Show explanation
Availability does not establish output suitability or effects in the intended use. The service can be reachable while its decisions or responses deteriorate.
Source and study boundary
Official standard overview. The reference identifies the topic being taught. Explanations and examples are original academy material; a topic link does not establish independent approval or sufficient exam mastery.