Academy module 3 · ISO/IEC 27090 · Lesson 3 of 8
Evaluate evasion and robustness carefully
About 8 minutes · Introductory draft · Checked 2026-10-23
AI-authored and AI editorially reviewed; independent review pending. Read course scope and remaining gaps.
Session only. Enable saving or download a backup from this course’s hub.
Completion is self-reported and does not certify understanding.
See it · Try it · Recall it
Picture the topic
Remember this: A clean test set and an adversarial test ask different questions.
Explore the anchors
Define attacker access and permitted input changes.
Concept matching activity
Match this task or description to its concept:
Evaluate relevant manipulated inputs.
Choose a concept before checking the explanation.
A quick way to study this topic
- Hide the explanation and redraw these three anchors from memory.
- Explain the difference between the anchors in your own words.
- Apply the distinction to the worked example below. Check the full lesson for assumptions and exceptions.
- Recall it again tomorrow, then after a few days. A memory hint is a retrieval cue, not a replacement for understanding.
Original academy illustrations and fictional exercises. These are learning simulations, not a real virtual machine, network assessment or measure of job readiness. Activities reset when you leave or reload; they do not change quiz scores.
What you will learn
- Define attacker access and permitted input changes.
- Evaluate relevant manipulated inputs.
- Record coverage limits and operational consequences.
Concepts and distinctions
Evasion concerns inputs crafted at inference time to induce an unwanted result. The attacker’s capabilities matter: can they observe confidence scores, query repeatedly, alter a few input features or control an entire document? A robustness claim without these assumptions is difficult to interpret or reproduce.
Tests should distinguish ordinary distribution changes from intentional manipulation, while acknowledging that a single observation may not establish intent. A defense that helps against one tested input family can fail under other conditions. Accuracy on a clean benchmark, average performance across users and resistance to a defined adversarial test are different claims. None alone proves safety in every deployment context.
Apply the process and collect evidence
Use approved test datasets and controlled environments. State acceptable modifications, system versions and success criteria, then examine effects on legitimate users as well as adversarial cases. Consider rejection, escalation and human review when confidence is insufficient. Document what was not tested and assess whether the mitigation creates unacceptable false alarms or denial of service.
Work through a scenario
Fictional facts for learning. This exercise is not scored.
A document classifier performs well on ordinary uploads but changes its decision when an attacker adds irrelevant-looking material. The proposed response is to reject all unusual documents automatically.
Compare your reasoning
Validate the threat assumption and affected behavior in a controlled test. Assess legitimate unusual documents and the consequences of rejection. Choose a response with suitable review and escalation, then retest both security effectiveness and ordinary service quality.
Remember
- A clean test set and an adversarial test ask different questions.
- Define attacker access and permitted input changes.
- Record coverage limits and operational consequences.
Orientation and further reading
- ISO: ISO/IEC 27090:2026 · Published AI-security guidance
- NIST AI 100-2 E2025: adversarial machine learning
Original examples and explanations. Background references do not establish complete exam coverage.
Try ISO/IEC 27090 practice questions · Report a content correction