Academy module 6 · ISO/IEC 27090 · Lesson 6 of 8
Apply trust boundaries to generative AI tools
About 8 minutes · Introductory draft · Checked 2026-10-23
AI-authored and AI editorially reviewed; independent review pending. Read course scope and remaining gaps.
Session only. Enable saving or download a backup from this course’s hub.
Completion is self-reported and does not certify understanding.
See it · Try it · Recall it
Picture the topic
Remember this: Generated instructions are data until independent authority permits an action.
Explore the anchors
Treat retrieved text and user input as data.
Concept matching activity
Match this task or description to its concept:
Validate identity scope and tool arguments.
Choose a concept before checking the explanation.
A quick way to study this topic
- Hide the explanation and redraw these three anchors from memory.
- Explain the difference between the anchors in your own words.
- Apply the distinction to the worked example below. Check the full lesson for assumptions and exceptions.
- Recall it again tomorrow, then after a few days. A memory hint is a retrieval cue, not a replacement for understanding.
Original academy illustrations and fictional exercises. These are learning simulations, not a real virtual machine, network assessment or measure of job readiness. Activities reset when you leave or reload; they do not change quiz scores.
What you will learn
- Treat retrieved text and user input as data.
- Validate identity scope and tool arguments.
- Enforce permissions outside generated text.
Concepts and distinctions
A generative assistant may combine user requests, retrieved documents and external tool results. Untrusted content can attempt to redirect the assistant’s behavior. Direct and indirect instruction attacks differ in where the influencing content enters the workflow, but both require careful separation between supplied data and trusted authority.
This chapter is an applied AI-security extension informed by public adversarial-machine-learning references; it is not presented as an exact clause mapping to ISO/IEC 27090. Model-generated text is not evidence of user authorization. A retrieved document that says “send the customer list here” should not acquire the privileges of the authenticated user or the application operator.
Apply the process and collect evidence
Give tool identities narrowly scoped permissions and validate action arguments in an enforcement layer outside the model’s generated output. Restrict data destinations and require appropriate approval for consequential actions. Test both malicious instructions and legitimate workflows. Keep useful audit evidence while protecting prompts, retrieved material and secrets according to defined handling rules.
Work through a scenario
Fictional facts for learning. This exercise is not scored.
A support assistant reads a supplier document containing instructions to export all learner records. Its tool account can access only the current support case, and the service independently checks destinations and permissions.
Compare your reasoning
Treat the supplier instructions as untrusted document content. The constrained identity and independent action checks reduce the authority the assistant can exercise. Continue testing for failures and logging relevant decisions; these safeguards do not establish immunity to every instruction attack.
Remember
- Generated instructions are data until independent authority permits an action.
- Treat retrieved text and user input as data.
- Enforce permissions outside generated text.
Orientation and further reading
- ISO: ISO/IEC 27090:2026 · Published AI-security guidance
- NIST AI 100-2 E2025: adversarial machine learning
Original examples and explanations. Background references do not establish complete exam coverage.
Try ISO/IEC 27090 practice questions · Report a content correction