Prompt injection and untrusted data

Prompt injection attempts to influence a model through hostile instructions embedded in a user request, retrieved document, or tool response. Delimit untrusted text and state its role, but do not rely on prompt wording alone. Keep secrets out of model context, narrow available tools, validate model outputs, and enforce authorization outside the model. Retrieval filters and content screening can reduce exposure but cannot prove that content is safe. Test direct and indirect attacks, including requests to reveal hidden instructions, ignore policies, or invoke an unrelated tool. Monitor attempts and define safe refusal or escalation behavior. A compromised answer should not be able to bypass application controls or perform an unapproved side effect. Indirect injection can arrive inside a document that appears relevant, a retrieved web page, or a tool response. Delimit and label these values as data, but assume the model may still follow hostile instructions. Keep secrets out of context, restrict the tool set, validate output against an allow-list, and enforce permissions outside the model. Create tests that ask the model to reveal hidden instructions, ignore policy, or call a write tool based on retrieved text. The desired security property is that even a manipulated response cannot cross the application’s authorization boundary. Implementation drill: trace one representative request through the stages in the diagram. Write down what enters each stage, what output or evidence it produces, and which identity is allowed to perform it. Then test an empty input, an unauthorized request, a dependency timeout, and an unexpected result. For each case, state whether the application should retry, reject, abstain, or ask a person for help. Keep the observed failure as a regression example so changes to prompts, models, data, or code can be checked against the same behavior. Practice task: Which is a reliable security boundary for a privileged action?