Instruction drift occurs when an AI agent’s actions, decisions, or tool calls move away from the user’s original instruction. The agent may still appear to be completing the task, but its behavior has changed in scope, purpose, or authorization.
For financial institutions, this matters because an agent can read sensitive data, call APIs, initiate payments, or change account settings. A small change in interpretation can therefore become an unauthorized action. The control question is direct: did the agent do what the customer authorized, and nothing beyond it?
Instruction drift usually occurs when new content, model outputs, memory, or tool results influence an agent more strongly than the original task constraints. The source may be malicious, ambiguous, irrelevant, or simply misinterpreted by the model.
These causes are related, but they are not identical. Prompt injection is a manipulation technique. Instruction drift is the resulting loss of alignment between the agent’s behavior and the authorized task.
These risks make it important to define and enforce the limits of delegated authority, so an agent cannot expand its permissions beyond the task the user approved.
Instruction drift is a change in what the agent is trying to do. Hallucination is an inaccurate or unsupported statement, interpretation, or result. An agent can hallucinate while following the right task, or drift while producing information that appears factually correct.
The distinction matters for control design. Accuracy checks can identify unsupported content, but they may not reveal that an agent has silently expanded the task. Scope validation must compare the planned and completed actions with the original instruction.
Instruction drift becomes a security issue when an agent can affect identity, money, access, or regulated data. An agent asked to summarize a transaction policy might instead send a message. An agent asked to prepare a payment could select a different beneficiary or exceed an approved limit.
In digital banking, the user’s identity is only one part of the trust decision. The institution also needs to understand the agent’s authority, the requested action, the transaction context, and any change between the original instruction and the final event.
This is closely related to trusted-agent design. A trusted agent needs verified identity, delegated authority, defined limits, revocation, and evidence of what it did. Instruction drift weakens that model by breaking the link between authority and action.
Entersekt explores the wider risks of agent adoption in When bots break the rules: How banks can safely harness AI for growth, including data exposure, access control, and evolvin fraud threats.
| Stage | Authorized task | Drifted behavior | Potential impact |
|---|---|---|---|
| Research | Compare payment providers | Contact a provider or disclose internal requirements | Unapproved external communication |
| Account support | Explain a failed login | Change recovery details | Account takeover risk |
| Payment preparation | Prepare a transfer for review | Submit or amend the transfer | Unauthorized movement of funds |
| Fraud investigation | Summarize alerts | Close an alert or alter a case record | Loss of control evidence |
| Document processing | Extract invoice details | Follow instructions embedded in the invoice | Data disclosure or tool abuse |
Detection requires an independent comparison between the user’s instruction, the agent’s plan, the tool calls it makes, and the final outcome. Checking only the final response can miss unauthorized intermediate actions.
Banks need to know when the bot has permission explains why accurate context and connected cuannel data matter when automated systems inform financial decisions.
Research on task drift describes a similar principle: the system should be assessed for changes in its objective after it processes external input, rather than relying only on the text it generates. See Microsoft Security Response Center research on task drift for the underlying concept.
Prevention works best as a layered control model. A system prompt alone cannot enforce authorization if tools accept unrestricted parameters or if external content is treated as a trusted instruction.
Put these controls into practice by treating external content as untrusted, in line with OWASP’s guidance on prompt injection, and limiting tools to the authorized task. Validate actions independently against transaction controls, using verifiable intent to connect sensitive actions to explicit approval. Keep delegated authority time-bound and revocable, and test the complete workflow for unauthorized actions and side effects.
Instruction drift is an emerging term, so organizations will usually map it to established security and governance categories rather than treat it as a standalone compliance requirement.
These references do not remove the need for institution-specific control design. The relevant safeguards depend on the agent’s permissions, data, channels, customers, and potential business impact.
These controls sit within the four shifts redefining trust in digital payments, which connect identity, intent, risk, and evidence as payment journeys become more automated.
Instruction drift is the loss of alignment between a user’s authorized objective and an AI agent’s subsequent reasoning, tool use, delegation, or action.
For financial institutions, the practical response is to make intent explicit, keep untrusted content separate, restrict agent authority, validate actions independently, and preserve evidence. Authentication confirms who is acting. Scope and intent controls help confirm what that actor authorized.
No. Prompt injection is one way to manipulate an agent. Instruction drift is the broader outcome in which the agent’s behavior diverges from the original task, regardless of whether the cause is malicious input, ambiguity, memory, or model error.
Yes. Ambiguous instructions, long contexts, weak tool boundaries, faulty planning, and excessive permissions can cause drift without deliberate exploitation. A secure design should therefore test benign failure modes as well as adversarial inputs.
Financial agents may interact with accounts, payments, identity records, and customer data. A drifted action can therefore create fraud, privacy, compliance, or operational risk. Independent authorization and auditable approval are essential for high-impact actions. For agent-initiated payments, a trusted surface provides a controlled interface where the customer can review the proposed action and give authenticated approval.
Entersekt connects authentication, device trust, transaction context, and risk-aware decisioning across digital banking and payment journeys. Those controls can help institutions assess who authorized an action, what was approved, and when additional verification is warranted.
Start by recording the user’s intent as structured constraints before the agent acts. Include the permitted action, target, amount, data, tools, time window, and approval conditions. Then compare every high-impact tool call with those constraints.