An agent is only as safe as the tools you gave it

Ask how an agent is kept safe and the answer usually involves the system prompt. It has been told not to do certain things, to refuse certain requests, to be careful with customer data. That instruction is real and it is the weakest layer in the system.

Almost all of the safety of an agent comes from what it can physically reach. That is the tool layer, and it is an engineering decision rather than a wording one.

Why prompt-level rules are not a security model

Instructions and data arrive in the same channel. An agent that reads an email is reading text somebody else wrote. If that text says to ignore previous instructions, the model has no structural way to know which of the two texts has authority. Careful prompting reduces the success rate; it does not change the shape of the problem.

Coverage is unknowable. A rule prohibits what you thought of. There is no way to enumerate what you did not, and no test that proves the list is complete.

The blast radius is unchanged. Even a perfectly obedient agent with broad database access is one bug away from doing something large. Instructions constrain intent, not capability.

What the tool layer does instead

The principle is ordinary software security, applied somewhere new: the agent gets narrow, explicit capabilities rather than general access, and the boundary is enforced by code that does not read prompts.

Typed operations, not general access. Not a database connection — get_order(order_id) and draft_reply(order_id, text). What is not exposed cannot be done, regardless of what any text persuades the model to attempt.

Permissions at the tool, checked server-side. The tool validates that this agent, for this user, may touch this record. Checked where the data is, not where the reasoning is.

Value and rate limits in code. A refund tool that refuses above a threshold does not need the agent to be careful. The limit holds whatever the model was told.

Irreversible actions behind approval. Anything that spends money, contacts a customer or makes a commitment returns a draft for a person rather than performing the action.

Data minimisation by construction. If the agent never needs salary fields, the tool never returns them. Data that is not reachable cannot leak, and this removes a whole class of incident rather than mitigating it.

Output validation. Structured results parsed and checked before anything acts on them, so a malformed or manipulated response fails closed rather than propagating.

The design question that matters

For each capability, ask what the worst outcome is if the model is fully manipulated into using it as badly as possible.

If the answer is a wrong draft, the tool can be open. If the answer is a payment, a deletion or a customer receiving something false, the tool needs a limit, an approval or both. That single question generates most of the design, and it is answerable by people who are not security specialists.

The unglamorous consequence

Building agents this way is more work than writing a good prompt, and most of the effort goes into integrations rather than into anything that looks like AI. It is also what makes the difference between something you can put in front of a real process and something that stays a demonstration because nobody will sign off the risk.

That tool layer is where most of our time goes in an AI agents engineering engagement, and it is the part we would ask any other supplier about first.