Human-approval steps in AI agents often aren't real safety controls
When a sensitive action like sending an email, moving money, or writing to a CRM has a human-approval step in front of it, teams often assume the system is safe — but that approval frequently isn't a real control. The first failure mode: the approval guards a path the agent never actually takes.
Teams coming from are used to an approval node sitting directly in front of a sensitive action node, genuinely blocking it. But once tools are attached to an agent, the agent's fire inside its own execution, downstream of that approval node — so the node ends up guarding an empty hallway, with nothing erroring to reveal the gap.
The second failure mode: the approval only confirms a description, not the actual call. Tool arguments are generated by the model, so when a human clicks "approve" on something like "send invoice to Acme for 1,200", they're approving the model's prose about its intent — not the actual argument values that get executed, which can change if the agent re-plans afterward.