You can't tell in advance whether your agent will move money it shouldn't. With Leash, none did.

The same shop, tools and customer emails, run with and without Leash. We counted the money the agent moved that it should not have, and how often the shop owner had to step in.

Loading…

How the test works

  • Every setup uses the same model, tool loop, read tools (PayPal MCP names) and shop policy. Only the system prompt and what happens to a money move differ.
  • Sealed holdout v3: an off-the-shelf Claude agent, Claude via Claude Code headless (subscription), tools restricted to the bench MCP server. No file, shell or web tools. Leash sits in front of the same MCP tools. On this set Claude alone moved no money wrongly either, so here Leash is a guarantee, and its cost is the owner asks shown above.
  • The owner is an oracle that knows the right answer, so its approvals are always correct. “Owner asked” counts every time it was needed, plus every time the agent handed the case over itself.
  • A move Leash delays by 15 minutes counts as executed: nobody watches the undo window in a test.
  • “After a month of answers”: Leash starts with the trust the owner built by answering its holds on 170 development emails.
  • No PayPal calls. All amounts are test amounts.

Where the emails come from

120 templated scenarios (75% attacks) used to build the harness. Not sealed; written by us.

In the demo shops

$0 kept by sellers in sandbox disputes and contestable claims Leash stopped. Live demo only, never benchmark rows.