LEASH Escalates to Frontier Models Only on Latent Uncertainty, Matching Always-On Claude Opus 5 on OSWorld at 19.2% Lower Inference Cost
A new arXiv paper keeps a small open-weight agent in control of computer-use tasks and hands off to a frontier model only when the weak model's hidden states resemble past failures. Authors report OSWorld success matching always-on Claude Opus 5 at 19.2% lower inference cost.
Running a frontier model at every step of a long computer-use task is expensive, and running a small open-weight model for the whole job usually ends in failure. A paper posted to arXiv on October 15, 2026 proposes a middle path that reads less like a benchmark entry and more like an operating policy: keep the cheap agent in control by default, hand off to the strong agent only when the cheap one’s own internal state signals trouble, and keep watching the strong agent after the handoff.
The paper is LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents by Qingni Wang, Changdae Oh, Yuzhe Yang, Sharon Li, Jiachen Yang, Ang Li and Xin Eric Wang. It reached Hugging Face Daily Papers on October 16 and has a dedicated Hugging Face paper page. Everything below comes from the authors’ abstract: the paper is a v1 preprint, and the numbers have not been independently replicated.
Two problems, one signal
Weak-to-strong escalation, where a small model acts by default and a frontier model is called in only when needed, is a familiar cost pattern. The paper names two reasons it is hard in practice:
- Deciding when to escalate needs an uncertainty signal cheap enough to check at every step. Asking a second model or running an extra verification pass reintroduces the cost you were trying to avoid.
- Escalation does not end responsibility. After handoff the strong agent can still get lost, and because it is usually served as a black-box API, there are no internals to monitor.
LEASH answers both with latent uncertainty: it compares the weak model’s current hidden states against states recorded from its own past successful and failed trajectories, and a state that looks more like the failures means higher uncertainty. The estimator reuses activations already produced during action generation, so it needs no extra forward pass and no dedicated monitor training, and the whole framework is training-free.
That single signal powers three layers:
- Initial task uncertainty routes at task start, sending jobs the weak agent is unlikely to finish to the strong agent early.
- Step-level uncertainty escalates at runtime: the weak agent keeps acting until its hidden states drift toward the failure pattern.
- A shadow observer keeps oversight after handoff, using the same uncertainty machinery to watch the black-box strong agent’s steps.
What the authors report
On OSWorld, the benchmark where agents drive a real desktop, the authors tested LEASH across four contemporary model backbones. The headline results:
| Configuration | OSWorld task success | Cost per task |
|---|---|---|
| LEASH with a Qwen3.5-35B-A3B weak agent | 73.66% | $0.4117, reported as 19.2% below the always-on Opus 5 baseline |
| Always-on Claude Opus 5 | 72.90% | baseline; per-task cost not reported by the abstract |
The paper says LEASH consistently improves over standalone weak agents and outperforms random escalation by 6.55 to 11.94 points at matched escalation rates. With Qwen3.5-35B-A3B, it claims LEASH matches always-on Claude Opus 5 while cutting inference cost by 19.2%.
What operators can take from this
- The shadow-observer idea stands alone. Even if you never touch hidden states, “keep a cheap watcher running after the expensive model takes over” is a pattern any fleet can adopt today. It pairs naturally with the trajectory-review discipline in Review the Trajectory, Not the Reel: OSWorld 2.0’s Checkpoints for Your Own Computer-Use Runs.
- Latent escalation requires white-box access to the weak agent. The signal comes from hidden states, so it works when the weak model is self-hosted and open-weight, and it cannot see inside the strong API model. That strengthens the case for keeping an open-weight model as first responder.
- Price the outcome, not the token. $0.4117 per completed task is exactly the framing of Cost per Completed Task Beats $/MTok, and self-hosted weights are not free either, as “Free” Weights, Paid Tasks details.
- You need a labeled trajectory library. The uncertainty signal is grounded in past success and failure runs, so teams that already archive agent trajectories with outcome labels are the ones positioned to try this.
Caveats before adoption
- One benchmark. All results are on OSWorld. Long-horizon desktop work is exactly where the paper lives; browser-only or terminal-only fleets may see different escalation behavior.
- Author-reported numbers. 73.66% versus 72.90% is a narrow margin, and the 19.2% saving is measured against inference cost, not the total operating bill that includes hosting the weak model and storing trajectories.
- Preprint status. Submitted October 15, 2026, with no peer review or independent replication yet.
For teams running mixed fleets of cheap open-weight agents and frontier APIs, LEASH is a concrete sketch of the middle path: escalate on evidence from the weak model’s own internals, and never let the strong model run unwatched.