Browser Use is one of the open stacks teams reach for when they want a browser agent without building a harness from scratch. The useful benchmark question is not “does it demo well on one site?” It is whether it survives a broad, live-web mix of reads and writes.
Halluminate included Browser Use Cloud API in the Web Bench launch study. This page is the honest version of that result: what we ran, where the comparison is imperfect, and how to evaluate Browser Use yourself without fooling your own roadmap.
What we ran
Web Bench’s open set is ~2,454 tasks across 452 real websites, drawn from a larger ~5,750-task pool. Tasks are tagged READ, CREATE, UPDATE, DELETE, and file manipulation. Launch grading was human review of trajectories and outputs.
Browser Use was executed through its Cloud API. Because of cost, that configuration used a 658-task subset, not the full open set used for several other agents. Any head-to-head with full-set runs needs that caveat in the same sentence as the score.
Other systems in the same study included Skyvern 2.0, OpenAI Computer Use, Anthropic Computer Use, and human-in-the-loop Operator / Convergence baselines. Charts and protocol: /blog/benchmark. Public board: webbench.ai.
What the study taught us about this category
Even without turning this page into a fake precision shootout on the subset, the category-level findings apply directly to Browser Use deployments:
- READ is the easy mode. Strong agents often clear >70% of extraction-style tasks.
- WRITE is the product. Best fully automated NON-READ in the launch study: Skyvern 2.0 at 46.6%. Overall fully automated SOTA: Anthropic Computer Use at 66.0%.
- Infrastructure is a co-author of your score. Proxy, captcha, and auth failures were a major bucket—later isolated in BrowserBench, where infrastructure alone moved accuracy by large margins.
If your Browser Use workflow is “log in, fill the form, submit, confirm,” a read-heavy internal eval will lie to you.
How to benchmark Browser Use without self-deception
- Match the task mix to the product. Overweight CREATE / UPDATE / DELETE if that is what you sell.
- Prefer live sites for the gate, simulated sites for the loop. Use Westworld or staging apps to iterate; use Web Bench-style tasks before you call a release done.
- Freeze infrastructure when you compare agent versions. Same proxy, same stealth provider, same auth strategy.
- Grade outcomes, not screenshots of the agent looking busy.
- Report N. If you sample 658 tasks, say 658. Subset results are useful; silent subsets are marketing.
Where this sits next to Skyvern and WebArena
Skyvern’s launch-study write-task leadership is documented separately in Skyvern Web Bench scores. WebArena remains the right tool for reproducible research clones—see WebArena vs Web Bench—but it will not tell you whether Browser Use clears bot checks on a random retailer.
For the full map: browser agent benchmarks in 2026. Dataset: Halluminate/WebBench. For custom eval design, jerry@halluminate.ai.