Score Gemini Computer Use Without Invented Claude Cells

Google put the Gemini 2.5 Computer Use model into public preview on the Gemini API on October 7, 2025, and you need a browser agent that clicks, types, and scrolls through real UIs without picking an API from a demo reel. Here is the shortest route that stays on what that launch post actually publishes.

Published Online-Mind2Web and latency rows beat another week of agent demos.

Google’s post already narrows the surface: primarily optimized for web browsers, strong promise on mobile UI control, and not yet optimized for desktop OS-level control. Start from that scope line and from the Online-Mind2Web latency claim Google attaches to the Browserbase harness, not from a product demo.

1. Open Google’s launch post and freeze the claim set. Read Google DeepMind’s October 7, 2025 announcement. Note three commitments only: the model is specialized on Gemini 2.5 Pro visual understanding and reasoning, it is exposed through a computer_use tool meant to run in a loop, and Google states it outperforms leading alternatives on multiple web and mobile control benchmarks with lower latency.
2. Copy Google’s benchmark framing before you invent an Anthropic column. Google’s table mixes self-reported numbers, evaluations Google ran, and evaluations run by Browserbase. The latency-and-quality claim Google highlights for browser control is performance on the Browserbase harness for Online-Mind2Web. Record every alternative Google names in that announcement. Google’s October 7, 2025 post claims outperformance against leading alternatives and does not publish Claude or Anthropic numeric rows for that harness. Leave Anthropic blank on every axis without a cited same-harness figure. Do not fill gaps from memory.
3. Score only the axes the launch post supports in prose. Google states the model outperforms leading alternatives on multiple web and mobile control benchmarks with lower latency, and that it offers leading quality for browser control at the lowest latency as measured on the Browserbase harness for Online-Mind2Web. The announcement text does not reproduce the numeric table cells, so log those qualitative claims with source and date rather than fabricating percentages. Scope from Google’s own limit line: primarily optimized for web browsers, strong promise on mobile UI control, not yet optimized for desktop OS-level control. API price cards are outside this announcement, so leave cost unscored here.
4. Keep Anthropic computer use off the scored axes until a same-harness citation exists. The buyer question is still Gemini versus Anthropic, but Google’s Computer Use launch post does not supply Anthropic Online-Mind2Web cells, Anthropic tool I/O specs, confirmation-gate rules, or Anthropic pricing. Mark those fields “not disclosed in Google’s October 7, 2025 Computer Use announcement” and keep going. Do not import Claude API details from memory to complete the sheet.
5. Run one client loop only after the sheet is filled. For Gemini, follow Google’s published path: Gemini API in Google AI Studio or Vertex AI, computer_use tool, local Playwright loop or a Browserbase-hosted environment. Feed user request, screenshot, and recent action history. Execute the returned UI action in your client. Return a new screenshot plus current URL. Stop when the task completes, errors, hits a safety stop, or asks for end-user confirmation. Log step count and wall time. One scripted browser task is enough for this audit.

If Google’s announcement left an alternative unnamed on the Online-Mind2Web harness, that vendor stays blank on your sheet.

The Gemini computer_use tool expects a client loop with screenshots and action history.

Google’s post is explicit about the runtime contract. Google’s published path puts execution in client-side code: the model returns a UI action, and your loop performs it.

Inputs to the tool are the user request, a screenshot of the environment, and a history of recent actions. You may exclude functions from the supported UI action list or add custom functions. The model typically returns a function call (click, type, scroll, and similar). For some actions, including making a purchase, the response may instead request end-user confirmation before your client proceeds.

After your client executes an allowed action, you send a fresh screenshot and the current URL back as the function response. The loop continues until completion, error, safety termination, or a user decision.

“The Gemini 2.5 Computer Use model is primarily optimized for web browsers, but also demonstrates strong promise for mobile UI control tasks. It is not yet optimized for desktop OS-level control.” — Google DeepMind, October 7, 2025

Google also lists production-shaped uses already inside its own stack: UI testing, versions powering Project Mariner, the Firebase Testing Agent, and some agentic capabilities in AI Mode in Search. Early-access users in Google’s program cite personal assistants, workflow automation, and UI testing. Those are category signals, not your benchmark substitute.

Safety is part of the published path, not an appendix. Google says it trained guardrails for intentional misuse, unexpected model behavior, and prompt injections or scams in the web environment, with details in the Gemini 2.5 Computer Use System Card. Developers also get a per-step, out-of-model safety service that assesses each proposed action before execution, plus system instructions that can force a refuse or a confirmation on high-stakes classes of action. Google’s examples of actions developers may want to block or gate include harming system integrity, compromising security, bypassing CAPTCHAs, and controlling medical devices.

Missing Claude cells and self-reported rows produce false API swaps.

The first failure mode is a spreadsheet with invented Anthropic scores. Google’s announcement claims outperformance against leading alternatives and points to Browserbase plus its own evaluation info for details. That October 7, 2025 post does not publish Anthropic or Claude numeric rows for the Browserbase Online-Mind2Web harness. If your Anthropic cell has no cited same-harness source, you do not have a comparison. You have a Gemini marketing claim with empty counterpart cells.

What it looks like: a slide that says “Gemini wins Mind2Web” with no harness name, no date, and no sourced Anthropic figure. What to do: reopen Google’s announcement, paste only claims and named evaluation sources Google prints, and mark undisclosed alternatives as undisclosed.

The second failure mode is mixing self-reported vendor scores with third-party harness scores in one average. Google states the table includes self-reported numbers, Browserbase evaluations, and Google-run evaluations, and that unless otherwise indicated the scores are for computer-use tools exposed via API. Averaging those as if they were one protocol invents a precision nobody published.

What it looks like: a single “accuracy” percentage in your design doc with no footnote. What to do: keep three columns (self-reported, Browserbase, vendor-run) and decide on the Browserbase Online-Mind2Web column when latency is in scope, because that is the harness Google uses for the quality-at-lowest-latency claim.

The third failure mode is treating confirmation prompts and the per-step safety service as random flakiness. Google documents that certain actions require end-user confirmation and that an inference-time safety service can interpose before execution. Your wall-clock “latency win” disappears when the loop is waiting on a human or a deny.

What it looks like: a CI job that hangs on checkout or login flows, or a sudden refuse on CAPTCHA-adjacent steps. What to do: classify steps that need confirmation up front, wire the confirmation UI, and exclude blocked action classes in tool config instead of retry-spamming the model.

The fourth failure mode is demo contamination. Google’s own demos (pet-care signup into a spa CRM, sticky-note board sorting) show multi-site browser work at 3× playback and illustrate the loop; only harness rows from Google’s cited evaluations support a vendor comparison against Anthropic computer use for production traffic.

Desktop OS agents fail against a browser-optimized computer use model.

If your agent must control a desktop OS (native installers, OS settings panes, non-browser apps) as a first-class surface, Google’s launch post already tells you Gemini 2.5 Computer Use is the wrong primary tool today. The model is optimized for browsers, shows promise on mobile UI control, and is explicitly not yet optimized for desktop OS-level control. Ranking it against Anthropic computer use on Online-Mind2Web will not repair that scope gap.

The same wrong-tool case applies when the sheet still has blank Anthropic cells because Google’s announcement never published them. Running the five checks above against empty counterpart rows does not produce a Gemini-versus-Anthropic ranking. It only reconfirms Google’s own browser-scope claims.

Skip the entire Gemini-versus-Anthropic benchmark procedure when the work is desktop OS control. Google’s October 7, 2025 post already places that workload outside this model’s optimization target, so the Online-Mind2Web latency contest is the wrong gate for that ship decision.