Google Rolls Out Gemini 2.5 Pro With Top AI Benchmark Scores and Cost-Effective Performance

By
Paras Guglani
-
0
8

Google has officially rolled out the Gemini 2.5 Pro (Preview 06-05 Thinking) model, just weeks after unveiling its capabilities at Google I/O 2025. The updated model builds significantly on its predecessor with leading performance scores, particularly in AI reasoning, coding, and multilingual tasks, while maintaining an efficient pricing structure.

The latest release shows a 24-point Elo gain on LMArena, reaching a top-tier score of 1470, and adds 35 Elo points on WebDevArena, now leading that leaderboard at 1443. These enhancements affirm Gemini 2.5 Pro’s dominance across various AI benchmarks, surpassing major competitors like OpenAI’s o3 and o4-mini, Claude Opus 4, and DeepSeek R1.

Performance-wise, Gemini 2.5 Pro leads in multiple categories:

  • GPQA (Science): 86.4%
  • AIME 2025 (Math): 88.0%
  • LiveCodeBench (code generation): 69.0%
  • Aider Polyglot (code editing): 82.2%
  • SWE-bench: 59.6% (single-agent), 67.2% (multi-agent)
  • HLE (Humanity’s Last Exam): 21.6%
  • SimpleQA (Factuality): 54.0%
  • FACTS Grounding: 87.8%
  • MMMU (Visual Reasoning): 82.0%
  • Vibe-Eval (Image): 67.2%
  • VideoMMMU (Video Reasoning): 83.6%
  • MRCR v2 (Long Context Handling): 58.0% at 128K, 16.4% at 1M
  • Global MMLU (Multilingual Understanding): 89.2%

Beyond performance, the update includes features aimed at developers such as “thinking budgets”, allowing better control over cost and latency during model use.

The pricing model is also notably competitive:

  • $1.25 per million input tokens, with caching rates between $0.50 and $2.00
  • $10 per million output tokens, with caching from $1.00 to $15.00

This undercuts rivals:

  • OpenAI o3: $10 input / $40 output
  • Claude Opus 4: $15 input / $75 output
  • DeepSeek R1: $0.55 input / $2.19 output (but with weaker benchmarks)

According to Tulsee Doshi, Senior Director of Product Management at Google, the improvements reflect community feedback—delivering better formatting, creative responses, and control features for developers and enterprises alike.

Though still in preview, Gemini 2.5 Pro is available to developers via the Gemini API on Google AI Studio and Vertex AI. Additionally, the Gemini app began rolling out this version on June 5, 2025, and a stable release is expected soon for general enterprise use.

For more updates on cutting-edge AI advancements, visit PassionateGeekz.com.

1 COMMENT