Hugging Face's logo
ml-explorer
Profile Inbox (0) Settings Billing Get PRO
Organizations
Demo Org Create New
Resources
Getting Started Documentation Forum Tasks Learn
Back to feed
HuggingNews 
published a story in AI Models about 11 hours ago

Opus 5.5 Tops New SemiAnalysis GDPx AI Benchmark With 12% Score

AI Models• AI Research • 16 sources • 17 posts
via @semianalysis_ on X

SemiAnalysis has introduced a new performance test called GDPx to assess how effectively artificial intelligence models can handle autonomous tasks across 11 specialized fields, including chip design, cybersecurity, and RSI. The highest score recorded on the benchmark was 12%, achieved by the model Opus 5.5, highlighting the difficulty of the new set of tests.

Google CEO Sundar Pichai noted the 'cost line' and efficiency of the company's Carbon model in response to the results, which he suggested provide a measurement of the remaining development potential in agentic AI. The benchmark's low top score indicates that even frontier models struggle to complete the complex professional tasks the test is designed to simulate.

Sources
  • @semianalysis_ @semianalysis_ ·about 12 hours ago ·Source Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵 https://t.co/RNnkcuUkih GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)
  • @dylan522p @dylan522p ·about 11 hours ago ·Commentary Official apology to Google from me, they cooked with Carbon. It's spikier in terms of domains and capabilities, but it's really good in a few areas. Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵 https://t.co/RNnkcuUkih they cooked with Carbon. It's spikier in terms of domains and capabilities, but it's really good in a few areas.
  • @jordannanos @jordannanos ·about 9 hours ago ·Commentary and just like that, all my cybersecurity alpha is gone https://t.co/tcIZ74K1Nw Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵 https://t.co/RNnkcuUkih and just like that, all my cybersecurity alpha is gone
  • @techmeme @techmeme ·about 9 hours ago ·Commentary SemiAnalysis launches GDPx, an agentic capability benchmark measuring 11 domains, including chip design, cybersecurity, and RSI; Opus 5.5 tops with a 12% score (SemiAnalysis) (Visit Techmeme dot com for the link and full context!) SemiAnalysis launches GDPx, an agentic capability benchmark measuring 11 domains, including chip design, cybersecurity, and RSI
Read the full story on HuggingNews →
·
semianalysis_

Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains.

GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵 https://t.co/RNnkcuUkih

dylan522p

Official apology to Google from me, they cooked with Carbon.
It's spikier in terms of domains and capabilities, but it's really good in a few areas.

sundarpichai

Great to see high quality benchmarks with headroom and look at that cost line for Carbon🚀

jordannanos

and just like that, all my cybersecurity alpha is gone https://t.co/tcIZ74K1Nw

In this story