HomeTagsBenchmarks

Benchmarks - Page 5

Stories from Benchmarks come first, newest first, then stories from connected topics (AI Models, LLM, AI, coding agents, preprint, open weights), each labelled with where it comes from. This view is not indexed by search engines.

Latest

Arena Raises $200M at $3.1bn Valuation to Rank AI Models

Arena, the public AI model leaderboard, raised $200m at a $3.1bn valuation and says it has passed $100m annualised revenue, up from $30m in January.

Scale AI’s DistressBench: Chatbots Spot Crises but Often Skip Help

Scale AI built and graded a benchmark of 718 clinician-written crisis conversations. Pooled across 25 models, in 35.3% of conversations where a model fully recognised the crisis, it still failed to route the user towards help. What the company research shows, how it was scored, and its limits.

Claude Agents Help Build a UV Sky Map as Studies Question Agent Scores

Anthropic says a team of Claude Science agents helped build the first complete UV map of the sky, about a third of it predicted rather than observed. A preprint finds agent benchmark scores can move because of the grader, not the agent, and Exa previews a search benchmark it designed and runs itself. What each one shows, and what it doesn't.

AI Agent Money, 8 October: Hone, Vesta and Catalyst Raise Funding; Crossmint Launches Agent Checkout

Three AI-agent start-ups announced funding on 8 October: Hone ($60M seed), Vesta ($30M, reported) and Catalyst ($30M seed, reported). Crossmint launched a toolkit that lets AI agents pay by card. What each company says, who reported what, and which figures are company claims.

AI Research Tools: TasteVal Measures “Research Taste”, Karotte Targets Reward Hacking and Zephon Fixes Data Order

A new benchmark, TasteVal, says the best AI model now reaches expert-level results on eight AI research tasks with 2.3 times less compute, though its authors list reasons that may overstate progress. Preference Model has open-sourced Karotte, a framework for training environments that are harder to game, and DatologyAI has released Zephon, a data loader that keeps experiments repeatable when the GPU count changes. What each claims, and who measured it.

AI Trails People on Visual Reasoning Test; Base Models Reason on Cue

Scale AI and Elorian's Humanity's Sixth Sense benchmark finds people score 93.1% on everyday visual questions while the best model, GPT-6 Astra, scores 53.6%, mostly because models misread the picture. A separate preprint, not yet peer reviewed, says base models can reason almost as well as RL-trained ones when their answer starts with the right word.

Grok 4.7 on VulcanBench Frontier v4: what secondary trade press says

Secondary CryptoBriefing coverage says xAI’s Grok 4.7 holds 1st–3rd on VulcanBench Frontier v4. Primary VulcanBench/xAI confirmation was not checked here.

Mistral Large 4 Preview: Artificial Analysis scores it 38

Third-party firm Artificial Analysis scored Mistral Large 4 Preview 38 on Intelligence Index v4.3.2. Company training and open-weight claims are labelled separately.

GitHub’s ReviewBench: A Shared Scorecard for AI Code Reviewers

GitHub has released ReviewBench, an open benchmark that scores AI code-review agents on 219 real public pull requests. Here is what it measures, and why the fine print matters.

EU Education Ministers Adopt AI Skills Council Conclusions

EU education ministers adopted non-binding conclusions on skills to engage with AI on 19 November, with no new funding, inviting use of existing tools.

Latest on TSN

Explore the TSN Atlas: Every topic we cover, mapped. Click a bubble to dive in.