HomeTagsBenchmarks

Benchmarks - Page 25

Stories from Benchmarks come first, newest first, then stories from connected topics (AI Models, LLM, AI, coding agents, preprint, open weights), each labelled with where it comes from. This view is not indexed by search engines.

Latest

Underdog’s Saluki 27B Shrinks Qwen3.8-27B to 7.89 GB, at a Maths Cost

Underdog's Saluki 27B is a 7.89 GB, 2-bit version of Qwen3.8-27B. Underdog says it passes more tool-calling tasks than the full model on its own 120-task test but scores 15 to 18% lower on competition maths. What quantisation means, and what the numbers do not prove.

Do AI Agents Overstep or Fib? Two New Ways of Checking

Two new attempts to measure whether AI agents behave: Arena's Alignment Index preview, which scores 27 models on real user sessions (Arena's own claims, published the day it raised $200 million), and FAR.AI's "Caught in the Act" preprint on probes that read a model's internals to spot deception. What each found, and the limits.

Firmus Pulls Its ASX IPO as Isomorphic Labs Eyes a $40bn Valuation

Three AI business items from 8–9 October 2026: Nvidia-backed Firmus withdraws its ASX listing and turns to private money; Bloomberg reports Isomorphic Labs is in early funding talks at a valuation of at least $40 billion; and StepFun's Step 5 Preview model is now listed on OpenRouter and Vercel's AI Gateway. What is confirmed, and what is only reported.

Harness Buys Augment Code Assets, Plus TRL 1.15 and Genkit Dart 1.0

Four AI developer-tool updates from 8–9 October 2026: Harness buys selected Augment Code assets (not the whole company, price undisclosed), Hugging Face's TRL v1.15.0 claims up to 6.9× longer training sequences, Google's Genkit Dart reaches 1.0, and NVIDIA shows its Boro kernel-review tool. What each says, and what it doesn't.

Minimal Coding Agent Matches Elaborate AI Harnesses, Preprint Finds

A preprint from EPFL researchers, listed on Apple's research site, finds a single minimal coding-agent session matched or beat elaborate machine-learning agent systems in its tests. Pasqal releases skills that let coding agents prepare quantum experiments, and Microsoft makes 30 Dynamics 365 CRM skills generally available, with Autopilot still in private preview.

Robotics Roundup: How Robot World Models Scale, a Beijing Funding Round and a Shop-Floor Humanoid

Three robotics items from 8 October 2026: a FAIR at Meta and Mila preprint on how robot "world models" scale, a reported Series A for Beijing's Livsyn Robotics, and CP AI Robotics' C1 store humanoid, which the company plans to roll out in Japanese convenience stores "as early as 2027". What each shows, and what it doesn't.

Two Open Models From 8 October: a Fast Coding Agent and a Gene Finder

JetBrains has released Mellum2.1, a small open model for coding agents, and Hugging Face has released Carbon-A, a 1.2B-parameter model that predicts protein-coding regions in DNA, plus a database of 566 million predicted loci. What each does, and which numbers are the developers' own.

Same AI Model Scores 33% or 62% Depending on Its Coding Tools

Hugging Face's FineEnvs team has released an open framework that turns unmodified coding-agent harnesses such as Claude Code, Codex and OpenCode into reinforcement-learning environments. In its own tests, one small model scored 62% in one harness and 33% in another. What a harness is, how the training works, and what the results do and do not show.

NVIDIA Jetson Demo Runs a Gemma 4 Vision Kiosk On-Device

An NVIDIA tutorial shows OpenAI's Codex coding agent turning a Jetson Orin Nano into a self-starting kiosk that captions a live webcam feed with Google's small Gemma 4 E2B model, running on the device. What the demo shows, what "No Mac required" means, and why local inference matters.

Quantum Hardware Check: DARPA’s Final Test Stage, Universal Quantum’s $100m Round and New Error-Correction Results

DARPA has moved Atom Computing, Diraq, IBM and IonQ into the final, hands-on stage of its Quantum Benchmarking Initiative, and says it increasingly expects a useful quantum computer by 2033. UK trapped-ion firm Universal Quantum has raised more than $100m, IonQ claims big speed-ups for error-corrected operations in simulations, and a peer-reviewed Cornell paper shows a full gate set on a qubit that partly tolerates its main error. Plus a Jülich preprint and Infineon's ion-trap chips. What each confirms, and what is only claimed.

Latest on TSN

Explore the TSN Atlas: Every topic we cover, mapped. Click a bubble to dive in.