preprint

A new benchmark, TasteVal, says the best AI model now reaches expert-level results on eight AI research tasks with 2.3 times less compute, though its authors list reasons that may overstate progress. Preference Model has open-sourced Karotte, a framework for training environments that are harder to game, and DatologyAI has released Zephon, a data loader that keeps experiments repeatable when the GPU count changes. What each claims, and who measured it.