SatBench: 63% of the AI benchmarks labs cite are now beaten
- What happened
- At 00:46 Paris time, Wharton professor Ethan Mollick published SatBench, which tracks how fast major AI benchmarks become saturated (effectively beaten). Of 173 benchmarks cited by the labs since 2023, 63% are now saturated, and saturation is happening faster each year. Mollick says Fable 5.2 built the analysis in about four prompts.
- Why it matters
- Public leaderboards are losing their power to separate frontier models, so model choices increasingly rest on your own task evaluations. Relevant to you: benchmark claims in vendor pitches deserve less weight than a test on your own processes.