Sign in Subscribe

benchmarks

OpenAI wants new AI benchmarks to replace Scale's "broken" tests

Image credit: https://unsplash.com/@joshua_hoehne

OpenAI

OpenAI wants new AI benchmarks to replace Scale's "broken" tests

"We retract our earlier recommendation to adopt SWE-Bench Pro."

Jul 9, 2026 2 min read

MLPerf's new inference benchmarks put NVIDIA on top, but...

AI

MLPerf's new inference benchmarks put NVIDIA on top, but...

Apr 3, 2026 3 min read

Claude is much better at calling bull on nonsense prompts

AI

Claude is much better at calling bull on nonsense prompts

Mar 9, 2026 3 min read

LLMs can't cook and can't adapt, says new paper, and more compute won't help

AI

LLMs can't cook and can't adapt, says new paper, and more compute won't help

Nov 4, 2025 2 min read

AI agent leaderboard challenges classic benchmarking

agentic ai

AI agent leaderboard challenges classic benchmarking

Kiera Fields

Oct 20, 2025 2 min read

Microsoft's ExCyTin benchmark targets agentic cybersecurity

Microsoft

Microsoft's ExCyTin benchmark targets agentic cybersecurity

Oct 16, 2025 2 min read

Hallucination evaluation is broken, says OpenAI - stop "suppressing" it

AI

Hallucination evaluation is broken, says OpenAI - stop "suppressing" it

Sep 9, 2025 4 min read

An exam hall with chairs and desks organised in a grid. Microsoft's new AI benchmark framework provides a series of tests to AI models

Microsoft

Microsoft has a new AI benchmarking model, but why should you care?

May 23, 2025 5 min read

AI benchmarking scandal: Were top models caught gaming the system?

AI

AI benchmarking scandal: Were top models caught gaming the system?

Jan 13, 2025 6 min read