benchmarks
Claude is wildly outperforming GPT and Gemini in BullshitBench. Its creator thinks some vendors may be losing touch with fundamentals.
"Reasoning models cannot determine when or how to revise what they have learned"
The holistic method addresses holes in AI agent benchmarks like cost and methods used.
Open-source benchmark uses almost real-world data and a simulated SOC.
Trillion-dollar industry can't even measure its biggest problem -- "calibration" not the answer, says OpenAI
The AI exam could be used to inform new regulations
Intentional? Or incidental to the nature of large-scale data scraping?