benchmarks
"Reasoning models cannot determine when or how to revise what they have learned"
The holistic method addresses holes in AI agent benchmarks like cost and methods used.
Open-source benchmark uses almost real-world data and a simulated SOC.
Trillion-dollar industry can't even measure its biggest problem -- "calibration" not the answer, says OpenAI
The AI exam could be used to inform new regulations
Intentional? Or incidental to the nature of large-scale data scraping?