Federal Agencies to Establish New Benchmarking Process for AI Models by August 1

Federal agencies are required to create a classified benchmarking process to assess the capabilities of frontier AI models by August 1, 2026. This initiative comes as current testing methods are deemed inadequate for evaluating the evolving abilities of AI systems. Anthropic announced it is collaborating with Amazon, Google, Microsoft, and others to develop a standardized benchmark focusing on the outcomes of AI jailbreaks. Meanwhile, the testing lab Irregular has introduced a new cyber benchmark that evaluates AI models on their ability to perform offensive cyber tasks. Industry experts, including David Slater from Armadin, have noted that existing benchmarks are quickly becoming obsolete, as advanced AI models are surpassing them in capabilities. They emphasize the need for new benchmarks that reflect real-world environments and more complex cyberattack scenarios. The ongoing developments highlight the urgency for updated evaluation methods to ensure the safe deployment of AI technologies.