The verification economy: Why AI is creating demand for proof
As Anthropic's Fable 5 standoff drags on, a wave of new startups is making the case for proof over trust
Over the past four years on the AI beat, I often find myself circling a theme that I spend months—sometimes years—trying to understand, and then suddenly it starts showing up everywhere.
At the moment, that theme is AI verification. For years, I’ve followed as the AI industry has focused on AI security and safety best practices including training, testing, red-teaming, and guardrails to make models safer and more reliable.
But a growing number of people I’ve spoken with argue that the goal shouldn’t be to trust AI at all. It should be to verify its outputs and behavior independently. This is becoming a trendy topic, thanks to Anthropic’s latest feud with the US government in which the Trump administration cut off foreign access to Anthropic’s powerful new AI model, Fable 5, on June 9 due to cybersecurity concerns — export controls that forced the lab to pull the model’s access entirely.
In this era, the question is: How do you prove that an AI system is safe enough to deploy?
‘Stop trying to trust AI models’
Back in 2024, I began chatting with computer scientist and former Meta engineering leader and researcher Erik Meijer, whose core idea is that we should stop trying to trust AI models and start verifying their outputs.
“I think the genie is out of the bottle,” he recently told me for probably the tenth time, arguing that efforts to contain AI capabilities may ultimately be futile because the underlying knowledge is already widely available and AI systems are fundamentally software that can be recreated by determined actors. “Nothing we can do about it anymore,” he said.
Rather than trying to prevent jailbreaks or perfectly align increasingly powerful models (which I reported last week is, apparently, impossible), Meijer believes the industry should focus on proving that AI systems behave safely in the contexts where people depend on them.
His idea, outlined in a 2025 paper called Guardians of the Agents, is that AI systems should be required to generate evidence that their actions are safe before they are allowed to execute them. The approach borrows from formal verification techniques long used in computing, where mathematical proofs are used to establish that certain properties hold before software runs.
Provably-correct decision making is trending
The idea seemed super-academic and in the weeds when Meijer first started explaining it to me. To be honest, I didn’t really understand it, so I filed it away in my brain folder marked “interesting, but huh?”
But over the past couple of weeks, I’ve been hearing versions of the same argument everywhere. For example, I spoke earlier this week to Anjana Rajan, CEO & co-founder of Atalanta, a startup whose tagline is “provably correct decision-making for the world’s most important missions.”
