Years to Hours: Chris Hsu on the Inflection That Just Made Software Provable

Formal verification spent forty years as the most trustworthy technique nobody could afford. AI-assisted formalization is collapsing its cost curve, and with it, any excuse for unproven software.

By Exec Edge Editorial Staff

Almost all software reaches the world on the strength of testing. Engineers run a program through the cases they can think of, watch for failures, and trust that the cases they tried resemble the ones they didn’t. Formal verification asks for something testing can never give: a mathematical proof that a program obeys its specification for every possible input, including the inputs no one anticipated.

That certainty has been available in principle for decades, but it was simply too manual, or expensive, to buy anywhere outside a few domains where failure was catastrophic. Thanks to AI, the price of formalization is now falling, but the software most in need of proof is increasingly the software that AI writes, which is running with less and less human supervision.

The prior price of certainty

The seL4 microkernel is the standard illustration of what mathematical certainty used to cost. When researchers completed the first machine-checked functional-correctness proof of a general-purpose operating system kernel in 2009, the kernel itself was 8,700 lines of C; the proof apparatus ran to roughly 200,000 lines of a proof assistant called Isabelle, software that mechanically checks every step of a proof the way a compiler checks code. Writing the C took about 2.2 person-years. Proving it correct took about twenty.

Later accounts put the core engineering effort at twelve person-years and roughly $350 per line of code, and identified the deeper problem: verification effort scales with the square of the specification size, so bigger systems get disproportionately more expensive to prove.

At those prices, formal methods survived only where failure was unthinkable: avionics, rail switching, cryptographic protocols. CompCert, the formally verified C compiler proven correct in the Coq proof assistant, took years of work by some of the field’s best researchers and won the 2021 ACM Software System Award for it. The economics confined proof to a niche while the rest of the world’s software shipped on testing and hope.

What changed

Two bottlenecks kept proofs expensive: writing formal specifications of what software should do, and constructing the proofs that it meets those specifications. Both were artisanal work for scarce experts, but both are now being automated.

The clearest peer-reviewed datapoint comes from competition mathematics. DeepMind’s AlphaProof system, published in Nature in November 2025, took roughly one million math problems written in plain language and rendered them into a training set of some eighty million problems in the rigid, machine-checkable form that automated proof requires. The particular theorems matter less than the demonstrated capacity: translation from informal statements into machine-checkable formal language, at a scale no human workforce could approach, performed by AI. That translation step is precisely the one that made verification uneconomic for four decades.

Government programs are running the same play on legacy code. DARPA’s TRACTOR program aims to automatically translate legacy C into memory-safe Rust, combining static and dynamic analysis with large language models, with MIT Lincoln Laboratory as independent evaluator.

Leonardo de Moura, creator of Lean, a system for writing mathematical proofs a computer can verify line by line, has meanwhile announced a moonshot to verify the Signal protocol and its Rust implementation, a project that not long ago would have been dismissed as a decade of work. Amazon, making the largest donation in the Lean Focused Research Organization’s history, described the goal as proving that software and AI agents behave correctly for every possible input.

WATCH MORE

Activist Investors Shift Back to Basics: ICR Global Head of Governance Gabriel Hasson, Live at NYSE

The underinvested bedrock

Among those who saw this sea change coming is Chris Hsu, founder and CEO of Kilometre Capital and family office Rocketeer Management. Hsu has described formal methods as an underinvested but critical bedrock: a discipline long relied on by industries that cannot afford to be wrong, now urgently scalable because AI delivers orders-of-magnitude efficiency on specification and auto-formalization. His Infinitude Foundation’s grantees in this direction include the Lean FRO, the Stanford Center for AI Safety, and Convergent Research, each doing scalable formal-verification-focused work.

Hsu has maintained that taken together, developments point to a deeper shift than improved theorem proving. For decades, the dominant cost of formal verification was neither computation nor logic, but the human labor required to translate informal intent into formal specifications and machine-checkable proofs. AI is now attacking precisely those bottlenecks through auto-formalization, specification generation, proof search, and equivalence verification. What once demanded years of work by small teams of specialists is increasingly becoming tractable in days or hours, fundamentally altering the economics of formal methods. If that trend continues, mathematical proof may cease to be a rare property of exceptional software and become a routine stage in the software development lifecycle.

A 2025 white paper authored by Hsu, proposing a Stanford-anchored center for AI alignment and consciousness science, names formal verification and safe deployment as one of four critical research directions, arguing that for AI systems operating with decreasing human oversight, proof is the only approach that provides guarantees rather than probabilities.

Final Agenda: 2nd LA CorpGov Forum Sep 18 Featuring Activism, Sports, Entertainment

Verification researchers are equally careful to distinguish what formal methods can and cannot guarantee. A proof establishes that an implementation conforms to its formal specification, not that the specification itself is complete or correct, and AI-generated specifications inherit AI’s capacity for omission and error. Skeptics also note that, despite rapid advances in AI-assisted verification, the level of end-to-end assurance demonstrated by projects such as the seL4 microkernel has yet to be replicated for software systems orders of magnitude larger.

But what the skeptics and the converts share is the sense that the question has changed. For forty years the question was whether proof could ever be affordable outside a few life-or-death niches. The new question is how far the collapsed cost curve extends: whether verification becomes a routine layer that AI-generated code passes through on its way to production, the way compilation is today. Google’s published principles for secure AI agents already gesture at this distinction, separating probabilistic confidence from enforceable guarantees.

Hsu’s belief is that the second question should ultimately answer itself: once certainty becomes economically attainable, it becomes economically inevitable. His broader thesis is that the transition to provable software is driven as much by economics as by scientific progress. As the cost of formal verification continues to fall, mathematical proof ceases to be a specialist capability and becomes foundational infrastructure for the next generation of software, autonomous agents, and critical digital systems.