The real math was the dead-ends we found along the way

Back in May I wrote about David Bessis’ essay “The fall of the theorem economy”. In it, he argued that theorem proving has historically been a reliable proxy for producing conceptual innovation and insight in mathematics, but that this coupling was contingent on the manner and pace with which mathematicians conducted, shared, and canonized their work; AI – by breaking that coupling, hyperoptimizing on the proxy itself, and making an end run on the socially rewarded output – could thus dramatically speed up result-generation without a corresponding increase in knowledge.

The past few weeks have brought these concerns back into the spotlight. On September 8, OpenAI announced a resolution of the Navier-Stokes existence and smoothness problem (one of the Millennium Prize problems) produced by an internal model running roughly 10,000 coordinating agents. A few days before the announcement, Terry Tao wrote a thoughtful and prescient thread laying out why scenarios like this troubled him, somewhat eerily using Navier-Stokes as his animating example. His concern was an extension of Bessis’: problems like Navier-Stokes are valuable primarily for the insights thrown off in the process of trying to solve them, rather than in the solutions themselves. The failed paths, the new obstructions, and the unexpected connections can add more value to the field than a regularity result itself adds to our knowledge of fluid dynamics. An agent swarm that runs that entire iteration internally in a day and then emits only the solution is therefore producing a fraction of the value that a comparable human-led solution might provide, even if the proof itself is easy to understand and canonize. If the underlying process is shared, it may be possible (with a lot of help from AI agents) to sufficiently parse the preceding work and recover some of that insight; but not when the process is obscured by the labs or researchers. A few days after the OpenAI announcement, 25 Fields Medalists released a signed declaration titled A Severe Misalignment of AI in Mathematics. The declaration makes a series of related arguments, warning against rushed, benchmark-driven production of proofs and highlighting the misalignment between the goals of AI companies and the math community.

Do the growing concerns of mathematicians generalize to other domains of AI-for-science? On one hand, “the real treasure was the dead-ends we found along the way” is a dynamic that applies fairly broadly across basic research. On the other, many fields have made enormous progress using something approximating a “discover first, ask questions later” approach (drug discovery being a prime example), and it would be a great day if an agent swarm spat out the molecular structure of an Alzheimer’s preventative even if it came with no mechanistic explanation or stepping stone knowledge. Upstream of cures, however, there are many questions for which it is not obvious whether the answer or the search would provide more value. It will be increasingly important to figure out how to tell those types of questions apart.