AI and the proxies of science

Our scientific ecosystem often prizes outputs that, themselves, don’t necessarily drive scientific progress, but instead are useful proxies for the more intangible processes that do. Three reads this week have me thinking about what happens if AI gets good at creating these outputs before (or instead of) getting good at the things they’ve been proxying for. First is David Bessis’s fascinating essay on AI in math. He discusses, among many other things, the idea that theorem-proving has historically served as a legible demonstration of underlying conceptual innovation, because solving major open problems typically required first building a new framework that made the solution tractable and expressible; AI may break that coupling, producing correct but unintelligible proofs and capturing social rewards without the accompanying distillation and canonization that allow the field to build on the new knowledge. Second is Engzell and Wilmers’ recent preprint, “The Paper Factory,” which, though different in goal and tone, describes a related dynamic in social science. The authors develop a multi-agent LLM workflow that automates most of the steps of creating a publishable empirical paper, but struggles with the intangibles that form the foundation of meaningful outputs, like problem selection and judgment. For a more speculative take on the topic, read Ted Chiang’s prescient short story “Catching crumbs from the table,” published in 2000 in Nature, which imagines human scientists reduced to interpreting the incomprehensible discoveries of superintelligent successors.