For several months now, OpenAI and other artificial intelligence companies have been using mathematics as a testing ground for their models. This discipline allows them to tout the capabilities of their systems, but it alone does not explain the interest these companies have in it. The creator of ChatGPT has been making numerous announcements since last year about its models’ progress in this field.
In May, an OpenAI model refuted a conjecture related to Paul Erdős’s unit distance problem, which dates back to 1946. A conjecture is a proposition considered true based on observations or analogies, but which has not been formally proven. For decades, mathematicians believed that arranging points in a square grid yielded the greatest number of pairs at the same distance. A model found other ways to organize them to obtain more. In a blog post, the company mentions “an infinite family of examples” refuting the old conjecture. The proof was verified by a group of external mathematicians.
Other announcements followed. In September, OpenAI mobilized, in its own words, “more than 10,000 AI agents” for one of the Millennium Prize Problems, an initiative contested by researchers who accuse the company of plagiarism. In early October, a cutting-edge model enabled the solving of hundreds of previously unsolvable problems.
A Showcase for Models
For François Willot, a mathematician and research director at Mines Paris-PSL, the interest is primarily commercial. “It gives them enormous publicity,” he told BFM Tech, explaining that companies can thus tell their clients they have “the best tools that exist at the moment.” The mathematician also highlights the difficulty of the subjects tackled: “Mathematical conjectures that have not yet been proven are among the most difficult problems that, until recently, humans were capable of solving and machines were not.”
François Charles, a mathematician and head of the Department of Mathematics and Applications at ENS-PSL, a member of AGMAI, an independent committee of nine world-renowned mathematicians advising AI giants, draws the same conclusion. He notes that large language models (LLMs) were recently capable of solving “competitions, albeit very difficult ones, but for high school students,” before, in his view, cutting-edge research results were solved autonomously since the end of May.
The fact that a proof is either true or false also makes these domains relevant for measuring performance. “There is no middle ground, so it’s a good benchmark,” explains François Willot, who considers it more difficult to rank models on scientific questions that admit multiple answers.
Concerns About Transparency
The mathematician points out that these systems were trained on “several million articles in the sciences in general,” which, in his view, allows them to connect little-known old results to new problems. He also compares these advances to those of Galileo, who “in a few nights” made more discoveries in astronomy than in the previous thousand years. He believes that with AI, “we will likely prove many things in mathematics that had not been proven until now.”
He is concerned, however, about a scenario where proofs would be so long that we could no longer understand the idea, even if their validity were verified by automatic means. He also criticizes OpenAI for providing few details about its methods. For François Charles, researchers must be able to “appropriate the knowledge” derived from these proofs, whereas, for the first time, the pace of research is not set by an open community, as frontier models are “controlled by a small number of companies.”
To address these criticisms, OpenAI consulted the group of mathematicians tasked with giving it recommendations on the review and communication of its results. On October 26, the company also announced three discoveries in computer science and physics, sparking new criticism from researchers about their communication methods. Despite these tensions, François Charles states: “There is no scientific revolution without scientists being involved.”
