Helen Toner, Executive Director of Georgetown University’s Center for Security and Emerging Technology (CSET) and a former OpenAI board member, argues that policymakers need reliable information on artificial intelligence. In an interview with host Nilay Patel for The Verge’s Decoder podcast, she discusses lab governance, the concept of AGI, and the rivalry with China.
CSET, which has existed since 2019 and has about 50 people, presents itself as a non-academic analysis center. It produces long reports, as well as short briefings for busy policymakers. According to Helen Toner, the gap between Washington and Silicon Valley is real, but it’s not just a matter of expertise: in her view, Congressional staffers are ‘generally smarter than they get credit for.’
The OpenAI Precedent
Helen Toner was one of the four OpenAI board members who voted to fire Sam Altman. This episode, sometimes nicknamed ‘the blip,’ ended with Sam Altman’s return as CEO, Helen Toner’s departure, and the establishment of a new board. OpenAI has since been completely restructured.
The non-profit board’s mission was to ensure the company did not sacrifice its goal of building safe AI for profit and growth. According to the host, when the board tried to pull that lever, this accountability mechanism broke. Helen Toner thus saw, in her view, what happens when safety commitments clash with commercial incentives. She believes the future should not rely on the trustworthiness of a single person: if it did, ‘we’d be in a bad spot.’
AGI, Self-Improvement, and Singularity
Asked about AGI, or artificial general intelligence, Helen Toner says she is not claiming that milestone has been reached. She speaks rather of a ‘fuzzy cloud’ of definitions, some of which have already been crossed. Others, like that of a conscious AI capable of everything a human can do, remain distant. She wrote a post on the subject on her Substack.
She also describes an idea widespread in Silicon Valley: recursive self-improvement, or RSI (recursive self-improvement), meaning the possibility that AIs participate in creating more capable systems, up to superintelligence. According to her, companies are now using AI in a growing share of their research and development, without entrusting it with the entire process. She cites Demis Hassabis at Google, who said that ‘we are in the foothills of the singularity.’
On the notion of alignment, which aims to make models conform to their designers’ intentions, Helen Toner finds the term too broad. The most coherent version, in her view, is to make the AI do what the user wants. She believes this direction is ‘not on a good path’ to being solved. She stresses, however, that these issues are part of scientific research that takes time.
Regarding regulation, she says she is more in favor of rules targeting the most advanced systems than a text covering all of AI. She cites California, New York, and Illinois, which have adopted nearly identical texts, and notes that this dynamic could continue at the state level.
