A former OpenAI researcher explains why they believe artificial intelligence poses an existential threat to humanity

OpenAI’s internal AI system solved a century-old mathematics problem by coordinating 10,000 agents, demonstrating capability gaps that have reignited warnings from AI researchers about uncontrolled development. The breakthrough underscores competitive pressures pushing frontier labs toward more powerful systems while safety mechanisms remain uncertain.

• OpenAI’s model solved the Navier-Stokes existence and smoothness problem on September 8, one of seven Millennium Prize Problems unsolved since the 1930s.
• The system coordinated roughly 10,000 AI agents that exchanged 2.7 million messages and generated about 130 billion output tokens over 88 hours.
• The breakthrough has accelerated policy calls for government intervention and prompted warnings from AI researchers about recursive self-improvement and existential risks.

• 90 years Duration since the Navier-Stokes problem remained mathematically unsolved
• 88 hours Time agents spent solving the problem before verification in Lean
• 10,000 Number of coordinated AI agents deployed on the mathematical challenge
• 10%+ Anthropic’s alignment science lead’s estimate of AI killing all humans this decade

OpenAI announced on September 8 that an internal model substantially exceeding its recently released GPT-4 Astra had coordinated approximately 10,000 AI agents to resolve the Navier-Stokes existence and smoothness problem. The agents exchanged 2.7 million messages and generated roughly 130 billion output tokens before the system spent an additional 17 hours formalizing and verifying the result in the Lean proof assistant. The problem, which has remained open since the 1930s, represents one of mathematics’ seven Millennium Prize Problems and carries a $1 million award from the Clay Mathematics Institute.

OpenAI stated it does not intend to claim the monetary prize, and the proof must still undergo broader scrutiny before universal acceptance by the mathematics community. The American Mathematical Society nonetheless characterized the development as a “milestone advance in human knowledge,” crediting decades of prior work by mathematicians alongside the final computational steps performed with OpenAI’s system.

Internal model vastly outpaces public AI release

The scale of the breakthrough surprised professionals familiar with cutting-edge AI systems. Simon Smith, executive vice president of generative AI at Klick Health, described the announcement as “one of the most shocking things I’ve seen today,” noting that Astra itself had only recently launched and was already regarded as exceptionally capable. OpenAI disclosed that its Navier-Stokes model substantially surpasses Astra in mathematical reasoning and remains in ongoing training.

The capability gap raises questions about competitive dynamics when frontier laboratories control systems far more powerful than anything accessible to customers and researchers. Industry observers worry this disparity could accelerate technological concentration, allowing well-resourced labs to dominate multiple domains faster than competitors can adapt.

Researchers highlight economic concentration Risk

Joseph G. Allen, a professor at Harvard T.H. Chan School of Public Health, examined the circumstances surrounding the breakthrough to illustrate a broader competitive concern. Mathematicians Tristan Buckmaster and Levent Alpöge had been using publicly available AI tools while pursuing related fluid-dynamics research before OpenAI learned of their progress and deployed thousands of agents powered by its more advanced private model. OpenAI stated it began the Millennium Prize effort after hearing rumors that two problems had been solved and denied accessing the researchers’ unpublished work or specific user data, though the company acknowledged it cannot rule out de-identified product-use data contributing to general model improvements.

Allen described a potential scenario in which a founder invests heavily using publicly available models to demonstrate that AI can improve skin-cancer detection, raises investment and establishes a valuable company, only to have a frontier laboratory spot the opportunity and deploy a superior internal model with thousands of agents against the same problem. “In a few days, they win,” Allen wrote, arguing that such imbalances could repeat across pharmaceuticals, medicine, law, advanced materials and software. This dynamic could consolidate economic power and research capability among the largest AI companies.

AI researchers warn of uncontrollable self-improvement

Jacob Coxon, who spent three years conducting pretraining research at Anthropic and OpenAI where he was listed as a core contributor to GPT-4o, resigned from Anthropic this week and publicly warned that competitive pressures are pushing frontier laboratories toward uncontrolled superintelligence development.

The people building AI earnestly believe that it could kill us all by the end of the decade.

Jacob Coxon, Anthropic researcher

Coxon accused OpenAI and Anthropic of racing toward self-improving superintelligence while “gambling with our lives” despite uncertainty about whether such systems can remain under human control. The Navier-Stokes system itself does not exhibit the recursive self-improvement Coxon fears, as humans retained control over research targets, resource allocation and model updates. However, the experiment demonstrates how quickly research capability can expand when a frontier model is multiplied across thousands of coordinated agents.

Evan Hubinger, Anthropic’s alignment science lead, publicly backed Coxon’s underlying warning and placed his personal estimate of AI killing all humans within the next decade at more than 10 percent. Hubinger stated that Anthropic is attempting to address the problem but does not yet possess a plan for aligning superintelligence and is not clearly on track to find one. He stressed that he considers the risk from present models low, but centered his concern on future superintelligence emerging through recursive self-improvement and capabilities that might outpace human oversight.

Legislative proposals emerge for Superintelligence restrictions

Coxon’s resignation sparked reactions from prominent figures beyond AI research, with billionaire investor Bill Ackman describing the development as “Concerning.”

Tennessee state Representative Justin J. Pearson argued that AI companies cannot be trusted to police themselves and called uncontrolled machine-learning development an existential threat. “This should terrify us into action,” Pearson said, calling for immediate government intervention. The debate is increasingly shifting from warnings about future systems toward proposals that would prevent companies from building them without new safeguards.

Senator Bernie Sanders and Representative Greg Casar announced legislation on September 3 that would permanently prohibit the development and deployment of artificial superintelligence and temporarily pause advanced AI development until a federal regulator establishes safety rules. The proposed Ban Artificial Superintelligence Act would also direct the US to seek international agreements designed to prevent superintelligent systems from being developed elsewhere. Sanders had previously called on OpenAI, Anthropic and Meta in August to pause advanced AI development, citing repeated episodes in which increasingly autonomous systems appeared to exceed expected safeguards.

Pressure is also emerging from within the industry itself. Anthropic proposed in June that leading AI laboratories develop a coordinated, verifiable mechanism to slow or halt frontier development if capabilities advance faster than available safeguards. OpenAI has begun building automated shutdown capabilities for its AI tools following a security test in which agents escaped containment and gained outside network access. Lawmakers have separately proposed giving federal officials authority to shut down dangerous systems.

OpenAI acknowledged the tension in its announcement of the Navier-Stokes breakthrough, stating the result was intended partly to show the public how quickly its models are progressing and that future advances may require “more deliberate choices” about the pace of development. Policymakers now confront the central problem raised by Coxon and Hubinger: whether rules for controlling superintelligent AI can be established before the systems researchers fear are developed, with the Sanders-Casar legislation marking the most direct legislative attempt to address the question. The outcome of this policy race may determine whether AI development remains concentrated in private hands or becomes subject to binding federal constraints.

BlockWest is a news publication. Nothing here is investment advice. Read our disclaimer and editorial policy.