Jacob Coxon, a researcher who worked on the pretraining of artificial intelligence models at OpenAI and, for the past three years, at Anthropic, where he worked on developing Claude, has resigned from Anthropic. Coxon argued that the two companies are engaged in “a straight race toward self-improving superintelligence” and are gambling with humanity’s future.
Coxon said that many people developing artificial intelligence sincerely believe the technology could wipe out humanity by the end of the decade, and that this is not a marketing ploy. Coxon said the risks are known at Anthropic but that the company is continuing development because of the race to be first, adding that this race carries the risk of producing systems that could spiral out of control.
Coxon said he was optimistic about coordination and noted that developments such as the Hugging Face attack could make it possible for US laboratories to reach agreements on setting the pace. Coxon said costly steps, such as a temporary ban on improving model capabilities, might be necessary to prevent a global race, and urged researchers to consider the consequences of operating superintelligent systems without adequately understanding them.
Why it matters
This statement shifts the artificial intelligence safety debate from an abstract risk issue to a governance problem directly linked to how laboratories make decisions. The fact that companies are continuing the race to be first despite knowing the risks raises concerns that technical capabilities and safety assessments may not advance at the same pace. The issue concerns not only researchers but also coordination among companies, which will determine the conditions under which these systems are developed and their possible limits. Whether developments such as the Hugging Face attack can make agreements on setting the pace possible, and whether the proposal to temporarily pause efforts to increase model capabilities is feasible, remain open questions.
Background
Anthropic is not a new name in the FikirPilot archive: over the past 90 days, we have published 7 stories mentioning the name; the latest is dated September 6, 2026.