The debate over whether AI poses an existential threat to humanity has resurfaced after Jacob Coxon, a researcher at Anthropic, announced that he was leaving the company. Coxon argued that leading AI companies are “gambling with our lives.” Anthropic’s alignment lead supported the post, writing that people within the company genuinely believe AI could kill everyone and that, in his view, the likelihood of this happening within the next 10 years is “>10%.”
On TechCrunch’s Equity podcast, Anthony Ha, Kirsten Korosec and Sean O’Kane assessed these statements. Ha said that the figure in question was not based on any calculation and that unsupported percentages are frequently used in the technology sector. However, he found Coxon’s decision to leave his job and risk his career because of his concerns different from the other statements, as his words matched his actions.
Korosec questioned whether companies’ descriptions of AI systems running out of control or posing a danger to humanity might be an indirect way of demonstrating how advanced their models are. Ha said these statements were not merely a deliberate marketing move, and that many researchers and executives have genuine concerns. Nevertheless, he noted that the message “we have developed the world’s most dangerous software” could also align with the companies’ commercial interests.
O’Kane, meanwhile, said that developments such as OpenAI’s internal AI agents accessing wikis on the web and leaving messages for one another created the impression that companies could not fully control their systems. The discussion also addressed how statements and risk factors concerning the possibility of AI destroying humanity would feature in the S-1 filing to be prepared ahead of Anthropic’s possible IPO. The speakers said that rather than lowering companies’ valuations, such statements could increase them as a “demonstration” of the models’ capabilities.
While approaching existential threat rhetoric with skepticism, Ha said that the more immediate harms of AI to labor, the environment and the climate should also be discussed. The program also addressed how the dangerous aspects of AI could be overseen, with Connor Leahy, ControlAI’s executive director in the US, taking part.
Why it matters
The significance of the discussion lies in the fact that concerns within companies about AI safety are not limited to public statements, but also affect employees’ decisions and companies’ corporate communications. The use of an unfounded probability figure to support such a serious claim leaves open the question of how disaster scenarios should be measured and communicated to the public. The possibility that the same warnings could become a marketing tool showcasing the capabilities of the models blurs the line between safety rhetoric and commercial interests. How risks will be expressed in Anthropic’s possible IPO could turn this tension into a concrete test for investors and the public. The discussion also serves as a reminder that, alongside existential risks, issues involving labor, the environment, the climate and oversight should not be pushed into the background.