Safety Was Anthropic's Whole Product. On Tuesday, Its Maker Walked Out.

Jacob Coxon resigned from Anthropic on Tuesday. His job was pretraining, the foundation work under Claude's models, first at OpenAI, then at the lab that markets itself as the careful one. Both employers, he now writes, are “racing straight to self-improving superintelligence and gambling with our lives.”^1,2

What he actually said

In his exit note on X, he says the fear is not for show: “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.”¹ His split of the two labs is blunt. OpenAI has not absorbed what it is building. Anthropic has absorbed it and builds anyway, on the theory that no rival will be careful, so it must win.²

He is not alone

He is not alone inside the building. A colleague, Evan Hubinger, posted publicly that risk from current models is low but compounding with “superintelligence arising from recursive self-improvement,” which is “happening faster than we thought.” Coxon calls for pacing agreements between US labs, or failing that, a temporary ban on capability improvements.²

The money did not blink

The money did not blink. OpenAI shipped its new Astra model the same morning.⁸ Legal AI startup Harvey closed $550 million at $15.5 billion the same day.⁹ And Bill Gurley, echoed by Chamath Palihapitiya to 301,000 views, told the AI doomers to calm down and carry on, because things will be amazing.^4,5

Why a board should care

Anthropic's commercial edge is trust, and trust is measurable. Claude has 245 million monthly users, third place, and converts 13% of them to paid subscriptions, the best rate in the industry.^6,7

A safety brand that loses its safety researchers keeps the slogan and loses the substance.

The race has three scoreboards now: models shipped, capital raised, and researchers who leave. Watch the third one. It moves first.