---
title: "Safety Was Anthropic's Whole Product. On Tuesday, Its Maker Walked Out."
description: "A pretraining researcher quit the lab that sells caution, warning that both his old employer and his new one are “gambling with our lives.” The money did not blink."
publisher: "The Inference"
section: "Model"
published: 2026-09-10T01:49:03.000Z
modified: 2026-09-16T16:29:05.743Z
canonical: https://theinference.org/article/safety-was-anthropics-whole-product-on-tuesday-its-maker-walked-out
language: en
keywords: "Safety, Anthropic, OpenAI, Talent"
---

# Safety Was Anthropic's Whole Product. On Tuesday, Its Maker Walked Out.

> A pretraining researcher quit the lab that sells caution, warning that both his old employer and his new one are “gambling with our lives.” The money did not blink.

Jacob Coxon resigned from Anthropic on Tuesday. His job was pretraining, the foundation work under Claude's models, first at OpenAI, then at the lab that markets itself as the careful one. Both employers, he now writes, are “racing straight to self-improving superintelligence and gambling with our lives.”[1, 2]

## What he actually said

In his exit note on X, he says the fear is not for show: “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.”[1] His split of the two labs is blunt. OpenAI has not absorbed what it is building. Anthropic has absorbed it and builds anyway, on the theory that no rival will be careful, so it must win.[2]

## He is not alone

He is not alone inside the building. A colleague, Evan Hubinger, posted publicly that risk from current models is low but compounding with “superintelligence arising from recursive self-improvement,” which is “happening faster than we thought.” Coxon calls for pacing agreements between US labs, or failing that, a temporary ban on capability improvements.[2]

## The money did not blink

The money did not blink. OpenAI shipped its new Astra model the same morning.[8] Legal AI startup Harvey closed $550 million at $15.5 billion the same day.[9] And Bill Gurley, echoed by Chamath Palihapitiya to 301,000 views, told the AI doomers to calm down and carry on, because things will be amazing.[4, 5]

## Why a board should care

Anthropic's commercial edge is trust, and trust is measurable. Claude has 245 million monthly users, third place, and converts **13% of them to paid subscriptions, the best rate in the industry**.[6, 7]

*Chart: Chart of Anthropic's paid-conversion rate against rivals*

*Claude converts 13% of monthly users to paid, the best rate in the industry, and the exact asset a safety exodus puts at risk.*

A safety brand that loses its safety researchers keeps the slogan and loses the substance.

The race has three scoreboards now: models shipped, capital raised, and researchers who leave. Watch the third one. It moves first.

## Takeaway

A safety brand that loses its safety researchers keeps the slogan and loses the substance. There are three scoreboards now: models shipped, capital raised, researchers who leave. The third moves first.

## Sources

1. [TechCrunch, Sep 9 2026 (resignation, full Coxon thread).](https://techcrunch.com/2026/09/09/gambling-with-our-lives-anthropic-researcher-quits-warns-against-self-improving-ai/)
2. X: Jacob Coxon resignation thread.
3. X: Jason Calacanis repost.
4. X: Chamath Palihapitiya quoting Bill Gurley, Sep 9 2026.
5. X: Bill Gurley post (quoted).
6. [TechCrunch, Jun 16 2026 (Sensor Tower data).](https://techcrunch.com/2026/06/16/chatgpts-market-share-slips-below-50-for-first-time/)
7. [Sensor Tower State of AI Report 2026.](https://sensortower.com/blog/state-of-ai-2026)
8. The Information, Sep 9 2026 (Astra launch; math data-sharing concerns).
9. [TechCrunch, Sep 9 2026 (Harvey $15.5B).](https://techcrunch.com/2026/09/09/harvey-hits-15-5b-valuation-months-after-reaching-11b/)
10. TechCrunch, Feb 4 2026 (lab feud tone).
11. TechCrunch (state AG investigation, policy pressure).
12. TechCrunch, Feb 9 2026 (ad rollout, commercial pressure).


---

Safety Was Anthropic's Whole Product. On Tuesday, Its Maker Walked Out. — The Inference. Canonical: https://theinference.org/article/safety-was-anthropics-whole-product-on-tuesday-its-maker-walked-out