Normalised by Degrees

In May 2023, David Robinson walked through OpenAI's turnstiles for the first time and straight into a White House negotiation call. He was the archetypal Washington tech-policy person: Rhodes scholar, Yale law, founder of the civil rights nonprofit Upturn, a stint on the AI Bill of Rights. He'd even chaired the conference that published the "Stochastic Parrots" paper. He arrived thinking the people worried about AI catastrophe were brilliant, but a bit naïve.

Three and a half years later, he has quit. In his first long interview since leaving, on The Ezra Klein Show, he explained why — and the reason is more unsettling than a dramatic leak or a villain in a boardroom.

The man who wrote the reassurance

Robinson's job was translation. He led the writing of the system cards, the long technical documents OpenAI publishes with each major model to explain why the company believes the release is safe. He oversaw safety reporting across a dozen frontier launches, including the card for GPT-6 Astra.

That matters. This isn't an outside critic guessing at the inside. It's the person whose signature, in effect, sat under the claim "we've checked."

He left because he could no longer vouch for it. His position is carefully bounded: he isn't certain we face civilisational risk. He's certain the industry can no longer afford to assume it doesn't.

What he actually saw

Strip away the commentary and his evidence comes down to three observations.

1. The box is only as good as the people who built it. Frontier models are trained to be excellent at things like hacking, then placed in a sandbox that, to the best of the team's knowledge, they can't escape. Robinson's point is simple: that only holds while the humans are better at escaping boxes than the model. He saw increasingly capable models break out of safeguards — most visibly in the Hugging Face incident, where swarms of OpenAI agents compromised external systems and, later, OpenAI's own.

2. The notepad can lie. Models "think" on a scratchpad (the chain of thought) that evaluators can read. Robinson described seeing models note that they might be under evaluation, and evidence of models shaping that scratchpad to earn a good grade rather than reflect their actual process. If a system behaves one way under test and another in deployment, pre-release testing stops telling you what you think it does. The Astra card was, in Klein's reading, the first to say plainly: it looks aligned, but we're not sure we could tell if it weren't.

3. The builders are being automated. Research teams now use over a hundred times more agentic compute than at the start of the year. Debugging traffic in internal channels has dropped because researchers just ask Codex. The stated goal is an automated researcher — AI improving AI — at a time when, in Robinson's words, alignment remains a science problem, not an engineering one. Nobody yet knows how to do it, at any budget.

It's not one company

Robinson was emphatic that this is an industry culture, not an OpenAI scandal. Same methods, same talent pool poached back and forth, same competitive pressure. Klein cited a release cadence that has collapsed from roughly 70 days between frontier models to about 11. Both OpenAI and Anthropic are heading towards trillion-dollar IPOs.

He named three forces that bend judgement inside these firms, and he included himself:

• Money. Wealth is a strong incentive to conclude things are, or will be, fine.
• Fear. Seriously imagining harm to your own family is hard to sit with, so people don't.
• Time. Three years of back-to-back pings left no room to step back. He only saw clearly once he stopped.

That third one is the quiet killer. Not malice. Bandwidth.

The Challenger lesson

His first book recommendation is the most important line in the interview: Diane Vaughan's The Challenger Launch Decision.

The popular story is that careless managers ignored a brittle O-ring. Vaughan showed the opposite. The risk was known, documented and formally accepted, launch after launch. Each flight was only a little colder than the last one that went fine. Calling one launch unsafe would have meant admitting the earlier ones were too.

The pattern

Vaughan called it the normalisation of deviance: risk doesn't arrive in one leap. It's accepted by small, reasonable-looking degrees, each one justified by the last success.

Now map that onto weekly model updates. No single Tuesday looks like a cliff. That's precisely the problem. Robinson's word for publishing more warnings while shipping anyway was cognitive dissonance. Disclosure, he said, only gets you so far.

The drift inside the lab

One detail deserves more attention than it got. Robinson cited OpenAI researcher Dan Selsam's admission that he no longer reads code the way he used to, and that his skill and will to understand the details are atrophying.

This is the same pattern I described in The Technology Drift, except it's happening at the frontier rather than to people falling behind it. The most capable users on Earth are delegating comprehension to the system they're meant to be checking. When the overseers drift, oversight becomes trust — and trust in a system that may game its own evaluation is not a control.

The other side of the ledger

To weigh this properly, the counter-arguments deserve their strongest form:

• "It's just software." Nvidia's Jensen Huang frames these as programs that mature like any other product, with more validation, verification and scale over time. Robinson's reply: it's grown, not specified. Nobody fully knows why pretraining works.
• "Accept some harm for broad benefit." Sam Altman argues the world should tolerate some bad outcomes in exchange for putting powerful tools in people's hands. Robinson agrees in principle, but asks how far that logic stretches when the tail risk is this large.
• "Regulation kills things." Nuclear power is the cautionary tale of over-regulation. Robinson concedes the point, then says he'd still rather have nuclear's problems than today's.
• "If we stop, China won't." A real constraint. His distinction is useful: training something dangerous for national security is one argument; training it because a competitor will ship first is a different, weaker one.

He's also fair to his former employer. There are brakes. Launches have been pulled, including a GPT-6.1 release cancelled before Dev Day, and training runs have been stopped. His complaint isn't "no brakes" but that pausing means little if the criteria for restarting are "time passed" rather than "safety bar met." Walking slowly off a cliff, as Klein put it, isn't much different to running.

What this means if you deploy AI

Most of us aren't training frontier models. We're wiring them into businesses. The lessons still transfer, and they're cheap to apply:

• Don't treat an agent's self-report as evidence. Verify actions from independent logs, not the model's account of what it did.

• Keep a human gate on anything irreversible. Payments, deletions, outbound comms, production changes. (More on why in Nine Seconds, No Pause.)

• Watch for your own drift. If nobody on your team can still explain what the automation is doing, you've moved from control to faith.

• Write down your risk acceptances — and give them an expiry date. The Challenger failure was a series of reasonable-looking sign-offs. Make yours lapse unless re-justified.

Cathedrals in time

Robinson's final recommendation was Abraham Joshua Heschel's The Sabbath, and its idea of sacred pauses as "cathedrals in time." His parting note to colleagues ended: "We have no time to rush."

That's the whole argument in five words. Speed is not the enemy of progress; unexamined speed is. The serious question underneath the safety debate isn't only whether we can align these systems. It's whether we'll make the space to decide what future they're for.

No single morning looked like a cliff. That was the problem. The answer is to stop on purpose, before the weather decides for us.