Industry story
OpenAI Chief Scientist: No Lab Has Solved Alignment Sufficiently to Keep Scaling
alignment governance safety-standards scaling
Jakub Pachocki, OpenAI's Chief Scientist, stated publicly that no AI lab has solved alignment and monitoring to a degree that justifies continuing to scale AI capabilities at maximum speed. He expressed hope that voluntary slowdowns would become common until shared safety standards are established, and called international coordination on AI development a top government priority. This is a remarkable statement from a sitting chief scientist at the world's most prominent AI lab.
This view was echoed by 'roon,' an OpenAI employee, who noted that alignment research has produced no fundamental advances since RLHF (Reinforcement Learning from Human Feedback — a technique for training AI systems using human preference signals) and that most of the field remains at the level of informal intuitions rather than rigorous science or engineering. Roon warned that continuing to race toward recursive self-improvement (where AI systems help design the next generation of AI) without solving alignment would be 'unprecedented folly in world history.'
Analysis
Showing the shorter version.
OpenAI Chief Scientist Jakub Pachocki said what the whole field has been dancing around: nobody has solved AI alignment well enough to justify scaling at full speed "for much longer," and he wants voluntary slowdowns until labs agree on shared safety standards. An OpenAI employee posting as "roon" went further, saying alignment research has produced nothing fundamental since RLHF, the technique that trains models on human feedback.
Here is the problem with taking this seriously. Pachocki's statement costs nothing to make and nothing to reverse. OpenAI is closing a large funding round, converting its nonprofit structure, and fighting Anthropic for the "we're the careful ones" brand. Sounding worried is free marketing. If the lab actually believed scaling was reckless, the move would be to cancel the next training run and cut chip orders. Neither is happening. Watch what they buy.
Roon's point is more interesting precisely because an employee has no funding round to protect. If interpretability, scalable oversight, and debate have not moved the needle in five years of serious money, then alignment is harder than the scaling bet assumed. Not a few careful months harder. Harder in kind. And that creates a structural problem for any regulatory framework that wants to gate on "alignment progress": you cannot pause on a standard nobody has defined. Pachocki handed regulators a reason to act and no yardstick to act with.
The enterprise consequence is concrete. Legal and procurement teams at banks, hospitals, and defense contractors will clip this quote. A vendor's own chief scientist just said the product has not been proven safe to scale, and any general counsel who wants to stall an internal approval or demand indemnification the vendor cannot give now has the sourcing they need. Expect "AI pause" and "safety representation" language to start appearing in enterprise contracts over the next two quarters.
On compute, the damage is in the word "voluntary." The top-three labs slowing while xAI, Mistral, and the Chinese labs keep sprinting does not shrink chip demand. It moves it. NVIDIA's order book does not care which logo trains the cluster.
The call: OpenAI will announce or begin training a new frontier model more capable than GPT-5, with no binding multi-lab safety agreement in place first. Confidence is high. The funding incentives, the competitive pressure, and the $205 billion hyperscaler capex cycle all point the same direction. The blog post is the stated belief. The chip commitments are the revealed one.
Revisit by 2027-04-15. We're right if OpenAI ships or confirms training of a post-GPT-5 model with no shared safety bar preceding it. We're wrong if OpenAI publicly pauses its next frontier training run pending shared standards.
OpenAI's Chief Scientist Jakub Pachocki said out loud what the whole field has been dancing around: nobody has solved the problem of keeping powerful AI systems pointed where their builders want, and that means scaling at full speed "for much longer" is not responsible. He wants voluntary slowdowns to become normal until labs agree on shared safety standards. An OpenAI employee posting as "roon" went further, saying alignment research has produced nothing fundamental since RLHF, the technique that trains models on human thumbs-up and thumbs-down.
Here is the frame. This is a statement, not a policy, and it is almost perfectly easy to undo. Pachocki can walk it back in a sentence, and OpenAI can keep buying chips while he says it. What is actually being decided is whether "we haven't solved alignment" becomes a line that regulators, enterprise lawyers, and competitors can quote back at the whole industry. Nothing sets a deadline here. There is no shutdown date and no pause clock. So the useful question is what a business that builds on these models does with a confession that the people building the models can't vouch for them.
The Skeptic. OpenAI is closing a giant funding round, converting its nonprofit structure, and fighting Anthropic for the "we're the careful ones" brand. Pachocki's line costs nothing and buys credibility with regulators and nervous investors. If OpenAI actually believed scaling was reckless, the move is to cancel the next training run and slow chip purchases. Neither is happening. The company that is racing hardest just discovered that sounding worried is free marketing. Watch what they buy, not what they blog. Roon's post is more honest precisely because an employee has no round to close.
The Safety Lens. A sitting chief scientist at a top-three lab saying current safety work can't justify full-speed scaling crosses a line nobody at that level had crossed. EU AI Act enforcers and the UK's AI safety people will cite this when they set expectations for the largest models. But here is the trap roon exposes: if there has been nothing fundamental since RLHF, then any international agreement built on "wait for alignment progress" has no agreed way to measure progress. You cannot gate on a number nobody has defined. So the quote hands regulators a reason to act and no yardstick to act with. That ambiguity is the dangerous part.
The Researcher. Pachocki admitted a problem without naming what solving it would look like. "We haven't solved alignment" with no test for what counts as solved is a confession, not a research plan. Roon's claim is the one with teeth: if interpretability, scalable oversight, and debate have not moved the needle in five years of serious money, alignment is harder than the scaling bet assumed. Not "a few more careful months." Harder in kind. Treating a statement of ignorance as a roadmap is the mistake everyone will make this week.
The Enterprise Buyer. Legal and procurement teams at banks, hospitals, and defense contractors will clip this quote. Not because they read AI blogs, but because the vendor's own chief scientist just said the product isn't proven safe to scale. That is gold for a cautious general counsel who wants to stall an internal approval or demand indemnification the vendor can't give. Expect "AI pause" and "safety representation" language to start showing up in enterprise contracts over the next two quarters. The deployments that get hurt are the ones waiting on sign-off from a careful lawyer who now has a quote to point at.
The Compute Pragmatist. The hyperscaler spending cycle is priced on every major lab training the next generation as fast as the chips allow. A real coordinated slowdown would soften that demand in the 12-to-18-month window. But "voluntary" is the word doing the damage. The top three labs slowing while xAI, Mistral, and the Chinese labs keep sprinting does not shrink chip demand, it moves it. NVIDIA's order book does not care which logo trains the cluster. A pause that only binds the people who agree to it is a handoff to whoever didn't sign.
Where they disagree. The Safety Lens reads Pachocki's words as a genuine threshold crossing. The Skeptic reads the same words as free marketing from the lab racing hardest. Both can't be right about intent, and the tie-breaker is behavior: does OpenAI's compute spending or training cadence actually bend? The second split is between the Safety Lens and the Researcher. Safety wants this to seed an international framework; the Researcher says you can't build a framework on "alignment progress" when nobody can define the finish line. The decision the field actually faces lives in that gap. Everyone wants a gate, and there is no agreed metric to put on it.
What this hinges on is simple. Does a stated worry change what anyone buys or builds? The council leans hard toward no. A statement that is costless to make and costless to reverse, with no deadline and no metric, does not slow a $205-billion capex cycle or a locked product roadmap. The thing to verify is OpenAI's own behavior: its next training run and its chip commitments. If those hold steady while the safety language escalates, you have your answer about which signal is real.
Prediction: OpenAI will announce or begin training a new frontier model, more capable than GPT-5, on or before OpenAI's next major model release following 2026-10-15, with no public multi-lab "shared safety bar" agreement in place first.
Confidence: High — the funding and competitive incentives all point the opposite way from a pause.
Why: Pachocki's statement is costless and reversible, and it arrived while OpenAI is closing a large round, converting its structure, and fighting Anthropic for the safety-conscious brand, so the words buy credibility without binding behavior. The "voluntary slowdown" he hopes for has no agreed metric to gate on, which roon's point about zero fundamental progress since RLHF makes concrete: you cannot pause on a standard nobody has defined. For a real slowdown to happen instead, OpenAI would have to cancel or defer its next training run and let xAI, Mistral, and the Chinese labs pull ahead, which no lab racing for a funding round and a product lead has ever done voluntarily. The hyperscaler capex cycle priced on continuous scaling is the revealed belief; the blog post is the stated one.
Revisit by 2027-04-15: We're right if OpenAI ships or publicly confirms training of a model more capable than GPT-5 with no binding multi-lab safety-bar agreement preceding it. We're wrong if OpenAI publicly pauses or defers its next frontier training run pending shared safety standards.
Also covered this issue
-
Wave of AI Lab Defectors Testify at NYC Hearing on AI Danger
zvi-vase
Former AI researchers testifying on record that labs could lose control by 2027 will start appearing in vendor risk questionnaires and compliance requirements within months.
-
OpenAI Safety Researcher David Robinson Resigns, Blows Whistle
zvi-vase
A safety framework's author just quit saying it doesn't actually solve the core problem, so vendor safety claims now need sharper scrutiny in your contracts
Comments