Refacto AI

Industry story

Chain-of-Thought Monitorability at Risk as Key Authors Depart OpenAI

chain-of-thought evals guardrails interpretability model-pricing

Tomek Korbak and Mikita Balesni, the lead authors of what observers called the 'extremely influential Chain of Thought Monitorability paper,' have been forced out of OpenAI at a moment when the field's ability to monitor AI reasoning is described as 'collapsing.' Chain-of-thought monitorability refers to the ability for researchers and auditors to inspect the intermediate reasoning steps AI models produce — a critical tool for detecting when models behave unexpectedly or deceptively. Their departure also ends ongoing work to establish a cross-laboratory industry standard on monitorability.

The timing is notable because OpenAI recently deployed GPT-6 Astra, which critics say has reduced monitorability. Observers noted the irony that the researchers who were most vocal about preserving this safety property — and who were working on embedding it as an industry norm — are now gone, potentially setting back both internal and cross-industry efforts on this front.

Analysis

Showing the shorter version.

Two OpenAI researchers who led chain-of-thought monitorability work, Tomasz Korbak and Bogdan Balesni, are out. They were also the people driving a cross-lab standard that would have made inspectable reasoning a requirement across frontier labs, not just a feature one lab drops when it gets expensive.

The question underneath this is whether chain-of-thought traces were ever faithful to what the model actually computes. The scratchpad reasoning a model writes out may be plausible-looking output, not a window into the actual inference. If it was always theater, losing two researchers costs little. If customers and regulators are already treating these traces as audit-grade evidence of how a decision got made, the gap between "looks inspectable" and "is inspectable" is a real liability, and the people best positioned to close it just left.

The economics make this worse. Faithful, long reasoning traces cost tokens and add latency. As serving costs dominate product decisions, the pressure is to compress or internalize reasoning rather than verbalize it. GPT-6 Astra is harder to read than its predecessors, and that is what happens when cheap inference wins the tradeoff. Korbak and Balesni were, in effect, defending a compute tax the product side had little appetite to pay. Remove them and the friction disappears. Nobody has to decide to reduce monitorability. It happens by default.

The cross-lab standard is the part that does not rebuild easily. Voluntary coordination between competitors requires insiders at each lab carrying it through meeting after meeting. Remove the insiders and the work stalls. No single lab will take on the cost of inspectable reasoning while competitors ship faster, cheaper models and face no requirement to do the same.

For enterprise buyers who purchased OpenAI's models partly because they could log reasoning for compliance: ask your provider, in writing, whether the chain-of-thought they expose is faithful to the actual inference. Watch how fast they decline to put that in a contract. If your legal team thinks those scratchpad traces are evidence of how a decision got made, you have exposure you have not priced.

A practical test: feed the model cases where you know the right answer and a tempting wrong path. Check whether the stated reasoning actually predicts the output or just decorates it. If the trace and the answer come apart, you have been logging theater.

The call: No frontier lab (OpenAI, Anthropic, Google DeepMind, Meta, xAI) will publish a signed cross-lab commitment to a shared chain-of-thought monitorability standard before 2027-07-01. The two people driving it are gone, the economics cut against it, and voluntary standards between rivals only survive when insiders keep pushing them through. Medium confidence.

Also covered this issue

Comments