Refacto AI

Podcast episode

From Restoring Sight to Reimagining the Brain, with Max Hodak

interpretability mechanistic-interpretability neuroscience open-weights

Sarah Guo's No Priors brought on Max Hodak, co-founder of Science Corp, to talk about a retinal implant that just received European regulatory approval. The AI angle is secondary: Hodak uses the episode to argue that neuroscience and large language models are converging in ways that matter for both fields.

The claim worth sitting with is the platonic representation hypothesis, a 2024 MIT paper arguing that as models scale and train on more data, their internal representations (the geometric patterns the model uses to organize concepts) converge toward each other, and possibly toward biological brain recordings. Hodak's Science uses that alignment as an engineering signal, matching animal neural data against model activations. He also argues human cognition is bottlenecked at roughly 10 bits per second of serial throughput, meaning faster inference past a certain point produces output the user simply cannot absorb.

The model-to-model convergence result is real and peer-reviewed. The extension to biological brains is a hypothesis wearing that paper's credibility. Worth tracking, not worth betting a roadmap on.

Full analysis

Max Hodak went on No Priors to talk about a retinal implant that just got European approval, and the AI content is a footnote to a biotech story. That's the frame worth being honest about up front. Nothing here changes your model stack, your inference bill, or next quarter's architecture. The one idea a technical AI lead should actually chew on is the platonic representation hypothesis: the empirical claim that the internal representations inside large models line up, geometrically, with recordings from biological brains.

Reversibility: Not a decision. This is a briefing on a research idea, so treat it as background reading, not a fork in the road. Nobody's asking you to bet a roadmap on brain implants.

What's actually on the table: whether "neuroscience on the models" and representation alignment are a real signal you should track, or a nice dinner-party story from a founder selling a different product.

The council

The Skeptic. Hodak has a retinal chip to sell and a 10-to-15-year runway to fill with narrative, so treat the AI claims as ambient support, not evidence. The platonic representation idea is genuinely interesting, but "internal representations align with neural recordings" is doing a lot of quiet work about which representations, whose recordings, and how you measured alignment. Geometric similarity between two high-dimensional spaces is easy to find and easy to fool yourself with. For a PM: two very different systems can end up organizing information in similar-looking ways, and it's hard to tell a deep truth from a coincidence you went looking for. None of this touches what you ship.

The Researcher. The platonic representation hypothesis is a real 2024 paper (Huh, Isola, and colleagues at MIT), and the core observation is that as models get bigger and train on more modalities, their representations converge toward each other. Hodak extends that to biological brains, which is the spicier and less settled claim. His practical use is the tell worth respecting: Science aligns animal neural recordings with model activations as an engineering signal, not a metaphor. That overlaps directly with mechanistic interpretability, the work of figuring out what features a model actually computes inside. If the overlap holds, interpretability tooling has a second customer in neuroscience.

The Open-Source Advocate. Here's the quietly good news for anyone outside a frontier lab. Hodak's line is that it's easier to do neuroscience on models than on brains, because you get full access, full control, and infinite repeatability. That describes open-weight models exactly. You cannot run activation-alignment studies against GPT-5 or Claude through an API. You need the weights, the activations, every layer. So this research bridge runs on Llama, Qwen, and Mistral, not on the closed labs, even though Hodak name-drops OpenAI and Anthropic as employers. For a PM: the science needs to open the hood, and only open models let you.

The Compute Pragmatist. The 10-bit-per-second cognitive bottleneck is the one claim with a concrete engineering consequence. Hodak argues human cognition is stuck at roughly 10 bits a second of serial throughput, that language already saturates it, and that high-bandwidth brain-to-AI interfaces therefore hit a wall on the human side no matter how good the model gets. For anyone building always-on or ambient AI: your ceiling isn't tokens per second on the GPU, it's how fast a person can absorb output. Push more at them and it spills on the floor. That reframes latency budgets. Past a point, faster inference buys you nothing the user can use.

Where they disagree

The Researcher sees a genuine, publishable convergence result and a real bridge to interpretability. The Skeptic sees a founder generalizing a paper about model-to-model similarity into a much bigger model-to-brain claim, using it to signal that AI "isn't hitting a wall" while he raises for a decade-long biotech program. Both are right, which is the point: the model-to-model version is solid, the model-to-brain version is a hypothesis wearing the first one's credibility.

The second split is about who this helps. The Open-Source Advocate says the whole research program is only possible on open weights, because you need the activations. The frontier labs get named as prestige employers but can't actually host the experiment behind an API. So the talent flows one way and the reproducible science flows the other.

What it hinges on

For your team, almost nothing. Three takeaways survive contact with a real roadmap. One, interpretability work has scientific value beyond safety, which is a mild recruiting and morale point if you have people doing it. Two, representation-alignment research needs open weights, so it strengthens the case for keeping capable open models in your evaluation loop. Three, the 10-bit bottleneck is a real design constraint for ambient and streaming AI products, and it's free to internalize now: stop optimizing human-facing throughput past what a human can read.

If you want to verify anything, read the original platonic representation paper and check whether the model-to-brain extension has independent replication yet. It mostly doesn't. Treat Hodak's version as a promising claim, not settled ground.

The Prediction

Prediction: By the NeurIPS 2026 proceedings (December 2026), the platonic representation hypothesis will still have no independently replicated result establishing that frontier-model internal representations predict biological neural recordings better than a strong non-deep baseline, on a held-out task the aligning team did not choose.

Confidence: Medium. The model-to-model result is solid; the model-to-brain extension lacks a clean, pre-registered test.

Why: The 2024 platonic representation paper documented convergence between AI models, which is well-supported, but Hodak's stronger claim is that those representations align with biological brains, and that version rests on post-hoc geometric similarity between two high-dimensional spaces, which is notoriously easy to find by accident and hard to falsify without a pre-committed baseline. Alignment scores go up almost automatically as both systems get more expressive, so a positive result that beats a simple baseline on data the team didn't cherry-pick is a much higher bar than the papers currently clear. The opposite outcome, a clean replication landing in the next cycle, would require someone to run the adversarial version of this test and publish it, and the field's incentive right now is to report the exciting alignment, not to try to break it.

Revisit by 2026-12-31: We're right if no peer-reviewed, independently replicated study shows frontier-model representations beating a non-deep baseline at predicting neural recordings on a held-out, team-independent task. We're wrong if such a study appears and holds up.

Low-impact episode for an AI operator, and worth saying so plainly. The retinal implant is the real news. The AI angle is one good paper, one founder stretching it, and one bottleneck number you can actually use.

Comments