World Models and the Future of Spatial AI with Justin Johnson - #775

Full analysis

Justin Johnson, co-founder of World Labs and a Michigan professor, spent an hour on TWIML explaining "world models," the systems that generate navigable 3D scenes from a photo or a text prompt. For a business reader, this is a research tutorial, not news you act on this quarter. Johnson himself says the field has no agreed definition, no settled recipe, and no answer on whether explicit 3D or pure pixel-generation wins. That's the story. Not a product you can buy, but a map of how far off the buyable version is.

What's actually being decided: whether "world model" belongs on your vendor-evaluation radar yet, and how to read the claims when it arrives. Easy to undo, nothing to commit to. No deadline. So the useful work is learning to tell a real capability from a demo.

The Skeptic Watch the input count. Marble makes worlds from a single image, which sounds magical until you remember the model is inventing everything the camera never saw. That's a learned guess, not observed geometry. Great for a game backdrop or a mood board. Not great for anything where being wrong about the room behind you matters. And "single image to 3D world" demos are the most cherry-picked genre in AI right now. Johnson is candid that even his own team runs two contradictory approaches, Marble and RTFM, because nobody knows which scales. When the lab hedges that hard, don't let a vendor tell you it's solved.

The Researcher Johnson's taxonomy is the takeaway worth keeping: a world model either renders pixels, plans actions, or evolves world state. Three different jobs, often sold under one word. When a robotics vendor and a game-asset vendor both say "world model," they mean different machines. His second real point is that the architecture isn't changing. Transformers stay. The bottleneck is context length: an LLM rarely needs millions of tokens, a 3D scene needs hundreds of millions routinely. That reframes progress. Progress here means a cheaper way to hold enormous scenes in memory, not a smarter model.

The Compute Pragmatist That context number is the whole cost story. Attention memory, the running scratchpad a transformer keeps of everything it's seen, grows with the sequence. Hundreds of millions of tokens per scene means these workloads eat far more memory and compute per query than a chatbot. Johnson's own split proves it: Marble uses explicit 3D points because they're "consistent by construction" and cheaper, while he bets the implicit pixel route "scales to infinity" if you can afford the data and compute. Translation for a buyer: the good version runs on frontier-lab budgets, and per-query cost will stay high long after the demos look finished.

The Builder What's usable today is narrow and real. Gaussian splats, the cloud of colored semi-transparent points Marble outputs, export to meshes for game engines and VFX pipelines. If you make 3D assets, environments, or previz, that's a live format to test now, alongside images and video. Everything else is a research preview. There's no API contract to depend on, no latency you can quote a client, no consistency guarantee. RTFM makes frames in real time but promises nothing about whether the world stays the same when you turn around. Ship the asset-generation use case. Do not put the navigable-world piece in front of a paying customer's workflow yet.

The Open-Source Advocate Gaussian splatting itself came out of open research and early Meta demos, which is why splats are a shared primitive and not a walled format. That's the good news: the output type is portable. The bad news is the generative model that produces novel worlds from one image is proprietary to World Labs, and it took a lab co-founded by Fei-Fei Li to build. There's no open checkpoint within 80% of Marble to grab. For now the interface is open, the capability is closed.

Where they disagree. The Compute Pragmatist and the Open-Source Advocate split on when this reaches ordinary buyers. If the winning approach is the implicit pixel route that "scales to infinity," it stays a frontier-budget game for years and only a few labs play. If explicit splats, cheaper and consistent, turn out to be enough for most commercial jobs, the capability commoditizes faster because the output format is already open. Johnson refuses to call it, and that refusal is the most useful thing he says. The Skeptic and the Builder agree on the one safe move: use the export format, ignore the navigable-world promise.

What this hinges on: whether the cheap, consistent explicit-3D approach is good enough for real commercial work, or whether you need the expensive implicit route that only frontier budgets can run. That single fork decides whether spatial AI is a tool you buy in two years or a lab demo you admire for five.

No high-conviction prediction this week.

This is a pre-paradigm research conversation. Johnson names no ship date, no price, no benchmark anyone can grade, and explicitly says the field doesn't know which approach wins. There's no mechanism here that resolves into a checkable real-world event on a clean date. Forcing a call off a tutorial would be inventing precision the material doesn't support. The one thing worth tracking without betting on it: whether a splat-native output format shows up in a mainstream game engine or VFX toolchain, because that's the first place this capability turns into something an operator actually buys.