Podcast episode

🔬Causal Models Need Causal Data - Xaira’s X-Cell model for Drug Discovery (Bo Wang & Ci Chu, Chief Discovery Officer & Chief AI Scientist)

by Ken Rona ·July 22, 2026 ·Latent Space

causal-inference domain-specific-models fine-tuning open-weights

TL;DR

This episode is primarily a deep-dive into computational and wet-lab biology — specifically Xaira Therapeutics' X-Cell virtual cell model and X-Atlas dataset for AI-driven drug discovery. It is largely off-topic for the core AI frontier-lab/infrastructure beat, though it contains meaningful AI research signal for practitioners building or tracking foundation models applied to biology and drug discovery.

What was covered

Xaira Therapeutics overview: A $1B+ AI-native drug discovery company co-founded by protein-design pioneer David Baker (UW), with three AI platforms: protein design, virtual cell / foundation models of biology, and patient representation learning — aimed at end-to-end AI acceleration from target ID through Phase 3 clinical trials.
X-Cell model release: Xaira's first virtual cell model (4.9B parameters), built to predict how silencing individual genes alters gene expression across an entire cell — a counterfactual/causal prediction task, not merely descriptive. Preprint published; model and data are open-sourced.
X-Atlas dataset: A proprietary causal perturbation dataset ("PISCES") comprising ~25M high-quality cells across 16 biological contexts and 7 genome-wide CRISPR perturbation campaigns. Described as ~30× more information-rich than training on the public CELLxGENE corpus (168M observational cells) for causal tasks.
Architecture shift — autoregressive → diffusion: Xaira moved away from GPT-style autoregressive generation (used in Bo Wang's earlier scGPT model) toward a diffusion language model (BERT-style, iterative refinement), arguing gene expression has no natural sequential order, making masked/diffusion approaches a better fit. Ablations show significant gains on out-of-distribution generalization tasks.
Beating the linear baseline: A persistent benchmark problem in the field — foundation models trained on static/observational data (CELLxGENE) routinely fail to outperform simple linear models on perturbation prediction. X-Cell, trained on causal data, outperforms linear baselines on Pearson-delta metrics and generalizes to unseen cell types (e.g., activated T cells, primary donor T cells, held-out iPSC-derived cell types).
Scaling law insight — data information ceiling: Test loss flatlined at ~1.5B parameters on a single small dataset while training loss kept dropping — a clear signal that the model was data-information-limited, not compute- or parameter-limited. Scaling the causal dataset ~30× restored the expected scaling trend through at least 4.9B parameters.
Five biological priors baked in: X-Cell incorporates LLM-derived gene embeddings (gene-PT, via GPT querying), protein-protein interaction networks, DepMap cancer essentiality scores, morphology features, and scGPT cell-type embeddings — injected via cross-attention as conditioning signals.

Notable claims & predictions

Bo Wang: "From empirical experience, we find that the quality and amount of the datasets matters the most… training on perturbation datasets already gives a big boost. Switching from autoregressive training to diffusion language models gives a significant improvement over harder tasks, particularly generalization to unseen tasks."
Ci Chu: "Models trained on descriptive [observational] data do not yet outperform linear models on causal tasks… The correlation data in the descriptive dataset can be fit with many, many possible causal structures." — articulating why CELLxGENE-trained models fail at perturbation prediction.
Ci Chu: "This is the first time that someone can put together not just one perturbation campaign, but seven genome-wide perturbation campaigns together" — on the scale advantage of X-Atlas vs. prior public datasets.
Bo Wang (bottleneck prediction): "I hope to see a breakthrough in sequencing technology that can sequence the same cells at different time points… everything we model is extremely static. That would be the real virtual cell."
Ci Chu (bottleneck prediction): "If I can do high-throughput protein measurement at the same scale we can do genomics — their abundance, post-translational modifications, localizations, conformational states — such datasets will be incredibly useful to train the next generation of foundation models."

Why this matters for AI operators

Data quality > scale for specialized domains: The X-Cell scaling law result is a concrete, quantified demonstration that information content in training data — not raw token/cell count — determines whether foundation models can scale. This pattern likely generalizes: operators building domain-specific models (legal, scientific, medical) should audit whether their data is causal/interventional or merely observational/descriptive before assuming more compute will help.
Diffusion vs. autoregression for non-sequential data: The Xaira team's architectural pivot away from autoregressive LLMs toward diffusion models for data without natural token order (gene expression matrices) is a reusable design principle for AI practitioners working in tabular or high-dimensional scientific domains where sequence order is arbitrary.
The "beat the linear baseline" problem as a benchmark signal: The field-wide failure of foundation models trained on observational biology data to outperform linear models is a warning for applied AI teams: if your training distribution is purely observational and your task is counterfactual, model complexity alone won't save you. Causal or interventional training data is a prerequisite.
Open-source release of 4.9B biology foundation model + 25M-cell causal dataset: Practitioners building AI tools in pharma, biotech, or life-sciences adjacent verticals have a new open artifact (X-Cell weights, X-Atlas data on GitHub/Xaira) that could serve as a base model or benchmark, similar to how AlphaFold and scGPT seeded downstream work. Worth tracking for anyone operating at the AI × biology intersection.

Full analysis

Xaira Therapeutics — the $1B+ drug-discovery startup co-founded by protein-design pioneer David Baker — released X-Cell, a 4.9-billion-parameter "virtual cell" model, plus X-Atlas, a 25-million-cell causal perturbation dataset. Both are open-sourced. The headline for AI builders isn't the biology; it's the methodology claim: that causal (interventional) training data, not more compute or parameters, is what lets a foundation model actually beat a dumb linear baseline on counterfactual tasks. That's a general lesson dressed in a specialized domain.

Reversibility: N/A — this is a briefing, not a decision. The reusable takeaways (audit your data for causal vs. observational structure; consider diffusion for non-sequential data) are cheap Type-2 experiments for most teams.

What's actually being decided by the reader: whether the "data information content beats scale" result changes how you think about your own domain-specific model plans — and whether X-Cell/X-Atlas is worth pulling down.

The Skeptic. The load-bearing claim is "our causal data lets us beat a linear baseline that everyone else's models can't." Fine — but the honest read is that the field's foundation models have been embarrassingly bad at this. Beating a linear model on Pearson-delta is clearing a low bar, not landing AlphaFold-2. The "30× more information-rich" number compares their curated CRISPR campaigns against a general observational corpus (CELLxGENE) — apples to oranges, chosen to flatter. And generalization to "unseen cell types" is exactly the claim that dissolves when an independent lab runs it on their held-out biology. For a PM: the demo works, but "works better than the trivial baseline" is a weaker sentence than the headline implies.

The Researcher. The genuinely transferable result is the scaling-law diagnostic. Test loss flatlined at ~1.5B parameters while training loss kept dropping — a textbook signature of a data-information ceiling, not a compute ceiling. Scaling the causal data ~30× restored clean scaling through 4.9B. That's a clean, quantified demonstration you can cite in any domain. The autoregressive→diffusion pivot is also principled: gene-expression vectors have no natural token order, so a masked/iterative-refinement (BERT-style diffusion) objective fits better than GPT-style left-to-right generation. Ci Chu's framing — "correlation data can be fit by many possible causal structures" — is the cleanest one-line explanation of why observational-only training caps out on counterfactuals. In plain terms: you can't learn what happens if I flip this switch by only watching switches that were already flipped by someone else.

The Open-Source Advocate. This is the part that actually matters beyond biology. A 4.9B model and a 25M-cell causal dataset, both released — that's the AlphaFold/scGPT playbook: seed the downstream ecosystem, become the default base model, own the benchmark. The dataset is arguably the bigger gift than the weights; causal perturbation data at this scale is nearly impossible for an academic lab to generate. For anyone at the AI×bio intersection, X-Atlas is a free evaluation harness and a fine-tuning base. The catch: check the license and whether the "open" data covers all seven CRISPR campaigns or a subset. Open weights from a company that raised $1B usually come with a moat somewhere — here it's likely the next dataset they don't release.

The Compute Pragmatist. Refreshing story: they hit a ceiling at 1.5B params and their answer was not "rent more H100s." It was "generate better data." That inverts the default frontier-lab reflex and it's the cheaper lesson to internalize — wet-lab CRISPR campaigns are expensive, but for most software teams "audit whether your data is interventional" costs an afternoon and zero GPU-hours. The uncomfortable flip side: in biology, the bottleneck moves from silicon to the lab bench. Both founders named the same wall — you can't cheaply measure the same cell over time, or measure proteins at genomic throughput. Compute is not the constraint here; assay technology is. For a builder: money spent on the right 25M examples beat money spent on the next parameter decade.

The Builder. What would I actually do Tuesday? Not deploy X-Cell into a product — it predicts gene-silencing effects, that's a research instrument, not an API you wire into an app. What I'd steal is the diagnostic recipe: plot train loss vs. test loss as you scale, and if test flatlines while train keeps falling, stop buying compute and go fix your data. And the second reusable move — if your features have no inherent order (tabular rows, sets of attributes, sparse event logs), try a masked/diffusion objective before defaulting to a next-token transformer. Both are cheap to test and could save a quarter of wasted scaling.

Where they disagree:

Skeptic vs. Researcher on "beating linear." The Researcher sees a clean, quantified causal-data result worth citing. The Skeptic notes the baseline being beaten is trivially weak and the 30× comparison is rigged in Xaira's favor. Both are right: the method lesson is real even if the performance headline is modest.
Open-Source Advocate vs. Compute Pragmatist on where the moat is. The Advocate celebrates the open release; the Pragmatist points out that when data is the bottleneck, releasing one dataset while hoarding your data-generation pipeline is a stronger moat than closed weights ever were. Xaira gave away the artifact and kept the factory.
Everyone vs. the Builder on relevance. The bio result is a great parable, but for the median AI product team the directly actionable payload is two lines — a scaling diagnostic and an architecture heuristic. The rest is signal to track, not act on.

What it hinges on: whether "causal data beats scale" is a law or a special case. The mechanism (correlation underdetermines causal structure) is sound and general. But most software teams can't generate interventional data the way a CRISPR lab can — so the lesson often reduces to "know that observational data caps your counterfactual accuracy," not "go run experiments." Verify before over-generalizing: does the X-Cell generalization claim survive an independent held-out biology test, or is it in-distribution flattery?

Prediction: Within roughly 9 months — by the time follow-up virtual-cell preprints cite X-Cell (mid-2027) — at least one independent group will reproduce the core "causal data beats observational data on perturbation prediction" finding on a different dataset, while the specific "generalizes to unseen cell types" claim draws public pushback or fails to replicate cleanly.

Confidence: Medium — the mechanism is robust; the generalization claim is the fragile part.

Why: The underlying principle — that observational correlation data is consistent with many causal structures, so models trained on it can't reliably predict interventions — is mathematically sound and already an accepted pain point in the field (models failing to beat linear baselines is a documented field-wide problem, not Xaira's invention), so the directional result is very likely to hold up when others test it. But out-of-distribution generalization to unseen cell types is exactly the claim that historically collapses under independent scrutiny in both genomics and mainstream ML, because held-out performance inside one lab's data pipeline rarely transfers to another lab's assays and batch effects. The opposite outcome — clean replication of both claims with no pushback — is less likely precisely because open weights plus open data make the generalization claim easy for rivals to stress-test, and there's academic incentive to find the crack.

Revisit by 2027-04-30: We're right if a third-party paper confirms causal-data superiority on a new dataset and the cross-cell-type generalization claim is publicly challenged or narrowed. We're wrong if both claims replicate uncontested, or if no independent group engages with X-Cell at all.

Listen to the episode →