A tabular predictor trained on synthetic data absorbs causal structure as a side effect

The central claim here is architectural: standard tabular foundation models organize prediction around a single designated target column, and that confinement limits what they can learn about the joint structure of a dataset. LimiX-2 replaces that target-centric objective with something broader — Contextual Mechanism Networks (CMNs), trained via Context-Conditional Masked Modeling (CCMM), which forces the model to predict any masked variable given the rest, not just a fixed label column. The result is denser supervision and a model that handles classification, regression, missing-value imputation, and causal discovery from a single set of pretrained weights.

The architecture keeps cell-level representations rather than collapsing each row into one vector. That choice turns out to matter for more than prediction accuracy: because each feature retains its own embedding, the attention scores between features and a target variable can be read as a proxy for direct causal adjacency. On six causal discovery benchmarks — ranging from the Sachs protein network to the PIGS genetic pedigree — LimiX-2 achieves a mean skeleton F1 of roughly 0.80, outperforming dedicated causal discovery methods like PC, GES, and AVICI on most datasets, and doing so without any task-specific training. That is a genuinely surprising result: a tabular predictor absorbing causal structure as a side effect of its pretraining objective.

Pretraining data comes entirely from synthetic tables generated by structural causal models. The generation engine samples DAGs, propagates values through diverse functional mechanisms (MLPs, decision trees, kernel functions, piecewise and periodic functions, and their compositions), then applies stochastic observation transformations before handing the data to the model. Nothing real is in the pretraining corpus.

On the three benchmarks reported — TabArena (51 datasets), TALENT (288 datasets after exclusions), and BCCO (156 datasets) — LimiX-2 tops the Elo leaderboard in every category. On TabArena it reaches an Elo of 1935, beating the nearest foundation-model competitor TabFM+ by 117 points before rounding, with an improvability of 3.3% against TabFM+'s 6.2%. On BCCO regression it achieves an average rank of 3.10, well ahead of TabFM at 3.93. The model does this at 406M parameters — roughly four times smaller than TabFM — and scaling experiments across six model sizes from 12.5M to 406M show clean log-linear Elo growth with no sign of saturation, implying the gains continue into the billion-parameter range.

The honest caveat is that every comparison is against models trained on real data or pretrained differently, so parameter efficiency claims are confounded by architecture, training compute, and data distribution simultaneously. The scaling extrapolation to 2B parameters is explicitly flagged as a forecast. Still, the combination of top predictive performance across three independent benchmarks and emergent causal awareness from a purely synthetic pretraining regime makes this a substantive result, not just an incremental leaderboard entry.