Xiaomi-TabLDM: One Model for All Tabular Tasks via Synthetic Pretraining & Test-Time Scaling

Introduction

Xiaomi has unveiled Xiaomi-TabLDM , a general-purpose tabular foundation model designed to bring the "pretrain once, adapt across tasks" paradigm to structured data. Core domains such as finance, healthcare, manufacturing, and logistics rely heavily on tabular data, yet traditional machine learning (XGBoost, LightGBM, CatBoost) requires retraining, hyperparameter tuning, and ensembling for each new dataset, incurring high production and deployment costs. Xiaomi-TabLDM aims to eliminate this per-dataset overhead by leveraging a single pretrained model with a unified default configuration that directly adapts to different tabular datasets for classification and regression.

Three Technical Directions

Xiaomi-TabLDM advances tabular foundation models along three axes:

Large-scale synthetic data pretraining. The model is pretrained entirely on synthetic tabular tasks. By structurally generating diverse data scales, variable types, dependency relationships, and functional relationships, the pretraining distribution covers a wide variety of table structures, letting the model encounter diverse patterns upfront.

Efficient model scaling. The architecture introduces dual-stream Feature Groups, lightweight Attention Residual connections, and a Sparse Mixture-of-Experts (MoE) layer. These components model feature relationships at different granularities, selectively reuse historical information in deeper networks, and expand model capacity via sparse experts so that performance gains do not require proportional compute increases.

Test-Time Scaling. Without changing pretrained parameters, Xiaomi-TabLDM can increase inference compute by leveraging different feature permutations, data transformations, and prediction perspectives to produce complementary results. These are then adaptively selected and combined per dataset to further boost predictive performance.

Benchmark Results

Xiaomi-TabLDM was evaluated on four public benchmarks — TALENT, TabArena, BCCO, and OpenML — covering binary classification, multi-class classification, and regression across sample sizes up to 100k.

Regression excellence: Ranked 1st on OpenML-CTR23; 2nd on TALENT, TabArena, and BCCO Regression.

Binary classification: Ranked 1st on TALENT.

Performance–efficiency trade-off: On TabArena Regression, Xiaomi-TabLDM achieves 1900 Elo with only 3.12 seconds per 1k validation samples, significantly faster than TabFM's 9.67 seconds.

Industrial Applications

Real-world validation in manufacturing scenarios demonstrates substantial gains over traditional ML:

Material performance prediction: Without fine-tuning, Xiaomi-TabLDM improves prediction accuracy by 130% and reduces invalid trial-and-error tests by ~90% , dramatically accelerating R&D.

Zero-piece weight prediction: Using production line sensor and process data, the model reduces misprediction sample ratio by ~31% compared to XGBoost.

Production component prediction: Average error drops by ~54% versus XGBoost.

Rapid adaptation to shifting conditions: When production conditions change, adding only ~30 new samples as context cuts average error by ~62% . Even with 200 new samples , traditional ML still exhibits far higher error. Xiaomi-TabLDM achieves better results with only ~15% of the new samples and no fine-tuning , showcasing strong few-shot generalization and fast condition adaptation.

Open Source Release

Model weights, code, and technical report are publicly available:

Code: https://github.com/xiaomi-research/xiaomi-tabldm Weights: https://huggingface.co/occams/Xiaomi-TabLDM Technical report:

https://arxiv.org/abs/2609.03880