Introducing TabFM: A zero-shot foundation model for tabular data

Skip to main content

Google

Research

Search

Introducing TabFM: A zero-shot foundation model for tabular data

June 30, 2026<br>Weihao Kong and Abhimanyu Das, Research Scientists, Google Research

We’ve seen a massive shift in how people handle time-series forecasting since we launched TimesFM. Now, we’re bringing that same "zero-shot" logic to tabular data.<br>We introduce TabFM, a new foundation model for tabular data integrated directly into BigQuery ML to simplify classification and regression workflows.

Quick links

Hugging Face

GitHub

Share

Copy link

Tabular data constitutes the backbone of enterprise data infrastructure and powers a significant fraction of critical predictive machine learning applications. From predicting customer churn to identifying financial fraud, tabular regression and classification tasks are ubiquitous. For years, supervised tree-based algorithms like AdaBoost, XGBoost and random forests, to name a few, have historically dominated this space, offering robust performance on structured data.<br>However, the lifecycle of deploying these traditional models presents a significant bottleneck. Fitting an XGBoost model to a new dataset is not merely a matter of a single .fit() step; it invariably requires tedious manual effort. Data scientists must invest countless hours into extensive hyperparameter optimization and domain-specific feature engineering just to extract a reliable signal from the raw data.<br>On the other hand, recent advances in the broader machine learning landscape — particularly the evolution of large language models (LLMs) — have changed how we interact with novel tasks. LLMs have demonstrated the remarkable power of zero-shot prediction through in-context learning (ICL). This technique lets a pretrained model learn a new task by providing examples and instructions in the input context, without updating any underlying model weights.<br>Today, we introduce TabFM, a foundation model designed specifically for tabular data classification and regression. By framing tabular prediction as an ICL problem, TabFM eliminates the need for manual model training, hyperparameter tuning, and complex feature engineering. We are excited to share how this approach allows users to generate high-quality predictions on previously unseen tables in a single forward pass. TabFM is now available on our Hugging Face and GitHub repos.

How it works<br>The traditional ML paradigm relies on updating model parameters specific to a given dataset's distribution. In contrast, the ICL paradigm bypasses this completely. Instead of undergoing a traditional training phase for each new task, TabFM takes the entire dataset — comprising both the historical training examples and the target testing rows — as a single unified prompt. The model learns to interpret the relationships between columns and rows directly from this context at inference time.<br>However, applying ICL to tabular data is not as straightforward as tokenizing natural language. Standard language models process one-dimensional, ordered sequences, but tables are fundamentally two-dimensional and inherently orderless: swapping two rows or two columns does not change the underlying meaning of the data. To effectively process these diverse tabular structures while enabling scalable zero-shot prediction, TabFM synthesizes the strengths of architectures like TabPFN and TabICL into a novel hybrid design. This architecture, visualized below, relies on three key mechanisms:<br>Alternating row and column attention: First, the raw table is processed through a multilayer attention module. Similar to TabPFN, this step applies alternating attention across both columns (features) and rows (examples). By continuously attending across these two dimensions, the model learns rich representations that natively capture complex feature interactions and dependencies. This deep contextualization effectively performs the heavy lifting that would otherwise require tedious manual feature crafting by data scientists.<br>Row compression: Following this contextualization, the rich, cross-attended information for each individual row is compressed into a single, dense vector representation.<br>In-context learning (ICL): Finally, a dedicated Transformer operates on this sequence of compressed embeddings. Adopting the highly efficient approach of TabICL, performing attention over these compressed row vectors — rather than the raw, uncompressed grid — drastically reduces the computation cost. This ensures the prediction step remains highly computationally efficient, even for much larger datasets.

TabFM model architecture.

Training on synthetic data at scale<br>A typical recipe for building foundation models is to use a high-capacity neural network trained on vast amounts of diverse data. However, a major hurdle in tabular ML is that high-quality, diverse tabular datasets — especially the massive tables required to reflect true...