TabFM: Google’s New Zero-Shot Foundation Model for Tabular Data (Hands-On Guide)

Google Research just released something that a lot of data folks have been waiting for without quite knowing it — a foundation model built specifically for tables. It’s called TabFM, and it’s doing for spreadsheets and structured data what models like TimesFM already did for time-series forecasting.

In this post, I’ll walk through what TabFM actually is, why it matters, how to install and run it yourself, and — since I ran into a real bug while testing it — how I debugged it live, because that’s part of the honest story of using a model that’s only days old.

https://youtu.be/-LpTbS3mvFw

The problem TabFM is solving

We’ve had incredible frontier models for text, code, images, even video for a while now. But structured tabular data — the stuff that actually runs banks, hospitals, retail, and logistics — has mostly stayed in the hands of classic algorithms like XGBoost, LightGBM, and random forests.

Foundation models never got much traction here, and there’s a good reason: tables are structurally weird compared to language. Rows and columns can be shuffled without changing the meaning of the data, there’s no natural left-to-right “sequence” the way there is in a sentence, and a regular transformer doesn’t map onto that cleanly.

TabFM’s whole pitch is bringing the same “just show me examples and I’ll figure it out” magic that works for LLMs into this boring-but-massive world of rows and columns.

What TabFM actually does

Instead of grabbing XGBoost and grinding through feature engineering, hyperparameter tuning, and cross-validation every time you hit a new dataset, you hand TabFM your training rows and your test rows together, and it predicts the missing values in a single forward pass.

It treats tabular prediction as an in-context learning problem — the same idea that lets large language models pick up a new task just from a few examples in the prompt.

How the architecture works

Under the hood, TabFM combines ideas from two existing lines of research — TabPFN and TabICL:

1. Alternating row/column attention. The model looks down each column to understand individual features, then across each row to see how those features combine — back and forth, automatically. This replaces the manual feature engineering step data scientists usually have to do by hand.
2. Row compression. Each row gets squeezed down into a single dense summary vector.
3. In-context learning transformer. A 24-block causal transformer runs over those compressed row vectors. Rows that already have known answers act as “context,” and the model predicts the missing ones in one pass — no training loop involved.

One detail I think is genuinely the most interesting part of this release: TabFM was trained entirely on synthetic data — hundreds of millions of synthetic datasets generated using structural causal models (SCMs). Google went this route because genuinely diverse, high-quality, open tabular datasets at scale basically don’t exist — most real industrial tables are private or full of sensitive information. So instead, they built a synthetic universe of tables to teach the model general patterns of how rows and columns tend to relate to each other, and that generalizes surprisingly well to real-world data.

Why .fit() shows up even though there’s “no training”

If you look at the code below, you’ll notice .fit() is still called — which seems to contradict the “no training” pitch. Here’s the actual distinction, straight from the model documentation:

• Traditional models (XGBoost, etc.): .fit() runs gradient descent or builds trees specific to your dataset. The model’s parameters change. You end up with a dataset-specific model.
• TabFM: .fit() just prepares ordinal encoders for categorical columns, fits numerical scalers, and stores your training rows to use as context later. No gradient updates happen. No weights change.

The actual model only runs at .predict() time — and that’s when it processes your training rows and test rows together, using the same frozen pretrained weights every single time, regardless of what dataset you throw at it. TabFM reuses the familiar scikit-learn API for convenience, but don’t let the naming fool you — nothing is being trained on your data.

Installing TabFM

You’ll need Python 3.11 or newer. Clone the repo and install with the backend of your choice:

git clone https://github.com/google-research/tabfm.git
cd tabfm
pip install -e .[pytorch]

For PyTorch specifically, TabFM pins torch==2.12.1 (or a matching GPU build for your CUDA version) — check requirements.txt in the repo for the full pinned dependency list.

Hardware note: you don’t need much. TabFM is a small model (256-dim embeddings, 24 transformer blocks) and comfortably runs under 4GB of VRAM. A laptop GPU with 6–8GB is plenty. The only thing that scales your memory usage is how many training rows you pass in as context — the model size itself stays fixed.

Weights download automatically from Hugging Face the first time you load the model.

Running it: classification example

Here’s a simple credit-risk style classification example, copied straight from the official repo:

import numpy as np
import pandas as pd
from tabfm import TabFMClassifier
from tabfm import tabfm_v1_0_0_pytorch as tabfm_v1_0_0

model = tabfm_v1_0_0.load()
clf = TabFMClassifier(model=model)

X_train = pd.DataFrame({
    "age": [25.0, 45.0, 35.0, 50.0],
    "job": ["engineer", "manager", "engineer", "manager"],
    "income": [80000, 120000, 90000, 130000]
})
y_train = np.array(["low_risk", "high_risk", "low_risk", "high_risk"])

X_test = pd.DataFrame({
    "age": [30.0, 48.0],
    "job": ["engineer", "manager"],
    "income": [85000, 125000]
})

clf.fit(X_train, y_train)
predictions = clf.predict(X_test)
probabilities = clf.predict_proba(X_test)

print("Predictions:", predictions)
print("Class Probabilities:\n", probabilities)

This ran cleanly for me — no training loop, just fit (encoders + scaling) and predict, with the model correctly separating low-risk from high-risk applicants and reporting high confidence on both.

Running it: regression (and a real bug I hit)

The regression example follows the same pattern:

import numpy as np
import pandas as pd
from tabfm import TabFMRegressor
from tabfm import tabfm_v1_0_0_pytorch as tabfm_v1_0_0

model = tabfm_v1_0_0.load()
reg = TabFMRegressor(model=model)

X_train = pd.DataFrame({
    "sqft": [1200, 2500, 1500, 3000],
    "neighborhood": ["A", "B", "A", "C"]
})
y_train = np.array([250000, 550000, 310000, 620000])

X_test = pd.DataFrame({
    "sqft": [1800, 2800],
    "neighborhood": ["A", "B"]
})

reg.fit(X_train, y_train)
predictions = reg.predict(X_test)

print("Predicted Prices:", predictions)

While testing this on a fresh Colab environment, reg.predict(X_test) threw:

ValueError: cannot select an axis to squeeze out which has size not equal to one

Digging into the traceback, this wasn’t a bug in the sample code — it was failing inside the library’s own internals (tabfm/src/classifier_and_regressor.py), where the model’s output tensor had a last dimension that wasn’t size 1, and the library’s squeeze(-1) call choked on it.

Since TabFM is only days old at the time of writing, this looks like a release-week bug rather than anything wrong with the usage pattern. If you hit the same error, here’s what to try:

1. Pull the latest commit and reinstall — there’s a decent chance it’s already patched: cd tabfmgit pull origin mainpip install -e .[pytorch] --upgrade
2. Isolate whether it’s a batch-size issue by predicting one row at a time: for i in range(len(X_test)): print(reg.predict(X_test.iloc[[i]]))
3. If it persists, it’s worth checking or filing an issue on the google-research/tabfm GitHub repo with your exact traceback and environment details.

I’m including this because I’d rather show you the real, current state of a brand-new release than pretend everything works perfectly out of the box.

Benchmark results

Google evaluated TabFM on TabArena, a benchmark spanning 51 datasets (38 classification, 13 regression) ranging from 700 to 150,000 rows, ranked by Elo score. In pure zero-shot mode — no tuning, no hyperparameter search — TabFM sits right alongside heavily-tuned gradient-boosted tree baselines, and the ensemble variant (TabFMClassifier.ensemble(), which adds feature crosses, SVD features, and NNLS blending) pushes even higher.

That’s a strong result for a model that requires no dataset-specific training loop whatsoever.

Licensing — read this before you plan anything commercial

This is a detail that’s easy to miss, and I want to be upfront about it: the code on GitHub is Apache 2.0, so that part is genuinely open source. But the pretrained weights on Hugging Face are released under a separate TabFM Non-Commercial License. That means it’s free to use for research, learning, and personal projects — exactly what we’re doing here — but if you’re planning to ship this inside a commercial product, you’ll want to review that license carefully first, since commercial use isn’t currently covered.

What’s next: BigQuery integration

Google has also said they’re integrating TabFM directly into BigQuery, so before long you’ll be able to run this kind of prediction with a plain SQL command via AI.PREDICT, without needing a separate ML pipeline at all. That’s a meaningful signal about where this is headed — from a research checkpoint to something analysts can use without touching Python.

Wrapping up

TabFM is a genuine attempt to bring foundation-model-style, zero-shot prediction to plain rows and columns — the same way we’ve had it for text and images for a few years now. It’s not perfect yet (see: the regression bug above), and the license means commercial use needs a second look, but as a research tool and a preview of where tabular ML might be headed, it’s worth trying on your own data.

If you try it out, I’d genuinely like to hear what dataset you threw at it and how it did — drop a comment or reach out.

Google Colab: https://colab.research.google.com/drive/1d4E2ZiRf5bLYcDZ40sqieX2UV4TnOl__?usp=sharing

---

Watch the full hands-on video walkthrough [link to your YouTube video] for a live demo of installation, classification, regression, and the debugging process.