Large‑Scale Table Pretraining Model SPACE‑T: Background, Architecture, and Applications

This article introduces Alibaba DAMO Academy's large‑scale table pretraining model SPACE‑T, covering its background, technical trends, architecture, deployment, and future outlook.

01. Background and Technical Trends of Table QA

TableQA and Text2SQL have attracted significant academic attention in recent years. Enterprises store knowledge in various formats, especially tables, forming data middle‑platforms that support OA and other information systems. Extracting decision‑making information from these large‑scale tables can reduce costs and improve efficiency, but requires intelligent systems such as knowledge graphs, dialogue, and BI analysis.

Despite heavy investment in data middle‑platforms, companies still need to manually curate business knowledge to build intelligent applications, posing the challenge of directly leveraging data platforms for AI solutions.

Utilizing two‑dimensional table data (spreadsheets, web tables, relational databases) directly for AI can dramatically lower development costs.

Table QA Technology (TableQA) converts natural language queries into SQL, enabling users to interact with tabular knowledge via voice or text and receive accurate answers through a pipeline of language understanding, state tracking, SQL generation, and result retrieval.

Text‑to‑SQL aims to translate natural language questions into executable SQL statements, evolving from single‑table single‑turn models to large‑scale table pretraining approaches.

Practical deployment faces four main challenges: effectiveness, cost, efficiency, and language.

02. Large‑Scale Table Pretraining Model SPACE‑T

Building a large‑scale table pretraining model requires massive data and suitable pretraining techniques. Alibaba Cloud collected billions of tables across 17 industry categories, providing high‑quality, diverse training data.

The model addresses two challenges: high annotation cost and domain‑specific table knowledge. Instead of memorizing table facts, SPACE‑T learns to understand table schemas and content by jointly processing the question and table, enabling zero‑shot generalization to unseen tables.

SPACE‑T employs Linking Loss and Schema Loss to align table content with queries, producing SQL outputs that solve the aforementioned challenges.

03. SPACE‑T @ ModelScope

Users can search for SPACE‑T or SQL on ModelScope to find the pretrained model and try an online demo with multiple domain table examples. Developers can load the model via code, run queries against an in‑memory database, and obtain structured SQL results for further processing.

The demo illustrates the workflow from experience to development and customization, with interfaces for tailored applications.

04. SPACE @ ModelScope

SPACE‑T belongs to the broader SPACE family, which includes dialogue and document models. The dialogue models use semi‑supervised pretraining, combining high‑quality supervised data with large‑scale unsupervised data, achieving SOTA on 11 international dialogue benchmarks.

SPACE‑3, now available on ModelScope, offers four models: dialogue generation, intent recognition, pretrained dialogue, and state tracking, with ready‑to‑run code for reproducing SOTA results.

05. Summary and Outlook

Tables are the most common structured knowledge across industries; leveraging them directly for intelligent systems can greatly reduce costs.

SPACE‑T, trained on billions of tables, provides strong out‑of‑the‑box capabilities and is deployed in ModelScope and Alibaba Cloud Intelligent Customer Service, supporting multiple domains.

Users can access Chinese and English versions of SPACE‑T on ModelScope, experience it online, or integrate it via notebooks or code for custom applications.

The SPACE family also offers various dialogue models for building conversational AI.

Future work will continue to improve SPACE‑T’s performance and expand its abilities.

Thank you for attending the session.