GraphCLIP and 2D‑TPE: Enhancing Transferability of Graph Models and Table Understanding for Large Language Models

Structured data such as text‑attributed graphs (TAGs) and tables are central to many domains, yet traditional methods struggle with strong label dependence, poor cross‑domain transfer, and loss of spatial information.

At the ACM Web Conference (WWW) 2025 in Sydney, Ant Group has two papers accepted: GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text‑Attributed Graphs and 2D‑TPE: Two‑Dimensional Positional Encoding Enhances Table Understanding for Large Language Models . Both papers address the above challenges from graph learning and table understanding perspectives.

GraphCLIP proposes a self‑supervised graph‑summary contrastive pre‑training approach that leverages large‑scale graph‑summary pairs generated by LLMs and incorporates invariance learning. It also introduces a graph‑prompt tuning technique aligned with the pre‑training objective to mitigate catastrophic forgetting in few‑shot scenarios. Experiments show GraphCLIP outperforms a 72B‑parameter LLM in zero‑shot settings and surpasses existing graph‑prompt methods in few‑shot learning, with strong results on downstream tasks such as node classification and link prediction.

2D‑TPE tackles the problem that LLMs typically accept only one‑dimensional token streams, causing flattening of tables and loss of spatial context. The proposed two‑dimensional positional encoding allows each attention head to dynamically choose token ordering (row‑wise, column‑wise, etc.), preserving table structure while keeping computational efficiency. Extensive experiments on five benchmark datasets demonstrate that 2D‑TPE consistently exceeds strong baselines, highlighting the importance of retaining table geometry for accurate understanding and showing superior scalability on large tables.

The live paper session (Paper Show Live #16) will feature the first authors, Zhu Yun (Ant Group research intern, Zhejiang University PhD) and Li Jianan (Ant Technology research intern, Renmin University PhD), who will share design ideas and validation processes. The session is scheduled for March 6 2025, 18:00‑20:00, and will be streamed on WeChat Video Channels, AntTech, and Bilibili.