Graph Foundation Models: Concepts, Progress, and Future Directions

The rapid advancement of large language models (LLMs) has highlighted the importance of the Transformer architecture across text, video, and audio domains, prompting researchers to explore how graph models can benefit from similar large‑scale pre‑training. Graph Foundation Models (GFMs) are introduced as a new class of models pre‑trained on extensive graph data to serve a variety of downstream graph tasks.

1. Foundations of GFMs – A foundation model is a large‑scale model trained on broad data that can be fine‑tuned for many downstream tasks. In the graph domain, GFMs aim to inherit the emergence and homogenization properties observed in LLMs, enabling a single model to handle diverse graph problems.

2. Historical Development of Graph Machine Learning – From Euler’s Seven Bridges problem to modern graph neural networks (GNNs), the field has evolved through matrix‑factorization methods, random‑walk based embeddings, and deep GNN architectures such as GCN. Recent work focuses on self‑supervised pre‑training and prompt‑based adaptation.

3. Related Work – Existing approaches are categorized into three streams: (a) GNN‑centric models (e.g., Graph‑BERT, GROVER, Graph‑Prompt), (b) LLM‑centric models that convert graphs to tokens or text (e.g., InstructGLM, NLGraph, LLM4Mol), and (c) hybrid GNN+LLM models (e.g., SimTeG, TAPE, ConGrat, G2P2, Graph‑Toolformer) that align graph representations with language representations.

4. Our Team’s Contributions PT‑HGNN (KDD 2021) : Introduces same‑scale contrastive learning and vanilla fine‑tuning for heterogeneous graphs, preserving semantic and structural knowledge while enabling scalable pre‑training. Specformer (ICLR 2023) : Combines spectral graph theory with Transformers by encoding Laplacian eigenvalues and using a channel‑wise decoder to learn expressive graph filters, achieving strong performance on synthetic and real‑world benchmarks. GraphTranslator (WWW 2024) : Proposes a two‑stage framework that translates graph embeddings into token sequences for LLM alignment (Translator) and generates chain‑of‑thought descriptions using LLMs (Producer), demonstrating significant gains on zero‑shot e‑commerce and QA tasks.

5. Summary and Outlook – The authors emphasize the need for larger, higher‑quality graph datasets, improved backbone architectures, better training strategies, and robust evaluation pipelines to unlock the full potential of GFMs. Future research directions include data scaling, architectural innovation, and finding killer applications in domains such as drug discovery, urban computing, and recommendation systems.

6. Q&A Highlights – Discussed interpretability of GNNs, the challenges of knowledge transfer across domains, the distinction between homogeneous and heterogeneous graph models, and the promise of multimodal integration with graph structures.