Exploring Graph Foundation Models: Concepts, Techniques, and Future Directions

The rapid development of large language models (LLMs) has highlighted the power of the Transformer architecture across text, video, and audio, prompting the exploration of Graph Foundation Models (GFMs) that combine graph learning with foundation model techniques.

GFMs are pretrained on massive graph data and can be adapted to a variety of downstream graph tasks. Their key characteristics—emergence and homogenization—mirror those of LLMs, enabling a single model to handle many heterogeneous graph problems.

The article reviews the evolution of graph machine learning, from early graph theory to modern graph neural networks (GNNs), graph signal processing, and network representation learning. It categorises existing approaches into three families: GNN‑based models, LLM‑based models, and hybrid GNN+LLM models, citing representative works such as Graph‑BERT, GROVER, GraphCL, InstructGLM, NLGraph, SimTeG, and ConGrat.

The authors’ own contributions are highlighted: (1) PT‑HGNN, which uses same‑scale contrastive learning and vanilla fine‑tuning to improve heterogeneous graph representation; (2) Specformer, a spectral‑domain model that encodes Laplacian eigenvalues and leverages a Transformer to learn expressive graph filters; (3) GraphTranslator, a framework that aligns graph embeddings with LLM token sequences through a translator‑producer pipeline, achieving strong zero‑shot performance on e‑commerce and academic datasets.

Experimental results demonstrate that PT‑HGNN, Specformer, and GraphTranslator outperform existing baselines on node classification, link prediction, and knowledge‑transfer tasks, while also offering better interpretability.

Finally, the article outlines future research directions for GFMs, including scaling data quantity and quality, improving backbone architectures and training strategies, developing robust evaluation metrics and killer applications, and exploring multimodal integration, transferability, and security aspects of graph models.