Unlocking Pre‑trained Graph Models: The UniPrompt Approach to Graph Prompt Learning

Background

Graph Prompt Learning (GPL) connects pre‑trained graph neural networks (GNNs) with downstream tasks, aiming to reduce reliance on labeled data and to bridge the gap between upstream pre‑training objectives and downstream goals.

Key Limitations of Existing GPL Methods

Unclear underlying mechanisms: Existing approaches intervene at different stages—input‑level augmentation, layer‑wise prompt injection, or representation‑level prompts—without a unified theory describing how prompts interact with the frozen pre‑trained model.

Limited adaptability: Many methods fail to generalize when the downstream data distribution shifts (e.g., from homogeneous to heterogeneous graphs) or when the downstream task differs substantially from the pre‑training task.

Theoretical Insight

By formalizing representation‑level prompts as additive or multiplicative transformations applied to the hidden representations of a frozen encoder, we show that such prompts are mathematically equivalent to fine‑tuning a simple linear classifier on top of the encoder. Consequently, the prompt’s primary function is to expose the latent knowledge of the pre‑trained model, while the downstream classifier handles task‑specific adaptation.

UniPrompt: A Unified GPL Framework

Guided by the above insight, UniPrompt is designed to work with any pre‑trained graph model. Its pipeline consists of:

Preserving the original graph structure (node features, edge indices) and feeding it unchanged into the frozen encoder.

Extracting the encoder’s final node/graph embeddings.

Optionally applying a lightweight prompt module (e.g., a learnable bias vector) that does not modify the encoder weights.

Training only a downstream classifier (e.g., a linear layer or MLP) on the prompted embeddings.

This separation ensures that the encoder’s parameters remain untouched, reducing computational cost and avoiding catastrophic forgetting.

Experimental Validation

UniPrompt was evaluated on multiple benchmark datasets covering both homogeneous and heterogeneous graphs, and on several downstream tasks such as node classification, graph classification, and link prediction. Experiments included:

Integration with three popular pre‑trained GNNs (e.g., GraphCL, GCC, and G‑Transformer).

In‑domain evaluation where training and test graphs share the same distribution.

Cross‑domain evaluation where the test graphs differ in node/edge types or label space.

Results consistently showed that UniPrompt matches or exceeds the performance of state‑of‑the‑art GPL methods, often improving accuracy by 2–5 % in cross‑domain settings while using fewer trainable parameters.

Practical Recommendations

Keep the pre‑trained encoder frozen; only the classifier (and optional prompt bias) should be updated.

When transferring to a heterogeneous graph, retain the original adjacency and feature matrices; UniPrompt’s prompt does not need to be redesigned.

Use standard optimization settings (e.g., Adam with learning rate 1e‑3) and early stopping based on validation loss.

Code example

来源：专知
本文
约1000字
，建议阅读
5
分钟
我们提出图提示学习应当聚焦于释放预训练模型的能力，而由分类器来适应下游场景。