Unleashing Large Language Models for Graph Continual Learning: The UNIT Framework

Problem Setting

In multimedia web scenarios graph data arrive as a continuous stream (e.g., social‑network relationships, knowledge‑graph connections, recommendation interactions). Graph continual learning (GCL) must learn new tasks without forgetting previous knowledge.

Method Overview (UNIT)

UNIT consists of three sequential steps that keep the large language model (LLM) frozen after an initial adaptation:

Semantic instruction tuning of the LLM on the first task only.

Uncertainty‑aware semantic anchor generation for each task.

Lightweight structural anchor injection to complement semantic anchors.

Module 1 – Semantic Instruction Tuning

A graph prompt is built for each node containing task description, node text, local neighbour context, and candidate classes. The LLM is fine‑tuned with LoRA on the first task, aligning its pretrained knowledge with the graph distribution while keeping later tasks parameter‑free.

Module 2 – Uncertainty‑Aware Anchor Generation

For each labelled node the tuned LLM predicts a class distribution; entropy is used to compute a confidence weight. High‑confidence embeddings receive larger weights when updating class‑level semantic anchors. Anchors are merged across tasks using an exponential decay factor 𝛾_s, which balances retention of historic knowledge against adaptation to new tasks.

Module 3 – Structural Confluent Modeling

For each node lightweight structural descriptors (degree, average neighbour degree, clustering coefficient) are extracted and embedded as structural anchors. Class‑level structural anchors are updated across tasks with the same decay mechanism (factor 𝛾_t), providing explicit topological cues that complement semantic anchors in the classifier.

Experimental Setup

Evaluation uses five public text‑attributed graph datasets: Cora, Citeseer, WikiCS, Photo, and Products. The task is node‑level class‑incremental learning. Baselines cover three groups:

Traditional GCL methods: GCN, EWC, LwF, Cosine, TPP.

LLM‑based classifiers: BERT, RoBERTa, LLaMA, SimpleCIL.

Graph‑language‑model methods: GCN‑LLMEmb, ENGINE, GraphPrompter, GraphGPT, LLaGA, SimGCL.

Metrics are average accuracy over all tasks (ACC_avg) and final‑task accuracy (ACC_N).

Main Results

UNIT achieves the highest scores on all datasets. Example numbers (standard setting):

Cora – 88.7 ACC_avg / 85.9 ACC_N

Citeseer – 79.4 / 68.5

WikiCS – 76.1 / 65.0

Photo – 83.9 / 76.5

Products – 73.2 / 62.5

Performance curves show stable accuracy across task sequences, unlike traditional GNNs that degrade sharply.

Few‑Shot Scenario

Under limited labelled data, UNIT still leads or matches the strongest baselines, delivering up to 9.1 % improvement in ACC_avg and 14.6 % improvement in ACC_N over the best competing method.

Ablation Study

Removing any of the three modules reduces both ACC_avg and ACC_N:

Without semantic instruction tuning, the LLM’s adaptation to the graph task weakens.

Without uncertainty‑aware weighting, historic knowledge migration becomes less reliable.

Without structural confluent modeling, the synergy between semantics and topology collapses.

The full model consistently outperforms each variant.

Parameter Sensitivity

Two decay factors control the trade‑off between historic and new knowledge. Low values cause forgetting; high values cause over‑reliance on past knowledge. UNIT remains relatively stable across a broad range of decay settings, indicating robustness to hyper‑parameter choice.

Limitations

Evaluations are limited to node‑classification on text‑attributed graphs; richer tasks such as edge prediction, graph classification, or multimodal node attributes are not explored. The structural descriptors are simple local features; modeling higher‑order or dynamic topology remains open. Scaling to larger LLM backbones may affect deployment cost, latency, and storage.

Code example

来源：专知
本文
约4500字
，建议阅读
7
分钟
大语言模型具备强语义理解能力，能不能真正用于图持续学习？