Leveraging Large Language Models for Graph Recommendation System Optimization

Overview – The rapid interest in applying large language models (LLMs) to improve graph recommendation systems is explored through four major strategies: embedding graph nodes with LLMs, deep fusion of graph structures and LLMs, pure model‑driven graph data training, and enhancing recommendation algorithms with textual modalities.

1. LLMs as Graph Prefixes – GNNs act as tokenizers, converting graph data into rich token sequences for LLMs. Two approaches are discussed: node‑level tokenization (fine‑grained node information) and graph‑level tokenization (global structural semantics via pooling).

2. LLMs for Prefix – Embedding vectors generated by LLMs serve as inputs to GNNs, improving node embeddings and training supervision. This includes using LLM‑generated labels as additional signals.

3. LLM‑Graph Interaction – Three categories of hybrid methods are presented: (a) joint training of GNN and LLM parameters, (b) alignment of representations between GNN and LLM, and (c) LLM‑based autonomous agents for graph tasks.

4. LLM‑Only Approaches – Techniques that let LLMs directly ingest graph structures without fine‑tuning, either by designing prompts that encode graphs or by converting graphs to token sequences and aligning them with natural language tokens.

Representation Learning with Text Modality – User‑item interaction data are encoded as feature vectors, then combined with textual descriptions (product or user profiles). Mutual information between text and collaborative‑filtering embeddings is maximized via an InfoNCE lower‑bound, using a critic function f.

High‑Quality Text Feature Extraction – Large language models transform raw product descriptions, reviews, or attributes into concise item profiles; a similar process generates user profiles. Various embedder models (Contriever, Instructor, Text‑embedding‑ada‑002) are evaluated, with the latter chosen as default.

Alignment Methods – Two alignment strategies are implemented: (a) Contrastive Alignment, projecting GNN embeddings to the same dimension as text embeddings and applying cosine similarity with an exponential kernel; (b) Generative Alignment, masking node features, reconstructing them via an MLP, and computing similarity similarly.

Explainable Recommendation – The system generates natural‑language explanations for recommendations by feeding combined GNN and text embeddings into an LLM fine‑tuned on ground‑truth explanations derived from GPT‑3.5 and user reviews. An MOE Adapter aligns GNN embeddings with LLM token space.

Experimental Validation – Experiments on Yelp, Amazon‑Book, and Steam datasets show that integrating the proposed RLMRec method improves baseline recommendation performance by up to 8%. Additional studies examine the impact of text encoder quality, alignment type, pre‑training, and data sparsity on both recommendation accuracy and explanation quality (measured by GPTScore and BERTScore). Ablation tests confirm the contribution of each module (embedding injection, GNN embedding, text modality).

Conclusion – The work demonstrates that large language models can substantially enhance graph‑based recommendation systems through richer representation learning, effective multimodal fusion, and generation of interpretable recommendation reasons, with all code and data publicly released.