Can Large Language Models Learn Recommendation Knowledge? A NL‑Simulated Auxiliary Task

Problem

Large language models (LLMs) are increasingly used as backbones for recommender systems, but on standard retrieval and ranking tasks they often lag behind traditional methods. The authors attribute this to a mismatch between the generic knowledge encoded in LLMs and the domain‑specific knowledge required for recommendation, such as detailed user‑item interaction patterns.

Proposed Method

Inspired by Masked Item Modeling and Bayesian Personalized Ranking (BPR), the paper proposes to generate auxiliary recommendation‑task data by phrasing recommendation operations as natural‑language instructions. Example prompts include “Choose an item for the user from the candidates.” These prompts are used to create text samples that encode item relevance and user preference signals. The generated samples—called recommendation‑task data—are then used to fine‑tune LLMs, thereby injecting recommendation‑specific knowledge without relying solely on raw user/item IDs.

Data Generation Details

Each training instance consists of a textual instruction, a set of candidate items, and the correct item(s) according to the underlying interaction data.

Masked‑item modeling is simulated by masking the target item in the instruction and asking the model to predict it.

Personalized ranking signals are incorporated by conditioning the instruction on user identifiers or recent interaction histories expressed in natural language.

Experimental Setup

Experiments are conducted on two FLAN‑T5 variants: FLAN‑T5‑Base and FLAN‑T5‑XL. Three Amazon domains are used: Toys & Games, Beauty, and Sports & Outdoors. The evaluation covers three tasks:

Item retrieval (recall@k).

Ranking (NDCG@k).

Rating prediction (RMSE).

Results

The fine‑tuned models consistently outperform both traditional baselines (e.g., matrix factorization, BPR) and existing LLM‑based approaches. The most pronounced gains are observed in the retrieval task, where the proposed method achieves higher recall across all three domains.

Conclusion

Generating natural‑language‑based auxiliary tasks effectively bridges the knowledge gap between LLMs and recommendation domains, enabling LLMs to acquire and apply recommendation‑specific expertise.

For full technical details, see the paper: https://arxiv.org/abs/2404.00245