Applying Large Language Models to Recommendation Systems at Ant Group

The presentation introduces Ant Group's work on leveraging large language models (LLMs) for recommendation scenarios, aiming to reduce exposure and popularity bias by injecting world knowledge into the recommendation loop.

Background: Traditional recommendation pipelines suffer from bias and limited semantic understanding. Ant Group proposes a two‑stage approach where LLMs first generate structured or textual knowledge, which is then consumed by downstream recommendation models.

Knowledge Extraction: LLMs act as knowledge extractors, producing new entities and relations beyond existing annotations. The process involves determining target relation types, generating entities, and filtering relations using carefully designed prompts to ensure relevance and novelty.

Teacher‑Student Model: Large models (e.g., ChatGPT, GPT‑4) are distilled into smaller models (LLAMA‑2) to provide reasoning capabilities for online inference. A chain‑of‑thought (CoT) prompt generates recommendation rationales, which are then used to fine‑tune the student model.

Experimental Findings: Various backbones (GRU4Rec, SASRec, SRGNN) were evaluated. The LLM‑enhanced models showed improved long‑tail recommendation, reduced popularity bias, and better handling of sparse user data. Ranking‑based distillation and embedding alignment further boosted performance.

Challenges & Solutions: Key challenges include balancing computational cost, aligning semantic and collaborative signals, and ensuring reliable knowledge generation. Solutions involve two‑stage integration, prompt engineering, KG‑BERT ranking, and utility‑aware retrieval‑augmented generation (RAG).

Q&A Highlights: The distilled LLAMA‑2‑7B model is used for offline inference, with subsequent lightweight sequence models deployed online. Knowledge extraction helps alleviate data sparsity, and the framework supports both offline and online recommendation scenarios.

The authors invite collaborators interested in LLMs, knowledge graphs, and recommendation systems to join their efforts.