Three Representative Ant Group Papers at NeurIPS 2024

NeurIPS 2024 will be held in Vancouver, Canada from December 10‑15, and Ant Group has 20 papers accepted. This article introduces three representative papers.

AMOR: A Recipe for Building Adaptable Modular Knowledge Agents Through Process Feedback proposes a finite‑state‑machine‑based knowledge agent that integrates factual and domain knowledge, interacts with external knowledge bases, and allows users to give feedback to modules. A two‑stage fine‑tuning strategy first trains a base language model on public data, then adapts it with process feedback, achieving superior knowledge correctness across multiple domains.

Rethinking Memory and Communication Costs for Efficient Data Parallel Training of Large Language Models addresses the challenges of distributed LLM training. The authors introduce the Partial Redundant Optimizer (PaRO) with finer‑grained sharding and PaRO‑CC for optimized collective communication, providing a quantitative guide for selecting data‑parallel strategies. Experiments show PaRO can speed up training by up to 266 % and PaRO‑CC improves model‑parallel speed by 17 %.

LLMDFA: Analyzing Dataflow in Code with Large Language Models leverages LLMs’ semantic understanding to perform data‑flow analysis without relying on compiler‑generated intermediate representations. By designing prompts and decomposing analysis tasks, LLMDFA detects unreachable source‑sink flows, supporting vulnerability detection such as null‑pointer dereference and division‑by‑zero. It integrates parsers and SMT solvers via LLM‑generated code, achieving accuracy and recall comparable or superior to traditional techniques.

To deepen understanding, the authors of the three papers will host a live session on December 12, 2024, 12:00‑14:00, streamed on WeChat Video, AntTech, and Bilibili. Attendees can interact with the presenters: Guo Jian (Associate Researcher, Ant Tech Institute), Zhang Hanxiao (Senior Development Engineer, Ant Group), and Wang Chengpeng (Postdoctoral Researcher, Purdue University).