Gradient-Based Multi-Objective Deep Learning: Theory, Algorithms, and LLM Applications

Overview

Gradient‑based multi‑objective optimization (MOO) for deep learning aims to optimize several criteria (e.g., performance, safety, efficiency) simultaneously. The tutorial first presents the theoretical foundation of gradient‑based MOO and then classifies three principal solution families:

Compute a single balanced solution that aggregates objectives (for example, weighted‑sum or scalarization).

Generate a discrete set of Pareto‑optimal solutions that approximate the Pareto front.

Learn a continuous Pareto manifold so that any trade‑off can be queried at inference time.

For each family the tutorial details the underlying algorithms (such as MGDA, PCGrad, Pareto MTL, and continuation‑based methods), discusses convergence conditions (e.g., Lipschitz continuity, convexity assumptions), and reviews known generalization results for multi‑task settings.

Application to Large Language Models

The second part shows how gradient‑based MOO can be incorporated into the fine‑tuning and alignment pipeline of large language models. By treating performance, safety, and computational cost as separate objectives, the methods provide a principled way to navigate the inherent trade‑offs. The tutorial demonstrates recent state‑of‑the‑art techniques, including Pareto‑front construction during RL‑HF, multi‑objective reward weighting, and continuation‑based alignment, and highlights practical considerations such as gradient scaling, stability, and evaluation metrics.

For the full tutorial materials and additional references, see https://gradnexus.github.io/IJCAI25_tutorial/.

Code example

来源：专知
本文
约1000字
，建议阅读
5
分钟
本教程基于我们的综述论文
《基于梯度的多目标深度学习：算法、理论、应用及未来展望
》
。