REVIEW 5 cited by
Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Many modern deep learning applications require balancing multiple objectives that are often conflicting. Examples include multi-task learning, fairness-aware learning, and the alignment of Large Language Models (LLMs). This leads to multi-objective deep learning, which tries to find optimal trade-offs or Pareto-optimal solutions by adapting mathematical principles from the field of Multi-Objective Optimization (MOO). However, directly applying gradient-based MOO techniques to deep neural networks presents unique challenges, including high computational costs, optimization instability, and the difficulty of effectively incorporating user preferences. This paper provides a comprehensive survey of gradient-based techniques for multi-objective deep learning. We systematically categorize existing algorithms based on their outputs: (i) methods that find a single, well-balanced solution, (ii) methods that generate a finite set of diverse Pareto-optimal solutions, and (iii) methods that learn a continuous Pareto set of solutions. In addition to this taxonomy, the survey covers theoretical analyses, key applications, practical resources, and highlights open challenges and promising directions for future research. A comprehensive list of multi-objective deep learning algorithms is available at https://github.com/Baijiong-Lin/Awesome-Multi-Objective-Deep-Learning.
Forward citations
Cited by 5 Pith papers
-
Improved Convergence Rate for Stochastic Multi-Gradient Descent: A Proof Discovered with AI
Vanilla stochastic multi-gradient descent achieves Õ(T^{-1}) squared Pareto-stationarity under linearly growing mini-batches, improving the prior Õ(T^{-1/4}) bound.
-
ML-Assisted Bulk Resource Allocation: Custom Outage-Based Loss Function and Reliability Analysis
A new ranking-aware loss for ML-based bulk resource allocation cuts bulk outage probability by roughly 15-40% versus pointwise losses in simulated 6G-style settings.
-
Non-conflicting Energy Minimization in Reinforcement Learning based Robot Control
PEGrad projects energy-minimization gradients orthogonal to task-reward gradients in RL, achieving 64% torque reduction in simulation and reduced battery draw on a Unitree Go2 without sacrificing task reward.
-
Identifying Offline Metrics that Predict Online Impact: A Pragmatic Strategy for Real-World Recommender Systems
A Pareto-front-conditioned single recommender model serves multiple online test groups; the paper reports significant offline-to-online alignments, but the significance analysis treats a five-group covariate as though...
-
SAMO: A Lightweight Sharpness-Aware Approach for Multi-Task Optimization with Joint Global-Local Perturbation
SAMO jointly uses global and local perturbations with forward-only task gradient approximation to improve multi-task learning performance at lower cost than F-MTL.
Discussion (0). Sign in to comment.