Pareto Set Learning for Multi-Objective Reinforcement Learning

Chao Qian; Chengrui Gao; Erlong Liu; Ke Xue; Ren-Jian Wang; Xiaobin Huang; Yu-Chang Wu

REVIEW 4 major objections 4 minor 1 cited by

Pareto Set Learning for Multi-Objective Reinforcement Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single hypernetwork generates a personalized policy for every preference weight, covering the full Pareto front of a multi-objective RL problem in one training run.

desk verdict A useful empirical MORL combination that overclaims a capacity theorem built on the wrong function class; worth a revision, not acceptance as is. read the letter →

arxiv 2501.06773 v2 pith:LITBYFNU submitted 2025-01-12 cs.LG

Erlong Liu , Yu-Chang Wu , Xiaobin Huang , Chengrui Gao , Ren-Jian Wang , Ke Xue , Chao Qian This is my paper · ORCID

classification cs.LG

keywords multi-objectivereinforcementlearningParetosethypernetworkfrontapproximationparameterfusionRademachercomplexitypreference-conditionedpoliciesdecomposition-basedMORL

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multi-objective reinforcement learning has long faced a trade-off between retraining a fresh policy for every preference weight and using one shared network that serves all preferences poorly. This paper tries to resolve it with PSL-MORL, a decomposition-based framework in which a single hypernetwork takes a preference weight vector as input and emits the parameters of a policy network tailored to that weight. The paper claims that one training run can thereby cover the entire Pareto front with dense, preference-specific policies, that the approach has provably larger model capacity than the most competitive baseline PD-MORL, and that each generated policy is optimal for its preference under linear scalarization. A reader should care because, if the claims are right, real-time preference switching and dense trade-off coverage would come from a single model that wraps any existing RL algorithm.

What carries the argument

The carrying object is the hypernetwork, a small MLP that takes the preference vector omega in R^m as input and outputs the complete parameter set of the main policy network, so that each preference receives its own policy in a single forward pass. The policy actually executed is the parameter-fused network $\theta$ = (1 - $\alpha$) theta_1 + $\alpha$ theta_2, where theta_1 is a shared policy network co-trained with the hypernetwork and $\alpha$ controls how much of the generated weights enter the final policy; this fusion is the mechanism that stabilizes hypernetwork training. The value-based instantiation augments the Q-update with a multi-dimensional preference interpolator and a cosine-similarity alignment term, and the two theoretical claims rest on Rademacher complexity bounds for neural networks and on the Banach fixed-point theorem applied to the preference-conditioned Bellman operator.

What would settle it

Compute the empirical Rademacher complexity of the trained PSL-MORL and PD-MORL function classes on the same benchmark data with their realized parameter norms; Theorem 2 predicts a ratio strictly greater than one, and a ratio at or below one would show the proof's norm-bound construction, not the architecture, caused the reported capacity gap. Alternatively, run PSL-MORL on a small tabular MOMDP and measure the maximum Bellman error of the preference-conditioned Q-update across iterations: if the error does not decrease monotonically, the contraction assumed by Theorem 1 fails for the parameter-fused update.

Watch

Extended reading notes

Core claim

PSL-MORL's central claim is that the Pareto front of a multi-objective RL problem can be approximated by training one hypernetwork phi to map each preference vector omega to the parameter vector theta_2 = $\varphi$(omega) of a policy network, instead of training one policy per weight or a single network for all weights. The executed policy fuses the generated parameters with a shared base policy via $\theta$ = (1 - $\alpha$) theta_1 + $\alpha$ theta_2, a parameter-fusion step that the ablations credit for stable training. On the theory side, Theorem 1 asserts that the preference-conditioned Q-learning update is a contraction on a complete metric space of value functions, so its unique fixed point is the optimal multi-objective value function and the generated policy is optimal for every preference; Theorem 2 asserts that the Rademacher complexity of the composed hypernetwork-policy function class is strictly larger than that of PD-MORL's class, giving PSL-MORL greater capacity to output distinct policies for distinct preferences. Empirically, the paper reports that PSL-MORL attains the best hypervolume and sparsity among seven methods on the MO-MuJoCo continuous-control tasks and the Fruit Tree Navigation discrete tasks.

Load-bearing premise

The capacity guarantee rests on assuming that the Rademacher complexity of the composed hypernetwork-plus-policy class is at least the product of the two classes' individual complexities and that the comparison may endow the hypernetwork with arbitrarily large norm bounds; the optimality guarantee separately assumes, rather than proves, that the parameter-fused preference-conditioned Q-update is a contraction.

Editorial extensions

If this is right

A single PSL-MORL training run yields a dense, near-continuous set of policies spanning the preference simplex, removing the need to retrain a policy for each newly requested trade-off.
Because the framework only changes how policy parameters are generated, any existing single-objective RL algorithm can be turned into a multi-objective method by wrapping it with the hypernetwork.
At deployment, an arbitrary preference vector can be converted into a policy instantly by one hypernetwork forward pass, enabling real-time preference switching with no fine-tuning.
Theorem 2 implies the PSL-MORL model class can express strictly more policy variation across preferences than PD-MORL, which the experiments associate with denser Pareto-front coverage and lower sparsity scores.
Theorem 1 implies that a converged PSL-MORL run is optimal for every preference it was trained over, so the Pareto front it traces is not just a feasible set but an optimal one.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The capacity argument compares function classes, not trained solutions; a natural test beyond the paper is whether the hypervolume advantage persists when PSL-MORL's total parameter budget (hypernetwork plus fused policy) is matched against PD-MORL's single network.
The reported grid search shows the best parameter-fusion coefficient alpha varies across environments, so an adaptive alpha scheduled during training is a plausible, testable improvement the paper leaves implicit.
Because the strongest baselines collapse to a single policy on the discrete benchmark, part of PSL-MORL's margin may come from its uniform preference sampling rather than the hypernetwork itself; an ablation that keeps the sampling but replaces the hypernetwork with a shared conditional network would isolate this.
The framework is stated for linear scalarization; extending the same hypernetwork to non-linear or learned utility functions, which the conclusion names as future work, would make it applicable to agents with non-linear risk or fairness preferences.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a reasonable empirical MORL paper with an unsound theoretical add-on. The hypernetwork-to-policy parameter mapping for preference-conditioned MORL is a natural combination of known ideas, and the parameter fusion trick seems to do real work based on the ablations. The experiments cover the standard MO-MuJoCo and FTN suites and the method looks competitive on hypervolume and sparsity.

The main soft spot is Theorem 2. The proof in Appendix B.3 treats PSL-MORL as a composition of two MLPs on the state x, but the hypernetwork in Algorithm 1 maps the preference ω to policy parameters; the actual policy is π_{ϕ(ω)}(x). So the Rademacher lower bound is computed on the wrong function class. On top of that, the proof sets M'_F(j) = max(||W'_j||_2, C) + 1, effectively choosing the norm bound after seeing the weights, which makes the ratio R_hat2/R_hat1 > 1 vacuous. That is a load-bearing flaw: the paper's only formal claim of capacity superiority rests on it. Theorem 1 is cited from Yang et al. and Basaklar et al. without showing the contraction holds for the hypernetwork-update rule used here.

The empirical reporting is thinner than I would like: means over six seeds with no error bars, and the FTN d=7 row has sparsity N/A in Table 3 while the text claims a 0.01 sparsity. That's at minimum a missing explanation and looks like an internal inconsistency.

On the plus side, the ablation study is informative and the parameter fusion idea is worth trying. The paper is positioned well in the literature and the writing is clear.

Who is it for: people building multi-policy MORL systems will get a useful technique. The theory needs to be either corrected into a real statement about the preference-to-parameter class, or removed. I'd send it to peer review because the empirical contribution is plausible and the flaws are identifiable and fixable, but the current form should not be accepted. My own verdict would be reject-and-resubmit.

Referee Report

4 major / 4 minor

Summary. The paper proposes PSL-MORL, a framework for multi-objective reinforcement learning that uses a hypernetwork to generate policy-network parameters for each preference vector, combined with a parameter-fusion technique and instantiations with DDQN and TD3. The authors claim three contributions: a general MORL framework that covers the whole preference space with personalized policies, theoretical guarantees of capacity superiority over PD-MORL (Theorem 2) and optimality of the generated policy (Theorem 1), and experimental superiority on MO-MuJoCo and Fruit Tree Navigation benchmarks in hypervolume and sparsity metrics.

Significance. If the theoretical claims were valid, PSL-MORL would be an appealing contribution: a single training run producing dense, preference-specific policies is practically valuable, and the framework's compatibility with any single-objective RL algorithm is a real strength. The parameter-fusion ablation is a useful empirical component. However, the paper's central theoretical guarantee is not established: the Rademacher-complexity proof analyzes a different function class, and the optimality theorem is cited rather than proved for the actual PSL-MORL update. The empirical evidence is also under-reported, with means only, no variance measures, and a concrete inconsistency in the FTN d=7 results. These issues undermine the paper's main claims in their current form.

major comments (4)

[Appendix B.3, Theorem 2] The proof models PSL-MORL as x ↦ (W'_d2 σ'_{d2-1}(...σ'_1(W'_1))) ∘ (W_{d1-1} σ_{d1-2}(...σ_1(W_1 x))), i.e., as a sequential composition of two MLPs on the state x. This is not the model in Algorithm 1: the hypernetwork maps the preference ω to policy parameters, so the policy class is {x ↦ π_{ϕ(ω)}(x) : ϕ ∈ Φ, ω ∈ Λ}, not a composition of two MLPs on the same input x. Consequently, the factorization R̂_2 = R̂_{2,1} · R̂_{2,2} is not a valid Rademacher-complexity lower bound for the PSL-MORL policy class. Furthermore, the proof sets M'_F(j) = max(||W'_j||_2, (γ²√n(√(2 log 2)d1+1)M_F(d1)/(c1 c2 B))^{1/d2}) + 1, which makes the norm bound depend on n, γ, and M_F(d1) and forces R̂_2/R̂_1 > 1 by construction rather than deriving it from the actual capacity of the models. Theorem 2 is therefore not established as written.
[Theoretical Analysis, Theorem 1] Theorem 1 states that (Q, d) is a complete metric space and that a contraction C with modulus γ has Q* as a unique fixed point, citing Yang et al. 2019 and Basaklar et al. 2023. However, the PSL-MORL update uses parameter fusion θ = (1−α)θ1 + αϕ(ω), a multi-dimensional interpolator I(ω), and a cosine-similarity term Sc in the target value; none of these components is shown to produce a contraction on Q with modulus γ for the proposed algorithm. The theorem is quoted for a different algorithm and does not, by itself, guarantee convergence of the hypernetwork-generated policies. The claim of 'optimality of the obtained policy network' is therefore unsupported.
[Table 3] The FTN d=7 row reports PSL-MORL sparsity as N/A. By the paper's own definition, sparsity is N/A only when the solution set contains a single solution. This is inconsistent with the abstract's claim of dense Pareto-front coverage and with the text's account that PSL-MORL obtains a 0.01 sparsity value. The authors should report the number of solutions found and explain the d=7 result, since a single-solution output at d=7 would contradict the dense-coverage claim.
[Tables 1–3, 7] All experimental tables report only mean values over six random seeds, with no standard deviations, confidence intervals, or significance tests. The paper repeatedly claims that PSL-MORL 'significantly outperforms' the baselines, but the data as presented do not support such a statistical claim; for example, Table 1 shows PD-MORL exceeding PSL-MORL on MO-Walker2d-v2 hypervolume (5.41 vs 5.36). Please provide variance measures and, where appropriate, paired significance tests for the hypervolume and sparsity comparisons.

minor comments (4)

[Algorithm 1, line 14] Line 14 says 'Update ϕ and θ1 by conducting a single-objective RL algorithm to maximize the scalarized return by weight ωi, i∈{1,2,...,N}', but N is the batch size of transitions, not the number of sampled weights; the notation and the update rule should be clarified.
[References] The TD3 instantiation in Section 4.3 is cited as [Dankwa and Zheng 2019], but the original TD3 algorithm is by Fujimoto et al. 2018, which is listed in the references but not cited at this location.
[Appendix A.6, Table 7] The grid search over α reports only means; for instance, on MO-Hopper-v2 the hypervolume at α=0.03 is 1.95×10^7 versus 1.77×10^7 at α=0.01, but without variance estimates it is unclear whether the chosen α is significantly better.
[Abstract and Conclusion] The abstract claims 'significantly outperforming' while the conclusion restricts the method to linear scalarization; given the missing error bars and the N/A sparsity at d=7, the strength of the performance claims should be moderated or supported with additional evidence.

Circularity Check

1 steps flagged · score 7.0 of 10

Theorem 2's capacity superiority is forced by the proof's choice of the hypernetwork norm bound M'_F(j), making the central theoretical guarantee true by construction.

self definitional [Appendix B.3, Proof of Theorem 2 (definition of M'_F(j))]
"Let M'_F(j) = max{ ||W'_j||_2, (γ²·√n(√(2 log 2)d1+1) M_F(d1))/(c1·c2·B) )^{1/d2} } + 1, which is an obvious upper bound on ||Wj||2. As M'_F(j) > (γ²·√n(√(2 log 2)d1+1) M_F(d1))/(c1·c2·B) )^{1/d2}, we can come to the conclusion that R_hat2/R_hat1 > 1, and complete the proof."

The threshold inside the max is exactly the value that makes the preceding lower bound R_hat2/R_hat1 ≥ c1·c2·B·(∏_j M'_F(j))/(γ²·√n(√(2 log 2)d1+1)·M_F(d1)) exceed 1. Since M'_F(j) is a free norm bound, not determined by PSL-MORL's architecture or trained weights, the proof can set it arbitrarily large; the conclusion R_hat2/R_hat1 > 1 is then guaranteed by this definition rather than by any capacity property of PSL-MORL relative to PD-MORL. The same argument would 'prove' capacity superiority for any MLP whose declared norm bound is chosen above the threshold, so the theorem is equivalent to the choice of M'_F by construction.

full rationale

The paper's advertised theoretical guarantee of model-capacity superiority (Theorem 2) reduces by construction. In Appendix B.3, after deriving a lower bound for the hypernetwork's Rademacher complexity, the proof defines M'_F(j) to be larger than the exact threshold needed to make R_hat2/R_hat1 > 1. Because this is a free norm bound rather than a quantity fixed by the PSL-MORL architecture, the proof does not compare the actual capacity of PSL-MORL with PD-MORL; it merely shows that a sufficiently large declared norm bound forces the desired inequality. That is a self-definitional step: the conclusion is built into the definition of M'_F(j). The proof also models PSL-MORL as x ↦ (hypernetwork MLP) ∘ (policy MLP) on the state, whereas in Algorithm 1 the hypernetwork takes the preference vector as input and outputs policy parameters, so the Rademacher lower bound is for the wrong function class. The optimality guarantee (Theorem 1) is quoted from prior work and asserted to apply to the hypernetwork with parameter fusion, but the paper does not show that the PSL-MORL update is a contraction; this is an unsupported-citation gap rather than a circular reduction, and the cited works are not by the present authors. The empirical benchmark comparisons are independent and would support the experimental claims, but the central theoretical result is forced. Score 7 reflects one load-bearing circular/self-definitional step in the core theoretical contribution, with independent empirical content elsewhere.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central empirical result depends mainly on the tuned fusion coefficient alpha and standard RL assumptions. The theoretical capacity claim depends on an unproved factorization of Rademacher complexity and on norm bounds chosen to force the result. No new physical or formal entities are introduced.

free parameters (2)

Parameter fusion coefficient alpha = 0.01 to 0.10 per benchmark (Tables 4 and 5, Appendix A.6)
Chosen by grid search on the test benchmarks to maximize hypervolume; it controls how much of the final policy comes from the hypernetwork and is central to the reported performance.
Hypernetwork Frobenius norm bounds M'_F(j) = max(||W'_j||_2, (...)^(1/d2)) + 1
Defined in Appendix B.3 precisely to make the Rademacher complexity ratio greater than 1; the capacity result is forced by this choice.

assumptions (5)

standard math The activation function sigma is 1-Lipschitz, positive homogeneous, and element-wise, with sigma(0) = 0.
Used in Lemma 1 and Theorem 2; standard for ReLU networks.
standard math The policy network class and hypernetwork class have parameter matrices with bounded Schatten norms, with input norms bounded by B and weight vectors bounded by M.
These are the premises of the Rademacher bounds in Lemmas 1 and 2.
ad hoc to paper The Rademacher complexity of the composed hypernetwork-policy function class factors as the product of the individual complexities.
Assumed in Appendix B.3 when R_hat2 = R_hat2,1 * R_hat2,2. The paper provides no proof, and composition of function classes does not generally factor this way.
domain assumption The preference-conditioned Q-learning update with hypernetwork-generated parameters is a contraction on a complete metric space.
Theorem 1 is cited from prior work and requires this contraction property, but the paper does not show that its parameter-fusion and interpolator updates satisfy it.
domain assumption Only linear scalarization u = omega^T J is considered.
Stated as a limitation in the conclusion; the method and theory do not cover nonlinear utilities.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Pareto Set Learning for Multi-Objective Reinforcement Learning." pith.science (2026). https://pith.science/paper/LITBYFNU

@misc{pith2026250106773,
  author       = {Pith},
  title        = {Pith review of: Pareto Set Learning for Multi-Objective Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LITBYFNU}},
  note         = {Machine review of arXiv:2501.06773}
}

read the original abstract

Multi-objective decision-making problems have emerged in numerous real-world scenarios, such as video games, navigation and robotics. Considering the clear advantages of Reinforcement Learning (RL) in optimizing decision-making processes, researchers have delved into the development of Multi-Objective RL (MORL) methods for solving multi-objective decision problems. However, previous methods either cannot obtain the entire Pareto front, or employ only a single policy network for all the preferences over multiple objectives, which may not produce personalized solutions for each preference. To address these limitations, we propose a novel decomposition-based framework for MORL, Pareto Set Learning for MORL (PSL-MORL), that harnesses the generation capability of hypernetwork to produce the parameters of the policy network for each decomposition weight, generating relatively distinct policies for various scalarized subproblems with high efficiency. PSL-MORL is a general framework, which is compatible for any RL algorithm. The theoretical result guarantees the superiority of the model capacity of PSL-MORL and the optimality of the obtained policy network. Through extensive experiments on diverse benchmarks, we demonstrate the effectiveness of PSL-MORL in achieving dense coverage of the Pareto front, significantly outperforming state-of-the-art MORL methods in the hypervolume and sparsity indicators.

Figures

Figures reproduced from arXiv: 2501.06773 by the authors.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimization
cs.LG 2026-02 conditional novelty 6.0 of 10

D3PO learns a single preference-conditioned policy via per-objective PPO losses, late preference weighting, and a preference-distance-scaled diversity regularizer, reporting improved Pareto fronts on most tested MORL ...

Reference graph

Works this paper leans on

56 extracted references · 48 canonical work pages · cited by 1 Pith paper

[1]

Abdolmaleki, A.; Huang, S.; Hasenclever, L.; Neunert, M.; Song, F.; Zambelli, M.; Martins, M.; Heess, N.; Hadsell, R.; and Riedmiller, M. 2020. A distributional view on multi-objective policy optimization. In Proceedings of the 37th International Conference on Machine Learning, 11--22. Vienna, Austria

work page 2020
[2]

M.; Lenaerts, T.; Nowé, A.; and Steckelmacher, D

Abels, A.; Roijers, D. M.; Lenaerts, T.; Nowé, A.; and Steckelmacher, D. 2019. Dynamic weights in multi-objective deep reinforcement learning. In Proceedings of the 36th International Conference on Machine Learning, 11--20. Long Beach, CA

work page 2019
[3]

Amodei, D.; Olah, C.; Steinhardt, J.; Christiano, P.; Schulman, J.; and Man \'e , D. 2016. Concrete problems in A I safety. arXiv:1606.06565

arXiv 2016
[4]

Andrychowicz, M.; Wolski, F.; Ray, A.; Schneider, J.; Fong, R.; Welinder, P.; McGrew, B.; Tobin, J.; Pieter Abbeel, O.; and Zaremba, W. 2017. Hindsight experience replay. In Advances in Neural Information Processing Systems 30, 5055–5065. Long Beach, CA

work page 2017
[5]

B.; and Trabelsi, W

Azzouz, R.; Bechikh, S.; Said, L. B.; and Trabelsi, W. 2018. Handling time-varying constraints and objectives in dynamic evolutionary multi-objective optimization. Swarm and Evolutionary Computation, 39: 222--248

work page 2018
[6]

Basaklar, T.; Gumussoy, S.; and Ogras, U. 2023. PD - MORL : Preference-driven multi-objective reinforcement learning algorithm. In Proceedings of the 11th International Conference on Learning Representations. Kigali, Rwanda

work page 2023
[7]

Chen, X.; Ghadirzadeh, A.; Bj \"o rkman, M.; and Jensfelt, P. 2019. Meta-learning for multi-objective reinforcement learning. In Proceedings of the 2019 International Conference on Intelligent Robots and Systems, 977--983. Macau, China

work page 2019
[8]

Dankwa, S.; and Zheng, W. 2019. Twin-delayed DDPG : A deep reinforcement learning technique to model a continuous movement of an intelligent robot agent. In Proceedings of the 3rd International Conference on Vision, Image and Signal Processing, 1--5. Vancouver, Canada

work page 2019

Show all 56 references

[9]

Daskalakis, C.; Tzamos, C.; and Zampetakis, M. 2018. A converse to Banach's fixed point theorem and its CLS-completeness. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, Los Angeles, CA , 44--50

2018
[10]

Fujimoto, S.; Hoof, H.; and Meger, D. 2018. Addressing function approximation error in actor-critic methods. In Proceedings of the 35th International Conference on Machine Learning, 1587--1596. Stockholm, Sweden

2018
[11]

Golowich, N.; Rakhlin, A.; and Shamir, O. 2018. Size-independent sample complexity of neural networks. In Proceedings of the 31st Conference on Learning Theory, 297--299. Stockholm, Sweden

2018
[12]

D., David; and Le, Q

Ha, A. D., David; and Le, Q. V. 2017. HyperNetworks. In Proceedings of the 5th International Conference on Learning Representations. Toulon, France

2017
[13]

a llstr \

Hayes, C. F.; R a dulescu, R.; Bargiacchi, E.; K \"a llstr \"o m, J.; Macfarlane, M.; Reymond, M.; Verstraeten, T.; Zintgraf, L. M.; Dazeley, R.; Heintz, F.; et al. 2022. A practical guide to multi-objective reinforcement learning and planning. Autonomous Agents and Multi-Agen...

2022
[14]

Hong, W.; Tang, K.; Zhou, A.; Ishibuchi, H.; and Yao, X. 2019. A scalable indicator-based evolutionary algorithm for large-scale multiobjective optimization. IEEE Transactions on Evolutionary Computation, 23(3): 525--537

2019
[15]

C.; Hern \'a ndez-Garc a, A.; Rector-Brooks, J.; Bengio, Y.; Miret, S.; and Bengio, E

Jain, M.; Raparthy, S. C.; Hern \'a ndez-Garc a, A.; Rector-Brooks, J.; Bengio, Y.; Miret, S.; and Bengio, E. 2023. Multi-objective gflownets. In Proceedings of the 40th International Conference on Machine Learning, 14631--14653. Honolulu, HI

2023
[16]

Lin, X.; Yang, Z.; and Zhang, Q. 2022. Pareto set learning for neural multi-objective combinatorial optimization. In Proceedings of the 10th International Conference on Learning Representations. Virtual

2022
[17]

Lin, X.; Yang, Z.; Zhang, Q.; and Kwong, S. 2020. Controllable P areto multi-task learning. arXiv:2010.06313

2020 arXiv
[18]

Lin, X.; Yang, Z.; Zhang, X.; and Zhang, Q. 2022. Pareto set learning for expensive multi-objective optimization. In Advances in Neural Information Processing Systems 35. New Orleans, LA

2022
[19]

Liu, F.; and Qian, C. 2021. Prediction guided meta-learning for multi-objective reinforcement learning. In Proceedings of the 2021 IEEE Congress on Evolutionary Computation , 2171--2178. Krak \' o w, Poland

2021
[20]

Lu, Y.; Li, B.; and Zhou, A. 2024. Are you concerned about limited function evaluations: Data-augmented Pareto set learning for expensive multi-objective optimization. In Proceedings of the 38th AAAI Conference on Artificial Intelligence, 14202--14210. Vancouver, Canada

2024
[21]

Mannor, S.; and Shimkin, N. 2001. The steering approach for multi-criteria reinforcement learning. In Advances in Neural Information Processing Systems 14, 1563 -- 1570. Vancouver, Canada

2001
[22]

A.; Veness, J.; Bellemare, M

Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M.; Fidjeland, A. K.; Ostrovski, G.; et al. 2015. Human-level control through deep reinforcement learning. Nature, 518(7540): 529--533

2015
[23]

M.; Roijers, D

Mossalam, H.; Assael, Y. M.; Roijers, D. M.; and Whiteson, S. 2016. Multi-objective deep reinforcement learning. arXiv:1610.02707

2016 arXiv
[24]

Navon, A.; Shamsian, A.; Fetaya, E.; and Chechik, G. 2021. Learning the Pareto front with hypernetworks. In Proceedings of the 9th International Conference on Learning Representations. Vienna, Austria

2021
[25]

Neyshabur, B.; Li, Z.; Bhojanapalli, S.; LeCun, Y.; and Srebro, N. 2019. The role of over-parametrization in generalization of neural networks. In Proceddings of the 7th International Conference on Learning Representations. New Orleans, LA

2019
[26]

Ortiz, J. J. G.; Guttag, J.; and Dalca, A. V. 2024. Magnitude invariant parametrizations improve hypernetwork learning. In Proceedings of the 12th International Conference on Learning Representations. Vienna, Austria

2024
[27]

Parisi, S.; Pirotta, M.; and Peters, J. 2017. Manifold-based multi-objective policy search with sample reuse. Neurocomputing, 263: 3--14

2017
[28]

Parisi, S.; Pirotta, M.; Smacchia, N.; Bascetta, L.; and Restelli, M. 2014. Policy gradient approaches for multi-objective sequential decision making. In Proceedings of the 2014 International Joint Conference on Neural Networks, 2323--2330. Beijing, China

2014
[29]

Pirotta, M.; Parisi, S.; and Restelli, M. 2015. Multi-objective reinforcement learning with continuous P areto frontier approximation. In Proceedings of the 29th AAAI Conference on Artificial Intelligence, 2928--2934. Austin, TX

2015
[30]

Qian, C.; Shi, J.-C.; Yu, Y.; Tang, K.; and Zhou, Z.-H. 2017. Subset selection under noise. In Advances in Neural Information Processing Systems 30, 3563--3573. Long Beach, CA

2017
[31]

Qian, C.; Yu, Y.; Tang, K.; Yao, X.; and Zhou, Z.-H. 2019. Maximizing submodular or monotone approximately submodular functions by multi-objective evolutionary algorithms. Artificial Intelligence, 275: 279--294

2019
[32]

Qian, C.; Yu, Y.; and Zhou, Z.-H. 2013. An analysis on recombination in multi-objective evolutionary optimization. Artificial Intelligence, 204: 99--119

2013
[33]

Qian, C.; Yu, Y.; and Zhou, Z.-H. 2015 a . On constrained B oolean P areto optimization. In Proceedings of the 24th International Joint Conference on Artificial Intelligence, 389--395. Buenos Aires, Argentina

2015
[34]

Qian, C.; Yu, Y.; and Zhou, Z.-H. 2015 b . Pareto ensemble pruning. In Proceedings of the 29th AAAI Conference on Artificial Intelligence, 2935--2941. Austin, TX

2015
[35]

Qian, C.; Yu, Y.; and Zhou, Z.-H. 2015 c . Subset selection by Pareto optimization. In Advances in Neural Information Processing Systems 28, 1765--1773. Montreal, Canada

2015
[36]

Qian, H.; and Yu, Y. 2021. Derivative-free reinforcement learning: A review. Frontiers of Computer Science, 15(6): 156336

2021
[37]

M.; Vamplew, P.; Whiteson, S.; and Dazeley, R

Roijers, D. M.; Vamplew, P.; Whiteson, S.; and Dazeley, R. 2013. A survey of multi-objective sequential decision-making. Journal of Artificial Intelligence Research, 48: 67--113

2013
[38]

M.; Whiteson, S.; Oliehoek, F

Roijers, D. M.; Whiteson, S.; Oliehoek, F. A.; et al. 2014. Linear support for multi-objective coordination graphs. In Proceedings of the 2014 International Conference on Autonomous Agents & Multiagent Systems, 1297--1304. Paris, France

2014
[39]

Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal policy optimization algorithms. arXiv:1707.06347

2017 arXiv
[40]

Shalev - Shwartz, S.; and Ben - David, S. 2014. Understanding Machine Learning - From Theory to Algorithms. Cambridge University Press

2014
[41]

S.; and Barto, A

Sutton, R. S.; and Barto, A. G. 2018. Reinforcement Learning: An Introduction. MIT press

2018
[42]

Todorov, E.; Erez, T.; and Tassa, Y. 2012. Mujoco: A physics engine for model-based control. In Proceedings of the 2012 International Conference on Intelligent Robots and Systems, 5026--5033. Algarve, Portugal

2012
[43]

Van Hasselt, H.; Guez, A.; and Silver, D. 2016. Deep reinforcement learning with double Q-learning. In Proceedings of the 30th AAAI Conference on Artificial Intelligence, 2094--2100. Phoenix, AZ

2016
[44]

M.; and Now \'e , A

Van Moffaert, K.; Drugan, M. M.; and Now \'e , A. 2013. Scalarized multi-objective reinforcement learning: Novel design techniques. In Proceedings of the 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, 191--199. Singapore

2013
[45]

Vapnik, V. 2000. The Nature of Statistical Learning Theory. Springer

2000
[46]

Xu, J.; Tian, Y.; Ma, P.; Rus, D.; Sueda, S.; and Matusik, W. 2020. Prediction-guided multi-objective reinforcement learning for continuous robot control. In Proceedings of the 37th International Conference on Machine Learning, 10607--10616. Vienna, Austria

2020
[47]

Xue, K.; Xu, J.; Yuan, L.; Li, M.; Qian, C.; Zhang, Z.; and Yu, Y. 2022. Multi-agent dynamic algorithm configuration. In Advances in Neural Information Processing Systems 35. New Orleans, LA

2022
[48]

Yang, P.; Zhang, L.; Liu, H.; and Li, G. 2024. Reducing idleness in financial cloud services via multi-objective evolutionary reinforcement learning based load balancer. Science China Information Sciences, 67(2): 120102

2024
[49]

Yang, R.; Sun, X.; and Narasimhan, K. 2019. A generalized algorithm for multi-objective reinforcement learning and policy adaptation. In Advances in Neural Information Processing Systems 32, 14636--14647. Vancouver, Canada

2019
[50]

Zhang, Q.; and Li, H. 2007. MOEA/D : A multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 11(6): 712--731

2007
[51]

Zhang, X.; Lin, X.; Xue, B.; Chen, Y.; and Zhang, Q. 2023. Hypervolume maximization: A geometric view of pareto set learning. In Advances in Neural Information Processing Systems 36. New Orleans, LA

2023
[52]

Zhou, Z.-H.; Yu, Y.; and Qian, C. 2019. Evolutionary Learning: Advances in Theories and Algorithms. Singapore: Springer

2019
[53]

T.; and Gibson, B

Zhu, X.; Rogers, T. T.; and Gibson, B. R. 2009. Human rademacher complexity. In Advances in Neural Information Processing Systems 23, 2322--2330. Vancouver, British Columbia

2009
[54]

Zuluaga, M.; Krause, A.; and P \"u schel, M. 2016. -pal: An active learning approach to the multi-objective optimization problem. Journal of Machine Learning Research, 17(1): 3619--3650

2016
[55]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
[56]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Abdolmaleki, A.; Huang, S.; Hasenclever, L.; Neunert, M.; Song, F.; Zambelli, M.; Martins, M.; Heess, N.; Hadsell, R.; and Riedmiller, M. 2020. A distributional view on multi-objective policy optimization. In Proceedings of the 37th International Conference on Machine Learning, 11--22. Vienna, Austria

work page 2020

[2] [2]

M.; Lenaerts, T.; Nowé, A.; and Steckelmacher, D

Abels, A.; Roijers, D. M.; Lenaerts, T.; Nowé, A.; and Steckelmacher, D. 2019. Dynamic weights in multi-objective deep reinforcement learning. In Proceedings of the 36th International Conference on Machine Learning, 11--20. Long Beach, CA

work page 2019

[3] [3]

Amodei, D.; Olah, C.; Steinhardt, J.; Christiano, P.; Schulman, J.; and Man \'e , D. 2016. Concrete problems in A I safety. arXiv:1606.06565

arXiv 2016

[4] [4]

Andrychowicz, M.; Wolski, F.; Ray, A.; Schneider, J.; Fong, R.; Welinder, P.; McGrew, B.; Tobin, J.; Pieter Abbeel, O.; and Zaremba, W. 2017. Hindsight experience replay. In Advances in Neural Information Processing Systems 30, 5055–5065. Long Beach, CA

work page 2017

[5] [5]

B.; and Trabelsi, W

Azzouz, R.; Bechikh, S.; Said, L. B.; and Trabelsi, W. 2018. Handling time-varying constraints and objectives in dynamic evolutionary multi-objective optimization. Swarm and Evolutionary Computation, 39: 222--248

work page 2018

[6] [6]

Basaklar, T.; Gumussoy, S.; and Ogras, U. 2023. PD - MORL : Preference-driven multi-objective reinforcement learning algorithm. In Proceedings of the 11th International Conference on Learning Representations. Kigali, Rwanda

work page 2023

[7] [7]

Chen, X.; Ghadirzadeh, A.; Bj \"o rkman, M.; and Jensfelt, P. 2019. Meta-learning for multi-objective reinforcement learning. In Proceedings of the 2019 International Conference on Intelligent Robots and Systems, 977--983. Macau, China

work page 2019

[8] [8]

Dankwa, S.; and Zheng, W. 2019. Twin-delayed DDPG : A deep reinforcement learning technique to model a continuous movement of an intelligent robot agent. In Proceedings of the 3rd International Conference on Vision, Image and Signal Processing, 1--5. Vancouver, Canada

work page 2019