Willem Röpke, Willem Röpke, Mathieu Reymond, Mathieu Reymond, Patrick Mannion, Patrick Mannion, Diederik M. Roijers, Diederik M. Roijers, Ann Nowe, Ann Nowe, Roxana Radulescu, Roxana Radulescu
An important challenge in multi-objective reinforcement learning is obtaining a Pareto front of policies to attain optimal performance under different preferences. We introduce Iterated Pareto Referent Optimisation (IPRO), which decomposes finding the Pareto front into a sequence of constrained single-objective problems. This enables us to guarantee convergence while providing an upper bound on the distance to undiscovered Pareto optimal solutions at each step. We evaluate IPRO using utility-based metrics and its hypervolume and find that it matches or outperforms methods that require additional assumptions. By leveraging problem-specific single-objective solvers, our approach also holds promise for applications beyond multi-objective reinforcement learning, such as planning and pathfinding.
Röpke, W, Reymond, M, Mannion, P, Roijers, DM, Nowe, A & Radulescu, R 2025, Divide and conquer: Provably unveiling the pareto front with multi-objective reinforcement learning. in Y Vorobeychik, S Das & A Nowe (eds), Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2025. Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), pp. 1774-1783, The 24th International Conference on Autonomous Agents and Multiagent Systems, Detroit, United States, 19/05/25. <https://arxiv.org/abs/2402.07182>
Röpke, W., Reymond, M., Mannion, P., Roijers, D. M., Nowe, A., & Radulescu, R. (2025). Divide and conquer: Provably unveiling the pareto front with multi-objective reinforcement learning. In Y. Vorobeychik, S. Das, & A. Nowe (Eds.), Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2025 (pp. 1774-1783). (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS). https://arxiv.org/abs/2402.07182
@inproceedings{16ac252a2a374d72b23f3177e31a3bf5,
title = "Divide and conquer: Provably unveiling the pareto front with multi-objective reinforcement learning",
abstract = "An important challenge in multi-objective reinforcement learning is obtaining a Pareto front of policies to attain optimal performance under different preferences. We introduce Iterated Pareto Referent Optimisation (IPRO), which decomposes finding the Pareto front into a sequence of constrained single-objective problems. This enables us to guarantee convergence while providing an upper bound on the distance to undiscovered Pareto optimal solutions at each step. We evaluate IPRO using utility-based metrics and its hypervolume and find that it matches or outperforms methods that require additional assumptions. By leveraging problem-specific single-objective solvers, our approach also holds promise for applications beyond multi-objective reinforcement learning, such as planning and pathfinding.",
author = "Willem R{\"o}pke and Mathieu Reymond and Patrick Mannion and Roijers, \{Diederik M.\} and Ann Nowe and Roxana Radulescu",
note = "Publisher Copyright: {\textcopyright} 2025 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org).; The 24th International Conference on Autonomous Agents and Multiagent Systems ; Conference date: 19-05-2025 Through 23-05-2025",
year = "2025",
language = "English",
series = "Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS",
publisher = "International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)",
pages = "1774--1783",
editor = "Yevgeniy Vorobeychik and Sanmay Das and Ann Nowe",
booktitle = "Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2025",
}