In this paper, we investigate the effects of opponent modelling on multi-objective multi-agent interactions with non-linear utilities. Specifically, we consider multi-objective normal form games (MONFGs) with non-linear utility functions under the scalarised expected returns optimisation criterion. We contribute a novel actor-critic formulation to allow reinforcement learning of mixed strategies in this setting, along with an extension that incorporates opponent policy reconstruction using conditional action frequencies. Our empirical results demonstrate that opponent modelling can drastically alter the learning dynamics in this setting.
Zhang, Y, Radulescu, R, Mannion, P, Roijers, D & Nowe, A 2020, Opponent Modelling for Reinforcement Learning in Multi-Objective Normal Form Games: Extended Abstract. in B An, A El Fallah Seghrouchni & G Sukthankar (eds), Proceedings of the 19th International Conference on Autonomous Agents and Multi-Agent Systems, AAMAS 2020. Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 2020-May, IFAAMAS, pp. 2080-2082, The 19th International Conference on Autonomous Agents and Multi-Agent Systems, Auckland, New Zealand, 9/05/20. <http://www.ifaamas.org/Proceedings/aamas2020/pdfs/p2080.pdf>
Zhang, Y., Radulescu, R., Mannion, P., Roijers, D., & Nowe, A. (2020). Opponent Modelling for Reinforcement Learning in Multi-Objective Normal Form Games: Extended Abstract. In B. An, A. El Fallah Seghrouchni, & G. Sukthankar (Eds.), Proceedings of the 19th International Conference on Autonomous Agents and Multi-Agent Systems, AAMAS 2020 (pp. 2080-2082). (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS; Vol. 2020-May). IFAAMAS. http://www.ifaamas.org/Proceedings/aamas2020/pdfs/p2080.pdf
@inproceedings{d48afc6d1cfe4b709be0396e77c344a1,
title = "Opponent Modelling for Reinforcement Learning in Multi-Objective Normal Form Games: Extended Abstract",
abstract = "In this paper, we investigate the effects of opponent modelling on multi-objective multi-agent interactions with non-linear utilities. Specifically, we consider multi-objective normal form games (MONFGs) with non-linear utility functions under the scalarised expected returns optimisation criterion. We contribute a novel actor-critic formulation to allow reinforcement learning of mixed strategies in this setting, along with an extension that incorporates opponent policy reconstruction using conditional action frequencies. Our empirical results demonstrate that opponent modelling can drastically alter the learning dynamics in this setting.",
author = "Yijie Zhang and Roxana Radulescu and Patrick Mannion and Diederik Roijers and Ann Nowe",
year = "2020",
language = "English",
series = "Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS",
publisher = "IFAAMAS",
pages = "2080--2082",
editor = "Bo An and {El Fallah Seghrouchni}, Amal and Gita Sukthankar",
booktitle = "Proceedings of the 19th International Conference on Autonomous Agents and Multi-Agent Systems, AAMAS 2020",
note = "The 19th International Conference on Autonomous Agents and Multi-Agent Systems, AAMAS 2020 ; Conference date: 09-05-2020 Through 13-05-2020",
url = "https://aamas2020.conference.auckland.ac.nz/, https://aamas2020.conference.auckland.ac.nz",
}