In many settings, as for example wind farms, multiple machines are instantiated to perform the same task, which is called a fleet. The recent advances with respect to the Internet of Things allow control devices and/or machines to connect through cloud-based architectures in order to share information about their status and environment. Such an infrastructure allows seamless data sharing between fleet members, which could greatly improve the sample-efficiency of reinforcement learning techniques. However in practice, these machines, while almost identical in design, have small discrepancies due to production errors or degradation, preventing control algorithms to simply aggregate and employ all fleet data. We propose a novel reinforcement learning method that learns to transfer knowledge between similar fleet members and creates member-specific dynamical models for control. Our algorithm uses Gaussian processes to establish cross-member covariances. This is significantly different from standard transfer learning methods, as the focus is not on sharing information over tasks, but rather over system specifications. We demonstrate our approach on two benchmarks and a realistic wind farm setting. Our method significantly outperforms two baseline approaches, namely individual learning and joint learning where all samples are aggregated, in terms of the median and variance of the results.
Verstraeten, T, Libin, P & Nowe, A 2020, Fleet Control using Coregionalized Gaussian Process Policy Iteration. in G De Giacomo, A Catala, B Dilkina, M Milano, S Barro, A Bugarin & J Lang (eds), Proceedings of the 24th European Conference on Artificial Intelligence (ECAI 2020). vol. 325, Frontiers in Artificial Intelligence and Applications, IOS Press, pp. 1571-1578, European Conference on Artificial Intelligence (ECAI 2020), Santiago De Compostela, Spain, 29/08/20. https://doi.org/10.3233/FAIA200266
Verstraeten, T., Libin, P., & Nowe, A. (2020). Fleet Control using Coregionalized Gaussian Process Policy Iteration. In G. De Giacomo, A. Catala, B. Dilkina, M. Milano, S. Barro, A. Bugarin, & J. Lang (Eds.), Proceedings of the 24th European Conference on Artificial Intelligence (ECAI 2020) (Vol. 325, pp. 1571-1578). (Frontiers in Artificial Intelligence and Applications). IOS Press. https://doi.org/10.3233/FAIA200266
@inproceedings{c15553f3ac2c48fcb56fb7d43ca23778,
title = "Fleet Control using Coregionalized Gaussian Process Policy Iteration",
abstract = "In many settings, as for example wind farms, multiple machines are instantiated to perform the same task, which is called a fleet. The recent advances with respect to the Internet of Things allow control devices and/or machines to connect through cloud-based architectures in order to share information about their status and environment. Such an infrastructure allows seamless data sharing between fleet members, which could greatly improve the sample-efficiency of reinforcement learning techniques. However in practice, these machines, while almost identical in design, have small discrepancies due to production errors or degradation, preventing control algorithms to simply aggregate and employ all fleet data. We propose a novel reinforcement learning method that learns to transfer knowledge between similar fleet members and creates member-specific dynamical models for control. Our algorithm uses Gaussian processes to establish cross-member covariances. This is significantly different from standard transfer learning methods, as the focus is not on sharing information over tasks, but rather over system specifications. We demonstrate our approach on two benchmarks and a realistic wind farm setting. Our method significantly outperforms two baseline approaches, namely individual learning and joint learning where all samples are aggregated, in terms of the median and variance of the results.",
author = "Timothy Verstraeten and Pieter Libin and Ann Nowe",
year = "2020",
month = aug,
day = "24",
doi = "10.3233/FAIA200266",
language = "English",
isbn = "978-1-64368-100-9",
volume = "325",
series = "Frontiers in Artificial Intelligence and Applications",
publisher = "IOS Press",
pages = "1571--1578",
editor = "{De Giacomo}, Giuseppe and Alejandro Catala and Bistra Dilkina and Michela Milano and Senen Barro and Alberto Bugarin and Jerome Lang",
booktitle = "Proceedings of the 24th European Conference on Artificial Intelligence (ECAI 2020)",
address = "Netherlands",
note = "European Conference on Artificial Intelligence (ECAI 2020), ECAI ; Conference date: 29-08-2020 Through 02-09-2020",
url = "http://ecai2020.eu/",
}