Sample-efficiency is crucial in reinforcement learning tasks, especially when a large number of similar yet distinct tasks have to be learned. For example, consider a smart wheelchair learning to exit many differently-furnished offices on a building floor. Sequentially learning each of these tasks from scratch would be highly inefficient. A step towards a satisfying solution is the use of transfer learning: exploiting the knowledge acquired in previous (or source) tasks to tackle new (or target) tasks. Existing work mainly focuses on exploiting only one source policy as an advisor for the fresh agent, even when there are several expert source policies available. However, using only one advisor requires artificial mechanisms to limit its influence in areas where the source task and the target task differ, in order for the advisee not to be misled. In this paper, we present a novel approach to transfer learning in which all available source policies are exploited to help learn several related new tasks. Moreover, our approach is compatible with tasks that differ by their transition functions, which is rarely considered in the transfer reinforcement learning literature. Our in-depth empirical evaluation demonstrates that our approach significantly improves sample-efficiency.
Plisnier, H, Steckelmacher, D, Roijers, D & Nowe, A 2019, Transfer Reinforcement Learning across Environment Dynamics with Multiple Advisors. in Proceedings of the 31st Benelux Conference on Artificial Intelligence (BNAIC 2019). vol. 2491, 11, CEUR Workshop Proceedings, CEUR Workshop Proceedings, 31st Benelux Conference on Artificial Intelligence and the 28th Belgian Dutch Conference on Machine Learning, BNAIC/BENELEARN 2019, Brussels, Belgium, 6/11/19. <http://ceur-ws.org/Vol-2491/paper11.pdf>
Plisnier, H., Steckelmacher, D., Roijers, D., & Nowe, A. (2019). Transfer Reinforcement Learning across Environment Dynamics with Multiple Advisors. In Proceedings of the 31st Benelux Conference on Artificial Intelligence (BNAIC 2019) (Vol. 2491). Article 11 (CEUR Workshop Proceedings). CEUR Workshop Proceedings. http://ceur-ws.org/Vol-2491/paper11.pdf
@inproceedings{49b4f0e24ff6401a8cf39a6504f822e5,
title = "Transfer Reinforcement Learning across Environment Dynamics with Multiple Advisors",
abstract = "Sample-efficiency is crucial in reinforcement learning tasks, especially when a large number of similar yet distinct tasks have to be learned. For example, consider a smart wheelchair learning to exit many differently-furnished offices on a building floor. Sequentially learning each of these tasks from scratch would be highly inefficient. A step towards a satisfying solution is the use of transfer learning: exploiting the knowledge acquired in previous (or source) tasks to tackle new (or target) tasks. Existing work mainly focuses on exploiting only one source policy as an advisor for the fresh agent, even when there are several expert source policies available. However, using only one advisor requires artificial mechanisms to limit its influence in areas where the source task and the target task differ, in order for the advisee not to be misled. In this paper, we present a novel approach to transfer learning in which all available source policies are exploited to help learn several related new tasks. Moreover, our approach is compatible with tasks that differ by their transition functions, which is rarely considered in the transfer reinforcement learning literature. Our in-depth empirical evaluation demonstrates that our approach significantly improves sample-efficiency.",
author = "Helene Plisnier and Denis Steckelmacher and Diederik Roijers and Ann Nowe",
year = "2019",
month = nov,
day = "6",
language = "English",
volume = "2491",
series = "CEUR Workshop Proceedings",
publisher = "CEUR Workshop Proceedings",
booktitle = "Proceedings of the 31st Benelux Conference on Artificial Intelligence (BNAIC 2019)",
note = "31st Benelux Conference on Artificial Intelligence and the 28th Belgian Dutch Conference on Machine Learning, BNAIC/BENELEARN 2019 ; Conference date: 06-11-2019 Through 08-11-2019",
}