Joris De Winter, Albert De Beir, Ilias El Makrini, Greet Van de Perre, Ann Nowe, Bram Vanderborght
The assembly industry is shifting more towards customizable products, or requiring assembly of small batches. This requires a lot of reprogramming, which is expensive because a specialized engineer is required. It would be an improvement if untrained workers could help a cobot to learn an assembly sequence by giving advice. Learning an assembly sequence is a hard task for a cobot, because the solution space increases drastically when the complexity of the task increases. This work introduces a novel method where human knowledge is used to reduce this solution space, and as a result increases the learning speed. The method proposed is the IRL-PBRS method, which uses Interactive Reinforcement Learning (IRL) to learn from human advice in an interactive way, and uses Potential Based Reward Shaping (PBRS), in a simulated environment, to focus learning on a smaller part of the solution space. The method was compared in simulation to two other feedback strategies. The results show that IRL-PBRS convergesmore quickly to a valid assembly sequence policy and does this with the fewest human interactions. Finally, a use case is presented where participants were asked to program an assembly task. Here, the results show that IRL-PBRS learns quickly enough to keep up with advice given by a user, and is able to adapt online to a changing knowledge base.
De Winter, J, De Beir, A, El Makrini, I, Van De Perre, G, Nowe, A & Vanderborght, B 2019, 'Accelerating Interactive Reinforcement Learning byHuman Advice for an Assembly Task by a Cobot', Robotics, vol. 8, no. 4, 104. https://doi.org/10.3390/robotics8040104, https://doi.org/10.3390/robotics8040104
De Winter, J., De Beir, A., El Makrini, I., Van De Perre, G., Nowe, A., & Vanderborght, B. (2019). Accelerating Interactive Reinforcement Learning byHuman Advice for an Assembly Task by a Cobot. Robotics, 8(4), Article 104. https://doi.org/10.3390/robotics8040104, https://doi.org/10.3390/robotics8040104
@article{7b5952f9081144e881ea77c845302ecc,
title = "Accelerating Interactive Reinforcement Learning byHuman Advice for an Assembly Task by a Cobot",
abstract = "The assembly industry is shifting more towards customizable products, or requiring assembly of small batches. This requires a lot of reprogramming, which is expensive because a specialized engineer is required. It would be an improvement if untrained workers could help a cobot to learn an assembly sequence by giving advice. Learning an assembly sequence is a hard task for a cobot, because the solution space increases drastically when the complexity of the task increases. This work introduces a novel method where human knowledge is used to reduce this solution space, and as a result increases the learning speed. The method proposed is the IRL-PBRS method, which uses Interactive Reinforcement Learning (IRL) to learn from human advice in an interactive way, and uses Potential Based Reward Shaping (PBRS), in a simulated environment, to focus learning on a smaller part of the solution space. The method was compared in simulation to two other feedback strategies. The results show that IRL-PBRS convergesmore quickly to a valid assembly sequence policy and does this with the fewest human interactions. Finally, a use case is presented where participants were asked to program an assembly task. Here, the results show that IRL-PBRS learns quickly enough to keep up with advice given by a user, and is able to adapt online to a changing knowledge base.",
keywords = "Interactive Reinforcement Learning, Cobots, Programming by Advice, Assembly planning",
author = "{De Winter}, Joris and {De Beir}, Albert and {El Makrini}, Ilias and {Van De Perre}, Greet and Ann Nowe and Bram Vanderborght",
year = "2019",
month = dec,
day = "16",
doi = "10.3390/robotics8040104",
language = "English",
volume = "8",
journal = "Robotics",
issn = "2218-6581",
publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",
number = "4",
}