Gaoyuan Liu, Joris De Winter, Yuri Durodié, Denis Steckelmacher, Ann Nowe, Bram Vanderborght
Task and motion planning (TAMP) for robotics manipulation necessitates long-horizon reasoning involving versatile actions and skills. While deterministic actions can be crafted by sampling or optimizing with certain constraints, planning actions with uncertainty, i.e., probabilistic actions, remains a challenge for TAMP. On the contrary, Reinforcement Learning (RL) excels in acquiring versatile, yet short-horizon, manipulation skills that are robust with uncertainties. In this letter, we design a method that integrates RL skills into TAMP pipelines. Besides the policy, a RL skill is defined with data-driven logical components that enable the skill to be deployed by symbolic planning. A plan refinement sub-routine is designed to further tackle the inevitable effect uncertainties. In the experiments, we compare our method with baseline hierarchical planning from both TAMP and RL fields and illustrate the strength of the method. The results show that by embedding RL skills, we extend the capability of TAMP to domains with probabilistic skills, and improve the planning efficiency compared to the previous methods.
Liu, G, De Winter, J, Durodié, Y, Steckelmacher, D, Nowe, A & Vanderborght, B 2024, 'Optimistic Reinforcement Learning-Based Skill Insertions for Task and Motion Planning', IEEE Robotics and Automation Letters, vol. 9, no. 6, pp. 5974-5981. https://doi.org/10.1109/LRA.2024.3398402
Liu, G., De Winter, J., Durodié, Y., Steckelmacher, D., Nowe, A., & Vanderborght, B. (2024). Optimistic Reinforcement Learning-Based Skill Insertions for Task and Motion Planning. IEEE Robotics and Automation Letters, 9(6), 5974-5981. https://doi.org/10.1109/LRA.2024.3398402
@article{a6af9764cad349198a85e41bbd0bf0f3,
title = "Optimistic Reinforcement Learning-Based Skill Insertions for Task and Motion Planning",
abstract = "Task and motion planning (TAMP) for robotics manipulation necessitates long-horizon reasoning involving versatile actions and skills. While deterministic actions can be crafted by sampling or optimizing with certain constraints, planning actions with uncertainty, i.e., probabilistic actions, remains a challenge for TAMP. On the contrary, Reinforcement Learning (RL) excels in acquiring versatile, yet short-horizon, manipulation skills that are robust with uncertainties. In this letter, we design a method that integrates RL skills into TAMP pipelines. Besides the policy, a RL skill is defined with data-driven logical components that enable the skill to be deployed by symbolic planning. A plan refinement sub-routine is designed to further tackle the inevitable effect uncertainties. In the experiments, we compare our method with baseline hierarchical planning from both TAMP and RL fields and illustrate the strength of the method. The results show that by embedding RL skills, we extend the capability of TAMP to domains with probabilistic skills, and improve the planning efficiency compared to the previous methods.",
keywords = "Task and Motion Planning, Reinforcement Learning, Manipulation Planning",
author = "Gaoyuan Liu and {De Winter}, Joris and Yuri Durodi{\'e} and Denis Steckelmacher and Ann Nowe and Bram Vanderborght",
note = "Publisher Copyright: IEEE",
year = "2024",
month = may,
day = "9",
doi = "10.1109/LRA.2024.3398402",
language = "English",
volume = "9",
pages = "5974--5981",
journal = "IEEE Robotics and Automation Letters",
issn = "2377-3766",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "6",
}