Liu, G. (2025). Hybrid solutions for behavior planning in robotic manipulation. [PhD Thesis, Vrije Universiteit Brussel]. Crazy Copy Center Productions.

@phdthesis{359a602c66fa478d9a5a88d634f39f7f,
title = "Hybrid solutions for behavior planning in robotic manipulation",
abstract = "Autonomous robotic manipulators are becoming increasingly essential across a wide range of domains, including manufacturing, logistics, healthcare, and agriculture. Although robotic hardware has seen significant advancements, a central challenge persists: enabling robots to generate behaviors intelligently in unstructured, real-world environments. Various schemes to generate robots{\textquoteright} behavior are widely studied: reinforcement learning (RL), task planning, motion planning, and servoing-based control, etc. However, single-paradigm approaches usually concentrate on one specific capability for a certain constraint. In contrast, real-world manipulation tasks usually pose multiple constraints that require a broader and more balanced set of features, making hybrid solutions essential. The main objective of this PhD dissertation is to explore how hybrid planning methods can be designed and leveraged to improve the effectiveness and efficiency of behavior planning in robotic manipulation tasks. We deliberated the main objective into four research questions by identifying the single-paradigm schemes{\textquoteright} limits in the specific real-world problem. First, we highlight the safety gap of implementing RL on real robots. An RL workflow tends to be unsafe when directly deployed on physical robot systems because of the intrinsic randomness in the exploration. We designed an RL training platform by deploying the motion planning library into the RL workflow. This design enables collision-aware exploration in RL. With the training platform, we provide a use case of an RL task planner in a human-robot coexistence environment. The RL policy chooses the next tasks according to the human{\textquoteright}s position, trying to avoid interfering with the human. Second, the existing hybrid planning method, i.e., sampling-based task and motion planning (TAMP) faces limitations when applied to cluttered manipulation tasks, particularly due to its inability to effectively generate non-prehensile actions with probabilistic effects. In contrast, RL excels at acquiring such skills. To address this gap, we propose a hybrid approach that combines the structured planning capabilities of TAMP with the adaptability of RL. In this framework, RL is used to learn non-prehensile decluttering strategies that guide the environment toward configurations within the capacity of the sampling TAMP. We validate the approach on a cluttered bin-picking task, where experimental results highlight the critical role of RL in enhancing the overall planning efficiency and task success of sampling-based TAMP. Third, we address the challenge of enhancing the versatility of current TAMP methods, which are inherently limited by their sampling-based design and reliance on deterministic actions. To overcome this, we introduce a framework for integrating RL skills into TAMP via logical interfaces. Each RL policy is encapsulated with symbolic representations, enabling it to be invoked by a high-level logic-based task planner. We evaluate this approach on a table rearrangement task, which requires coordinated use of both prehensile and non-prehensile actions. Experimental results demonstrate the complementarity between TAMP and RL: TAMP excels at long-horizon planning over structured tasks, while RL offers robust handling of probabilistic and uncertainty-aware actions over shorter horizons. Fourth, we investigate how robotic manipulators can be equipped with effective planning and control strategies to perform agricultural tasks. We specifically focused on the orchard pruning task for apple trees. We identified two main challenges in generating behaviors in such complex orchard environments: intricate obstacles and uncertainties. Existing motion planning approaches often exhibit low efficiency in such unstructured settings. To bridge this gap, we develop a pruning workflow that goes beyond perception. We introduce a novel formulation of high-level pruning commands and exploit task redundancies, thereby relaxing constraints and expanding the feasible planning space. Our system integrates sampling-based motion planning with servoing-based control, enabling it to meet the demanding requirements of both efficiency and precision. With the redundancies, we further deployed a goal region motion planning, and use two perception modes in different stages to tackle the issue of uncertainty. These approaches are validated with datasets of trees scanned from a real orchard. The workflows are also implemented in real-world laboratory setups, demonstrating their potential for practical deployment in agricultural robotics. The effectiveness of these hybrid methods is demonstrated through a range of manipulation tasks, from indoor applications like bin unpacking and object rearrangement to outdoor, domain-specific challenges such as pruning in orchards. These tasks highlight the need for hybrid behavior planning, which requires comprehensive capacities across logical reasoning, collision avoidance, and accurate execution. We implement and validate the proposed systems in both simulation and real-world scenarios, and compare them with their counterpart baseline methods. The results show that, in general, the hybrid solutions improve performance in behavior planning for robotic manipulation, enhancing both effectiveness, measured by success ratio, and efficiency, measured by planning time. Through this exploration, the thesis provides new insights into behavior synthesis for robotic manipulators and contributes toward the development of more general, robust, and intelligent robotic systems for real-world deployment.",
author = "Gaoyuan Liu",
year = "2025",
language = "English",
isbn = " 9789493461222",
publisher = "Crazy Copy Center Productions",
address = "Belgium",
school = "Vrije Universiteit Brussel",
}