Publication Details
Jessica Coto Palacio, Yailen Martínez Jiménez, Yailen Martinez Jimenez, Schietgat, Leander, Bart Van Doninck,

Procedia CIRP

Contribution To Journal


In this work we propose a Reinforcement Learning approach for a real-world flexible job shop scheduling scenario, where a two-armed robot and a human operator share two workstations in order to assemble light switches. The approach returns a schedule of predefined assembly actions, taking into account given constraints specific to the scenario, such as which actions should precede others, which actions should occur together, or which actions should occur separately. In order to avoid collisions, the work area is divided into zones and all these constraints are taken into account by the algorithm in order to propose a schedule that optimizes the time it takes to prepare the customer orders. The approach that is taken is a reinforcement learning approach. There both arms and the operator are considered to be agents, and they learn from experience who should do which job best, taking into account the above mentioned constraints, as well as the estimated timings to execute the tasks. These estimated timings can be online updated, as well as their uncertainty. The experiments show that the proposed algorithm outperforms the currently used algorithm, which is a constraint solver, in terms of computation time and it is more user friendly with respect to expressing constraints. It also allows to take into account the uncertainty on the execution times of the tasks, and to add slack at critical timeslots, without unnecessarily increasing the makespan. Moreover, the algorithm allows to recompute an updated solution dynamically and efficiently when it turns out an action cannot be performed within the proposed time due to unexpected events. While we show the benefits in a scenario with a two-armed robot collaborating with a human for assembling light switches, the approach is relevant for many assembly tasks in the manufacturing industry where machines and human operators work together.