Hierarchical Support Vector State Partitioning for Distilling Black Box Reinforcement Learning Policies
 
Hierarchical Support Vector State Partitioning for Distilling Black Box Reinforcement Learning Policies 
 
Senne Deproost, Ann Nowe, Mehrdad Asadi
 
Abstract 

We introduce State Vector Space Partitioning (SVSP), a novel method to mimic a black-box reinforcement learning policy using a set of human-interpretable sub-policies. By partitioning a distillation dataset of state–action pairs with linear support vector machine splits, SVSP constructs a compact and structured represen- tation of the original policy where linear models can be interpreted as a measure of feature importance. Our method improves mean return by +7.4\% over previous critic-driven state partitioning attempts such as Voronoi State Partitioning (VSP) and +2.8\% over the original TD3 policy, while reducing the number of required sub-policies against VSP by 82.1\%. Our results pave the path towards a more flexi- ble form of distillation where both the decision boundary and surrogate models can be chosen within a margin of the original black box behavior.