traditional compression methods have been used to facilitate distributed AI systems; in what follows we give more information about this research path: Recent successful results in the field of artificial intelligence (AI) are achieved with deep learning models that contain a large number of parameters and are trained with massive amounts of data. For example, FixResNeXt-101 32x48d, a state-of-the-art model for image classification, contains approximately 800 million parameters, and BERT (Bidirectional Encoder Representations from Transformers), a recent model for natural language processing, contains 110 million parameters. Training such deep networks in a single machine (given set of hyperparameters) can take weeks. A solution is data-parallel distributed training, where a model is replicated into several computational nodes that have access to different chunks of data. This approach, however, entails high communication rates and latency because of the computed gradients that need to be shared among nodes per iteration. Traditional compression methods have been proposed to address the gradient communication problem, including gradient sparsification, quantization, and entropy coding. While explainable AI methods have been used in some fields, the opposite direction in the interaction has never been established; namely, how can compression methods assist explainable AI?

The project’s core objective is to investigate the interaction of compression methods and explainability methods, particularly in the realm of large-scale (distributed) AI systems: using one to empirically evaluate the other or applying them jointly so that they can bootstrap or strengthen one another. Our particular aim is to develop a framework that addresses the aforementioned gaps in international research.