Interpretable Saliency Map for Deep Reinforcement Learning

Abstract

Deep reinforcement learning (deep RL) achieved big successes with the advantage of deep learning techniques, while it also introduces the disadvantage of the model interpretability. Bad interpretability is a great obstacle for deep RL to be applied in real situations or human-machine interaction situations. Borrowed from the deep learning field, the techniques of saliency maps recently become popular to improve the interpretability of deep RL. However, the saliency maps still cannot provide specific and clear enough model interpretations for the behavior of deep RL agents. In this paper, we propose to use hierarchical conceptual embedding techniques to introduce prior-knowledge in the deep neural network (DNN) based models of deep RL agents and then generate the saliency maps for all the embedded factors. As a result, we can track and discover the important factors that influence the decisions of deep RL agents.