The Need for Empirical Evaluation of Explanation Quality

Prototype networks (Li et al. 2018) provide explanations to users using a prototype vector; that is, a vector learned by the network representing a "typical" observation. In this work, we propose an approach that identifies relevant features in the input space used by the Prototype network. We find however that empirical evaluation of explanation quality is difficult without ground truth explanations. We include a discussion about developing methods for generating explanations, identifying when one explanation method is preferable to another, and the complications that arise when measuring explanation quality.