David Alvarez-Melis

PhD Candidate, MIT Computer Science and Artificial Intelligence Lab

Stata Center, Bldg 32-G496, Cambridge MA 02139

d_alv_mel_[at]_mit_[dot]_edu (humans: remove underscores)

Robustly Interpretable Machine Learning

Bridging the gap between model expressiveness and transparency
Abstract

(Under construction)

From Extrinsic to Intrinsic Explanations: Self-Explaining Neural Networks

Most recent work on interpretability of complex machine learning models has focused on estimating a posteriori explanations for previously trained models around specific predictions. Self-explaining models where interpretability plays a key role already during learning have received much less attention. We propose three desiderata for explanations in general – explicitness, faithfulness, and stability – and show that existing methods do not satisfy them. In response, we design self-explaining models in stages, progressively generalizing linear classifiers to complex yet architecturally explicit models. Faithfulness and stability are enforced via regularization specifically tailored to such models. Experimental results across various benchmark datasets show that our framework offers a promising direction for reconciling model complexity and interpretability.

A Self-Explaining Neural Network consists of three components: a concept encoder (green) that transforms the input into a small set of interpretable basis features; an input-dependent parametrizer (orange) that generates relevance scores; and an aggregation function that combines to produce a prediction. The robustness loss on the parametrizer encourages the full model to behave locally as a linear function on $h(x)$ with parameters $\theta(x)$, yielding immediate interpretation of both concepts and relevances.
A Self-Explaining Neural Network consists of three components: a concept encoder (green) that transforms the input into a small set of interpretable basis features; an input-dependent parametrizer (orange) that generates relevance scores; and an aggregation function that combines to produce a prediction. The robustness loss on the parametrizer encourages the full model to behave locally as a linear function on $h(x)$ with parameters $\theta(x)$, yielding immediate interpretation of both concepts and relevances.

Relevant Publications:

  1. Alvarez-Melis and Jaakkola. “A Causal Framework for Explaining the Predictions of Black-Box Sequence-to-Sequence Models”, EMNLP 2017.
  2. Alvarez-Melis and Jaakkola. “On the Robustness of Interpretability Methods”, WHI@ICML 2018.
  3. Lee, Alvarez-Melis and Jaakkola. “Game-theoretic Interpretability for Temporal Modeling”, FAT/ML@ICML2018.
  4. Alvarez-Melis and Jaakkola. “Towards Robust Interpretability with Self-Explaining Neural Networks”, NIPS 2018.