Technical Report # 27

Goal-based explanation evaluation

Leake, D.

Abstract

Explanation-based learning (EBL) is a powerful method for focusing on the important features of new situations. However, EBL does not in itself assure that an explainer will learn useful concepts: the value of what is learned depends on the particular explanations used. Even when the initial explanation is guaranteed to be valid, it may not provide the information that the explainer needs. In addition, it may be impossible to generate the complete explanations that EBL systems often require, which limits the applicability of explanation-based methods.

We argue that both wider applicability of EBL, and more useful learning when EBL is used, can be achieved by basing selection of explanations on goal-based criteria. This argument conforms to the common-sense view that people accept and learn from explanations precisely if those explanations give the information they need. We present a theory for evaluating whether a partial explanation provides the information an explainer needs, extending the applicability of EBL to situations for which complete explanation is impossible. We illustrate this theory by sketching its implementation in the computer program ACCEPTER, which does goal-based evaluation of the goodness of explanations for surprising events in news stories.