Faithfulness Without Anchors: A Position Paper on the Structural Misalignments of Explainable AI Faithfulness Metrics for Feature Importance

Explainable Artificial Intelligence (XAI) tools are often marketed as solutions to enhance trust in machine learning models by rendering their algorithms more comprehensible to humans. However, evidence gathered over the past decade indicates a general failure to achieve this goal, leading to an overreliance on inaccurate predictions and, consequently, algorithmic aversion. Instead, XAI seems to primarily serve as a debugging tool, providing anecdotal insights into features learned by the model. A significant issue contributing to this limitation is that XAI methods frequently oversimplify the model’s inner workings, resulting in a misalignment between the explanation and the model itself, a phenomenon known as unfaithfulness or infidelity. Faithfulness can be defined as the degree of approximation between an explanation and the model. Rigorously computing faithfulness would enable the quantification of one aspect of an explanation’s trustworthiness. However, current metrics exhibit considerable limitations, rendering this quantification either impractical or computationally intensive. This position paper identifies five key shortcomings of existing faithfulness metrics and suggests four potential research directions to enhance their reliability.

Source: arxiv.org