Comprehensive Attribution: Inherently Explainable Vision Model with Feature Detector
Research Poster Engineering 2025 Graduate ExhibitionPresentation by Xianren Zhang
Exhibition Number 12
Abstract
As deep vision models’ popularity rapidly increases, there is a growing emphasis on explanations for model predictions. The inherently explainable attribution method aims to enhance the understanding of model behavior by identifying the important regions in images that significantly contribute to predictions. It is achieved by cooperatively training a selector (generating an attribution map to identify important features) and a predictor (making predictions using the identified features). Despite many advancements, existing methods suffer from the incompleteness problem, where discriminative features are masked out, and the interlocking problem, where the non-optimized selector initially selects noise, causing the predictor to fit on this noise and perpetuate the cycle. To address these problems, we introduce a new objective that discourages the presence of discriminative features in the masked-out regions thus enhancing the comprehensiveness of feature selection. A pretrained detector is introduced to detect discriminative features in the masked-out region. If the selector selects noise instead of discriminative features, the detector can observe and break the interlocking situation by penalizing the selector. Extensive experiments show that our model makes accurate predictions with higher accuracy than the regular blackbox model, and produces attribution maps with high feature coverage, localization ability, fidelity and robustness.
Importance
Our study tries to make deep learning model explainable and understandable for general users. This work will help to improve the transparency of machine learning models.