Pith. sign in

REVIEW 47 cited by

Captum: A unified and generic model interpretability library for PyTorch

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2009.07896 v1 pith:ZZF64T4O submitted 2020-09-16 cs.LG cs.AIstat.ML

Narine Kokhlikyan , Vivek Miglani , Miguel Martin , Edward Wang , Bilal Alsallakh , Jonathan Reynolds , Alexander Melnikov , Natalia Kliushkina
show 3 more authors
Carlos Araya Siqi Yan Orion Reblitz-Richardson
This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords libraryalgorithmscaptummodelallowsattributionbuiltextensibility
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In this paper we introduce a novel, unified, open-source model interpretability library for PyTorch [12]. The library contains generic implementations of a number of gradient and perturbation-based attribution algorithms, also known as feature, neuron and layer importance algorithms, as well as a set of evaluation metrics for these algorithms. It can be used for both classification and non-classification models including graph-structured models built on Neural Networks (NN). In this paper we give a high-level overview of supported attribution algorithms and show how to perform memory-efficient and scalable computations. We emphasize that the three main characteristics of the library are multimodality, extensibility and ease of use. Multimodality supports different modality of inputs such as image, text, audio or video. Extensibility allows adding new algorithms and features. The library is also designed for easy understanding and use. Besides, we also introduce an interactive visualization tool called Captum Insights that is built on top of Captum library and allows sample-based model debugging and visualization using feature importance metrics.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 47 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 643 citations worldwide. Full citation record

  1. Scaling Inherently Interpretable Language Models

    cs.CL 2026-08 conditional novelty 7.0 of 10

    Training a language model with a built-in concept bottleneck preserves compute-optimal scaling and yields interpretability metrics that improve with scale, demonstrated on an 8B causal diffusion model.

  2. What is Missing? Explaining Neurons Activated by Absent Concepts

    cs.CV 2026-03 accept novelty 7.0 of 10

    DNNs encode concept absences that raise neuron activation; standard XAI misses them, but non-target attribution and minimization visualization uncover them and improve debiasing.

  3. MATCHED: Multimodal Authorship-Attribution To Combat Human Trafficking in Escort-Advertisement Data

    cs.CL 2024-12 conditional novelty 7.0 of 10

    A new multimodal escort-ad dataset shows that end-to-end joint text-image training outperforms unimodal and CLIP-aligned models for vendor linking.

  4. Saliency Methods are Encoders: Analysing Logical Relations Towards Interpretation

    cs.LG 2024-12 conditional novelty 7.0 of 10

    On synthetic AND/OR/XOR tasks, saliency methods frequently rank irrelevant baseline inputs above logically relevant ones, and retrained models can recover class information from masked inputs, indicating that score or...

  5. Multimodal deep learning framework to predict strain localization of Mg/LPSO two-phase alloys

    cond-mat.mtrl-sci 2026-08 conditional novelty 6.0 of 10

    A multimodal deep learning model predicts local strain in Mg/LPSO alloys from microstructure descriptors, revealing that strain localization is tied to 45-degree-oriented elongated LPSO phases.

  6. Does Explainability Transfer? A Controlled Benchmark of Attribution Methods on Vision Transformers and CNNs

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Attribution-method rankings are architecture-dependent: CNN conclusions do not reliably transfer to Vision Transformers, and apparent CAM localization wins under bounding boxes are largely metric saturation.

  7. Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering

    cs.CL 2026-07 reject novelty 6.0 of 10

    A token-attribution metric (ASI) and attribution-guided steering are proposed for LLM sycophancy, with large in-sample reductions reported.

  8. Scaling Time Series Classification via XAI-Driven Data Reduction

    cs.LG 2026-07 conditional novelty 6.0 of 10

    drXAI uses XAI attributions from a fast classifier to choose important channels/time points, achieving 80–90% data reduction with comparable classification accuracy.

  9. Imputation-free transformer learning enables robust Alzheimer's disease prediction and calibrated uncertainty quantification across heterogeneous clinical cohorts

    q-bio.NC 2026-07 conditional novelty 6.0 of 10

    An imputation-free transformer with masked and intersample attention predicts Alzheimer’s status and scores across cohorts with better calibration than tree ensembles.

  10. STAPO: Selective Trajectory-Aware Policy Optimization for LLM Agent Training

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Normalized entropy locates trajectory-neglect steps; STAPO selectively optimizes them with a trajectory-aware reward and trajectory-independent penalty, beating group-based RL baselines on agent tasks.

  11. Two Black Boxes, One Solver: Encoder Probing and Decoder Attribution for Neural Multi-Attribute VRP under Hard-Mask and Recourse Decoders

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Graph encoders and recourse training improve latent constraint representation and actionable decoder explanations in neural MAVRP solvers; make-feasible counterfactuals arise from the training regime, not the mask.

  12. Interpreto: An Explainability Library for Transformers

    cs.CL 2025-12 accept novelty 6.0 of 10

    Interpreto provides a unified, documented Python API for attribution and end-to-end concept-based explanation of HuggingFace classification and generation models.

  13. An Empirical Evaluation of Factors Affecting SHAP Explanation of Time Series Classification

    cs.AI 2025-09 conditional novelty 6.0 of 10

    Equal-length segmentation matches or beats seven more complex segmentation strategies for SHAP-based time series explanations, and length-weighted normalisation of segment attributions improves evaluation scores.

  14. Accept or Deny? Evaluating LLM Fairness and Performance in Loan Approval across Table-to-Text Serialization Approaches

    cs.LG 2025-08 conditional novelty 6.0 of 10

    Serialization format and in-context examples change both accuracy and gender fairness of LLM loan approvals, with finance-tuned models often showing larger disparities.

  15. DeepFaith: A Domain-Free and Model-Agnostic Unified Framework for Highly Faithful Explanations

    cs.LG 2025-08 unverdicted novelty 6.0 of 10

    A single trained explainer that unifies multiple faithfulness metrics and generates model-agnostic explanations with high measured faithfulness.

  16. POLARIS: Explainable Artificial Intelligence for Mitigating Power Side-Channel Leakage

    cs.CR 2025-07 conditional novelty 6.0 of 10

    An XAI-based masking-gate insertion flow is claimed to beat the state-of-the-art VALIANT in leakage reduction, speed, and overhead, but the comparison is weakened by test-set model selection and inference-only timing.

  17. TRACE: Training and Inference-Time Interpretability Analysis for Language Models

    cs.CL 2025-07 conditional novelty 6.0 of 10

    TRACE is a modular training-time interpretability toolkit that, combined with the ABSynth synthetic corpus generator, reveals phased emergence of syntax and semantics during transformer training.

  18. DeltaSHAP: Explaining Prediction Evolutions in Online Patient Monitoring with Shapley Values

    cs.LG 2025-07 conditional novelty 6.0 of 10

    DeltaSHAP attributes a monitoring model's prediction change between time steps to individual features using sampled Shapley values over the latest observed measurements.

  19. Model Fusion via Retrofitting

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A neuron-centric fusion method that clusters intermediate activations of independently trained models into importance-weighted centroids and fits the fused network to them, outperforming baselines in zero-shot non-IID...

  20. On the Necessity of Multi-Domain Explanation: An Uncertainty Principle Approach for Deep Time Series Models

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A time-frequency uncertainty bound can tell when a model's time-domain and frequency-domain attributions point to different features, making multi-domain explanations necessary.

  21. MANTIS: Detection of Zero-Day Malicious Domains Leveraging Low Reputed Hosting Infrastructure

    cs.CR 2025-02 conditional novelty 6.0 of 10

    MANTIS detects new malicious domains by monitoring the low-reputation hosting infrastructure around known malicious domains, using a graph neural network, and reports finding about 19,000 new malicious domains daily.

  22. Multi-megabase scale genome interpretation with genetic language models

    q-bio.GN 2025-01 conditional novelty 6.0 of 10

    Phenformer uses frozen Enformer embeddings of 512 gene windows to predict disease risk and cell type involvement, outperforming PRS baselines restricted to the same genomic regions.

  23. Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Media

    cs.AI 2024-12 conditional novelty 6.0 of 10

    A detector trained on a new multi-LLM social media benchmark estimates that AI-generated text on Medium and Quora grew from about 2% to roughly 37 to 39 percent between 2022 and 2024, while Reddit stayed near 2%.

  24. Self-Supervised Radiograph Anatomical Region Classification -- How Clean Is Your Real-World Data?

    cs.CV 2024-12 conditional novelty 6.0 of 10

    Self-supervised contrastive learning on 48,434 skeletal radiographs achieves 96.6% accuracy in anatomical region classification and can detect incorrect PACS labels.

  25. Can Input Attributions Explain Inductive Reasoning in In-Context Learning?

    cs.CL 2024-12 conditional novelty 6.0 of 10

    Using synthetic inductive reasoning tasks with a single 'aha' example, the paper shows simple gradient-norm attribution often beats integrated gradients for identifying the crucial example, while interpretability wors...

  26. A Physics-Flavored Transformer Network for Parametrizing Contraction Dynamics of Engineered Skeletal Muscle Tissues

    cs.LG 2026-08 conditional novelty 5.0 of 10

    A physics-flavored CNN-transformer automatically fits stretched-exponential parameters to engineered skeletal muscle contraction curves, using synthetic pre-training and unsupervised real-data alignment.

  27. Automated Solar Radio Burst Detection Using Deep Learning on Augmented e-Callisto Data

    astro-ph.SR 2026-07 conditional novelty 5.0 of 10

    FlareSense, a ResNet detector trained on 304,750 e-Callisto spectrograms with SpecAugment and TimeWarp, reaches 93% precision and 73.15% recall, outperforming routine expert cataloging at matched precision.

  28. A Preliminary Study on Explaining Risk of Code Changes using LLM-Based Prediction Models

    cs.SE 2026-07 conditional novelty 5.0 of 10

    Top-2 attention-highlighted hunks cover expert-labeled outage root causes 53.85% of the time while requiring review of 26.28% of changed lines.

  29. Rashomon in the Streets: Explanation Ambiguity in Scene Understanding

    cs.LG 2025-09 conditional novelty 5.0 of 10

    Equally accurate models trained on the same driving scenes often point to different features as the cause of an action, so the explanation you get depends on which model you happen to train.

  30. Exploring Pose-based Sign Language Translation: Ablation Studies and Attention Insights

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Pose normalization based on the signer's signing space substantially improves gloss-free sign language translation with a T5 model, while interpolation and augmentation give smaller, less certain gains.

  31. Navigating Shortcuts, Spurious Correlations, and Confounders: From Origins via Detection to Mitigation

    cs.LG 2024-12 accept novelty 5.0 of 10

    A unifying taxonomy and formal definition that connects shortcut learning, spurious correlations, Clever Hans behavior, and confounders across detection, mitigation, and datasets.

  32. Combining Machine Learning Defenses without Conflicts

    cs.CR 2024-11 conditional novelty 5.0 of 10

    A stage-and-risk-based decision rule predicts whether pairs of ML defenses conflict, with reported balanced accuracy of 90% on eight prior combinations and 81-86% on 30 new ones.

  33. Crushing the Evidence: A Dual-Penalty Evasion Framework for Fooling White-Box Explainable AI Auditors

    cs.LG 2026-08 conditional novelty 4.0 of 10

    A dual-penalty training loss that suppresses trigger-feature gradients hides tabular backdoors from Integrated Gradients, DeepSHAP, and CAD-Detect while maintaining high attack success.

  34. Functional Groups are All you Need for Chemically Interpretable Molecular Property Prediction

    cs.LG 2025-09 conditional novelty 4.0 of 10

    FGR encodes molecules as functional-group bit vectors, embeds them with an autoencoder, and reports competitive or better benchmark accuracy while enabling chemical attribution.

  35. Uncovering Latent Connections in Indigenous Heritage: Semantic Pipelines for Cultural Preservation in Brazil

    cs.HC 2025-07 conditional novelty 4.0 of 10

    The paper introduces two embedding pipelines and a visualization tool that reveal latent clusters and label errors in the Museu Nacional dos Povos Indígenas digital collection.

  36. PyG 2.0: Scalable Learning on Real World Graphs

    cs.LG 2025-07 conditional novelty 4.0 of 10

    PyG 2.0 is presented as a modular, scalable graph-learning framework with heterogeneous and temporal graph support, compilation-based speedups, and explainability.

  37. Why Uncertainty Calibration Matters for Reliable Perturbation-based Explanations

    cs.LG 2025-06 conditional novelty 4.0 of 10

    Calibration of a model under the exact perturbations used by an explanation method improves explanation fidelity, and ReCalX achieves this with per-perturbation-strength temperature scaling.

  38. Saliency Maps are Ambiguous: Analysis of Logical Relations on First and Second Order Attributions

    cs.LG 2025-01 conditional novelty 4.0 of 10

    On synthetic AND/OR/XOR datasets with perfectly accurate models, every tested saliency method sometimes ranks a truly irrelevant input above a necessary one, so the scores cannot be trusted as relevance rankings.

  39. The Process of Categorical Clipping at the Core of the Genesis of Concepts in Synthetic Neural Cognition

    cs.AI 2025-01 conditional novelty 4.0 of 10

    Words that strongly activate both a lower-layer neuron and its strongly connected upper-layer neuron in GPT-2XL form more semantically similar clusters, which the paper interprets as a clipping process.

  40. Advancing Stroke Risk Prediction Using a Multi-modal Foundation Model

    cs.CV 2024-11 conditional novelty 4.0 of 10

    A CLIP-style multimodal model with image-tabular matching achieves modest AUC gains over unimodal baselines for pre-stroke stroke risk prediction on a small UK Biobank test set.

  41. AnomalyExplainer Explainable AI for LLM-based anomaly detection using BERTViz and Captum

    cs.LG 2025-08 conditional novelty 3.0 of 10

    A chatbot that classifies HDFS log lines with fine-tuned RoBERTa and provides BERTViz/Captum visual explanations was benchmarked and tested with 13 users.

  42. ExplainBench: A Benchmark Framework for Local Model Explanations in Fairness-Critical Applications

    cs.LG 2025-05 reject novelty 3.0 of 10

    ExplainBench is a proposed open-source benchmarking suite for local explanation methods, but the paper provides no empirical evaluation.

  43. Quantized and Interpretable Learning Scheme for Deep Neural Networks in Classification Task

    cs.LG 2024-12 conditional novelty 3.0 of 10

    Saliency-guided training combined with PACT quantization keeps MNIST and CIFAR-10 accuracy near parity with a quantized baseline, while the claimed efficiency and interpretability gains are not directly measured.

  44. Establishing and Evaluating Trustworthy AI: Overview and Research Challenges

    cs.LG 2024-11 conditional novelty 3.0 of 10

    A semi-structured literature review synthesizing six trustworthy AI requirements and their evaluation methods, plus cross-cutting research challenges.

  45. xai_evals : A Framework for Evaluating Post-Hoc Local Explanation Methods

    cs.LG 2025-02 reject novelty 2.0 of 10

    A technical report introducing xai_evals, a Python package that wraps existing explainability and metric libraries without adding new methods or validated results.

  46. How Do Artificial Intelligences Think? The Three Mathematico-Cognitive Factors of Categorical Segmentation Operated by Synthetic Neurons

    q-bio.NC 2024-12 reject novelty 2.0 of 10

    The paper names three components of a neuron's aggregation function as cognitive factors and reports near-unity correlations in GPT-2XL, but the effects are largely true by construction.

  47. A Comprehensive Guide to Explainable AI: From Classical Models to LLMs

    cs.LG 2024-12 unverdicted novelty 1.0 of 10

    A survey-style XAI book with code examples, covering standard interpretability methods and models, but no new scientific contributions.

Pith tools