Pith. sign in

REVIEW 3 major objections 7 minor 54 references

Explaining Facial Expression Recognition

T0 review · 3 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Explanations built from facial action units help end-users understand when a facial-expression-recognition model will be right or wrong, and calibrate their trust, while LIME, SHAP, and saliency maps do not.

desk verdict The DEFAULTS study is a well-run empirical comparison undermined by label leakage: the FAU explanations are generated from the model's emotion output, so the headline gains likely reflect users decoding the label rather than understanding the model. read the letter →

arxiv 2501.15864 v2 pith:PQTAAXJT submitted 2025-01-27 cs.HC

Sanjeev Nahulanthran , Leimin Tian , Dana Kulić , Mor Vered This is my paper · ORCID
classification cs.HC
keywords facialexpressionrecognitionexplainableAIactionunitsuserunderstandingappropriatetrustexplanationmodalitiesDEFAULTShuman-modelprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that explaining a facial expression recognition (FER) model through Facial Action Units (FAUs)—the visible muscle movements behind an expression—lets non-expert users predict the model's output better than no explanation, and calibrates their trust to the model's actual accuracy. The claim is supported by a between-subject user study with 280 participants comparing seven conditions: no explanation, LIME, SHAP, saliency maps, textual FAUs, visual FAUs, and combined textual-visual FAUs. Textual-only and combined FAU explanations significantly improved users' ability to guess the model's emotion prediction, and all FAU modalities produced significantly higher appropriate trust than no explanation. The generic post-hoc methods did not significantly improve understanding or trust. The paper concludes that explanation content grounded in facial physiology matters more than explanation modality alone, with text doing the main work of conveying the model's reasoning.

What carries the argument

The central object is the Facial Action Unit (FAU), a muscle activation defined by the Facial Action Coding System, used here as the atomic unit of explanation. DEFAULTS converts the outputs of a deep network FAU predictor into a textual list of activated FAUs and a visual mask that highlights the facial landmarks associated with those FAUs. The architecture matters for interpreting the result: a CNN trained on CK+ produces eight emotion-class scores, and a separate DNN head receives the concatenation of the CNN's final convolutional features and the 1x8 emotion vector to predict 15 boolean FAUs. The evaluation machinery is the two-question survey protocol that yields Human Model Prediction (Hmp) accuracy and the appropriate-trust score.

What would settle it

Run the same study with a version of the FAU predictor trained without access to the emotion vector, or with FAU labels randomly shuffled relative to each image; if the gains in Hmp accuracy and appropriate trust disappear, the reported effects came from the leaked emotion label.

Watch

Extended reading notes

Core claim

The paper's central claim is that an explanation built from Facial Action Units—named muscle movements such as 'inner brow raised' or 'lips parted'—gives end-users a usable handle on what a facial-expression-recognition model will predict. In the study, users who saw textual-only or combined textual-visual FAU explanations predicted the model's emotion label significantly more accurately than controls, and every FAU-based explanation modality produced significantly higher appropriate trust, defined as trusting the model's prediction when it is correct and withholding trust when it is incorrect. LIME, SHAP, and saliency-map explanations produced no significant improvement on either measure. The paper concludes that FAU-based explanations, particularly those with a textual component, are more effective than generic attribution methods at supporting end-user understanding and calibrated trust in FER systems.

Load-bearing premise

The load-bearing premise is that the FAU explanation is an independent account of the face, but the FAU predictor receives the model's emotion-prediction vector as an extra input, so the explanation may leak which emotion the model predicted.

Editorial extensions

If this is right

  • Text is the load-bearing modality: textual-only FAU explanations significantly improved users' ability to predict the model's emotion label, and adding visuals to text improved that ability further.
  • Visual-only explanations, including FAU-based visual explanations, did not significantly improve understanding of the model, although FAU visual explanations did raise appropriate trust relative to all other visual-only methods and controls.
  • General-purpose post-hoc explanation methods (LIME, SHAP, saliency maps) did not significantly help users understand or calibrate trust toward a FER model in this setup.
  • If the finding holds, FAU-based explanations could give non-expert users a way to know when an emotion-recognition system is likely to be wrong, which matters for applications where over-reliance on emotion judgments carries consequences.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the FAU predictor takes the model's 1x8 emotion vector as one of its inputs, the textual FAU list can encode the predicted emotion; participants may have decoded the label from the explanation, so the Hmp accuracy gain is not necessarily evidence of deeper visual understanding. A version of the FAU predictor trained without the emotion vector would settle this.
  • The appropriate-trust metric is symmetric under always agreeing and always disagreeing with one's own emotion guess on a 50/50 mix of correct and incorrect model predictions, so the score measures alignment with model correctness relative to that design, not absolute trust calibration; a participant who uniformly distrusts the model can score 50 percent without calibrating anything.
  • The paper notes that its static landmark contours omit the 'action' that the textual labels convey; animated FAU visualizations are a natural extension that might close the gap between visual and textual modalities.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper presents DEFAULTS, an explanation method for facial expression recognition (FER) that generates textual and visual explanations from predicted Facial Action Units (FAUs), and reports a between-subject user study (N=280) over seven conditions: control, LIME, SHAP, saliency maps, FAU-textual (FAU-T), FAU-visual (FAU-V), and FAU-visual-plus-textual (FAU-VT). The headline claims are that FAU-T and FAU-VT explanations significantly improve participants' ability to predict the FER model's outputs (Hmp accuracy; F(3,156)=15.41, p<0.001), with FAU-VT also beating FAU-V, and that all FAU-based modalities yield significantly higher 'appropriate trust' than control (p<0.0001), while LIME, SHAP, and saliency maps show no significant benefit. The paper interprets these results as evidence that FAU-grounded multimodal explanations improve user understanding and trust calibration. The central problem is that the FAU predictor (Section 3.1) receives the FER model's 1x8 emotion prediction vector as a concatenated input, so the textual FAU list can encode the model's predicted emotion; because the training phase explicitly paired each FAU explanation with the model's prediction, participants in the textual conditions may be decoding the model's output rather than demonstrating understanding.

Significance. The contribution is potentially valuable: DEFAULTS grounds explanations in FACS-based facial action units, compares an explanation method across three XAI families in a pre-registered, a-priori powered between-subject design, and releases a participant dataset. The paper is also well organized around two explicit research questions and includes a balance check showing no group differences in human emotion recognition. If the reported effects were causally attributable to genuine understanding of the FER model, the finding that textual, and combined textual-visual, FAU explanations outperform visual-only and standard XAI methods would be an important result for the affective-computing and XAI communities. However, the empirical claims rest on Hmp accuracy and on an 'appropriate trust' metric defined entirely through Hmp; both are directly contaminated by the label-leakage confound described in the summary. The observed pattern—textual conditions improve Hmp while visual-only FAU-V does not—is precisely what a label-decoding account would predict.

major comments (3)
  1. [Sections 3.1, 3.4, and 4.1] The load-bearing flaw is the construction of the FAU explanation pipeline. In Section 3.1, the DNN used to predict FAUs is described as taking 'a concatenation of the final convolution layer of the CNN model (a 1x4032 Fully Connected node) and the 1x8 emotion prediction vector from the CNN' as input. The textual FAU list shown to participants is therefore a function of the model's predicted emotion, not an independent description of the face. In the training phase (Section 3.4), FAU-cohort participants were shown explanations together with the model's prediction for one correct and one incorrect example per emotion, which teaches the mapping from FAU text to model output. In the test phase, Question 2 (Hmp) asked participants to guess the model's prediction, and FAU-T and FAU-VT participants could answer by decoding the emotion implied by the FAU list (e.g., 'Nose Wrinkled' mapped to disgust). The reported Hmp results (F(3,156)=15.41, p<0.001; FAU-T vs. control p=0.0002; FAU-VT vs. control p<0.0001; FAU-VT vs. FAU-V p=0.0001) are exactly the pattern a label-decoding account predicts: the textual conditions improve Hmp accuracy while the visual-only FAU-V condition does not. Since Appropriate Trust (Section 3.5) is defined through agreement or disagreement between Hgtp and Hmp, the appropriate-trust effects (F(3,156)=17.26, p<0.0001 and F(4,195)=11.65, p<0.0001) are confounded in the same way. The central claim that FAU-based explanations improve user understanding and trust calibration therefore does not follow from the data; supporting it would require generating FAU explanations from a predictor that does not condition on the emotion output, or demonstrating empirically that the FAU output carries no information about the model's prediction beyond the image, and then re-running the user study.
  2. [Section 4] No means, standard deviations, confidence intervals, or effect sizes are reported for any dependent variable. Sections 4.1 and 4.2 report only F statistics, p-values, and Tukey HSD p-values (e.g., 'F(3,156)=15.41, p<0.001'; 'F(4,195)=11.65, p<0.0001'), and Section 4.1 describes a result as showing 'large significance,' which is not a meaningful property of a p-value. Without group means and effect sizes (e.g., partial eta-squared), readers cannot assess the practical magnitude of the FAU advantages, and the observed effects cannot be compared with the medium-effect-size assumption used in the G*Power sample-size calculation (Section 3.6). The boxplots are helpful but do not substitute for numeric reporting of these headline quantities.
  3. [Section 3.5] The 'Appropriate Trust' metric is defined purely by whether Hgtp and Hmp coincide on correct (GT=MP) images and differ on incorrect (GT≠MP) images; it does not measure whether users' reliance on the model tracks the model's actual competence. A participant who systematically answers Hmp differently from Hgtp receives credit on all incorrect images, and one who answers Hmp=Hgtp receives credit on all correct images, so the metric rewards these response biases even in the absence of any information leak. In addition, because Hmp is the single confounded quantity identified in Major Comment 1, the appropriate-trust comparisons do not constitute a valid test of trust calibration. The authors should justify this metric against standard calibration measures and, at minimum, report the Hgtp-Hmp agreement rates separately for correct and incorrect trials, along with any analysis that does not depend on Hmp.
minor comments (7)
  1. [Sections 3.3, 3.5, and 5.1] The hypothesis numbering is inconsistent: Section 3.3 defines H1.1, H1.2, H2.1, and H2.2, but Section 3.5 refers to 'RQ1 H2.1 and H2.2' and 'RQ2 H3.1 and H3.2,' and Section 5.1 says 'This results partially support H2.2' where the relevant hypothesis is H1.2.
  2. [Section 1] There is a typo in the introduction: 'as well assystem understanding' should read 'as well as system understanding.'
  3. [Section 3.4] The exclusion criterion is written as a conjunction: participants were removed only if they completed the survey in less than half of the median time and failed both attention checks. As written, slow participants who failed both attention checks are retained; typically, failing both attention checks should trigger exclusion regardless of completion time, and the authors should clarify and justify the stated conjunction.
  4. [Section 3.1] The FAU predictor was trained on CK+ (posed expressions) but applied to Aff-Wild2 test images (spontaneous, in-the-wild expressions); the paper does not discuss this domain shift or report FAU prediction quality on the test distribution, which matters for judging whether the explanations themselves are veridical.
  5. [Section 3.1] The FAU predictor's accuracy is reported as a 'minimum accuracy of 96.33%' across cross-validation folds; please report the mean and standard deviation across folds instead of only the minimum.
  6. [Section 3.4] The model outputs eight emotion classes including contempt, but participants choose among only seven emotions; the paper does not state what happens on test images for which the model predicts contempt, nor how many such images exist, so the reader cannot assess the resulting noise in Hmp accuracy.
  7. [Section 3.3] The pre-registration links (osf.io/2ez64 and osf.io/35upf) appear as bare URLs in the text; they should be cited with a formal reference including the registration date, and the manuscript should state which reported analyses are confirmatory versus exploratory relative to the pre-registration.

Circularity Check

1 steps flagged · score 8.0 of 10

FAU explanations are generated from the FER model's emotion prediction vector, so textual FAU explanations can encode the target label; Hmp accuracy and appropriate trust advantages are partly forced by construction.

  1. self definitional [Section 3.1 (FER Model & Explanation Methods) and Section 3.5 (Evaluation Metrics)]
    "It used a concatenation of the final convolution layer of the CNN model (a 1x4032 Fully Connected node) and the 1x8 emotion prediction vector from the CNN to predict the FAUs present in an image. [...] Human Model Prediction (Hmp) Accuracy : The number of times that participants answered Question #2 from the Survey Questions section in Figure 1 correctly out of the total number of questions. This measured a participant’s ability to guess the model’s prediction."

    The DNN that produces the FAU explanations takes the CNN's 1x8 emotion prediction vector as a direct input, so the textual FAU list is a function of the model's predicted emotion. Hmp accuracy is defined as the participant's ability to guess exactly that model prediction, and appropriate trust is defined via agreement between Hgtp and Hmp. Because the training phase explicitly pairs FAU explanations with correct and incorrect model predictions for each emotion, FAU-T and FAU-VT participants can decode the predicted emotion from the explanation rather than infer it from understanding the model's decision boundary. LIME, SHAP, saliency, and control participants have no comparable code.

full rationale

The central causal claim—that FAU-based textual and visual-textual explanations improve users' understanding and appropriate trust—depends on the explanation being an independent window into the FER model's behavior. Section 3.1 breaks this assumption: the DNN FAU predictor receives a concatenation of the CNN's final convolution layer and its 1x8 emotion prediction vector as input. The FAU outputs are therefore not causally independent of the emotion label; a textual FAU list can carry information about which emotion the model will output. The training phase reinforces this mapping by explicitly pairing each FAU explanation with the model's prediction for one correct and one incorrect example per emotion. In the test phase, FAU-T and FAU-VT participants can answer the Hmp question by decoding the implied emotion from the FAU text rather than by understanding the model's decision boundary. Because Hmp accuracy is the basis for RQ1, and appropriate trust is defined through agreement between Hgtp and Hmp, both headline effects are confounded; the significant Hmp advantage of FAU-T/FAU-VT over control and FAU-VT over FAU-V is exactly what a label-leakage account predicts. No other circularity was found: the authors' prior work [33] is used only to motivate the gap and is not load-bearing, and the comparison with LIME, SHAP, and saliency maps is otherwise an independent empirical study. However, because the main positive results reduce in part to the construction of the explanation from the target label, the circularity score is high.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No ad hoc numerical parameters were fitted to the user-study data; the central claim rests on the pretrained FER and FAU models, the external landmark mapping, and the assumption that participant responses reflect understanding rather than label leakage. The most fragile assumption is that the FAU explanation does not encode the model's emotion prediction, which is contradicted by the architecture in Section 3.1.

assumptions (4)
  • domain assumption The Kim and Kim CNN trained on CK+ is an adequate FER model for the study, and the DNN FAU predictor's outputs are valid explanations of that model.
    The study uses this pretrained model as the common reference; but the DNN FAU predictor takes the emotion prediction vector as input, so this assumption is questionable.
  • domain assumption The FAU-to-landmark mapping from Perveen and Mohan produces visual explanations that are faithful to the model's reasoning.
    The visual contours are derived from an external landmark model, not from the FER model's internal features.
  • ad hoc to paper Participants' answers to Question 2 (Hmp) measure understanding of the model rather than information leaked by the explanation.
    The study's inference from Hmp accuracy to understanding depends on this; the leak confound violates it.
  • domain assumption Aff-Wild2 ground truth labels are correct for computing GT/MP agreement.
    Standard dataset assumption; the 50/50 correct/incorrect split is based on these labels.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Explaining Facial Expression Recognition." pith.science (2026). https://pith.science/paper/PQTAAXJT

@misc{pith2026250115864,
  author       = {Pith},
  title        = {Pith review of: Explaining Facial Expression Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PQTAAXJT}},
  note         = {Machine review of arXiv:2501.15864}
}
read the original abstract

Facial expression recognition (FER) has emerged as a promising approach to the development of emotion-aware intelligent agents and systems. However, key challenges remain in utilizing FER in real-world contexts, including ensuring user understanding and establishing a suitable level of user trust. We developed a novel explanation method utilizing Facial Action Units (FAUs) to explain the output of a FER model through both textual and visual modalities. We conducted an empirical user study evaluating user understanding and trust, comparing our approach to state-of-the-art eXplainable AI (XAI) methods. Our results indicate that visual AND textual as well as textual-only FAU-based explanations resulted in better user understanding of the FER model. We also show that all modalities of FAU-based methods improved appropriate trust of the users towards the FER model.

Figures

Figures reproduced from arXiv: 2501.15864 by the authors.

Figure 1
Figure 1. An Overview Figure. (From top) Visualisation of [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Standardized masked image explanations generated [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Process Flow of Experiment Procedure. truth annotation label from the Aff-Wild2 dataset and “MP” represents the model prediction of our FER model. Trust is considered appropriate when either: (a) For the correct test images (where GT is the same as MP): Participant answers for Question #1 (Hgtp) and Question #2 (Hmp) from the Survey Questions section in [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Example images shown for the LIME cohort during [PITH_FULL_IMAGE:figures/full_fig_p005_3.png]
Figure 1
Figure 1. Figure 1: This shows that participants in this study were similar in [PITH_FULL_IMAGE:figures/full_fig_p006_1.png]
Figure 5
Figure 5. Figure 5: Hmp Accuracy between groups with different XAI [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Hmp Accuracy between groups with different XAI [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Appropriate Trust between groups for Method Type [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 36 canonical work pages

  1. [1]

    Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y Lim, and Mohan Kankanhalli. 2018. Trends and trajectories for explainable, accountable and intelligible systems: An hci research agenda. In Proceedings of the 2018 CHI conference on human factors in computing systems . 1–18

    work page 2018

  2. [2]

    Bader Aldughayfiq, Farzeen Ashfaq, NZ Jhanjhi, and Mamoona Humayun. 2023. Explainable AI for retinoblastoma diagnosis: interpreting deep learning models with LIME and SHAP. Diagnostics 13, 11 (2023), 1932

    work page 2023

  3. [3]

    Tadas Baltrusaitis, Amir Zadeh, Yao Chong Lim, and Louis-Philippe Morency

  4. [4]

    Lisa Feldman Barrett, Ralph Adolphs, Stacy Marsella, Aleix M Martinez, and Seth D Pollak. 2019. Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological science in the public interest 20, 1 (2019), 1–68

    work page 2019

  5. [5]

    Giovanna Castellano, Berardina De Carolis, and Nicola Macchiarulo. 2023. Automatic facial emotion recognition at the COVID-19 pandemic time. Multimedia Tools and Applications 82, 9 (2023), 12751–12769

    work page 2023

  6. [6]

    Arun Das and Paul Rad. 2020. Opportunities and challenges in explainable artificial intelligence (xai): A survey. arXiv preprint arXiv:2006.11371 (2020)

    arXiv 2020

  7. [7]

    Guillermo del Castillo Torres, Maria Francesca Roig-Maimó, Miquel Mascaró-Oliver, Esperança Amengual- Alcover, and Ramon Mas-Sansó. 2022. Understanding How CNNs Recognize Facial Expressions: A Case Study with LIME and CEM. Sensors 23, 1 (2022), 131

    work page 2022

  8. [8]

    Laurence Devillers. 2021. Human–robot interactions and affective computing: The ethical implications. Robotics, AI, and Humanity: Science, Ethics, and Policy (2021), 205–211

    work page 2021

Show all 54 references
  1. [9]

    Laurence Devillers and Roddy Cowie. 2023. Ethical considerations on affective computing: an overview. Proc. IEEE (2023)

    2023

  2. [10]

    Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017)

    2017 arXiv

  3. [11]

    Benjamin D Douglas, Patrick J Ewell, and Markus Brauer. 2023. Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA. Plos one 18, 3 (2023), e0279720

    2023

  4. [12]

    Lilian Edwards. 2021. The EU AI Act: a summary of its significance and scope. Artificial Intelligence (the EU AI Act) 1 (2021)

    2021

  5. [13]

    Paul Ekman and Wallace V Friesen. 1978. Facial action coding system. Environmental Psychology & Nonverbal Behavior (1978)

    1978

  6. [14]

    Dan-Mikael Ellingsen, Andrea Duggento, Kylie Isenburg, Changjin Jung, Jeungchan Lee, Jessica Gerber, Ishtiaq Mawla, Roberta Sclocco, Robert R Edwards, John M Kelley, et al . 2022. Patient–clinician brain concordance underlies causal dynamics in nonverbal communication and nega...

    2022

  7. [15]

    Peer Eyal, Rothschild David, Gordon Andrew, Evernden Zak, and Damer Ekaterina. 2021. Data quality of platforms and panels for online behavioral research. Behavior research methods (2021), 1–20

    2021

  8. [16]

    Prashant Gohel, Priyanka Singh, and Manoranjan Mohanty. 2021. Explainable AI: current status and future directions. arXiv preprint arXiv:2107.07045 (2021)

    2021 arXiv

  9. [17]

    Mara Graziani, Lidia Dutkiewicz, Davide Calvaresi, José Pereira Amorim, Katerina Yordanova, Mor Vered, Rahul Nair, Pedro Henriques Abreu, Tobias Blanke, Valeria Pulignano, et al . 2023. A global taxonomy of interpretable AI: unifying the terminology for the technical and socia...

    2023

  10. [18]

    Muhammad Anas Hasnul, Nor Azlina Ab Aziz, Salem Alelyani, Mohamed Mohana, and Azlan Abd Aziz. 2021. Electrocardiogram-based emotion recognition systems and their applications in healthcare—A review. Sensors 21, 15 (2021), 5015

    2021

  11. [19]

    Robert R Hoffman, Shane T Mueller, Gary Klein, and Jordan Litman. 2018. Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:1812.04608 (2018)

    2018 arXiv

  12. [20]

    Maryam Imani and Gholam Ali Montazer. 2019. A survey of emotion recognition methods with emphasis on E-Learning environments. Journal of Network and Computer Applications 147 (2019), 102423

    2019

  13. [21]

    Sunbin Kim and Hyeoncheol Kim. 2019. Deep explanation model for facial expression recognition through facial action coding unit. In 2019 IEEE International Conference on Big Data and Smart Computing (BigComp) . IEEE, 1–4

    2019

  14. [22]

    Abhiram Kolli, Alireza Fasih, Fadi Al Machot, and Kyandoghere Kyamakya

  15. [23]

    Dimitrios Kollias and Stefanos Zafeiriou. 2018. Aff-wild2: Extending the aff-wild database for affect recognition. arXiv preprint arXiv:1811.07770 (2018)

    2018 arXiv

  16. [24]

    Louisa Kulke, Dennis Feyerabend, and Annekathrin Schacht. 2020. A comparison of the Affectiva iMotions Facial Expression Analysis Software with EMG for identifying facial expressions of emotion. Frontiers in Psychology 11 (2020), 329

    2020

  17. [25]

    Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. 2020. Explainable ai: A review of machine learning interpretability methods. Entropy 23, 1 (2020), 18

    2020

  18. [26]

    Zachary C Lipton. 2018. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue 16, 3 (2018), 31–57

    2018

  19. [27]

    Patrick Lucey, Jeffrey F Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. 2010. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In 2010 ieee computer society conference on computer vision and patte...

    2010

  20. [28]

    Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems 30 (2017)

    2017

  21. [29]

    Joao Marques-Silva and Alexey Ignatiev. 2022. Delivering trustworthy AI through formal XAI. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 12342–12350

    2022

  22. [30]

    Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence 267 (2019), 1–38

    2019

  23. [31]

    Tim Miller. 2022. Are we measuring trust correctly in explainability, interpretability, and transparency research? arXiv preprint arXiv:2209.00651 (2022)

    2022 arXiv

  24. [32]

    Gregory Mone. 2015. Sensing emotions. Commun. ACM 58, 9 (2015), 15–16

    2015

  25. [33]

    I think you need help! Here’s why

    Sanjeev Nahulanthran, Mor Vered, Leimin Tian, and Dana Kulić. 2024. "I think you need help! Here’s why": Understanding the Effect of Explanations on Automatic Facial Expression Recognition. In Proceedings of the 12th International Conference on Affective Computing & Intelligen...

    2024

  26. [34]

    Andrés Páez. 2019. The pragmatic turn in explainable artificial intelligence (XAI). Minds and Machines 29, 3 (2019), 441–459

    2019

  27. [35]

    Chandana Panati, Simon Wagner, and Stefan Brüggenwirth. 2022. Feature relevance evaluation using grad-CAM, LIME and SHAP for deep learning SAR data classification. In Proceedings of the 23rd International Radar Symposium (IRS) . IEEE, 457–462

    2022

  28. [36]

    Nazil Perveen and Chalavadi Krishna Mohan. 2020. Configural Representation of Facial Action Units for Spontaneous Facial Expression Recognition in the Wild.. In VISIGRAPP (4: VISAPP). 93–102

    2020

  29. [37]

    Rosalind W. Picard. 2003. Affective computing: challenges. International Journal of Human-Computer Studies 59, 1 (2003), 55–64. https://doi.org/10.1016/S1071- 5819(03)00052-1 Applications of Affective Computing in Human-Computer Interaction

    2003 doi

  30. [38]

    Manish Rathod, Chirag Dalvi, Kulveen Kaur, Shruti Patil, Shilpa Gite, Pooja Kamat, Ketan Kotecha, Ajith Abraham, and Lubna Abdelkareim Gabralla. 2022. Kids’ Emotion Recognition Using Various Deep-Learning Models with Explainable AI. Sensors 22, 20 (2022), 8066

    2022

  31. [39]

    Why should i trust you?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " Why should i trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining . 1135–1144

    2016

  32. [40]

    Elena Ryumina, Denis Dresvyanskiy, and Alexey Karpov. 2022. In search of a robust facial expressions recognition model: A large-scale visual cross-corpus study. Neurocomputing 514 (2022), 435–450

    2022

  33. [41]

    Muhammad Sajjad, Mansoor Nasir, Fath U Min Ullah, Khan Muhammad, Arun Kumar Sangaiah, and Sung Wook Baik. 2019. Raspberry Pi assisted facial expression recognition framework for smart security in law-enforcement services. Information Sciences 479 (2019), 416–431

    2019

  34. [42]

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision . 618–626

    2017

  35. [43]

    Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)

    2013 arXiv

  36. [44]

    Chamberlain, Jennifer Healey, and Akane Sano

    Leimin Tian, Sharon Oviatt, Michal Muszynski, Brent C. Chamberlain, Jennifer Healey, and Akane Sano. 2022. Applied Affective Computing (1 ed.). Vol. 41. Association for Computing Machinery, New York, NY, USA

    2022

  37. [45]

    Jasper van der Waa, Tjeerd Schoonderwoerd, Jurriaan van Diggelen, and Mark Neerincx. 2020. Interpretable confidence measures for decision support systems. International Journal of Human-Computer Studies 144 (2020), 102493

    2020

  38. [46]

    Mor Vered, Piers Howe, Tim Miller, Liz Sonenberg, and Eduardo Velloso. 2020. Demand-driven transparency for monitoring intelligent agents.IEEE Transactions on Human-Machine Systems 50, 3 (2020), 264–275

    2020

  39. [47]

    Mor Vered, Tali Livni, Piers Douglas Lionel Howe, Tim Miller, and Liz Sonenberg

  40. [48]

    Giulia Vilone and Luca Longo. 2020. Explainable artificial intelligence: a systematic review. arXiv preprint arXiv:2006.00093 (2020)

    2020 arXiv

  41. [49]

    Daniel S Weld and Gagan Bansal. 2019. The challenge of crafting intelligible intelligence. Commun. ACM 62, 6 (2019), 70–79

    2019

  42. [50]

    Jacob O Wobbrock. 2012. Seven research contributions in HCI. studies 1, 1 (2012), 52–80

    2012

  43. [51]

    Zhihong Zeng, Maja Pantic, Glenn I Roisman, and Thomas S Huang. 2007. A survey of affect recognition methods: audio, visual and spontaneous expressions. In Proceedings of the 9th international conference on Multimodal interfaces . 126– 133

    2007

  44. [2011]

    In Proceedings of the Joint INDS’11 & ISTET’11

    Non-intrusive car driver’s emotion recognition using thermal camera. In Proceedings of the Joint INDS’11 & ISTET’11 . IEEE, 1–5

  45. [2018]

    In2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018)

    Openface 2.0: Facial behavior analysis toolkit. In2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018) . IEEE, 59–66

    2018

  46. [2023]

    Artificial Intelligence 322 (2023), 103952

    The effects of explanations on automation bias. Artificial Intelligence 322 (2023), 103952

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.