Pith. sign in

REVIEW 3 major objections 5 minor 43 references

Do Users' Explainability Needs in Software Change with Mood?

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that users' need for software explanations is largely subjective: only emotional reactivity and age showed weak correlations, and only for user-interface explanations.

desk verdict Re-analysis of a published survey gives a mostly null result, but the paper's own Bonferroni rule makes its two headline correlations mutually inconsistent—a load-bearing error that a referee must catch. read the letter →

arxiv 2502.06546 v1 pith:7L6VQUTA submitted 2025-02-10 cs.SE

Martin Obaidi , Jakob Droste , Hannah Deters , Marc Herrmann , Jil Klünder , Kurt Schneider This is my paper · ORCID
classification cs.SE
keywords explainabilityexplanationneedsmoodanalysissentimentemotionalreactivitydemographicsuserinterfaceexplanationscorrelation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a software user's mood or demographic profile can tell you how many explanations they want and what kind. Analyzing survey data from 66 participants, it finds that the answer is mostly no: 26 of 28 tested relationships were not significant. The exceptions are both tied to user-interface explanations: people with more emotionally reactive moods reported more such needs ($r=0.33$, $p=0.007$), and older users reported fewer ($r=-0.25$, $p=0.044$). Stable mood (sentiment), gender, and all other explanation categories showed no reliable link. The authors conclude that explanation needs are highly subjective and depend only partially on objective factors, so software teams should elicit explainability requirements directly from users rather than infer them from profiles.

What carries the argument

The analysis is carried by two instruments joined by standard correlation tests. The first is the Mood Survey's two dimensions, sentiment (general positive or negative disposition) and reactivity (intensity and frequency of mood swings), measured on 7-point Likert scales. The second is the five-category taxonomy of explanation needs—interaction, system behavior, domain knowledge, privacy/security, and user interface—derived from the original survey. Relationships between continuous variables are tested with Pearson's $r$; gender differences are tested with the Mann-Whitney-U test; and multiplicity is controlled with Bonferroni correction.

What would settle it

A preregistered replication on a larger, stratified sample (e.g., 500+ users) that measures mood immediately before a standardized software task, then records which explanation categories users request, would settle the claim. If the reactivity-UI correlation ($r=0.33$) and age-UI correlation ($r=-0.25$) do not reproduce, the paper's positive results are not stable; if mood or demographic factors predict other explanation categories, the conclusion that explanation needs are largely unpredictable would need revision.

Watch

Extended reading notes

Core claim

On its own terms, this paper's discovery is mostly an absence: the expected systematic relationships between mood, demographics, and explanation needs do not materialize. Across 28 null hypotheses, only two subhypotheses showed significant correlations, both confined to the user-interface category of the explanation-need taxonomy. Emotional reactivity correlated positively with the need for UI explanations, and age correlated negatively with that same category; gender, sentiment, and overall counts of explanation needs showed no significant associations. From this the paper concludes that explanation need is very subjective and cannot be reliably predicted from mood or demographic data, and that adaptive explanation systems should be driven by directly gathered user-specific requirements rather than inferred personas.

Load-bearing premise

The load-bearing premise is that the 66 participants who completed the mood questionnaire represent software users generally; since the survey was spread through the authors' networks and a university message board, and 17 participants were dropped for incomplete mood data without a comparison to completers, a non-representative sample would change the pattern of correlations.

Editorial extensions

If this is right

  • Software companies cannot reliably build explainability features from age, gender, or mood profiles; user-specific elicitation remains necessary.
  • The only mood signal that mattered was reactivity, not stable sentiment, and only for UI explanations; adaptive systems would need real-time emotional state rather than static mood scores.
  • Age is a weak negative predictor of UI explanation needs, so younger users may need more interface guidance, but the signal is too small to drive policy alone.
  • Category-level analysis matters: total explanation-need counts hide the two correlations that appeared, so future studies should keep the taxonomy's categories separate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the null pattern generalizes, the productive next variables to test are likely cognitive style, domain knowledge, and task context rather than demographics or mood, since those vary more within a person than between profiles.
  • The age-UI correlation may reflect generational interface expectations, but this sample has few participants in the middle age bands; a cohort-balanced replication would tell whether the effect is truly about age.
  • The two significant findings both concern UI explanations, suggesting that interface changes and confusing layouts are the place where personalization could pay off first.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper reports a secondary analysis of an existing survey dataset from Droste et al. (RE'24) to investigate whether users' mood (sentiment and reactivity) and demographic factors (age, gender) correlate with their self-reported needs for explanations in software. The authors test 28 null hypotheses via Pearson correlations and Mann-Whitney-U tests, reporting two uncorrected significant correlations: reactivity with UI-related explanation needs (r=0.33, p=0.007) and age with UI-related explanation needs (r=-0.25, p=0.044). They conclude that explanation needs are highly subjective and only partially depend on objective factors, recommending that companies actively elicit user-specific explainability requirements. The paper's central statistical claims are, however, internally inconsistent: Section 4.3 states that no main null hypothesis survives Bonferroni correction, while the abstract and discussion present the two correlations as significant without qualification.

Significance. If the results were robust, the finding that mood and demographics are weak predictors of explanation needs would be a useful contribution to explainability requirements engineering, supporting calls for direct elicitation of user-specific explanation preferences. The paper's transparency—showing the full hypothesis table and correlation results, and discussing threats to validity—is a strength. However, the inconsistent application of multiple-testing correction undermines the specific empirical claims and the central message. The paper does not ship machine-checked proofs or reproducible code, but the statistical analyses are simple and re-computable from the reported numbers, which is a positive feature for verification.

major comments (3)
  1. [Section 3.4, Section 4.3, Table 5, Abstract] The Bonferroni correction is applied inconsistently, making the reported significance decisions self-contradictory. The paper states in Section 3.4 that the correction is applied per main hypothesis, so for H30 (age vs. explanation need) and H50 (reactivity vs. explanation need), each of which has six subhypotheses, the corrected threshold is α_corr = 0.05/6 ≈ 0.0083. Under this rule, the reactivity-UI correlation (p=0.007) is below the threshold and should lead to rejection of H50, whereas the age-UI correlation (p=0.044) is above the threshold and would not reject H30. Therefore, Section 4.3's statement that "none of the main null hypotheses could be rejected after applying the Bonferroni correction" is wrong under the paper's own rule, and the abstract's "Two significant correlations" is not derivable from any single consistent application of the stated procedure. Please specify the correction family, recompute all decisions, and revise the abstract, Section 4.3, Section 5, and the conclusion accordingly.
  2. [Section 3.4, Section 5.2] The Mood Survey measures general, enduring mood traits (sentiment and reactivity) as acknowledged in Section 2.2, not momentary mood states. Parts of the discussion, such as "situational emotional intensity may heighten users' demand for guidance" and "designing adaptive systems that respond dynamically to real-time emotional feedback, such as increased reactivity," treat reactivity as if it were a transient state. This overstates the applicability of the findings to mood-as-state and should be aligned with the trait-based nature of the instrument.
  3. [Section 3.3, Section 4.1, Section 5.4] The paper excludes 17 of 83 participants due to incomplete mood data but provides no comparison of included versus excluded participants. The claim in Section 4.2 that "we consider the sample is representative of the overall population in terms of expressed explanation needs" is based solely on similarity to the original Droste et al. sample, which is tautological because the original sample is the parent dataset. This weakens the external validity claim and makes it difficult to assess whether the observed correlations and null results are biased by selective attrition. Please report reasons for missingness and compare demographic and explanation-need distributions between completers and non-completers.
minor comments (5)
  1. [Section 3.3 vs. Section 6] The number of survey participants is reported as 83 in Section 3.3 but as 84 in Section 6; please reconcile this discrepancy.
  2. [Table 2] The scale for "NFE by category X" is listed as "Ordinal 1..5," but the variable represents a count of explanation needs within a category; clarify whether this is a count or an ordinal rating.
  3. [Table 5] Please state whether the reported p-values are one-tailed or two-tailed, and specify this in the table caption or methodology section.
  4. [Figure 1] The caption uses "explainability needs" while the text predominantly uses "explanation needs"; please use consistent terminology.
  5. [Section 3.4] When describing the Bonferroni correction, the text gives only the example with two subhypotheses; it would be helpful to state explicitly that for hypotheses H30 and H50, which have six subhypotheses, the corrected threshold is 0.0083.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: secondary correlational re-analysis; the conclusion that explanation needs are subjective is an interpretation of the observed correlations, not a quantity that reduces to its inputs.

full rationale

The paper performs correlation analyses on an existing survey dataset (Section 3.2), testing null hypotheses about mood, demographics, and explanation needs. The central claim—that only two correlations are significant and that explanation needs are largely subjective—is an interpretation of Table 5's observed p-values rather than a quantity derived from the inputs by construction. No fitted parameter is renamed as a prediction, and no hypothesis is defined in terms of the outcome it is said to explain. Several self-citations appear: the taxonomy and dataset come from Droste et al. [17,18], and Section 4.2 uses the same prior study to assert that the sample is representative of the broader population in terms of explanation needs. These self-citations support the measurement instrument and a generalizability claim, but they are not load-bearing for the correlational results, which are computed from participant responses rather than from the cited taxonomy. The paper's internal inconsistency about which correlations survive Bonferroni correction (Section 3.4's rule versus Table 5 and the abstract's wording) is a statistical reporting and correctness issue, not circular reasoning. No pattern from the enumerated circularity kinds is present; the score reflects only minor self-citation without load-bearing circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no free parameters and no invented entities. The central analysis depends on standard statistical assumptions and on the representativeness and validity of the previously collected survey data. The main burden lies on the reliability of the mood scales and the decision to report uncorrected correlations as findings.

assumptions (4)
  • domain assumption The adapted Mood Survey items from Underwood and Froming validly measure the two dimensions 'sentiment' and 'reactivity'.
    Section 2.2 and Table 1 treat these scales as measuring the intended constructs; the paper does not report reliability (e.g., Cronbach's alpha) in this dataset, and Section 5.4 acknowledges construct validity threats.
  • domain assumption Pearson correlation is appropriate for the variables analyzed (ordinal Likert scores, counts of explanation needs).
    Section 3.4 chooses Pearson for continuous variables; sentiment, reactivity, and category needs are ordinal scales, which violates the interval-level assumption without discussion.
  • domain assumption The 66 participants with complete mood data are representative of the broader survey sample and of software users.
    Section 3.2 describes recruitment through personal networks and a university board; Section 3.3 notes that 17 of 83 participants were excluded for incomplete mood data, and no comparison of completers vs. non-completers is provided.
  • ad hoc to paper An uncorrected per-test significance level of alpha=0.05 is the appropriate criterion for declaring the reported correlations significant.
    Section 3.4 sets alpha=0.05 and also applies Bonferroni correction, but the abstract and Section 5 rely on the uncorrected p-values (0.007 and 0.044) to claim significant findings, even though Section 4.3 says no main hypothesis survives the correction. This inconsistency makes the significance criterion an ad hoc choice that the central claims depend on.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do Users' Explainability Needs in Software Change with Mood?." pith.science (2026). https://pith.science/paper/7L6VQUTA

@misc{pith2026250206546,
  author       = {Pith},
  title        = {Pith review of: Do Users' Explainability Needs in Software Change with Mood?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7L6VQUTA}},
  note         = {Machine review of arXiv:2502.06546}
}
read the original abstract

Context and Motivation: The increasing complexity of modern software systems often challenges users' abilities to interact with them. Taking established quality attributes such as usability and transparency into account can mitigate this problem, but often do not suffice to completely solve it. Recently, explainability has emerged as essential non-functional requirement to help overcome the aforementioned difficulties. Question/problem: User preferences regarding the integration of explanations in software differ. Neither too few nor too many explanations are helpful. In this paper, we investigate the influence of a user's subjective mood and objective demographic aspects on explanation needs by means of frequency and type of explanation. Principal ideas/results: Our results reveal a limited relationship between these factors and explanation needs. Two significant correlations were identified: Emotional reactivity was positively correlated with the need for UI explanations, while a negative correlation was found between age and user interface needs. Contribution: As we only find very few significant aspects that influence the need for explanations, we conclude that the need for explanations is very subjective and does only partially depend on objective factors. These findings emphasize the necessity for software companies to actively gather user-specific explainability requirements to address diverse and context-dependent user demands. Nevertheless, future research should explore additional personal traits and cross-cultural factors to inform the development of adaptive, user-centered explanation systems.

Figures

Figures reproduced from arXiv: 2502.06546 by the authors.

Figure 1
Figure 1. Distribution of sentiment (blue) and reactivity (red) among study participants. Need for Explanations The sample of 66 participants reported an average of 4.6 explainability needs in their survey answers. The distribution of explainabil￾ity needs across the sample is displayed in [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 41 canonical work pages

  1. [1]

    In: SEAA (2019)

    Andrade, H., Lwakatare, L.E., Crnkovic, I., Bosch, J.: Software challenges in het- erogeneous computing: A multiple case study in industry. In: SEAA (2019)

    work page 2019

  2. [2]

    In: ESEC/FSE’20

    Antinyan, V.: Revealing the complexity of automotive software. In: ESEC/FSE’20. Association for Computing Machinery (2020)

    work page 2020

  3. [3]

    In: DLRS’18

    Bellini, V., Schiavone, A., Di Noia, T., Ragone, A., Di Sciascio, E.: Knowledge- aware autoencoders for explainable recommender systems. In: DLRS’18

    work page

  4. [4]

    Zusammenstellung sozialwis- senschaftlicher Items und Skalen (ZIS) (1997)

    Bohner, G., Schwarz, N.: Stimmungsskala. Zusammenstellung sozialwis- senschaftlicher Items und Skalen (ZIS) (1997)

    work page 1997

  5. [5]

    Diagnostica37(2) (1991)

    Bohner, G., Hormuth, S.E., Schwarz, N.: Die stimmungs-skala: Vorstellung und validierung einer deutschen version des „mood survey “. Diagnostica37(2) (1991)

    work page 1991

  6. [6]

    Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS) (2016)

    Breyer, B., Bluemke, M.: Deutsche version der positive and negative affect schedule panas (gesis panel). Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS) (2016)

    work page 2016

  7. [7]

    JSS195 (2023)

    Brunotte, W., Specht, A., Chazette, L., Schneider, K.: Privacy explanations–a means to end-user trust. JSS195 (2023)

    work page 2023

  8. [8]

    Chazette, L., Brunotte, W., Speith, T.: Exploring explainability: a definition, a model, and a knowledge catalogue. In: RE. IEEE (2021)

    work page 2021

Show all 43 references
  1. [9]

    In: RE (2022)

    Chazette, L., Klös, V., Herzog, F., Schneider, K.: Requirements on explanations: A quality framework for explainability. In: RE (2022)

    2022

  2. [10]

    REJ25(4) (2020)

    Chazette, L., Schneider, K.: Explainability as a non-functional requirement: chal- lenges and recommendations. REJ25(4) (2020)

    2020

  3. [11]

    Noise reduction in speech processing (2009)

    Cohen,I.,Huang,Y.,Chen,J.,Benesty,J.,Benesty,J.,Chen,J.,Huang,Y.,Cohen, I.: Pearson correlation coefficient. Noise reduction in speech processing (2009)

    2009

  4. [12]

    CoRRabs/2006.11371 (2020)

    Das, A., Rad, P.: Opportunities and challenges in explainable artificial intelligence (XAI):A survey. CoRRabs/2006.11371 (2020)

    2020 arXiv

  5. [13]

    Deters, H., Droste, J., Fechner, M., Klünder, J.: Explanations on demand-a tech- nique for eliciting the actual need for explanations. In: REW. IEEE (2023)

    2023

  6. [14]

    In: NordiCHI’24

    Deters, H., Droste, J., Hess, A., Klös, V., Schneider, K., Speith, T., Vogelsang, A.: The x factor: On the relationship between user experience and explainability. In: NordiCHI’24. Association for Computing Machinery

  7. [15]

    In: REFSQ’24

    Deters, H., Droste, J., Obaidi, M., Schneider, K.: How explainable is your system? towards a quality model for explainability. In: REFSQ’24

  8. [16]

    ACM Trans

    Dominguez, V., Donoso-Guzmán, I., Messina, P., Parra, D.: Algorithmic and hci aspects for explaining recommendations of artistic images. ACM Trans. Interact. Intell. Syst.10(4) (nov 2020)

    2020

  9. [17]

    In: RE’24

    Droste,J.,Deters,H.,Obaidi,M.,Schneider,K.:Explanationsineverydaysoftware systems: Towards a taxonomy for explainability needs. In: RE’24

  10. [18]

    Ex- planations in Everyday Software Systems: Towards a Taxonomy for Explainability Needs

    Droste, J., Deters, H., Obaidi, M., Schneider, K.: Supplementary Material - "Ex- planations in Everyday Software Systems: Towards a Taxonomy for Explainability Needs" (RE’24) (Mar 2024)

    2024

  11. [19]

    Droste, J., Deters, H., Puglisi, J., Klünder, J.: Designing end-user personas for explainability requirements using mixed methods research. In: REW. IEEE (2023)

    2023

  12. [20]

    In: Advances in Industrial Design

    Gabbas, M., Ryu, Y., Park, J., Kim, K.: Understanding challenges of designing for complex users by adapting the existing framework. In: Advances in Industrial Design. Springer International Publishing (2021)

    2021

  13. [21]

    Springer, New York, NY, USA (2013)

    Haynes, W.: Bonferroni Correction. Springer, New York, NY, USA (2013)

    2013

  14. [22]

    John Wiley & Sons, Inc., Hoboken, NJ (2013) 16 Obaidi et al

    Hosmer, D.W., Lemeshow, S., Sturdivant, R.X.: Applied Logistic Regression. John Wiley & Sons, Inc., Hoboken, NJ (2013) 16 Obaidi et al

    2013

  15. [23]

    Kim, G., Yeo, D., Jo, T., Rus, D., Kim, S.: What and when to explain? on-road evaluation of explanations in highly automated vehicles. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.7(3) (sep 2023)

    2023

  16. [24]

    In: CHI’16

    Kizilcec, R.: How much information? effects of transparency on trust in an algo- rithmic interface. In: CHI’16. Association for Computing Machinery

  17. [25]

    International Journal of Human–Computer Interaction34(2), 177–186 (2018)

    Kortum, P., Oswald, F.L.: The impact of personality on the subjective assessment of usability. International Journal of Human–Computer Interaction34(2), 177–186 (2018)

    2018

  18. [26]

    In: RE’19

    Köhl, M.A., Baum, K., Langer, M., Oster, D., Speith, T., Bohlender, D.: Explain- ability as a non-functional requirement. In: RE’19

  19. [27]

    Empirical Software Engineering26(1) (2021)

    Levy, O., Feitelson, D.: Understanding large-scale software systems – structure and flows. Empirical Software Engineering26(1) (2021)

    2021

  20. [28]

    Computer45(08) (aug 2012)

    Mens, T.: On the complexity of software systems. Computer45(08) (aug 2012)

    2012

  21. [29]

    In: IUI’16

    Muhammad, K.I., Lawlor, A., Smyth, B.: A live-user study of opinionated explana- tions for recommender systems. In: IUI’16. Association for Computing Machinery

  22. [30]

    User Modeling and User-Adapted In- teraction 27 (2017)

    Nunes, I., Jannach, D.: A systematic review and taxonomy of explanations in decision support and recommender systems. User Modeling and User-Adapted In- teraction 27 (2017)

    2017

  23. [31]

    Obaidi, M., Fischbach, J., Herrmann, M., Deters, H., Droste, J., Klünder, J., Schneider, K.: How does users’ app knowledge influence the preferred level of detail and format of software explanations? In: REFSQ’25 (2025)

    2025

  24. [32]

    ICSE-SEIP’25 (2025)

    Obaidi,M.,Voß,N.,Droste, J.,Deters,H.,Herrmann, M.,Fischbach,J.,Schneider, K.: Automating explanation need management in app reviews: A case study from the navigation app industry. ICSE-SEIP’25 (2025)

    2025

  25. [33]

    IEEE Transactions on Artificial Intelligence 4(4) (2023)

    Oberste, L., Heinzl, A.: User-centric explainability in healthcare: A knowledge- level perspective of informed machine learning. IEEE Transactions on Artificial Intelligence 4(4) (2023)

    2023

  26. [34]

    In: HCI-COLLAB’21

    Ramos, H., Fonseca, M., Ponciano, L.: Modeling and evaluating personas with software explainability requirements. In: HCI-COLLAB’21. Springer

  27. [35]

    In: UMAP ’24

    Sadeghi, M., Pöttgen, D., Ebel, P., Vogelsang, A.: Explaining the unexplainable: The impact of misleading explanations on trust in unreliable predictions for hardly assessable tasks. In: UMAP ’24. Association for Computing Machinery

  28. [36]

    Journal of personality and social psy- chology 52 6(1987)

    Shaver, P.R., Schwartz, J.C., Kirson, D., O’Connor, C.: Emotion knowledge: fur- ther exploration of a prototype approach. Journal of personality and social psy- chology 52 6(1987)

    1987

  29. [37]

    In: UMAP’19

    Tran, T.N.T., Atas, M., Felfernig, A., Le, V.M., Samer, R., Stettinger, M.: Towards social choice-based explanations in group recommender systems. In: UMAP’19

  30. [38]

    Journal of Personality Assessment44(4) (1980)

    Underwood, B., Froming, W.J.: The mood survey: A personality measure of happy and sad moods. Journal of Personality Assessment44(4) (1980)

    1980

  31. [39]

    Unterbusch,M.,Sadeghi,M.,Fischbach,J.,Obaidi,M.,Vogelsang,A.:Explanation needs in app reviews: Taxonomy and automated detection. In: REW. IEEE (2023)

    2023

  32. [40]

    In: ICDM (2018)

    Wang, X., Chen, Y., Yang, J., Wu, L., Wu, Z., Xie, X.: A reinforcement learning framework for explainable recommendation. In: ICDM (2018)

    2018

  33. [41]

    Journal of personality and social psychology54(6) (1988)

    Watson, D., Clark, L.A., Tellegen, A.: Development and validation of brief mea- sures of positive and negative affect: the panas scales. Journal of personality and social psychology54(6) (1988)

    1988

  34. [42]

    Springer (2012)

    Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Ex- perimentation in software engineering. Springer (2012)

    2012

  35. [43]

    In: SIGIR’19

    Xu, L., Zhou, X., Gadiraju, U.: Revealing the role of user moods in struggling search tasks. In: SIGIR’19. Association for Computing Machinery

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.