{"id":"334da6e7-4659-4dad-87a1-eea6a933a368","arxiv_id":"2412.15834","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Anger-themed animated chat balloons from an AI assistant increased reversal of decision certainty in moral dilemmas, whereas sadness cues did not.","lead":"This study tested whether animated chat balloons showing anger or sadness from an AI assistant could change how certain people feel about a moral decision. The authors found that anger cues significantly increased shifts away from people's original choices, while sadness cues did not differ from a no-cue baseline.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"RCS sign convention in Eq. 1 is internally inconsistent: the worked example maps an 'Acceptable' first choice to x1=-3, and the formula's sign does not distinguish reversal from reinforcement; the reported anger effect could reflect reinforcement, so the central claim is not yet supported.","rationale":"The reader's weakest assumption is the same as the most load-bearing concern here: the definition and interpretation of RCS. The paper's main quantitative result, that anger cues increase RCS relative to baseline and sadness cues, is only meaningful if a positive RCS actually corresponds to a reversal of the initial moral decision. The formula and worked example are internally inconsistent, so the direction of the reported effect is indeterminate from the manuscript alone. This is not an external disagreement with consensus; it is an internal inconsistency in the dependent measure. The study has clear strengths: the AniBalloons stimuli are validated in prior work, the verbal responses are held constant across conditions, and the materials are described in enough detail for replication. But those strengths do not resolve the sign problem. A simple reanalysis of the raw data, applying the sign convention stated in prose, would settle the issue. If the anger effect persists as a reversal under that convention, the claim stands; if not, the central conclusion is inverted. The secondary issue of excluding five participants who failed the manipulation check (Section 4.1) is worth reporting but is less fundamental than the RCS direction. Accordingly, the verdict should remain CONDITIONAL: acceptance should require the authors to clarify the scale, correct the example, and show that the AC vs baseline difference remains a significant reversal under a consistent sign convention.","tokens_in":15896,"tokens_out":5942,"duration_ms":50821,"concrete_test":"Request the raw trial-level data (n, x1, x2 per participant per scenario) and recompute RCS under both possible sign conventions: (a) the formula as printed, and (b) the convention implied by the text that RCS > 0 means the second-round rating moved to the opposite side of the initial decision. Re-run the ANCOVA from Section 4.2 for each convention. If the AC vs baseline contrast is no longer significantly positive under convention (b), the central claim of anger-induced reversal is unsupported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim in Section 4.2 is that anger cues significantly increase Reversal Certainty Shift (RCS). This depends entirely on the sign of RCS in Eq. 1: RCS = (-1)^n (x2 - x1), with n=0 for an initial 'Acceptable' choice. The paper states RCS > 0 means certainty 'shifted toward reversing the first-round decision.' However, the worked example in Eq. 2 sets n=0, x1=-3, x2=-1, and calls RCS=2 a 'reversal.' If x is a standard acceptability rating (-3 = completely unacceptable, +3 = completely acceptable), then x1=-3 contradicts an initial 'Acceptable' choice. If x is a certainty rating (-3 to +3), then moving from -3 to -1 is an increase in certainty, which is reinforcement of the first choice, not reversal. In either reading, the example does not support RCS > 0 as reversal. More generally, for n=0 the formula leaves the sign of (x2-x1) unflipped, so any increase in the rated value (x2 > x1) yields positive RCS; but if the first choice was 'Acceptable' and the scale is anchored accordingly, an increase in that value moves toward the first choice, i.e., reinforcement. The reported AC mean RCS=0.811 vs baseline 0.343 would then indicate that anger increases reinforcement, not reversal, flipping the paper's theoretical interpretation. The post-hoc exclusion of five participants (Section 4.1) is a secondary concern; it does not need to be resolved if the RCS direction is undefined.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript reports an online experiment with 147 completions (142 after exclusions) in which participants faced moral dilemmas and received an AI assistant's opposing verbal response with either no nonverbal cue, an anger AniBalloon cue, or a sadness AniBalloon cue. The central outcome is Reversal Certainty Shift (RCS), defined in Eq. (1), which the authors interpret as movement toward reversing the first-round decision. The headline claim is that anger cues significantly increase RCS relative to both the baseline and sadness cues (Section 4.2: AC M=0.811 vs. baseline M=0.343, p<0.01; AC vs. SC M=0.453, p<0.05). Secondary claims concern gender differences in perceived AI influence on moral thinking. The paper concludes that subtle nonverbal emotional cues can shape moral decision certainty and discusses design and ethical implications.","tokens_in":16223,"tokens_out":4037,"duration_ms":37235,"significance":"If the central result holds, the paper is a worthwhile empirical contribution to human-AI interaction and affective computing: it demonstrates that a lightweight, language-independent nonverbal manipulation can shift decision certainty in moral dilemmas, and the gender interaction on perceived influence is thought-provoking. The study has clear strengths: the AniBalloons stimuli were validated in prior work, the verbal responses were held constant across conditions, the manipulation check indicates that the intended emotions were perceived, and the limitations section is candid. However, the bespoke RCS measure is the linchpin of the main claim, and its sign convention is internally inconsistent as written. Until that inconsistency is resolved and the analysis is re-expressed with an unambiguous directional measure, the headline result cannot be interpreted as evidence of reversal rather than reinforcement. The secondary findings about perceived influence are less affected, but they also need small clarifications in the statistical reporting.","major_comments":[{"comment":"The sign convention for RCS is internally inconsistent, and this is load-bearing for the paper's central claim. Equation (1) defines RCS = (-1)^n (x2 - x1) with n=0 for an initial 'Acceptable' decision, so for n=0 any increase in the rated value (x2 > x1) yields positive RCS. The interpretation text states that RCS > 0 means certainty shifted toward reversing the first-round decision, but the worked example in Eq. (2) uses n=0, x1=-3, x2=-1 and calls RCS=2 a shift 'away from the first-round decision.' If x is an acceptability scale anchored at -3 = completely unacceptable and +3 = completely acceptable, then x1=-3 contradicts an initial 'Acceptable' choice. If x is instead a certainty scale, moving from -3 to -1 is an increase in certainty that is consistent with reinforcing the initial 'Acceptable' choice, not reversing it. Under either reading, the example does not establish that positive RCS means reversal. The reported AC mean of 0.811 in Section 4.2 could therefore reflect reinforcement rather than reversal, which would invert the theoretical interpretation. Please specify the exact rating question and its anchors, correct the worked example, and re-analyze the data with a signed measure that unambiguously codes movement toward the opposite decision.","section":"Section 3.3.2, Eqs. (1)-(2)"},{"comment":"The exclusion of five participants is reported only by participant ID, without stating how the exclusions were distributed across the three conditions, whether the exclusion criterion was pre-specified, or whether the main conclusions survive when these participants are retained. Excluding participants who fail a manipulation check is defensible in principle, but if the exclusions fall unevenly across conditions, the group comparisons in Section 4.2 could be biased. Please report the per-condition exclusion counts and provide a sensitivity analysis that includes the excluded participants.","section":"Section 4.1"},{"comment":"The statistical reporting for the RCS analysis needs clarification. The values labeled SD (e.g., AC SD=0.123 on a -3 to +3 scale) are implausibly small for raw standard deviations given the sample size, which suggests they are standard errors or estimated marginal means; Figure 4's error bars are described as 95% confidence intervals. Relatedly, the scenario main effect is reported as F(3,396), which implies a repeated-measures structure, but the paper does not state whether RCS was pooled across the four scenarios for each participant or modeled with participant as a random effect. Please clarify the units, the error-bar definition, and the exact mixed-model specification used to obtain the reported F statistics and p-values.","section":"Section 4.2"}],"minor_comments":[{"comment":"The scale is described as 'ranging from strongly disagree (3) to strongly agree (3)'; this should presumably read -3 to 3.","section":"Section 3.3.2, Manipulation Check"},{"comment":"The interaction between gender and nonverbal emotional cues is reported as F(3,128)=5.522; for a two-way ANCOVA with three gender categories and three cue conditions, the interaction degrees of freedom do not match this value. Please verify and correct the reported degrees of freedom.","section":"Section 4.3.1"},{"comment":"The text says 'we chose two scenarios' but then lists four scenario versions; this wording should be adjusted to clarify that there are two dilemma stories, each with a proscriptive-norm version and a prescriptive-norm version.","section":"Section 3.2"},{"comment":"The discussion states that 'the reversal effect of AC is the same for participants of both genders,' but Section 4.2 does not report a gender-by-cue interaction on RCS. Either report that analysis or soften the claim to avoid overstating what the data show.","section":"Section 5.2"},{"comment":"The figures show significance stars and error bars, but the caption does not state whether the plotted values are estimated marginal means and whether error bars are standard errors or confidence intervals; adding this information would improve interpretability.","section":"Figure 4 and Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the RCS sign convention. I do not think this requires rejection, because the authors can likely clarify the rating scale and re-express the analysis, but the current manuscript does not support the reversal interpretation as written. The manipulation-check exclusion and the mixed-model reporting are secondary but should be addressed in the same revision. The paper is otherwise within scope for an HCI venue and the general idea is sound."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a tidy, well-motivated study of whether animated chat balloons expressing anger or sadness shift people's certainty in moral dilemma decisions. The headline result—anger cues move decision certainty more than sadness or no cue—is plausible, but it currently rests on a dependent variable whose sign convention and scale anchors the paper never actually pin down.\n\nWhat is genuinely good: the experimental control is unusually careful. The AI's verbal responses are identical across all three conditions; only the AniBalloons animation differs. The manipulation check confirms participants perceived the intended emotions, and prior AniBalloons work provides external validation of the stimuli. The scenarios come from the CNI-model corpus, so the stimulus base is defensible and reproducible. The gender-by-cue interaction on perceived AI influence is a genuinely interesting secondary finding, and the discussion of gender-emotion stereotypes is measured rather than speculative. The paper also owns its limitations.\n\nThe soft spot is load-bearing. RCS is defined in Eq. 1 as (-1)^n (x2 - x1), with the claim that positive RCS means certainty moved toward reversing the first-round decision. But the scale anchors for x are never stated, and the worked example in Eq. 2 sets n=0 ('Acceptable' first choice), x1=-3, x2=-1, calling the resulting +2 a reversal. On a conventional acceptability scale, x1=-3 contradicts an 'Acceptable' first choice; on a pure confidence scale, moving from -3 to -1 is increased certainty—reinforcement, not reversal. There is one reading that saves the formula: if the scale deliberately runs -3 = certain the action is acceptable to +3 = certain it is not, the sign convention works for both n=0 and n=1. But that anchoring is never stated, and the paper's language ('certainty shifts from -3 to -1') reads like a confidence scale. Until the authors provide the exact question and anchors for x, the direction of the main effect is unverifiable: AC > baseline could mean anger increases reversal, or it could mean anger increases reinforcement of one's initial stance. The title and the 'open-mindedness' framing hang on that.\n\nSecondary issues are minor: five participants were excluded post hoc on manipulation-check grounds without a robustness analysis, and the null trust/sympathy results are underpowered at roughly 49 per group.\n\nWho this is for: HCI and human-AI interaction researchers working on affective cues and decision-making; people in AI ethics will read the discussion. The study deserves a serious referee—the design is strong enough that the RCS question should be answerable with a short clarification, and the data could inform the field in either direction. But I would not cite the anger-reversal claim in its current form.","headline":"A well-designed study of AI anger cues on moral decision certainty whose headline result rests on a dependent variable with an unstated and likely inconsistent sign convention—worth refereeing, but not citable until the RCS anchors are clarified.","tokens_in":16750,"tokens_out":13981,"would_cite":false,"duration_ms":110212,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A conversational AI that displays anger through animated chat balloons can push people toward reversing their moral decisions, according to a controlled experiment.","keywords":["AI-Assisted Decision-making","Human-AI Collaboration","Emotion","Moral Dilemmas","Nonverbal emotional cues","AniBalloons","Decision certainty","Reversal Certainty Shift"],"falsifier":"Recode each participant's second-round certainty with a sign that reflects the final choice (e.g., positive when the final answer matches the original stance, negative when it flips), then rerun the group comparison. If anger no longer shows a positive reversal relative to baseline, or if the sign flips, the reported reversal effect is an artifact of the scoring convention. A direct audit of the excluded five participants' data would show whether their removal changes the conclusion.","tokens_in":15694,"feed_emoji":"😡","tokens_out":8014,"duration_ms":66066,"temperature":0.7,"pith_summary":"This paper sets out to show that a conversational AI's nonverbal emotional display can change how certain people feel about a moral decision. In an online study, participants read a moral dilemma, chose a stance, and then heard an AI assistant argue the opposite side; for some groups the assistant's message was wrapped in animated anger or sadness chat balloons, and for a baseline group the same words appeared without emotion. The authors report that anger cues produced a significantly larger Reversal Certainty Shift, a score measuring how much participants' confidence moved toward the opposite of their first-round choice, than either the no-cue baseline or sadness cues, while sadness cues were not significantly different from baseline. If the finding holds, it means a subtle visual design choice, not the content of the AI's advice, can move people toward reconsidering their moral stance, with implications for deliberation support and for manipulative design.","feed_headline":"Angry AI cues make people reverse moral choices","feed_subtitle":"Animated anger beats sadness and no-cue baselines in a moral dilemma test; perceived influence splits by gender.","key_machinery":"Three components carry the argument: Reversal Certainty Shift (RCS), a signed score $RCS=(-1)^n (x_2-x_1)$ that measures movement toward or away from the first-round decision; AniBalloons, animated chat-balloon cues that convey anger or sadness without changing the AI's words; and a devil's-advocate AI whose verbal response always argues against the participant's first choice. The RCS score is what turns raw confidence ratings into the reported reversal effect, and AniBalloons is what isolates emotion from content so that any effect can be attributed to the nonverbal cue.","core_discovery":"The paper's central claim is that an AI assistant expressing anger through animated chat balloons significantly increased Reversal Certainty Shift in moral dilemmas compared with a no-cue baseline ($M_{\\text{AC}}=0.811$, $SD=0.123$ vs $M_{\\text{Baseline}}=0.343$, $SD=0.113$, $p<0.01$) and compared with sadness cues ($M_{\\text{SC}}=0.453$, $SD=0.123$, $p<0.05$), while sadness cues did not differ significantly from baseline. The paper also claims that gender and cue type interact: self-identified men perceived more AI influence under anger cues than under sadness cues, self-identified women perceived more under sadness cues than baseline, and the two genders' perceived influence ran in opposite directions for anger and sadness. The authors conclude that subtle nonverbal cues can shape human moral decision certainty and perceptions of AI influence even when the verbal content is identical.","pith_inferences":["Because the AI always argued against the participant, the anger effect may be specific to adversarial or disagreement contexts; an obvious extension is to test supportive or neutral AI roles to see whether the cue alone, rather than opposition plus anger, drives the reversal.","The paper does not measure whether a reversal shift improves moral reasoning; if RCS is used as a design goal, future work should test decision quality, reflection depth, or downstream behavior.","The gender-by-emotion pattern in perceived influence suggests stereotype-consistent expectations may color how people report AI influence; measuring stereotype endorsement directly would test that mechanism.","A direct check of the RCS sign convention against a simpler 'did the final choice flip' outcome would clarify whether the reported result is about reversal or about reduced certainty generally."],"forward_implications":["Designers of conversational AI can shift decision outcomes without changing a single word of advice, merely by altering the emotional animation on the message bubble.","Anger cues are a candidate tool for reducing rigidity in deliberation, for example in creativity workshops or negotiation training, because they pushed participants toward the opposite stance.","The same mechanism is a manipulation risk: in customer service, sales, or political messaging, angry cues could pressure users into decisions they would not otherwise make.","Perceived influence and actual influence are separable; user self-report alone may not reveal whether an AI changed a decision.","Gender differences in perceived AI influence (males under anger, females under sadness) do not translate into gender differences in the actual reversal effect, so designers cannot rely on perception to predict behavior."],"supporting_citations":[{"why":"Supplies the chat-balloon animation set and early evidence that balloon animations carry affective affordances.","marker":"[1]"},{"why":"Validates that AniBalloons communicate named emotions without relying on message content, grounding the cue manipulation.","marker":"[2]"},{"why":"Provides the 48 CNI-model dilemma stories from which the four study scenarios were selected.","marker":"[47]"},{"why":"Shows most people do not change moral-dilemma stances over time, which the paper uses to interpret unchanged decision rates.","marker":"[53]"},{"why":"Supports the devil's-advocate design by showing that role-played opposition prevents premature decision closure.","marker":"[24]"},{"why":"Meta-analysis showing devil's advocacy promotes cognitive conflict, justifying the AI assistant's always-opposing responses.","marker":"[67]"},{"why":"Sufficiency-threshold hypothesis linking low certainty to more effortful processing, motivating RCS as an outcome.","marker":"[14]"},{"why":"Moral emotions theory that motivates choosing anger and sadness as the nonverbal cues.","marker":"[70]"},{"why":"Oxford Utilitarianism Scale used as a control variable in the ANCOVA analyses.","marker":"[41]"}],"fun_headline_variants":["AI anger cues reverse moral certainty more than sadness","Animated AI anger flips moral choices more than sadness or neutral","Angry AI chat cues make people reverse moral decisions","AI anger nonverbal cues shift moral certainty more than sadness"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The effect depends on the Reversal Certainty Shift sign convention correctly scoring movement toward the opposite of the first-round decision; if the convention is inverted, the anger condition would show reinforcement rather than reversal, and the post-hoc removal of five participants assumes the groups remain comparable.","fun_headline_variants_meta":{"raw":{"variants":["AI anger cues reverse moral certainty more than sadness","Animated AI anger flips moral choices more than sadness or neutral","Angry AI chat cues make people reverse moral decisions","AI anger nonverbal cues shift moral certainty more than sadness"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000675,"raw_usage":{"total_tokens":3046,"prompt_tokens":895,"completion_tokens":2151,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":2086}},"tokens_in":511,"tokens_out":2151,"duration_ms":13478,"temperature":1.0,"reasoning_tokens":2086,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:02:46.307509+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recode each participant's second-round certainty with a sign that reflects the final choice (e.g., positive when the final answer matches the original stance, negative when it flips), then rerun the group comparison. If anger no longer shows a positive reversal relative to baseline, or if the sign flips, the reported reversal effect is an artifact of the scoring convention. A direct audit of the excluded five participants' data would show whether their removal changes the conclusion.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the 48 CNI-model dilemma stories from which the four study scenarios were selected."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows most people do not change moral-dilemma stances over time, which the paper uses to interpret unchanged decision rates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supports the devil's-advocate design by showing that role-played opposition prevents premature decision closure."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Meta-analysis showing devil's advocacy promotes cognitive conflict, justifying the AI assistant's always-opposing responses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Sufficiency-threshold hypothesis linking low certainty to more effortful processing, motivating RCS as an outcome."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Moral emotions theory that motivates choosing anger and sadness as the nonverbal cues."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Oxford Utilitarianism Scale used as a control variable in the ANCOVA analyses."}],"review_version":1}