{"id":"5bffef3f-1908-47a3-b41b-0c8620e37636","arxiv_id":"2508.19193","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Ordinal representations based on the rate of change of ambiguous emotion annotations improve prediction of directional changes in continuous emotion traces, especially for unbounded labels like engagement.","lead":"This paper proposes representing emotions as ordinal signals that track the rate of change of annotator ratings, rather than just the average rating and its spread. The authors show that these representations improve prediction of relative changes in continuous emotion labels, especially for unbounded engagement traces.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ordinal-vs-interval comparisons are made on different targets: OG/OI predict time-derivatives while I predicts raw values, so the reported SDA/CCC superiority may be an artifact of target mismatch.","rationale":"The reader's weakest assumption (untested cognitive premise) is real but external; my concern is internal and directly verifiable from existing model outputs. The paper's own limitations mention normalization and hybrid interval+ordinal models, but never acknowledge that the evaluation metrics are applied to different target signals. Because the abstract claims a general superiority of ordinal representations on CCC and SDA, and because the GameVibe σ SDA result is openly inconsistent with that broad wording, the evidence as reported is not commensurable. I am not rejecting the framework: ordinal representations may well be useful, and the proposed test can rescue or refute the claim. For that reason I keep the reader's conditional posture; the condition should be 're-evaluate in a common measurement space,' not merely 'test the cognitive premise.'","tokens_in":14528,"tokens_out":10690,"duration_ms":117435,"concrete_test":"Recompute all comparisons in one target space. From each trained OG and OI model, reconstruct an absolute trace by cumulative summation of the predicted gradient outputs (the additive constant does not affect SDA; use training-set initialization for CCC). Then compute CCC and SDA between (a) all reconstructed absolute traces and the original ground-truth µ and σ traces, and (b) all first-differenced predictions and the ground-truth first differences, for both RECOLA and GameVibe. If OG/OI no longer consistently beat I on SDA in either common space—or if the Table II SDA_σ reversal changes—the abstract's superiority claim is an artifact of comparing different prediction targets. Report bootstrap confidence intervals across folds for these recomputed metrics.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on a comparison that is not like-for-like. The I representation targets the raw distribution parameters θ={µ,σ} (Eq. 3); OG targets their time-derivatives {dµ/dt,dσ/dt} (Eq. 7); OI targets the distribution of individual trace gradients (Eqs. 4-6). Tables I and II report CCC and SDA between each model output and its own representation target. As SDA [41] measures agreement of the signs of the first derivatives of two signals, for I the reported SDA compares sign(dµ̂/dt) with sign(dµ/dt), whereas for OG the same metric compares sign(d²µ̂/dt²) with sign(d²µ/dt²). The models are not solving the same prediction problem, so 'ordinal representations excel in SDA' is not established. The GameVibe σ row is a red flag: I's SDA is 0.586 vs OG's 0.112. This is exactly what happens when one target is a smooth raw σ trace and the other is its noisy derivative. Similarly, raw engagement traces drift over time (Fig. 4), making I's raw-µ prediction a non-stationary extrapolation while OG predicts a locally stationary derivative; the large CCC gain for OG may be a target-difficulty effect, not evidence about ordinal emotion representations.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a new family of ambiguity-aware emotion representations for continuous emotion recognition. Three representations are compared: an interval representation I, which models the per-time-step distribution of annotator ratings via its mean and standard deviation; an individual ordinal representation OI, which models the distribution of per-annotator trace gradients; and a group ordinal representation OG, which models the temporal gradients of the group-level distribution parameters (dµ/dt and dσ/dt). These are integrated into an LSTM-based CER system and evaluated on RECOLA (arousal, valence; bounded labels) and GameVibe (engagement; unbounded labels) using CCC and SDA. The paper claims that ordinal representations outperform interval representations on unbounded labels and on SDA for bounded traces, and that the group representation is generally superior to the individual representation.","tokens_in":14838,"tokens_out":4995,"duration_ms":56935,"significance":"If the central claim were established, the framework would be a useful contribution to ambiguity-aware affective computing: it explicitly connects inter-rater ambiguity with temporal dynamics, it is clearly formalized, and it is tested on one bounded and one unbounded corpus. The use of SDA as a complementary metric is appropriate for ordinal representations, and the authors are transparent about some inconsistencies in their own results. The main conceptual strength is that the paper takes seriously the idea that emotion annotations are relative and dynamic rather than absolute point values. However, the empirical comparison as presented is not like-for-like, and several summary claims go beyond what the tables show. The framework may still be valuable, but the current evidence does not support the headline conclusions without additional re-analysis on a common target.","major_comments":[{"comment":"The comparisons are not on a common prediction target. I predicts the raw distribution parameters θ_n = {µ_n, σ_n}; OG predicts their time derivatives {dµ/dt, dσ/dt}; OI predicts the distribution of per-annotator gradients. The reported CCC and SDA values are computed between each model's output and its own representation target. For example, the SDA of I compares sign changes of µ̂_t with sign changes of µ_t (first derivatives), whereas the SDA of OG compares sign changes of the predicted dµ/dt with sign changes of the true dµ/dt (second derivatives of µ). Superior ordinal performance may therefore reflect target stationarity or smoothness rather than a better emotion representation. The GameVibe σ row in Table II is a clear illustration: I achieves SDA σ = 0.586 while OG achieves 0.112, consistent with raw σ drifting monotonically (Fig. 4) while dσ/dt is oscillatory. To support the abs","section":"§III-A/B, Eqs. (3), (6), (7); Tables I–II"},{"comment":"RECOLA results are based on a single train/development split with no repeated runs, confidence intervals, or significance tests. The highlighted differences (e.g., arousal SDA µ: I=0.430, OI=0.482, OG=0.506; valence SDA µ: I=0.130, OI=0.221) may be within run-to-run or split-to-split variability. The paper should report variance over LSTM seeds and/or cross-validation folds and provide pairwise significance tests or effect sizes before ranking representations.","section":"§V-C, §VI-A, Table I"},{"comment":"The abstract states that ordinal representations 'outperform conventional ambiguity-aware models on unbounded labels, achieving the highest CCC and SDA scores' and 'excel in SDA' for bounded traces. Table II shows OG has the lowest SDA σ (0.112 vs I 0.586 and OI 0.534). On RECOLA, ordinal representations are better only on SDA µ, not on SDA σ, and I is substantially better on CCC. The discussion in §VI-B partially acknowledges the σ anomaly, but the abstract and the closing claims are stronger than the evidence. The claims should be revised to match the partial pattern, or a statistical basis for declaring overall superiority should be provided.","section":"Abstract and §VI-B, Table II"}],"minor_comments":[{"comment":"Typo: 'the sold line' should be 'the solid line'.","section":"Fig. 4 caption"},{"comment":"Typo: 'alternative models beyond LSMTs' should be 'beyond LSTMs'.","section":"§VII"},{"comment":"Incomplete sentence: 'it is essential to for improving the generalizability and fairness' should be revised.","section":"§VIII"},{"comment":"There is a visible encoding artifact after the author line (the '/uni00000013/...' sequence) that should be removed before publication.","section":"Author block / metadata"},{"comment":"The claim that 'people agree more on temporal change than on magnitude' is central to the motivation but is supported only by citations. Consider adding a small empirical demonstration from RECOLA/GameVibe (e.g., inter-annotator agreement on sign of change vs. raw value) to make the premise more concrete.","section":"§II-B"}],"recommendation":"major_revision","confidential_remarks":"The core comparison issue is fundamental: the current tables do not compare models on the same prediction target, so the headline claims are not supported. I would like to see a revised version that re-evaluates all representations on a common target (e.g., integrated raw traces) or explicitly reframes the contribution as predicting the chosen representation rather than outperforming the interval baseline. The paper is otherwise within the scope of ACII and the proposed representations are clearly described."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful thing here is the explicit attempt to treat inter-rater ambiguity as a dynamic quantity: instead of predicting the mean and spread of raw annotations (I), they predict either the distribution of each annotator's gradient (OI) or the gradients of the mean/spread traces (OG). That framing is genuinely new, and the writing is clear. The authors also deserve credit for not hiding the awkward results: on RECOLA the interval representation keeps the best CCC on both mu and sigma, and on GameVibe's sigma the ordinal group representation gets an SDA of 0.112 versus 0.586 for interval. The limitations section honestly acknowledges the bounded/unbounded distinction and the need for normalization.\n\nThe problem is that the headline comparison is not apples-to-apples. I targets {mu,sigma} at each time; OG targets {dmu/dt,dsigma/dt}; OI targets the distribution of individual trace gradients. The tables then report CCC and SDA between each model output and its own target. So when the abstract says ordinal representations outperform ambiguity-aware models on unbounded labels, it is partly comparing the difficulty of predicting a raw non-stationary trace with the difficulty of predicting a locally stationary derivative. The GameVibe sigma row is the giveaway: predicting sigma itself is easy because unbounded traces drift apart, while predicting dsigma/dt is noisy. The reported SDA gain for ordinal is therefore likely a target-difficulty effect, not evidence about ordinal emotion representation.\n\nThe underlying cognitive premise—that humans agree more on changes than on absolute levels—is cited from prior work but not tested here. That is fine for a building-on-citations paper, but it means the empirical claim cannot carry the load alone.\n\nWho is this for? People working on continuous affect prediction and ambiguity modeling. The paper is worth a serious referee because the idea is plausible and the evaluation framework can be fixed (e.g., integrate ordinal predictions back to raw traces, or compare all representations on the same target). As it stands, the central claim is conditional at best. I would not cite the current empirical result, but I would keep an eye on a corrected version. Bring it to reading group mainly as a case study in target mismatch.","headline":"A clean idea for treating ambiguity as dynamic, but the main comparison is between different prediction targets, so the headline claim about ordinal superiority isn't established.","tokens_in":15316,"tokens_out":2204,"would_cite":false,"duration_ms":23899,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Continuous emotion traces can be modeled by their rate of change, and this ordinal representation outperforms interval-based ambiguity models on unbounded labels.","keywords":["continuous emotion recognition","ordinal representation","inter-rater ambiguity","temporal dynamics","engagement prediction","rate of change","affective computing","unbounded labels"],"falsifier":"If the unbounded GameVibe traces were re-analyzed after removing the annotators' progressive divergence, for example by normalizing each trace to zero mean or detrending, and the group ordinal representation no longer beat the interval representation on SDA and CCC, then the claimed advantage would be an artifact of global drift rather than of ordinal structure.","tokens_in":14422,"feed_emoji":"📈","tokens_out":5287,"duration_ms":53935,"temperature":0.7,"pith_summary":"The paper argues that the ambiguity of emotion annotations should be modeled through how ratings change over time, not through their absolute values at each moment. It introduces two ordinal representations: one tracks each annotator's gradient and fits a distribution over those gradients, the other tracks the gradients of the group's mean and standard deviation. On the unbounded engagement labels of GameVibe, the group ordinal representation gives the highest concordance and directional-agreement scores; on the bounded arousal and valence labels of RECOLA, ordinal representations improve directional agreement even though interval representations keep the best absolute-value fit. The takeaway is that for predicting the direction of emotional change, relative change carries more signal than absolute magnitude.","feed_headline":"Rate-of-change emotion models beat interval models on unbounded traces","feed_subtitle":"Ordinal annotation representations predict directional swings in engagement and arousal better than static interval targets.","key_machinery":"The central object is the gradient, the rate of change of an annotation trace over time, used as the unit of emotional meaning. The individual representation computes each annotator's gradient at each time step and fits a probability distribution over those gradients, so the mean captures agreed-upon direction and the standard deviation captures disagreement about that direction. The group representation instead computes the gradients of the interval representation's own parameters, dμ/dt and dσ/dt, directly tracking how the group's central emotion and its ambiguity are moving. The gradient transform is what converts static ambiguity into dynamic, ordinal information, and it is what the pape","core_discovery":"The paper's central claim is that a continuous emotion trace is better represented, for recognition purposes, by the rate of change of its annotations than by the absolute interval values, and that ambiguity should be encoded as the spread of those changes. Concretely, it defines two ambiguity-aware ordinal representations: the individual representation fits a probability distribution over the temporal gradients of each annotator's trace, while the group representation takes the temporal gradients of the group's mean and standard deviation. Tested in an LSTM-based continuous emotion recognition system, the group representation outperforms interval and individual representations on GameVibe's","pith_inferences":["The gradient-based representation is not emotion-specific: any continuous subjective rating with multiple annotators, such as pain, fatigue, or product satisfaction, could be recast in the same ordinal form and might show similar directional-agreement gains.","The paper leaves implicit that the advantage of the group representation on sigma may hinge on the unbounded traces' tendency to diverge over time; a drift-normalized version of the data would test whether the advantage survives when global trends are removed.","A natural next step is a hybrid input that concatenates interval and ordinal representations; the paper mentions combining them as future work, and the complementary strengths seen on CCC versus SDA suggest such a hybrid could dominate both metrics.","The authors note that their regression objective does not reflect the ordinal nature of the labels; modeling this as a preference-learning or classification problem might amplify the ordinal advantage beyond what the LSTM regression setup shows."],"forward_implications":["For unbounded continuous emotion labels, ordinal representations should be a strong default target: group-level gradient modeling yielded the top CCC and SDA scores on GameVibe engagement.","For bounded traces, ordinal representations are complementary to interval ones: they improve SDA relative to interval representations even when absolute CCC drops.","The group representation is the more reliable variant: aggregating trends across annotators smoothed individual inconsistencies and consistently outperformed per-annotator gradient modeling.","Evaluation of ordinal emotion models should include SDA, since it directly measures agreement in directional trend rather than magnitude.","Existing interval-based ambiguity models may under-use the temporal information in rater disagreement."],"supporting_citations":[{"why":"Supplies the cognitive premise that human affective judgments are ordinal and relative, motivating the replacement of absolute ratings with rates of change.","marker":"[18]"},{"why":"Earlier version of the ordinal-emotion argument; grounds the claim that relative change is more reliably perceived than magnitude.","marker":"[19]"},{"why":"Provides the GameVibe corpus with unbounded, ordinal engagement annotations from RankTrace, the key testbed for the main claim.","marker":"[4]"},{"why":"Provides the RECOLA corpus with bounded arousal and valence traces, the testbed for the bounded case.","marker":"[5]"},{"why":"Establishes the ambiguity-as-distribution baseline (interval representation) that ordinal representations are compared against.","marker":"[11]"},{"why":"Represents the recent ambiguity-aware dynamical approach that the paper contrasts with static interval modeling.","marker":"[17]"},{"why":"Describes the RankTrace annotation tool that generates unbounded ordinal labels, making the GameVibe experiment possible.","marker":"[31]"},{"why":"Defines SDA, the directional agreement metric used to evaluate whether ordinal representations capture relative changes.","marker":"[41]"}],"fun_headline_variants":["Emotion ambiguity as rate of change beats static interval labels","Ordinal gradient representations top interval models on unbounded traces","For unbounded emotions, rate-of-change labels outperform static ones","Gradient-based emotion encoding outperforms on engagement","Track emotion change, not intervals—new model wins"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that people agree more on how an emotion changes over time than on its absolute magnitude, and that this relative agreement is the signal worth modeling; the paper takes this premise from prior work rather than testing it directly.","fun_headline_variants_meta":{"raw":{"variants":["Emotion ambiguity as rate of change beats static interval labels","Ordinal gradient representations top interval models on unbounded traces","For unbounded emotions, rate-of-change labels outperform static ones","Gradient-based emotion encoding outperforms on engagement","Track emotion change, not intervals—new model wins"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000978,"raw_usage":{"total_tokens":3952,"prompt_tokens":666,"completion_tokens":3286,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":410,"completion_tokens_details":{"reasoning_tokens":3207}},"tokens_in":410,"tokens_out":3286,"duration_ms":29032,"temperature":1.0,"reasoning_tokens":3207,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T15:51:33.762202+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"If the unbounded GameVibe traces were re-analyzed after removing the annotators' progressive divergence, for example by normalizing each trace to zero mean or detrending, and the group ordinal representation no longer beat the interval representation on SDA and CCC, then the claimed advantage would be an artifact of global drift rather than of ordinal structure.","supporting_citations":[{"cited_title":"The ordinal nature of emotions: An emerging approach,","cited_arxiv_id":null,"evidence_quote":"Supplies the cognitive premise that human affective judgments are ordinal and relative, motivating the replacement of absolute ratings with rates of change."},{"cited_title":"The ordinal nature of emotions,","cited_arxiv_id":null,"evidence_quote":"Earlier version of the ordinal-emotion argument; grounds the claim that relative change is more reliably perceived than magnitude."},{"cited_title":"GameVibe: a multimodal affective game corpus,","cited_arxiv_id":null,"evidence_quote":"Provides the GameVibe corpus with unbounded, ordinal engagement annotations from RankTrace, the key testbed for the main claim."},{"cited_title":"Introducing the recola multimodal corpus of remote collaborative and affective interactions,","cited_arxiv_id":null,"evidence_quote":"Provides the RECOLA corpus with bounded arousal and valence traces, the testbed for the bounded case."},{"cited_title":"From hard to soft: Towards more human-like emotion recognition by modelling the perception uncertainty,","cited_arxiv_id":null,"evidence_quote":"Establishes the ambiguity-as-distribution baseline (interval representation) that ordinal representations are compared against."},{"cited_title":"Ranktrace: Relative and unbounded affect annotation,","cited_arxiv_id":null,"evidence_quote":"Describes the RankTrace annotation tool that generates unbounded ordinal labels, making the GameVibe experiment possible."},{"cited_title":"Fifty shades of green: Towards a robust measure of inter-annotator agreement for continuous signals,","cited_arxiv_id":null,"evidence_quote":"Defines SDA, the directional agreement metric used to evaluate whether ordinal representations capture relative changes."}],"review_version":1}