{"id":"9a79bef6-9746-43dc-819b-87361b96c6c1","arxiv_id":"2507.22326","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Adding hand-coded PAD emotions and chain-of-thought rationales to LLM agents made their simulated delivery behavior look more like real rider data, though no quantitative test confirms it.","lead":"This paper proposes an emotion alignment framework for LLM-based agents in metaverse service simulations, combining a PAD emotion model with chain-of-thought prompting to make agent behavior emotionally consistent. The authors report that the framework produces more realistic social patterns in a simulated food delivery ecosystem, but the evidence is largely qualitative.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The realism claim in RQ2 is not quantitatively supported: Fig. 4(a) compares simulated involution to a Zomato 'real-world' curve whose construction is never defined, and the setup descriptions conflict (6 riders/20 days vs 100 riders/30 days).","rationale":"The paper proposes a plausible framework combining PAD emotional states, self-explanation rationales, and memory for LLM agents, and the idea of testing it in an O2O delivery simulation is reasonable. Credit is due for describing the agent architecture, the emotional evolution equations, and the three-way ablation contrast in Table III. However, the central claim that E-A matches real-world involution better than prior frameworks rests on a visual comparison in Fig. 4(a) with no quantitative metric, no definition of how the real-world curve was derived, and no confidence intervals. The contradictory descriptions of the experiment setup (6 riders over 20 days in Section IV-B versus 100 riders over 30 days in the Appendix) mean that even the qualitative curve cannot be trusted without additional information. The reader's identified RQ1 confound is valid, but I see RQ2's realism comparison as the more direct load-bearing weakness: even if Self-Explanation were cleanly isolated, the realism claim would still be unsupported because 'more consistent with real-world systems' is never measured. My recommendation to reject is consistent with the reader's verdict, though for a somewhat different primary reason.","tokens_in":11338,"tokens_out":3559,"duration_ms":42530,"concrete_test":"Provide a reproducible computation: specify the exact Zomato fields and aggregation used to produce the real-world Involution(t) curve, make the code and data available, and quantify the discrepancy between each framework's Involution(t) and the real-world curve using a defined metric (e.g., mean absolute log difference or KS distance on normalized curves) with at least 10 random seeds and bootstrap confidence intervals. If E-A is not significantly closer than Emotion-perceived and Traditional under that metric, the realism claim is unsupported.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is that the emotion-alignment framework (E-A) produces involution dynamics 'more consistent with real-world systems' (Section V-A). This is asserted from a visual overlay in Fig. 4(a), but no method is given for computing the real-world curve from the Zomato dataset: what field is used as 'riders' money', how the inverse coefficient of variation (Eq. 4) is calculated over delivery orders, and how time is aligned are all unspecified. Without that definition, the comparison cannot be checked, and 'more consistent' has no operational meaning. The problem is compounded by contradictory setup descriptions: Section IV-B says 6 rider agents over 20 days, while the Appendix says 100 agents over 30 days (3600 steps) and 4 experiment sets rather than 3. Under either reading, the plotted curves are not reproducible, and no confidence intervals or seed variation are reported. The RQ1 confound noted by the reader is also real (Base omits Zero-Shot-CoT), but even setting that aside, RQ2's realism evidence is a qualitative overlay rather than a measured result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes an \"Explainable Emotion Alignment Framework\" for LLM-based agents in Metaverse service ecosystems. The framework comprises four steps: emotional data clustering and sampling, construction of an emotional evolution system based on the PAD model, a Self-Explanation mechanism that asks the LLM to generate rationales via Zero-Shot-CoT, and emotional knowledge storage in a memory. The authors claim that this framework improves LLM response quality (RQ1) and produces more realistic emergent social behavior in an O2O food-delivery simulation, specifically an \"involution\" trend closer to real-world Zomato data (RQ2).","tokens_in":11618,"tokens_out":6075,"duration_ms":63259,"significance":"The paper addresses a timely and interesting problem: making LLM-based agents in social simulations behave in a more human-like, emotionally consistent manner. If the claims were well-supported, the framework would be a useful contribution to the Metaverse service and agent-based simulation literature. The authors are to be credited for proposing a concrete architecture, openly discussing limitations, and making an attempt to connect simulated outcomes with real-world data. However, as detailed in the major comments, the current empirical evidence is confounded, irreproducible, and partly circular, so the significance of the contribution is not yet established.","major_comments":[{"comment":"The RQ1 evaluation compares BaseLLMs directly with +SE versions, but the SE condition adds both rationale generation and zero-shot-CoT prompting, while the Base condition includes neither. Since the paper attributes the improvements to Self-Explanation (the central novelty), the experiment needs a control condition that uses Zero-Shot-CoT without the rationale requirement, or otherwise isolates the effect of the rationale-generation mechanism.","section":"Section IV-A, Table II, Algorithm 2"},{"comment":"The constants k in Pleasure = kΔIncome and Arousal = kΔHealth are never defined, and the time windows over which ΔIncome and ΔHealth are computed are not specified. Without these definitions the emotional-state computation is not reproducible, and the mapping from these scalar values to the PAD values used in the prompts is not made explicit.","section":"Section III-C, Equations (1) and (2)"},{"comment":"The central claim that the emotion-alignment framework yields involution dynamics 'more consistent with real-world systems' is based on a visual overlay of a simulated curve and a 'real-world' curve whose construction from the cited Zomato dataset is never described. The manuscript does not state how riders' money is derived from the dataset, how the inverse coefficient of variation (Eq. 4) is computed over the data, or how time is aligned between the simulation and the real-world data. Without this methodology, the comparison cannot be checked, and 'more consistent' has no operational meaning.","section":"Section V-A and Fig. 4(a)"},{"comment":"The experimental setup described in the main text (six rider agents, 20 days, three experiment sets) directly contradicts the Appendix (100 generative agents, 30 days, 4 sets of experiments, 3600 steps). Such a discrepancy makes the presented results irreproducible and must be reconciled in any revision.","section":"Section IV-B vs. Appendix"},{"comment":"The observed coupling between emotional states and order-acceptance behavior is imposed by construction: the PAD emotional state is computed from the agent's income and health changes and then explicitly fed into the LLM's decision prompt. Therefore, the finding that positive emotions correlate with acceptance (Section V-C) is not an emergent discovery but a near-tautological consequence of the design. A control condition with randomly assigned or hidden emotional states is needed to support the claim that the emotion model itself, rather than the prompting structure, drives the reported behavior.","section":"Section III-C and V-C"}],"minor_comments":[{"comment":"The text says 'four prominent BaseLLMs' but Table II lists six models (Spark-2.0, ChatGPT, Llama2, DeepSeek-V3, Qwen2.5, Gemma3); please correct this inconsistency.","section":"Section IV-A"},{"comment":"The sentence 'The vectors of the upper and lower cultures are averaged' is unclear; presumably this refers to some form of text preprocessing, but the intended meaning should be stated.","section":"Section III-B"},{"comment":"There are several typographical issues: 'MetaV erse' in the section heading II-A, 'sentence t ransformers' in Section III-B, and 'Scikit−learn' with a nonstandard dash.","section":"Throughout"},{"comment":"The statement '2349 (including food delivery orders from multiple cities)' is not self-explanatory; are these 2349 orders or some other quantity? Please clarify.","section":"Section IV-B"}],"recommendation":"major_revision","confidential_remarks":"The paper requires substantial additional work to substantiate its central claims. The RQ1 confound and the undefined real-world baseline are fixable with additional experiments and methodology, and the setup contradictions are correctable, so I recommend major revision rather than rejection. However, given the scope of the changes, the editor may wish to consider whether the authors can realistically address all of them within a revision cycle."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper has a sensible core: give LLM agents an explicit PAD emotional state, feed it into the prompt, and have the model generate and store rationales (Self-Explanation) to reinforce role-consistent decisions. Running that idea across six LLMs and three text-generation tasks is a reasonable sanity check, and the simulation setup—comparing bare LLM, emotion-perceived, and emotion-aligned agents in a delivery ecosystem—is a legitimate way to look for emergent differences.\n\nThe problem is that the evaluation never isolates the claimed new component. In RQ1, Base is the bare LLM; +SE adds both rationale generation and Zero-Shot-CoT. Any gain could be CoT alone. That's a decisive confound for the headline claim. RQ2 is in worse shape. The main text says six rider agents for 20 days; the appendix says 100 agents for 30 days and four experiment sets, not three. The real-world Zomato curve in Fig. 4(a) is never defined—no description of what field is used as 'riders' money', how the inverse coefficient of variation is computed over orders, or how time is aligned. There are no error bars or seed variation. The proportionality constants k in equations (1) and (2) are never specified, and no code or data are provided. So the assertion that the emotion-aligned framework produces involution 'more consistent with real-world systems' has no operational meaning.\n\nCredit where due: the paper's limitation section is candid about the narrow scope and acknowledges that other emotion models may be better. That honesty doesn't repair the absent controls.\n\nThis is a toolkit-assembly paper with a suggestive but confounded validation. It might be a useful starting point for researchers building emotion-aware social simulation agents, but as a contribution it's not ready. I would tell an editor to desk reject this version; it could become a real submission if the authors fix the RQ1 confound, resolve the setup contradictions, and provide a reproducible method for the real-world baseline.","headline":"Reasonable emotion-alignment framework, but confounded evaluation and unreproducible simulation; not ready as is.","tokens_in":12102,"tokens_out":5445,"would_cite":false,"duration_ms":57769,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding evolving emotional states and self-explanation makes LLM-based agents in social simulations behave more like real people.","keywords":["emotion alignment","LLM-based agents","metaverse service","social simulation","PAD emotion model","self-explanation","fact alignment","involution"],"falsifier":"Run a controlled experiment where the Base condition also includes Zero-Shot-CoT prompting, so the only difference from the emotion-aligned condition is the emotional state model and the rationale-generation loop; if the emotion-aligned agent then no longer tracks the real-world involution curve better than the CoT-only agent, the central claim collapses.","tokens_in":11155,"feed_emoji":"🤖","tokens_out":3750,"duration_ms":44253,"temperature":0.7,"pith_summary":"The paper proposes an 'emotion alignment' framework for LLM-based agents that adds a continuously updated emotional state and a self-explanation step to the agents' decision loop. It claims this makes agents in a metaverse service ecosystem act more realistically, specifically that the competition intensity ('involution') in a simulated food-delivery system tracks real-world delivery data more closely than agents without these components. The framework is tested on three generative tasks and in a 20-day rider-agent simulation. If the claim holds, emotion alignment offers a low-cost way to improve the fidelity of LLM-driven social simulations without retraining.","feed_headline":"Emotion-aware agents make simulations track real delivery data","feed_subtitle":"Adding evolving emotional states and self-explanation to LLM riders shifts competition and rejection rates closer to actual systems.","key_machinery":"The load-bearing machinery is a three-part loop: (1) an Emotional Evolution System that derives each agent's current PAD values from environmental inputs such as income change, stamina change, and income-rank, mapping them to one of seven emotion states; (2) Self-Explanation, implemented via Zero-Shot-CoT, which forces the LLM to output both a decision and a causal rationale for that decision; and (3) Emotional Thought Memory, which stores the (question, decision, reason) triplets and feeds them into later decisions with a timeliness check. Together these components let the agent reason about its own emotion before acting, so decisions are not made from raw knowledge alone but from a state-consistent perspective.","core_discovery":"The central claim is that LLM-based agents equipped with an Emotional Evolution System based on the PAD (Pleasure–Arousal–Dominance) model, plus a Self-Explanation mechanism where the agent generates a rationale for each decision, produce decisions that are consistent with their emotional state and thereby yield social emergence closer to real-world systems. The paper's key quantitative evidence is that the emotion-aligned experimental system exhibited a trend of involution (inverse coefficient of variation of riders' earnings) that was more consistent with real-world systems than the traditional and emotion-perceived baselines, and that these agents rejected fewer orders and clustered in fewer locations over the simulation.","pith_inferences":["A cleaner test of the Self-Explanation contribution would add Zero-Shot-CoT prompting to the Base condition while keeping all other variables fixed, since the reported +SE condition introduces both rationale generation and CoT at once; this is an editorial inference, not a claim in the paper.","The emotion alignment approach could plausibly transfer to other digital-labor or marketplace simulations, where the same PAD-style state variables could be redefined for the specific environment.","The framework's reliance on the top-10 nearest textual exemplars per emotional cluster makes its effectiveness sensitive to dataset quality and cluster count, a sensitivity the paper does not quantify."],"forward_implications":["If the framework's gains are real, adding hand-coded emotional states and a rationale-generation step to LLM agents is a cheap, training-free way to increase realism in social simulations.","The same recipe could extend to other 'factual dimensions' such as health or psychological states, giving a general template for multi-factor agent alignment beyond emotion.","The lower order-rejection rate and weaker clustering among emotion-aligned agents suggest that emotional state acts as a bounded-rationality constraint, making simulated workers balance earnings and fatigue more like humans.","The involution metric used in the paper offers a quantitative probe for competition intensity that could be reused in other gig-economy or crowd simulations.","The framework establishes a comparison pattern: simulate agents with and without the alignment components and check which emergent curve best matches real-world data."],"supporting_citations":[{"why":"Supplies the EDBE sentiment dataset that defines the six non-neutral emotional roles for the rider agents.","marker":"[19]"},{"why":"Provides the PAD emotion model with Pleasure, Arousal, and Dominance values for the seven emotion states used in the framework.","marker":"[20]"},{"why":"Is the Zero-Shot-CoT method that the Self-Explanation step uses to generate rationales.","marker":"[18]"},{"why":"Supplies the real-world food-delivery order data used as the target for the realism comparison in the simulation experiment.","marker":"[27]"},{"why":"Defines the generative-agent architecture (perception, planning, action, memory) that the framework builds on and extends with an emotional system.","marker":"[6]"},{"why":"Grounds the computational experiment and generative explanation methodology used to run the virtual food-delivery scenario.","marker":"[26]"}],"fun_headline_variants":["Emotion-aligned LLM agents make metaverse simulations more realistic","Emotion and self-explanation align LLM agents with real delivery behavior","Explainable emotion framework makes virtual riders act like real ones","PAD emotion model + self-explanation keep metaverse agents grounded"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that the improved realism in the simulation comes specifically from its emotion alignment framework, but its key ablation changes two things at once: the emotion-aligned condition adds both self-explanation and Zero-Shot-CoT prompting, while the base condition has neither, so the measured gain could be due to the prompting method alone rather than to emotion-based reasoning.","fun_headline_variants_meta":{"raw":{"variants":["Emotion-aligned LLM agents make metaverse simulations more realistic","Emotion and self-explanation align LLM agents with real delivery behavior","Explainable emotion framework makes virtual riders act like real ones","PAD emotion model + self-explanation keep metaverse agents grounded"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000369,"raw_usage":{"total_tokens":1938,"prompt_tokens":865,"completion_tokens":1073,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":481,"completion_tokens_details":{"reasoning_tokens":999}},"tokens_in":481,"tokens_out":1073,"duration_ms":10526,"temperature":1.0,"reasoning_tokens":999,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T11:48:37.242672+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled experiment where the Base condition also includes Zero-Shot-CoT prompting, so the only difference from the emotion-aligned condition is the emotional state model and the rationale-generation loop; if the emotion-aligned agent then no longer tracks the real-world involution curve better than the CoT-only agent, the central claim collapses.","supporting_citations":[{"cited_title":"A new model for emotion-driven behavior extraction from text,","cited_arxiv_id":null,"evidence_quote":"Supplies the EDBE sentiment dataset that defines the six non-neutral emotional roles for the rider agents."},{"cited_title":"Automatically select emotion for response via personality-affected emotion transition,","cited_arxiv_id":null,"evidence_quote":"Provides the PAD emotion model with Pleasure, Arousal, and Dominance values for the seven emotion states used in the framework."},{"cited_title":"Zomato delivery operations analytics dataset,","cited_arxiv_id":null,"evidence_quote":"Supplies the real-world food-delivery order data used as the target for the realism comparison in the simulation experiment."}],"review_version":1}