{"id":"e8e80a62-c563-4357-b03a-a980cbacc3b4","arxiv_id":"2608.03648","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DEAR trains two RL policies, peer selection and generation-behavior control, to regulate how LLM debaters reference each other, improving benchmark accuracy while cutting token use.","lead":"DEAR is a system for multi-agent LLM debate that learns which peer answers each agent should read and how to adjust each agent's generation randomness. It reports higher accuracy and lower token use than several debate baselines on math and question-answering benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DEAR's central claim that it mitigates blind conformity is never directly tested; experiments report only aggregate accuracy, and the RL reward is final accuracy, so the claimed mechanism remains unverified.","rationale":"Read in good faith, the paper makes two distinct claims: (1) DEAR improves accuracy and token efficiency over MAD baselines, and (2) it does so by mitigating blind conformity through group-level regulation of debate relationships. The first claim is well supported by extensive tables and ablations. The second is the scientific novelty and the title's promise, yet it is never operationalized. The reward function (Eq. 13) is final-answer correctness; the RL policies are therefore optimized for accuracy, not for conformity reduction. A policy can maximize accuracy by making agents explore more (via temperature), by selecting only high-quality peers, or by suppressing strong but wrong majority pressure. Any of these would improve accuracy without necessarily reducing 'blind conformity' in the sense defined in Section 1 (minority abandoning independent reasoning to align with majority). The case study in Appendix G is illustrative but not systematic. The ablation in Figure 4 shows each component contributes to accuracy, but does not show that the contribution is via conformity reduction. Therefore the central claim rests on an untested assumption. The proposed test—measuring answer-switching rates conditional on initial correctness—directly operationalizes blind conformity and would settle the issue. If the switch-to-incorrect rate is not lower for DEAR, the paper's framing should be revised, even if the accuracy gains are real. This is a load-bearing concern because the paper's contribution, title, and problem definition all hinge on the conformity mechanism. The reader's verdict of CONDITIONAL remains appropriate; no change in verdict is needed, but the requested revision should include a direct conformity measurement.","tokens_in":24470,"tokens_out":7500,"duration_ms":64570,"concrete_test":"Run MAD and DEAR with V=3, T=3 on a benchmark (e.g., GSM8K or GPQA Diamond). After round 1, label each agent's answer as correct/incorrect using ground truth. Track: (i) among initially-correct agents, the fraction whose final vote is incorrect (conformity events); (ii) among initially-incorrect agents, the fraction whose final vote is correct. Compare DEAR against MAD, DMAD, and a no-regulation control. Use at least 5 seeds and report 95% CIs. If DEAR does not significantly reduce (i) relative to MAD, the central 'mitigates blind conformity' claim fails, even if aggregate accuracy remains higher.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in Section 1, is that regulating debate relationships is the key to mitigating blind conformity. The experiments, however, never measure blind conformity. All reported outcomes (Tables 1-3, Figures 2-4) are final-answer accuracy and token consumption. The reward used to train both RL-Agents (Eq. 13) is R_debate, assigned solely on the correctness of the final answer. Thus the policies could improve accuracy by increasing output diversity via temperature/top_p adjustments, or by filtering noisy peers, without specifically reducing conformity. The only direct conformity evidence is a single case study (Appendix G), which is anecdotal. The ablation study (Fig. 4) shows accuracy drops when components are removed, but accuracy is not conformity. For the title claim to hold, DEAR must demonstrate a lower rate at which initially-correct agents abandon their answers in favor of an incorrect majority. Without such a measurement, the mechanism-specific claim is untested, regardless of whether the similarity metric is reliable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DEAR, a framework for multi-agent debate (MAD) that regulates which peers each LLM references and how each LLM generates (temperature/top_p) via two RL agents trained jointly with HAPPO. Group evidence is computed as pairwise cosine similarity between LLM conclusions and reasoning embeddings (Eqs. 2-3), decoupled by Subjective Logic into belief masses and uncertainty (Section 4.2), used to select reference peers (Section 4.3), with DST-based fusion guiding behavior adjustment (Section 4.4). Experiments on four math and four QA benchmarks against several MAD baselines report higher accuracy and lower token consumption (Tables 1-3, Figures 2-4), including closed-source models (Table 2). The authors claim this demonstrates that regulating debate relationships mitigates blind conformity.","tokens_in":24678,"tokens_out":1394,"duration_ms":14968,"significance":"If the central claim held, the paper would make a useful contribution: shifting MAD regulation from individual confidence/perplexity to group-level interaction evidence is an interesting direction, and the reported token-efficiency gains are practically relevant. The framework is nontrivial: it couples two heterogeneous RL agents with HAPPO and connects Subjective Logic and DST in a coherent pipeline. The paper reports consistent accuracy improvements over a wide range of baselines and models, and the cross-domain transfer experiment (Table 8) is a genuinely informative check on task-agnosticism. However, the headline claim that DEAR mitigates blind conformity is not directly measured, and the lack of error bars or significance tests weakens the empirical case, as noted below.","major_comments":[{"comment":"The paper's title and Section 1 claim that DEAR mitigates blind conformity, but no experiment measures blind conformity. All reported metrics are final-answer accuracy and token consumption, and the RL reward in Eq. (13) is assigned solely on final correctness. The only conformity evidence is a single anecdotal case study (Appendix G). The authors should either add a direct measurement—e.g., the rate at which initially-correct agents switch to an incorrect majority answer, or a comparison of answer-switch dynamics versus baselines—or reframe the central claim as improved accuracy/efficiency, not conformity mitigation.","section":"Section 1; Section 5.2; Appendix G"},{"comment":"All tables report single accuracy numbers without error bars, confidence intervals, or significance tests. AIME24 has only 30 problems and GPQA Diamond 150, so headline gains such as AIME24 16.7 vs 10.0 (Table 1) or GPQA 48.0 vs 42.0 could be within sampling noise. I ask for standard errors or hypothesis tests on at least the small benchmarks, and for a statement of how many seeds/runs the reported numbers average over.","section":"Tables 1-3"},{"comment":"The weakest assumption is that scaled cosine similarity between embeddings (Eqs. 2-3) is a valid, task-agnostic measure of consensus/divergence that can guide regulation. The paper does not show that similarity correlates with answer correctness, nor that the learned policies depend on similarity rather than spurious patterns (e.g., persona-specific wording or length). A concrete validation test would be to compare the distribution of similarity scores for correct-incorrect, correct-correct, and incorrect-incorrect answer pairs, and to check whether the Selection RL-Agent's masking decisions change when embeddings are replaced by a different encoder.","section":"Sections 4.1-4.4"},{"comment":"The RL formulation assigns a single terminal reward R_debate for the final answer, so the two RL agents are optimized only for accuracy, not for any conformity-related objective. This means the observed accuracy gains could arise from filtering noisy peers, increasing output diversity via temperature/top_p, or other mechanisms unrelated to reducing blind conformity. To support the mechanism claim, the reward or evaluation should include a conformity metric, or the paper should report ablation on initial-round correctness versus final-round correctness.","section":"Section 4.5 and Eq. (13)"},{"comment":"The comparison with RUMAD (6 rounds, 6 LLMs) uses configurations different from the main experiments (3 rounds, 3 LLMs in Table 4), and the paper does not state whether DEAR was retrained for the 6-agent/6-round setting or evaluated with the same policy. Without this detail, the claimed superiority over RUMAD is hard to interpret, since RL policies trained for one debate configuration may not transfer to another.","section":"Appendix E.2, Table 7"}],"minor_comments":[{"comment":"The text says DEAR is trained on GSM8K for math and ARC-C for QA, but does not state the size of the training subset or how many episodes/seeds were used for RL training; Appendix D gives total training time but not the number of trajectories.","section":"Section 5.1, Implementation Details"},{"comment":"The notation p_{i<-j} is used both as the Bernoulli parameter and as the preference matrix entry P; consider renaming one for clarity.","section":"Equation (7)"},{"comment":"The DST combination rule is written for two belief sets, but Eq. (8) applies it to V sets; the paper should clarify whether the combination is associative in this formulation or specify the order of pairwise fusion.","section":"Section 4.4, Definition 4.4.1"},{"comment":"The case study is illustrative but contains an apparent inconsistency: Agent 2's Round 1 reasoning says 'Agent 1 misinterprets' while Agent 3's Round 1 reasoning says 'Agent 1 correctly identifies'; this could confuse readers about the actual trajectories.","section":"Appendix G"},{"comment":"There are occasional typos and formatting issues, e.g., 'the results confirm' at the start of Section 5.4, and unrendered math symbols in the extracted text; a careful proofread is recommended.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper has an interesting mechanism and broad experiments, but the gap between the claimed conformity mitigation and the measured accuracy is the main editorial risk. I would ask the authors to add direct conformity metrics or soften the claim, and to provide uncertainty quantification for the small benchmarks. The paper does not ship code, but the appendix provides pseudocode and detailed hyperparameters, which mitigates reproducibility concerns somewhat."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: DEAR is a solid engineering contribution to multi-agent debate—coordinated peer selection and temperature/top-p control trained jointly with HAPPO—and it reports consistent accuracy gains across several benchmarks at lower token cost. But the central thesis, that it mitigates blind conformity, is never directly tested. The experiments report only aggregate accuracy and token counts, and the RL reward is final-answer correctness. So the accuracy gains could come from increased output diversity or filtering noisy peers rather than from reducing conformity specifically. The one case study in Appendix G is anecdotal. That gap is real and needs to be addressed before the claim can be taken at face value.\n\nWhat is new: the two-agent sequential regulation framework (Who and How) is a genuine departure from single-controller approaches like RUMAD and heuristic scoring like Free-MAD. The group evidence via cosine similarity, subjective logic decoupling, and DST fusion is a coherent pipeline, and the HAPPO-based joint training is a reasonable fit for the heterogeneous action spaces. The token savings are substantial and, if the method is reproducible, commercially relevant. I also note that the accuracy claim is not circular: training is done on GSM8K/ARC-C train splits and evaluation is on held-out test sets.\n\nSoft spots, in proportion: The biggest issue is the unmeasured mechanism. A direct conformity metric—for example, the rate at which initially-correct agents switch to an incorrect majority—would settle it and is straightforward to compute. Second, no error bars or significance tests anywhere; with AIME24 at 30 problems and GPQA Diamond at 150, several headline gains look like they could be within noise. Third, no code or trained policies are released, so the efficiency and accuracy numbers cannot be independently checked. The cosine-similarity evidence is a reasonable heuristic, but unvalidated against correctness; that is a minor concern relative to the mechanism gap.\n\nThe paper is worth a serious referee. It is well-written, honest about baselines, and the engineering novelty is real. But it needs a revision that measures conformity directly, reports variances, and releases code. If those are fixed, this could be a useful contribution to the MAD literature.\n\nMy recommendation: send to peer review, with the expectation of major revision.","headline":"DEAR is a plausible efficiency-plus-accuracy contribution to multi-agent debate, but its central claim about mitigating blind conformity is asserted rather than measured, and the experimental reporting lacks error bars and code.","tokens_in":25228,"tokens_out":2736,"would_cite":true,"duration_ms":23394,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DEAR regulates who each debater consults and how it responds, using group-level evidence to mitigate blind conformity in multi-agent LLM debate.","keywords":["multi-agent debate","blind conformity","debate relationships","group evidence","reinforcement learning","subjective logic","HAPPO","LLM reasoning"],"falsifier":"Take a constructed debate in which two debaters share high embedding similarity but give different final answers, while a third diverges in embedding but is correct; DEAR's selection policy, trained on similarity-based evidence, would be expected to sever the useful reference and keep the misleading one. Observing such a trajectory—or a near-zero correlation across a large sample between similarity scores and answer agreement—would falsify the central claim.","tokens_in":24265,"feed_emoji":"🤖","tokens_out":5575,"duration_ms":47560,"temperature":0.7,"pith_summary":"The paper's thesis is that blind conformity in multi-agent LLM debate is not a failure of any single debater's confidence but a group-level phenomenon, and that the right lever is the relationship between debaters: who is allowed to influence whom and how strongly. DEAR operationalizes this by extracting pairwise similarity between each debater's conclusion and reasoning, converting these similarities into consensus and uncertainty signals with Subjective Logic, and then training two policies—a selection policy that masks which peers are referenced and a behavior policy that sets temperature and top-p—jointly with HAPPO. Reported results show higher average accuracy and substantially lower token consumption than standard debate and individual-evaluation baselines across math and QA tasks. The contribution, if correct, is a way to get more reliable multi-model reasoning without trusting confidence scores or perplexity, which the paper argues are both misleading.","feed_headline":"A learned debate controller lifts accuracy and cuts token use","feed_subtitle":"Two trained policies decide who each debater listens to and how boldly it speaks.","key_machinery":"The central object is the group evidence matrix built from pairwise scaled cosine similarities between embedded conclusions and embedded reasoning processes of debating LLMs. Subjective Logic decouples this evidence into consultation tendencies (belief masses) and uncertainty, yielding the observation for the Selection RL-Agent, which outputs independent Bernoulli decisions about which peer references to keep. A Dempster-Shafer fusion rule then combines the selected peers' evidence, and the Behavior RL-Agent maps the fused evidence to a generation behavior—temperature and top-p—for each debater. Heterogeneous-Agent Proximal Policy Optimization (HAPPO) trains the two agents end-to-end by chaining their surrogate advantages, so the behavior policy's update is explicitly conditioned on the selection policy's update.","core_discovery":"DEAR's central claim is that regulating debate relationships—which peers each LLM may consult and with what generation behavior—mitigates blind conformity more effectively than individual-level evaluation. The paper shows that a fully connected debate amplifies erroneous majority reasoning, while confidence- or perplexity-based filters can propagate unreliable individual signals. DEAR instead builds a group evidence matrix from pairwise scaled cosine similarities of conclusions and reasoning, decouples it into belief and uncertainty via Subjective Logic, and trains Selection and Behavior RL-Agents that decide, each round, which references to keep and how to sample the next answer. The joint HAPPO training treats this as a sequential decision process. On GPT-4o-mini, DEAR obtains 65.1% average math accuracy and 75.0% average QA accuracy, against 57.9% and 69.6% for the closest baselines, while using roughly half the tokens.","pith_inferences":["A natural test is to replace the learned selection policy with random peer masking at the same token budget; DEAR's margin over that baseline would show whether learned selection, rather than mere sparsity, drives the gain.","If cosine similarity of embeddings is a reliable group signal, the same evidence pipeline could extend to multimodal debate, using embeddings of generated text and images.","The What-Who-How split suggests a general design pattern: any multi-agent reasoning system can be decomposed into perception of group state, routing of information, and control of generation behavior.","The paper does not show that its similarity scores correlate with answer correctness; if they do not, policies trained on them may latch onto superficial phrasing."],"forward_implications":["Multi-agent systems can be made more reliable without fine-tuning the LLMs themselves; only the two external policies are trained.","Token costs drop because the selection policy severs needless peer references; on GPT-4o-mini average tokens fall from about 22K for standard MAD to about 11K for DEAR on math tasks.","The framework transfers across domains: policies trained on math still improve QA accuracy, and vice versa, with some decay from distribution shift.","The gains persist when scaling the number of debaters or debate rounds, suggesting that learned regulation rather than a fixed topology is what helps."],"supporting_citations":[{"why":"Defines the multi-agent debate setup and fully connected interaction topology that DEAR modifies.","marker":"[1]"},{"why":"Confidence-based sparse debate baseline; represents the individual-evaluation approach DEAR argues is unreliable.","marker":"[10]"},{"why":"Perplexity-based memory filtering baseline; motivates DEAR's shift away from token-level uncertainty.","marker":"[14]"},{"why":"Supplies Subjective Logic, the formal mechanism used to decouple group evidence into consultation tendency and uncertainty.","marker":"[19]"},{"why":"Heterogeneous-agent PPO, the algorithm used to jointly train the two RL agents with a sequential advantage.","marker":"[20]"},{"why":"Reinforcement-learning debate baseline that also trains external policies without fine-tuning LLM weights.","marker":"[28]"},{"why":"Diverse multi-agent debate baseline; its accuracy is one of the main comparisons for DEAR.","marker":"[37]"},{"why":"Pre-trained encoder used to compute the cosine similarities that form the group evidence matrix.","marker":"[41]"}],"fun_headline_variants":["Group debate regulation stops conformity, saves tokens","DEAR: control peer influence to improve LLM debate","Regulate who debaters hear to cut conformity and cost","Debate relationships tuned to beat blind conformity","Smarter peer selection lifts accuracy, halves token use"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's load-bearing premise is that pairwise cosine similarity between embedded conclusions and reasoning is a reliable, task-agnostic measure of consensus and divergence; if that measure does not track who is actually right, both RL agents receive corrupted inputs and the claimed accuracy gain loses its foundation.","fun_headline_variants_meta":{"raw":{"variants":["Group debate regulation stops conformity, saves tokens","DEAR: control peer influence to improve LLM debate","Regulate who debaters hear to cut conformity and cost","Debate relationships tuned to beat blind conformity","Smarter peer selection lifts accuracy, halves token use"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000158,"raw_usage":{"total_tokens":1241,"prompt_tokens":977,"completion_tokens":264,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":190}},"tokens_in":593,"tokens_out":264,"duration_ms":3392,"temperature":1.0,"reasoning_tokens":190,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:47:50.900064+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a constructed debate in which two debaters share high embedding similarity but give different final answers, while a third diverges in embedding but is correct; DEAR's selection policy, trained on similarity-based evidence, would be expected to sever the useful reference and keep the misleading one. Observing such a trajectory—or a near-zero correlation across a large sample between similarity scores and answer agreement—would falsify the central claim.","supporting_citations":[{"cited_title":"Springer Publishing Company, Incorporated, 2018","cited_arxiv_id":null,"evidence_quote":"Supplies Subjective Logic, the formal mechanism used to decouple group evidence into consultation tendency and uncertainty."}],"review_version":1}