{"id":"1b32ba82-99ba-4760-bdc2-1dd5ff0f3372","arxiv_id":"2504.18085","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Random-Set LLMs predict belief functions over token clusters and report higher QA accuracy and credal-width uncertainty signals.","lead":"Large language models can be fine-tuned to output belief values over clusters of similar words instead of plain word probabilities, and this improves question-answering accuracy in the paper's tests. The same belief outputs can flag uncertain or hallucinated answers, although the evidence for that is weaker and the math has gaps.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 11 mass recovery over a subset-incomplete budget does not yield a genuine belief function, so credal-width uncertainty and hallucination detection lack a valid foundation.","rationale":"The reader's weakest assumption points directly at Eq. 11, and I agree this is the load-bearing fault line. The paper's headline contributions - second-level uncertainty via credal width and hallucination detection - are only meaningful if the predicted output is a belief function and the pignistic/credal machinery of Sec. 3 applies. The recursion in Eq. 11 is not Möbius inversion on the budget O unless O is closed under subsets; with hierarchical clusters plus singletons, non-singleton proper subsets of a cluster are absent, so their masses are silently folded into the parent cluster's residual. Because the loss trains each belief value independently, there is no structural guarantee of monotonicity or nonnegativity of the induced masses; the clipping and universal-set post-processing change the actual belief values, so the deployed predictor does not minimize the reported loss. Consequently, the pignistic probability is not provably the centre of mass of the credal set (6), and the claimed guarantee that BetP(t) lies in the interval [P(t), upper-P(t)] is unsupported. Table 2 and Fig. 5f could then reflect the ad-hoc inversion rather than a principled random-set model. This is an internal soundness issue, not a disagreement with consensus, and it is more fundamental than the lack of error bars and statistical testing. The accuracy improvements in Table 1 might survive a fix, but the paper's distinctive uncertainty contribution would need to be re-established. The reader's CONDITIONAL verdict - release code/data, report variance, and justify or correct the mass computation - remains appropriate, so no change to the verdict is needed.","tokens_in":17313,"tokens_out":13674,"duration_ms":134227,"concrete_test":"Restrict to the 200 most frequent Llama-2 tokens; let O' be all singletons plus the RS-Llama2 clusters contained in this set that have size at most 20 (larger clusters are subsampled to 20 tokens). For 100 test prompts, record the predicted belief vector on O'. Solve the linear program: find nonnegative mass m over all subsets of each cluster in O' such that the sum of m(B) over B subset of A equals the predicted Belief(A) for every A in O', and the total mass sums to 1. If the LP is infeasible for more than 5% of tokens, the predicted object is not a genuine belief function and the credal-width and hallucination claims fail. For feasible cases, compare the pignistic probability of the LP solution with the pignistic obtained via Eq. 11; an average L1 distance above 5% indicates the recursive inversion changes decision-relevant probabilities.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. 11 is the linchpin: it converts the predicted belief vector into the mass function used for pignistic transforms (Eq. 7) and credal widths (Eq. 14). The budget O is only singletons plus K hierarchical clusters; it is not closed under subsets. Möbius inversion (Eq. 3) requires Bel values on every subset of A, but for a cluster A={a,b,c} the pair {a,b} is typically absent from O, so the recursive subtraction in Eq. 11 omits m({a,b}) and the residual assigned to m(A) absorbs masses of all missing proper subsets. The recovered m is thus not the Möbius inverse of the predicted Bel, and the object is not a genuine belief function. Since the loss L_BCE trains each Bel(A) independently, nothing enforces monotonicity (A⊆C implies Bel(A)≤Bel(C)); negative masses are expected, and the clipping/universal-set post-processing changes the belief values, so the trained objective does not describe the deployed predictor. The pignistic entropy (13) and credal width (14) therefore inherit the invalidity: BetP is the centre of mass of the credal set only for genuine belief functions, and the bounds P(t) and upper-P(t) are not guaranteed to contain BetP. The hallucination separation in Fig. 5f and Table 2 may be an artifact of the ad-hoc inversion, not evidence of second-level uncertainty. The footnote in Sec. 4.3 that improper belief functions are normally used in the literature does not rescue the quantitative uncertainty claims, which rely on genuine credal-set semantics.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Random-Set Large Language Models (RS-LLMs), which replace the standard softmax next-token distribution with a belief function defined on a budget of focal sets: the singletons of the vocabulary plus K clusters obtained by hierarchical clustering of token embeddings. The model is trained with a binary cross-entropy loss on belief values plus two mass-regularization terms, and at generation time the predicted belief vector is converted into a mass function via a recursive subtraction (Eq. 11), then into a pignistic probability for token sampling. The authors also propose pignistic entropy and the width of the associated credal set as uncertainty measures, and apply them to hallucination detection. Experiments on CoQA and OBQA with Llama2-7b, Mistral-7b and Phi-2 report that RS-LLMs outperform standard fine-tuned models in accuracy and cosine similarity, and that credal width separates correct from incorrect responses.","tokens_in":17751,"tokens_out":6472,"duration_ms":64764,"significance":"If the claims hold, the paper would offer a single-model, fine-tuning-only method for extracting epistemic uncertainty and hallucination signals from LLMs, which is currently of high interest. The budgeting scheme (hierarchical clustering) is a practical contribution for scaling random-set output layers to large vocabularies, and the use of credal-set width as an uncertainty measure is conceptually appealing. The paper also makes an explicit empirical claim—accuracy improvements over standard fine-tuning on two QA benchmarks across three base architectures—which, if verified with proper statistics, would be a useful result. However, the significance is contingent on the mathematical validity of the predicted belief functions and on the statistical reliability of the reported gains; both need substantial strengthening.","major_comments":[{"comment":"The recursive mass computation over the budget O, which is not closed under subsets, does not produce the Möbius inverse of the predicted belief function. For a focal set A, the sum over B⊂A includes only sets present in O, so the masses of all missing proper subsets are silently absorbed into m(A) or produce negative masses. The subsequent post-processing (clipping negatives and adding a universal set) changes the belief values, so the loss L_BCE in Eq. (8) no longer describes the deployed model. Consequently, the predicted object is not a genuine belief function, the credal set in Eq. (6) is not the convex set whose center of mass is the pignistic probability of Eq. (7), and the claim in Sec. 4.4 that BetP(t) lies within [P(t), P(t)] is not guaranteed. This invalidates the pignistic entropy (Eq. 13) and credal width (Eq. 14) as measures of epistemic uncertainty, and therefore undermines the hallucination detection claims in Sec. 5.3 and Fig. 5. The footnote about improper belief functions used in the literature does not rescue these claims, because the quantitative uncertainty interpretation relies on genuine credal-set semantics.","section":"Sec. 4.3, Eq. (11)"},{"comment":"The central empirical claim that RS-LLMs 'clearly outperform' standard LLMs rests on a single run over a 500-sample test set, with no error bars, significance tests, or multiple seeds. The CoQA cosine-similarity differences are small (0.69 vs 0.71, 0.67 vs 0.72, 0.72 vs 0.73) and may lie within run-to-run variation, especially since the underlying model checkpoints are not specified. The authors should report confidence intervals (e.g., bootstrap over the test set) or results over several seeds, and apply a significance test (e.g., McNemar for OBQA accuracy) to support the accuracy claims.","section":"Sec. 5.1, Table 1"},{"comment":"The hallucination detection evidence is weak. For OBQA, the credal width under correct vs incorrect context is 0.00±0.04 vs 0.02±0.08, which is negligible in magnitude and likely not statistically significant; the histograms in Fig. 5f appear to overlap heavily. The statement that 'both models exhibit good separability' is not supported by these numbers. In addition, randomly replacing the question or answer choices is not a standard hallucination evaluation; the authors should either use a recognized hallucination benchmark or report a proper detection metric such as AUROC, and they should clarify what notion of hallucination this setup targets.","section":"Sec. 5.3, Table 2 and Fig. 5f"},{"comment":"The loss L_BCE trains each belief value Bel(A) independently, with no constraint enforcing monotonicity (A⊆C ⇒ Bel(A) ≤ Bel(C)). Even if the budget were subset-closed, the predicted vector would not generally be a belief function, so the mass regularization terms (Eqs. 9-10) and post-processing are essential but ad hoc. The trained objective therefore does not describe the deployed post-processed model. The authors should either reparameterize the output to guarantee a valid mass function, or add a projection step that is part of the training objective, so that the loss reflects the actual predictor.","section":"Sec. 4.3, Eq. (8) and Eq. (11)"}],"minor_comments":[{"comment":"The equation should read m(A) = Bel(A) - Σ_{B⊂A} m(B), not m(A). The current typesetting appears to contain a typo.","section":"Eq. (11)"},{"comment":"The abstract says 'high-quality tests' and the introduction says 'allows so efficiently'; both are likely typos for 'texts' and 'allows this efficiently', respectively.","section":"Abstract and Sec. 1"},{"comment":"The text says a vanilla RS-LLM would have '2N outputs' where N appears to be the vocabulary size; this should be 2^N (or 2^T) outputs, and the notation for the power set P(T) should be defined consistently.","section":"Sec. 4.1"},{"comment":"The phrase 'RS-LLMs clearly outperform the standard LLMs model on both datasets across all datasets' is confusing; the main table reports only CoQA and OBQA, so 'all datasets' is an overstatement. Also, the text refers to 'both models' while three base models are evaluated.","section":"Sec. 5.2"},{"comment":"The subfigure captions contain inconsistent capitalization ('Standard LLama2' vs 'Random-set LLama2') and the OBQA entropy subfigures show no clear separation, which should at least be commented on in the text rather than described as 'no real trend' without discussion.","section":"Fig. 5"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a potentially useful empirical idea, but the theoretical foundation of the uncertainty claims is seriously flawed: the mass recovery in Eq. (11) does not yield a genuine belief function on the non-subset-closed budget, so the credal-width and pignistic-entropy measurements do not have the claimed semantics. This is not a mere presentation issue; it affects the central novelty of the paper. The accuracy improvements, if real, could still justify publication after substantial revision, but the current manuscript gives no statistical evidence that the reported gains are reliable. I would ask the authors to either fix the mathematical pipeline (e.g., by training directly on masses or by projecting onto the set of valid belief functions in a way that is part of the training loss) or to reposition the claims so that they do not rely on genuine credal-set semantics. The hallucination results, particularly on OBQA, are too weak to support the current conclusions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read the RS-LLM paper. Short version: it applies the authors' random-set neural-network idea to LLM token prediction, using hierarchical clustering to build a budget of focal sets, and it reports consistent accuracy gains over standard softmax fine-tuning on CoQA and OBQA. That empirical result is plausible and worth a look, but the uncertainty and hallucination claims, which are the paper's real hook, rest on a mass-recovery step that doesn't do what the authors think it does.\n\nWhat's new: the application of RS-NN to token space, and a clustering budget that makes it feasible for 32k+ vocabularies. Predicting beliefs over token sets rather than just singletons is a genuine alternative to softmax, and the budget trick is a reasonable engineering choice. The accuracy numbers (Llama2 OBQA 83.2->89.6; Mistral CoQA cosine 0.67->0.72) are consistently in the right direction.\n\nThe problem is Sec. 4.3's Eq. 11. The budget O is singletons plus K clusters, not closed under subsets. The recursive formula m(A)=Bel(A)-sum_{B subset A} m(B) only inverts the belief function if all proper subsets of A are in O. They aren't, so the residual is absorbed into m(A), and the recovered mass is not the Möbius inverse of the predicted Bel. Also, each Bel(A) is trained independently with BCE, so nothing enforces monotonicity; negative masses are expected, and the clipping/renormalization post-processing changes the belief values. The pignistic probability and credal width computed afterward are not the center of mass and width of a legitimate credal set. The hallucination experiment is weak too: for OBQA, credal widths for correct vs incorrect contexts overlap nearly completely (0.00 +/- 0.04 vs 0.02 +/- 0.08).\n\nOther issues: no error bars, no significance tests, hyperparameters tuned on the test set, and no comparison with any other uncertainty method. The 'first and only' and 'conclusively demonstrate' language is overreach.\n\nThis paper is for researchers in LLM uncertainty quantification who want to know whether a single-model, belief-function approach can work. That's a real question, but in this form the theoretical foundation is unsound and the empirical evidence is too thin to carry the claims. I'd send it to a serious referee—the idea deserves engagement—but I'd expect a request for major revision. I wouldn't cite it as is.","headline":"Extends random-set neural nets to LLM token prediction with a practical clustering budget, but the mass-recovery step breaks the belief-function semantics, so the uncertainty and hallucination claims are unsupported.","tokens_in":18198,"tokens_out":10211,"would_cite":false,"duration_ms":99114,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Making an LLM output belief functions over token groups—instead of a single probability vector—both improves its answers and gives it a built-in signal for when it is guessing or hallucinating.","keywords":["random-set large language models","belief functions","credal sets","epistemic uncertainty","hallucination detection","pignistic probability","hierarchical clustering","question answering"],"falsifier":"Train an RS-LLM on a small vocabulary where the full power set is enumerable, compute the exact Möbius-inverse masses and exact credal sets, and compare them with the recursive budget approximation; if the approximate pignistic probabilities or credal widths differ materially, or if many predicted masses are negative and must be zeroed out by post-processing, the central uncertainty claims are not supported.","tokens_in":17146,"feed_emoji":"🎲","tokens_out":5224,"duration_ms":53427,"temperature":0.7,"pith_summary":"This paper tries to establish that an LLM can be made more trustworthy by changing what its final layer predicts: instead of a probability vector over the next token, it predicts a belief function over a small budget of token groups. The authors argue that this random-set output carries richer information about what the model knows versus what it is merely unsure about, and that this representation can be trained efficiently with a hierarchical-clustering trick. If correct, the approach would give a single fine-tuned model better answer accuracy plus two uncertainty signals—pignistic entropy and credal-set width—the latter of which separates correct from incorrect answers and rises when the context is deliberately wrong. The paper backs this with experiments on CoQA and OBQA across three base models, where the random-set variants outperform their standard counterparts in accuracy and cosine similarity.","feed_headline":"Belief-set LLMs answer better and flag their own hallucinations","feed_subtitle":"A final-layer change lets Llama2, Mistral, and Phi-2 score higher and expose second-level uncertainty.","key_machinery":"The load-bearing object is a belief function over a budget of focal sets: a non-additive assignment of mass to subsets of tokens, where the belief in a set is the sum of masses of its subsets, and the associated convex set of probability distributions is the credal set. The budget of focal sets is constructed by hierarchical clustering over token embeddings, then unioned with all singleton tokens, making the exponential power set tractable. The machinery also includes a recursive mass-computation formula used in the loss, a pignistic transformation that turns the belief function into a sampling distribution, and credal-set width as the measure of epistemic uncertainty.","core_discovery":"The central claim is that replacing the standard softmax next-token distribution with a predicted finite random set—a belief function over a curated budget of token subsets—improves both answer quality and the model's ability to express epistemic uncertainty. The belief function is trained with a binary cross-entropy loss over belief values plus regularization terms encouraging non-negative masses that sum to one. At generation time, the pignistic probability (the center of mass of the associated credal set) is used to sample the next token, while the size of the credal set, measured by the upper-minus-lower probability bound on the most likely token, acts as a second-level uncertainty estimate. The authors report that RS-LLMs outperform standard LLMs on both CoQA and OBQA across Llama2-7b, Mistral-7b, and Phi-2, and that credal width provides a usable hallucination-detection signal when the model is given incorrect context.","pith_inferences":["The accuracy gains may partly come from the implicit label smoothing and structured regularization induced by training on group memberships; a direct comparison against a label-smoothing baseline would say how much of the improvement is due specifically to the belief-function semantics.","Because credal width is computed only over the chosen focal-set budget, uncertainty hidden inside larger clusters remains invisible; a testable extension would adapt the budget dynamically, adding clusters where the model is least confident.","The belief-function representation could be combined with semantic-level uncertainty: instead of measuring width over token groups, one could measure credal width over paraphrased answer sets, which may yield an even stronger hallucination signal.","The paper fixes the number of focal sets K by hand and finds that both too-small and too-large budgets hurt; an automatic criterion based on cluster overlap or validation uncertainty could remove this hyperparameter and likely improve robustness."],"forward_implications":["Any autoregressive LLM can be converted into an RS-LLM by redefining the final layer; the rest of the architecture, including LoRA-based fine-tuning, stays intact.","Uncertainty and hallucination signals are obtained from a single forward pass, without sampling multiple continuations or maintaining an ensemble.","Credal width separates correct from incorrect predictions and increases when the context is wrong, giving a direct, trainable hallucination detector.","The hierarchical-clustering budget makes belief prediction feasible at vocabulary scales of 32K to 51K tokens, with an ablation showing that a medium budget works best.","The same recipe extends beyond question answering: on the XSum summarization task, RS-Mistral achieved higher cosine similarity than standard Mistral."],"supporting_citations":[{"why":"Supplies the random-set neural network training paradigm and loss design that RS-LLM adapts to language modeling.","marker":"(Manchingal et al., 2025)"},{"why":"Defines belief functions, mass functions, and Möbius inversion, which form the mathematical basis of the output representation.","marker":"(Shafer, 1976a)"},{"why":"Introduces the pignistic transformation used to turn a predicted belief function into the token-sampling distribution.","marker":"(Smets & Kennes, 1994)"},{"why":"Establishes the credal set as the convex set of probability distributions associated with a belief function, grounding the credal-width uncertainty measure.","marker":"(Cuzzolin, 2008)"},{"why":"Provides the hierarchical agglomerative clustering algorithm used to select the budget of focal token sets.","marker":"(Müllner, 2011)"},{"why":"Supplies the CoQA conversational question-answering dataset used for both training and evaluation.","marker":"(Reddy et al., 2019)"},{"why":"Supplies the OBQA multiple-choice question-answering dataset, the second evaluation benchmark.","marker":"(Mihaylov et al., 2018)"},{"why":"Provides the Llama2-7b base model used in the main experiments and ablations.","marker":"(Touvron et al., 2023)"},{"why":"Provides the Mistral-7b base model used as a second architecture in the experiments.","marker":"(Jiang et al., 2023)"},{"why":"Provides the Phi-2 base model used as the third architecture in the experiments.","marker":"(Javaheripi et al., 2023)"}],"fun_headline_variants":["Belief sets make LLMs sharper on QA and self-aware","Random-set LLMs outperform and expose their doubts","LLMs that know when they're guessing: RS-LLM wins","Belief-function LLMs: better answers, built-in caution","From softmax to belief sets: LLMs detect hallucination"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole result rests on the assumption that the approximate formula used to convert the model's set-belief scores into mass values, which only uses the limited dictionary of token groups, still yields a mathematically valid belief function; if it does not, the pignistic probability and credal-width uncertainty signals built on it are not well-defined.","fun_headline_variants_meta":{"raw":{"variants":["Belief sets make LLMs sharper on QA and self-aware","Random-set LLMs outperform and expose their doubts","LLMs that know when they're guessing: RS-LLM wins","Belief-function LLMs: better answers, built-in caution","From softmax to belief sets: LLMs detect hallucination"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000169,"raw_usage":{"total_tokens":1267,"prompt_tokens":948,"completion_tokens":319,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":234}},"tokens_in":564,"tokens_out":319,"duration_ms":3852,"temperature":1.0,"reasoning_tokens":234,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:24:48.097456+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train an RS-LLM on a small vocabulary where the full power set is enumerable, compute the exact Möbius-inverse masses and exact credal sets, and compare them with the recursive budget approximation; if the approximate pignistic probabilities or credal widths differ materially, or if many predicted masses are negative and must be zeroed out by post-processing, the central uncertainty claims are not supported.","supporting_citations":[{"cited_title":"K., Mubashar, M., Wang, K., Shariatmadar, K., and Cuzzolin, F","cited_arxiv_id":null,"evidence_quote":"Supplies the random-set neural network training paradigm and loss design that RS-LLM adapts to language modeling."},{"cited_title":"and Kennes, R","cited_arxiv_id":null,"evidence_quote":"Introduces the pignistic transformation used to turn a predicted belief function into the token-sampling distribution."},{"cited_title":"On the credal structure of consistent probabilities","cited_arxiv_id":null,"evidence_quote":"Establishes the credal set as the convex set of probability distributions associated with a belief function, grounding the credal-width uncertainty measure."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Phi-2 base model used as the third architecture in the experiments."}],"review_version":1}