{"id":"989729ee-70b9-43ce-ada4-627c2d928537","arxiv_id":"2501.08838","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ToMATO is a new Theory of Mind benchmark built from AI-generated conversations, covering five mental states, false beliefs, and personality traits, and it shows current LLMs still fall short of human-level social reasoning.","lead":"This paper introduces ToMATO, a benchmark that tests how well large language models infer the thoughts, feelings, and intentions of characters in conversations. The conversations are generated by two AI agents role-playing with different personalities, and the benchmark shows that even advanced models lag behind human performance.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The benchmark's ground truth is a single stochastic LLM sample; without evidence that the verbalized thoughts are stable and uniquely inferable, ToMATO may measure prompted text prediction rather than Theory of Mind.","rationale":"The reader's verdict is sound: ToMATO is a careful benchmark with released code, thorough analyses, and a human baseline, and it makes a plausible case that current LLMs underperform humans on this task. My stress-test does not overturn that verdict. The load-bearing point is exactly the ground-truth status of the Inner Speech thoughts. The paper is transparent that the thoughts are generated by Llama-3-70B-Instruct under prompts, and it validates that humans can often select the labeled thought from options, but that is also the behavior ToM is supposed to explain. The additional sharpening is that the sampling process is stochastic, so the 'correct' thought is not a stable target, and the validation protocol may be biased by revealing the answer. The ablation in Table 6 provides indirect evidence: even when the second-order agent can see the first-order agent's thought, it still generates a mismatched second-order thought 32% of the time (human-judged), indicating that some false beliefs are LLM inconsistencies rather than consequences of information asymmetry. However, this does not make the benchmark useless: it remains a challenging, transparent, and largely reproducible resource, and the false-belief subset could still be valid if the concretely tested stability and uniqueness conditions hold. The recommended action is to keep the conditional verdict and run the proposed checks before treating ToMATO scores as evidence about machine Theory of Mind.","tokens_in":20797,"tokens_out":10972,"duration_ms":124399,"concrete_test":"Pick 100 ToMATO conversations and their questions. (1) Run the released generation code with several random seeds (same temperature and prompts) and measure how often the generated thought for a fixed turn changes; if the correct thought changes in, say, more than 10% of turns, the ground truth is sampling-dependent. (2) For 100 questions, show only the conversation and the four options to MTurk workers who do not know which option is the generated thought, and ask them to pick the best answer; compare their free choices to the ToMATO label. If independent humans agree with the label at rates far below the reported 91.7% validation, or if regenerated thoughts diverge substantially, the benchmark needs a construct-validity study before its ToM claims can be accepted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on treating the (thought) tokens sampled in Sec. 4 (Eqs. 1-2) as the characters' actual mental states. Since πA and πB are stochastic LLMs, the same scenario and conversation prefix can yield different thoughts on different samples; there is no independent fact of the matter about which thought is correct. The quality-validation step asks annotators to confirm that the provided answer is correct and that the other options are incorrect, which anchors them to the sampled thought and cannot establish uniqueness. The false-belief subset doubles down: a 'false belief' is a disagreement between two separate samples (A's first-order thought and B's second-order thought), and Table 6 shows B generates a wrong second-order thought 32% of the time even when A's thought is visible, so part of ToMATO-FB may reflect generator inconsistency rather than information-asymmetry-induced false beliefs. If the labeled thoughts are not stable across sampling and are not the unique mental state a competent observer would infer, then model scores measure skill at predicting one particular LLM's prompted continuations, not human-like Theory of Mind.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces ToMATO, a Theory-of-Mind (ToM) benchmark built from LLM-LLM conversations. Role-playing LLMs with Big Five personality profiles and information asymmetry are prompted to verbalize their thoughts before each utterance, and those verbalized thoughts are used as ground-truth answers to multiple-choice questions about first- and second-order mental states across five categories: belief, intention, desire, emotion, and knowledge. A false-belief subset, ToMATO-FB, is created by comparing A's first-order thought with B's second-order thought, with both human annotators and GPT-4o mini as judges. The paper reports evaluations of nine LLMs plus a human baseline, finding that the best models, including GPT-4o mini and Llama-3.1-70B, underperform humans overall and especially on false beliefs, and that performance varies with the character's personality traits.","tokens_in":21009,"tokens_out":5546,"duration_ms":57613,"significance":"If the ground-truth thoughts are accepted as valid records of mental states, ToMATO is a substantial new resource: it broadens ToM assessment to five mental-state categories at two orders, includes false beliefs about mental states, introduces personality-trait controls, and includes useful robustness analyses against lexical-overlap and word-label artifacts. The release of data and code, the multi-run evaluation, and the human validation pipeline are strengths. The central empirical finding—that current LLMs lag behind humans on this benchmark, particularly on second-order false beliefs—would be important. However, the benchmark's validity rests on an assumption that the paper does not independently establish: that a single sampled verbalized thought from a stochastic LLM is the character's authoritative and uniquely inferable mental state.","major_comments":[{"comment":"The ground-truth answers are single samples from stochastic role-playing LLMs (πA and πB), and no evidence is provided that these sampled thoughts are stable or uniquely inferable. The quality validation in Section 4 and Appendix B asks annotators to check that the provided answer is the only correct option among the distractors, but this does not establish that the sampled thought is the character's actual mental state. Because the same conversation prefix can yield different thoughts across samples, the benchmark may measure text-continuation prediction rather than ToM as defined in psychology. I request (a) a resampling study reporting the stability of the generated thoughts and (b) a human study in which annotators read only the conversation, independently infer the character's mental state, and are compared against the sampled thought.","section":"Section 4 (Inner Speech Prompting; Multiple-choice QA Dataset Construction), Eqs. (1)-(2)"},{"comment":"The construction of both the general QA set and the false-belief subset requires GPT-4o mini's agreement: instances are included only when deemed valid by both the majority of human annotators and GPT-4o mini, and FB questions are added only when both the majority of annotators and GPT-4o mini agree that B misunderstands A's mental state. Since GPT-4o mini is one of the evaluated models and its FB score (60.2%) is a headline result, the measurement is partly circular for that model: it is scored on items it helped select. To support the claim that GPT-4o mini underperforms on false beliefs, the authors should reconstruct the labels with human-only judgments or with a judge model that is not included in the evaluation set, and then re-report the affected scores.","section":"Section 4 (False Belief Detection; Quality Validation) and Table 3"},{"comment":"The FB detection method labels a question as a false-belief question when B's second-order thought differs from A's first-order thought, but the ablation data show that such mismatches occur even when A's thoughts are visible to B: in the condition without thought asymmetry (Table 6, row 2), human judges still identify false beliefs in 32.0% of cases and GPT-4o mini identifies 40.4%. This suggests that a substantial portion of ToMATO-FB may reflect generator inconsistency—B failing to condition on the visible thought—rather than information-asymmetry-induced false beliefs. The paper should report the overlap between the current FB subset and human-judged FB items, and filter FB items by human judgment or by agreement across multiple generation seeds.","section":"Section 6, Table 6"}],"minor_comments":[{"comment":"The human performance numbers in Table 3 are reported as point estimates without confidence intervals or inter-annotator agreement; please report the number of annotators per item, variance across annotators, and the standard error of the human mean.","section":"Section 5, Human Baseline"},{"comment":"The paper acknowledges that evaluating Llama-3-70B on a dataset generated by Llama-3-70B is unfair, but Llama-3.1-70B is still highlighted as state-of-the-art; consider presenting the generator-model results with an explicit caveat or excluding them from the main ranking.","section":"Section 5, Experimental Setup"},{"comment":"The main text says ToMATO uses 15 personality trait patterns, while Section 6 generates all 32 combinations for the correlation analysis; please clarify in the main text whether the 15 evaluation patterns are a subset of the 32 and why they were chosen.","section":"Section 4 and Appendix A, Table 7"},{"comment":"For models without system-prompt support, the system prompt is concatenated into the user message; please confirm that the final text format is identical across all evaluated models and report whether any answer-parsing failures occurred.","section":"Appendix C"}],"recommendation":"major_revision","confidential_remarks":"The benchmark is potentially valuable, but the ground-truth validity issue is load-bearing: the paper's central claim that LLMs lack human-level ToM depends on treating sampled verbalized thoughts as authoritative mental states. I would need to see a human construct-validity study and a labeling pipeline that does not include the evaluated GPT-4o mini before I could support acceptance. The current version is best framed as a proposal for a benchmark rather than a definitive demonstration that LLMs lack ToM."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, it delivers a genuinely broader ToM benchmark than what we had: first- and second-order mental states across belief, intention, desire, emotion, and knowledge, plus false beliefs about those states and personality-robustness splits. Second, its ground truth is the role-playing LLM's own verbalized thoughts, and the paper never validates that those thoughts are stable or uniquely inferable. That is the soft spot, and it is the load-bearing one.\n\nWhat is new and good: the LLM-LLM conversation setup with information asymmetry is a real departure from FANToM's single-generator approach. The Inner Speech prompting is simple and effective at producing distinguishable text for each mental-state category, as the z-statistics show. The false-belief subset extends beyond beliefs to other mental states, which is novel. The dataset and code are released, human validation is substantial (conversation quality, QA pairs, false-belief detection), and the authors are careful about known pitfalls: they check lexical-overlap and word-label shortcuts, and they honestly flag the Llama-generation fairness issue. The ablation showing information asymmetry drives false-belief generation is nice.\n\nThe soft spots, in proportion. The central worry is that the 'thoughts' are a single stochastic sample from Llama-3-70B. The paper does not test whether re-sampling the same turn yields the same or even similar thoughts, and the human QA validation anchors annotators to the sampled thought. So the benchmark may measure skill at predicting one LLM's prompted continuations rather than human-like ToM. This is not a fatal flaw for a stress-test benchmark, but it does mean the headline result (GPT-4o mini at 73.5 vs. human 87.3) is conditional on an unproven assumption. The false-belief subset has an additional mild circularity: GPT-4o mini helps select the subset, and GPT-4o mini is then evaluated on it. The impact is small because humans also have to agree, but the authors should re-score GPT-4o mini on a subset selected without its participation. The human baseline may also be artificially low if the verbalized thoughts are idiosyncratic to the generator; a human study where annotators independently predict the character's mental state would clarify this.\n\nWho this is for: people building or consuming ToM benchmarks, and anyone interpreting LLM social-reasoning results. It deserves a serious referee. I would send it to peer review, but with a required revision: add a stability analysis (sample multiple thoughts per turn, report agreement) and an independent construct-validity study, or alternatively reframe the benchmark as measuring inference of LLM-consistent mental states rather than ToM proper.","headline":"A useful, more comprehensive ToM benchmark whose scores rest on unvalidated LLM self-reports as ground truth; worth reviewing with a construct-validity requirement.","tokens_in":21556,"tokens_out":2788,"would_cite":true,"duration_ms":33094,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ToMATO, a new Theory-of-Mind benchmark built from LLM-LLM conversations, uses prompted thoughts as ground-truth mental states and finds that even GPT-4o mini trails human performance (73.5% vs 87.3%).","keywords":["Theory of Mind","large language models","benchmark","false belief","mental states","Inner Speech prompting","personality traits","multiple-choice QA"],"falsifier":"One concrete test: strip the parenthetical thoughts from a random sample of conversations, have a different LLM generate its own mental-state inferences for each character, and check whether the two sets of thoughts agree beyond chance; near-chance agreement would show the thoughts are not stable reads of the characters' mental states.","tokens_in":20590,"feed_emoji":"🧠","tokens_out":10535,"duration_ms":96490,"temperature":0.7,"pith_summary":"ToMATO is a new benchmark for testing whether large language models can infer the mental states of a conversational partner—beliefs, intentions, desires, emotions, and knowledge—in both first-order form (what A feels) and second-order form (what B thinks A feels). The paper's key construction device is Inner Speech prompting: before each utterance, a role-playing LLM writes its thought in parentheses, and that text becomes both the recorded mental state and the correct multiple-choice answer. By hiding each agent's thoughts, goals, and personality from the other, the generation process naturally produces false beliefs about those five mental states. On the resulting 5.4k questions, even GPT-4o mini underperforms human annotators (73.5% vs 87.3%), with the largest gap on false-belief questions. Existing benchmarks mostly cover only beliefs, rarely test false beliefs about mental states, and ignore personality, so this result suggests LLM Theory of Mind is narrower and less reliable than prior tests indicated.","feed_headline":"GPT-4o mini scores 73.5% vs humans' 87.3% on new theory-of-mind test","feed_subtitle":"A 5.4k-question benchmark now covers five mental state types plus false beliefs; no tested model matched human accuracy.","key_machinery":"The central mechanism is Inner Speech prompting, a template that asks a role-playing LLM to prefix each utterance with a mental-state sentence in parentheses. For first-order mental states the prompt begins with I think, I will, I want, I feel, or I know; for second-order states it begins with I think that he/she thinks, will, wants, feels, or knows. These verbalized thoughts become the ground-truth answers to the benchmark's multiple-choice questions. The second half of the machinery is information asymmetry: each agent's system prompt, which contains its goal and Big Five personality, and its parenthetical thoughts are hidden from the other agent. That asymmetry induces false beliefs about the five mental-state types, which the ToMATO-FB subset then tests.","core_discovery":"The paper's central claim is that a Theory-of-Mind benchmark can be built by having two role-playing LLMs converse while verbalizing their mental states, and that the verbalized thoughts are valid answers to questions about the characters' mental states. ToMATO covers belief, intention, desire, emotion, and knowledge at both first and second order, and its ToMATO-FB subset isolates cases where character B misunderstands character A's mental state. The paper reports that no evaluated LLM reaches the human baseline, that false beliefs about emotions and intentions are the hardest category, and that accuracy varies with the assigned Big Five personality traits, falling for unconscientious, introversive, disagreeable, and neurotic characters. Read together, the experiments are offered as evidence that current LLMs lack human-level, personality-robust Theory of Mind.","pith_inferences":["Beyond the paper, a natural test is to rebuild ToMATO with a different generator model; if model rankings and the human gap persist, the findings reflect Theory of Mind difficulty rather than the style of one generator.","The information-asymmetry recipe suggests that false beliefs can be manufactured at scale in any conversational setting by hiding one agent's goals, personality, and inner speech, which could be used to build harder, more diverse ToM probes without narrative templates.","If the personality-robustness results transfer to real systems, a conversational AI that infers user mental states would be systematically less accurate for users who are less conscientious, more introverted, more disagreeable, or more neurotic; this is an equity-relevant implication the paper does not draw.","The fine-tuning result—improving ToMATO while degrading SocialIQa—predicts that other ToM training sets will also trade off against broad social reasoning unless training data spans multiple conversational formats; a cheap test is measuring SocialIQa before and after fine-tuning on ToMATO-FB alone."],"forward_implications":["ToMATO provides a single scoreboard for first- and second-order belief, intention, desire, emotion, and knowledge, plus a dedicated false-belief subset, so future model comparisons can separate which mental-state category is the bottleneck.","Every evaluated model falls short of the human baseline, and the gap is largest on false-belief questions, so current LLMs are not yet deployable for tasks where misreading another agent's thoughts has real cost.","LLM accuracy varies systematically with the assigned Big Five personality: unconscientious, introversive, disagreeable, and neurotic characters are harder to read, which means ToM performance in real deployments may differ across user groups.","The ablation results show that hiding thoughts, goals, and personality from the other agent raises false-belief frequency, giving a controllable recipe for generating false-belief data without hand-crafted stories.","ToMATO is less vulnerable to lexical-overlap and word-label shortcuts than FANToM and ToMBench, so improvements on it are less likely to be artifacts of surface correlations."],"supporting_citations":[{"why":"Introduces the concept of Theory of Mind that the benchmark is built to measure.","marker":"(Premack and Woodruff 1978)"},{"why":"Supplies the five-category inventory of mental states that defines ToMATO's question grid.","marker":"(Beaudoin et al. 2020)"},{"why":"Originates the false-belief paradigm that ToMATO-FB extends to five mental-state types.","marker":"(Wimmer and Perner 1983)"},{"why":"Provides ToMi, the prior false-belief QA benchmark that ToMATO contrasts with and extends.","marker":"(Le, Boureau, and Nickel 2019)"},{"why":"SOTOPIA contributes the 160 conversation scenarios and character profiles used to seed the LLM-LLM dialogues.","marker":"(Zhou et al. 2024b)"},{"why":"Describes Llama-3-70B-Instruct, the model that generates all conversations and thoughts.","marker":"(Dubey et al. 2024)"},{"why":"FANToM is the conversation-based ToM benchmark compared in Table 4 and supplies the quality-validation procedure.","marker":"(Kim et al. 2023b)"},{"why":"Raises the question of whether LLMs can simulate human participants, motivating the use of role-playing LLMs in place of human self-report.","marker":"Dillion et al. (2023)"},{"why":"Shows LLMs can express assigned personality traits and contributes the naive personality prompt that ToMATO extends to all Big Five factors.","marker":"(Jiang et al. 2023)"},{"why":"Provides the z-statistics test used to show ToMATO has few word-level spurious correlations in its options.","marker":"(Gardner et al. 2021)"}],"fun_headline_variants":["ToMATO: LLMs verbalize mental states to benchmark theory of mind","No LLM matches humans on ToMATO theory-of-mind test","False beliefs stump LLMs on ToMATO benchmark","ToMATO: LLM mental states revealed, humans still win on ToM","Personality traits mess with LLM theory-of-mind scores"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the parenthetical thoughts a role-playing language model produces under the paper's prompting are faithful reports of the character's mental state rather than merely plausible continuations; if that premise fails, ToMATO measures text inference, not Theory of Mind.","fun_headline_variants_meta":{"raw":{"variants":["ToMATO: LLMs verbalize mental states to benchmark theory of mind","No LLM matches humans on ToMATO theory-of-mind test","False beliefs stump LLMs on ToMATO benchmark","ToMATO: LLM mental states revealed, humans still win on ToM","Personality traits mess with LLM theory-of-mind scores"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000608,"raw_usage":{"total_tokens":2860,"prompt_tokens":999,"completion_tokens":1861,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":1768}},"tokens_in":615,"tokens_out":1861,"duration_ms":12700,"temperature":1.0,"reasoning_tokens":1768,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:16:10.969903+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete test: strip the parenthetical thoughts from a random sample of conversations, have a different LLM generate its own mental-state inferences for each character, and check whether the two sets of thoughts agree beyond chance; near-chance agreement would show the thoughts are not stable reads of the characters' mental states.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the concept of Theory of Mind that the benchmark is built to measure."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the five-category inventory of mental states that defines ToMATO's question grid."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Originates the false-belief paradigm that ToMATO-FB extends to five mental-state types."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Raises the question of whether LLMs can simulate human participants, motivating the use of role-playing LLMs in place of human self-report."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows LLMs can express assigned personality traits and contributes the naive personality prompt that ToMATO extends to all Big Five factors."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the z-statistics test used to show ToMATO has few word-level spurious correlations in its options."}],"review_version":1}