{"id":"47ecd213-00b9-46f4-a649-4052fac55ad2","arxiv_id":"2507.18342","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"EgoExoBench introduces 7,330 multiple-choice questions across 11 tasks that measure how well multimodal AI models align, associate, and temporally reason across paired first-person and third-person videos; the best model reaches 51.7% versus 90.1% human accuracy.","lead":"Researchers built a benchmark of over 7,300 multiple-choice questions that test how AI video models connect what they see from a first-person camera with what they see from a third-person camera. They tested 13 leading multimodal models and found all of them well below human accuracy on these cross-view tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"EgoExoBench's filtering only removes text-answerable QAs; without single-view controls, low MLLM scores may not isolate cross-view reasoning.","rationale":"Reader's CONDITIONAL verdict is appropriate. The most load-bearing concern is construct validity: EgoExoBench must demonstrate that its tasks cannot be solved without integrating both views. The text-only filtering step in §3.2 addresses only text-answerability, not single-view answerability, and the paper provides no matched single-view controls. The cross-benchmark comparison in Table 5 is confounded by task and dataset differences. However, the benchmark's design (e.g., EWI, where the wearer is absent from the ego view, and the reference-video ablation in Table 2 showing gains from cross-view context) suggests cross-view information is at least partially necessary and useful. These positives prevent rejection, but the missing view-dropping analysis means the central claim is not yet secured. Numerical inconsistencies (duplicate Qwen2.5-VL-72B rows in Table 1; Gemini 2.5 Pro in appendix only) further undermine the precise headline gap, though not the qualitative direction. A view-dropping diagnostic would settle whether the concern lands. Thus the verdict remains CONDITIONAL, not changed.","tokens_in":25919,"tokens_out":10366,"duration_ms":99692,"concrete_test":"Run a view-dropping diagnostic on the released benchmark (or a random sample of ~500 items): for each question, evaluate the same MLLMs under three conditions—(a) only the query/candidate set from view A, (b) only view B, (c) both views. Compute per-task accuracy and a cross-view gain score = acc(both) − max(acc(viewA-only), acc(viewB-only)). If average cross-view gain is near zero or not significantly positive, the benchmark does not isolate cross-view reasoning; if the gain is large and consistent, the concern is resolved. Additionally, have two independent annotators label a sample of items as 'answerable from one view alone' and compare with the filter's decisions to measure filtering precision and recall.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is that EgoExoBench's remaining 7,330 MCQs actually require cross-view integration. Section 3.2's 'Vision-Grounded Filtering' discards only questions answerable from text alone, using a text-only LLM (Qwen2.5-32B) prompt (Figure 11). This filter is structurally blind to single-view solvability: for tasks whose options are video clips (TR, AR, OR, PR, EWI, DP, SE, AP), there is no text for the filter to reason over, and no check that the correct answer is indeterminable from one view alone. The paper's evidence that MLLMs are strong on single-view tasks comes from separate benchmarks (Table 5: Video-MME, EgoSchema, etc.) with different videos, question formats, and difficulty; it is not a matched control. The preliminary multi-video-input test (Section B.1) only shows models can identify duplicate clips, not that they can combine information from two views. If a nontrivial fraction of items is solvable from a single view (e.g., by scene or object recognition), the reported 38-42 point human gap would overstate the cross-view deficit and the central claim would not be established. The numerical inconsistency in Table 1 (Qwen2.5-VL-72B appears twice, with Avg 48.5 and 44.7; Gemini 2.5 Pro's 51.7 appears only in the appendix) compounds the uncertainty about the headline gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces EgoExoBench, a benchmark for evaluating multimodal large language models (MLLMs) on video understanding that requires integrating first-person (egocentric) and third-person (exocentric) views. The benchmark contains 7,330 four-choice questions across 11 subtasks grouped into three dimensions: Ego-Exo Relation, Ego-Exo View Transition, and Ego-Exo Temporal Reasoning. Questions are constructed from six public ego-exo datasets using annotation-derived templates, LLM-generated QA from Qwen2.5-32B and DeepSeek-V3, and human annotation. The authors evaluate 13 (or 14, counting the appendix) open- and closed-source MLLMs and report that the best model scores around 48% while deliberate human performance is 90.1%, leaving a 42-point gap. They further find that chain-of-thought prompting does not help and that adding a cross-view reference video helps some tasks but not others. The paper's central claim is that current MLLMs, despite strong single-view performance, lack the ability to semantically align, spatially associate, and temporally reason across ego-exo viewpoints.","tokens_in":26200,"tokens_out":8503,"duration_ms":85684,"significance":"If established, EgoExoBench would fill a genuine gap as the first benchmark targeting cross-view ego-exo reasoning in MLLMs, and the reported human-model gap would be an important result for embodied AI and human-robot collaboration. The paper's strengths include construction from multiple public datasets, a structured task taxonomy, a human baseline, a sanity check that MLLMs can distinguish multiple video inputs, and release of the benchmark. However, the central quantitative claim is currently under-supported: the benchmark lacks matched single-view control conditions, so low scores cannot be confidently attributed to cross-view reasoning rather than to general video difficulty or answer-format artifacts. In addition, the main table contains internal numerical contradictions and omits the best-performing model reported in the appendix. These issues are fixable and the benchmark resource is valuable, but the paper requires substantive revision before its central claim can be accepted.","major_comments":[{"comment":"The headline human-model gap is not reproducible from the reported numbers. In Table 1, Qwen2.5-VL-72B appears twice, once with Avg 48.5 and once with Avg 44.7; the main text states that Qwen2.5-VL-72B achieves the highest overall accuracy at 47.0%, but Table 1 shows GPT-o4-mini at 48.0 as the best overall model and Qwen2.5-VL-72B at 44.7. Appendix B.2 then reports Gemini 2.5 Pro as best overall at 51.7, yet this model is absent from the main table. Depending on which table is used, the best-model-versus-human gap is approximately 38.4 to 42.1 points, and the claimed best model changes. Please reconcile these numbers, include Gemini 2.5 Pro in the main results, and report a single consistent best-model accuracy.","section":"Table 1, Section 4.2, Appendix B.2"},{"comment":"The 'Vision-Grounded Filtering' step discards only questions that a text-only LLM can answer from the question text alone (Figure 11/prompt in Appendix A). This does not remove questions that are solvable from a single video view, and for tasks whose options are video clips (TR, AR, OR, PR, EWI, DP, SE, AP) there is no text for the filter to reason over at all. The comparison with single-view benchmarks in Table 5 uses different videos, question formats, and difficulty levels, so it is not a matched control. Without a control in which the same items are presented with only one view (or with same-view distractors), the reported low MLLM scores could reflect general video comprehension difficulty or MCQ artifacts rather than a cross-view reasoning deficit. Please add per-task single-view controls and an analysis of single-view solvability, and use them to substantiate the claim that the benchmark isolates cross-view reasoning.","section":"Section 3.2 and Section 3.3"},{"comment":"The construction of negative candidates for Action Relation is underspecified regarding viewpoint. The text says the ground-truth clip is 'temporally aligned with the query but recorded from a different viewpoint,' while negative candidates are 'selected from different temporal segments of the same video.' If the negatives come from the same viewpoint as the query, then the correct answer is the only candidate from a different viewpoint, and the task reduces to viewpoint identification rather than semantic action matching. Please clarify the viewpoint composition of all options in AR and OR, and verify with an ablation that the tasks cannot be solved by low-level viewpoint or appearance cues alone.","section":"Section 3.3.1, Action Relation and Object Relation"},{"comment":"The human baseline rests on a small sample: 30 questions per subtask (330 total) answered by two graduate students. The paper reports only a single average (90.1%) with no confidence intervals, no per-annotator breakdown, and no chance-level comparison, even though some subtasks may have uneven answer distributions. This baseline is load-bearing for the central 42-point gap. Please report variance, inter-annotator agreement, and per-subtask human accuracies, and consider increasing the human evaluation set or at least providing error bars.","section":"Section 4.2 and Appendix A.5"}],"minor_comments":[{"comment":"The model name is inconsistent: the main text uses 'GPT-o4-mini' while the reference list and Appendix B.2 use 'GPT-4o mini' (or 'GPT-4o-mini'). Please unify the naming and the count of evaluated models, which is stated as 13 in the abstract but appears to be 14 including Gemini 2.5 Pro.","section":"Throughout"},{"comment":"The EgoExoBench column in Table 5 reports different values for the same models than Table 1 and Table 4 (e.g., Qwen2.5-VL-72B appears as 45.3 versus 44.7; InternVL3-78B as 40.5 versus 41.4). Please ensure that all tables report the same underlying evaluation numbers.","section":"Table 5"},{"comment":"The construction text says TF2023 provides 'synchronized egocentric–exocentric image pairs,' but the task is described as video-based. Please clarify whether EWI (and DP) use video clips or static images, and if any subtask is image-based, qualify the 'video understanding' framing accordingly.","section":"Section 3.3.2, Egocentric Wearer Identification"},{"comment":"The chain-of-thought analysis samples 100 questions per subtask and reports per-task accuracy differences without error bars or significance tests. Given the small sample and the magnitude of the observed differences, please add confidence intervals or a significance analysis to support the claim that CoT degrades performance.","section":"Section 4.3"},{"comment":"The caption states that correct answers are 'bold-highlighted,' but the figure appears to use green highlighting. Please align the caption with the actual figure formatting.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The benchmark is a potentially useful resource and the paper addresses a timely and underexplored problem. However, the central claim about a cross-view reasoning deficit is not yet established because the benchmark lacks matched single-view controls, and the headline numbers are internally inconsistent. Both issues are fixable in revision. The use of Qwen2.5-32B for question generation and filtering is a mild circularity risk, but the fact that closed-source models (GPT-o4-mini, Gemini 2.5 Pro) achieve the top scores partially mitigates it; still, an independent human or second-model verification of a sample of items would strengthen the benchmark's validity."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the useful part. EgoExoBench is a real contribution: a 7,330-question multiple-choice benchmark spanning three dimensions of ego-exo reasoning, assembled from six public datasets. No one has built a cross-view QA benchmark of this scope for MLLMs before. The main qualitative result—models that look strong on single-view benchmarks land around 30–50% here, far below the reported human accuracy—is almost certainly in the right direction. The multi-video-input sanity check (models can pick duplicate clips) is a sensible piece of validity evidence, and the CoT analysis is informative.\n\nWhere it gets soft. First, the reported numbers don't line up. Table 1 lists Qwen2.5-VL-72B twice, at 48.5 and 44.7; the main text says 47.0; Gemini 2.5 Pro's 51.7 appears only in the appendix. Those are easy to fix but they undermine confidence until they are fixed. Second, the 'vision-grounded filtering' step only discards questions a text-only LLM can answer. It never tests whether the correct answer is determinable from a single view. For tasks where the options are video clips, there is no text for the filter to reason over at all. The paper's evidence that MLLMs are strong on single-view tasks comes from other benchmarks with different videos and formats, so it is not a matched control. This means the reported gap could overstate the cross-view deficit—some fraction of the 7,330 items may be solvable by looking at one view only. Third, the human baseline is underspecified and internally inconsistent: the main text says 1-minute and 3-minute conditions; the appendix says evaluators had unlimited time and could replay. Fourth, no data or code is released yet, so independent verification isn't possible.\n\nI don't think these are fatal. The central idea is sound, and the benchmark is likely to become a reference point in embodied-AI video understanding. But as it stands, the paper asserts more than it establishes: the headline 'cross-view reasoning' claim is not fully pinned down until single-view controls are added or the filter is shown to exclude single-view-solvable items.\n\nWho is this for? Anyone evaluating MLLMs on paired ego-exo video. I'd send it to review, but the referee should insist on corrected numbers, a released artifact, and a matched single-view control or a clear limitation statement.","headline":"A genuinely useful new benchmark, but the headline cross-view-reasoning claim is not yet fully supported until the numeric inconsistencies are fixed and single-view solvability is controlled.","tokens_in":26746,"tokens_out":2948,"would_cite":true,"duration_ms":28686,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces EgoExoBench, a 7,330-question benchmark showing that current multimodal large language models fall roughly 42 points behind deliberate human accuracy when asked to reason across first- and third-person video views.","keywords":["egocentric video understanding","exocentric video understanding","multimodal large language models","cross-view reasoning","video question answering","benchmark construction","temporal reasoning","spatial correspondence"],"falsifier":"Give an LLM the text-only version of a random sample of EgoExoBench questions (question and options, with no video frames) under the same standardized answer-extraction protocol; if accuracy on that set rises substantially above the 25 percent random baseline, the vision-grounded filter has leaked text-solvable items and the reported gap is inflated.","tokens_in":25719,"feed_emoji":"🎥","tokens_out":12676,"duration_ms":108907,"temperature":0.7,"pith_summary":"The paper introduces EgoExoBench, a benchmark of 7,330 four-choice video questions built from six paired egocentric-exocentric datasets. Its aim is to test whether multimodal large language models can do what humans do routinely: align the same action, object, person, or task across first- and third-person views, translate spatial information between those views, and reason about the order of events across two streams. The paper reports that the best models reach only 48 to 52 percent accuracy while humans deliberately answering the same questions reach 90.1 percent, and that egocentric-only pretraining and chain-of-thought prompting do not close the gap. The central claim is that cross-view reasoning is a distinct, currently missing capability that single-view video benchmarks do not measure.","feed_headline":"Multimodal AI models trail humans by 42 points on cross-view video","feed_subtitle":"The best models score 48%, about 42 points below deliberate human accuracy on 7,330 cross-view questions.","key_machinery":"The carrying mechanism is the benchmark's construction pipeline rather than a single algorithm. EgoExoBench aggregates synchronized and asynchronous ego-exo video pairs from six public datasets; questions are produced by three strategies (annotation-derived templates, LLM generation with DeepSeek-V3, and human annotation for fine-grained spatial understanding); and a consistency check plus a vision-grounded filtering step together use Qwen2.5-32B to discard any question answerable from text alone. Each multiple-choice item presents one query video plus four candidate videos from the other viewpoint, with negative options chosen to share scene, action, or temporal cues so that only genuine cross-view reasoning selects the correct answer. This filtering is what lets the authors attribute low model scores to a missing cross-view capability rather than to language shortcuts.","core_discovery":"EgoExoBench is the paper's central discovery: a benchmark that makes cross-view video reasoning measurable. Its eleven subtasks are organized into three dimensions: Ego-Exo Relation (task, action, object, and person matching across views), Ego-Exo View Transition (egocentric wearer identification, direction prediction, and body-part action understanding), and Ego-Exo Temporal Reasoning (action prediction, action ordering, sequence alignment, and skill evaluation). On this benchmark, 13 state-of-the-art MLLMs evaluated in a zero-shot setting score between 29.5 and 48.0 percent on average, with Gemini 2.5 Pro reaching 51.7 percent in the appendix, while deliberate human accuracy on a 330-question subset is 90.1 percent. Models that are strong on single-view benchmarks drop substantially when asked to relate two views. EgoGPT, a model fine-tuned on a large egocentric corpus, gains little over similarly sized general models, and chain-of-thought prompting hurts more than it helps on most subtasks.","pith_inferences":["The authors do not measure how often the Qwen2.5-32B filter is wrong; leaking text-solvable questions would inflate model scores and shrink the reported gap, whereas discarding questions that need visual grounding would change what the benchmark measures.","A testable extension is to train an MLLM on the benchmark's paired videos with an explicit spatial-alignment objective, such as predicting the egocentric wearer's bounding box in the exocentric frame; the largest gains should appear on Egocentric Wearer Identification and Direction Prediction, the two spatial subtasks with the widest human-model gaps.","Because the answer options are deliberately matched for scene and action, a model that learns viewpoint-invariant semantics without true spatial mapping could score well on Relation tasks while still failing View Transition; that dissociation would be a useful diagnostic for future systems."],"forward_implications":["If the central claim is right, strong single-view video benchmarks do not predict performance on cross-view tasks, so EgoExoBench should be used alongside them when evaluating embodied and assistant models.","Egocentric pretraining alone is not a substitute for training objectives that explicitly require associating first- and third-person views.","Chain-of-thought-style language reasoning is not the fix; architectures or prompting methods that interleave visual and textual evidence across streams are needed.","Providing a reference video from the other viewpoint can improve next-action prediction by up to about 9 points, so paired demonstrations may help future assistants.","The benchmark gives the field a concrete 7,330-question yardstick: closing the 42-point gap to deliberate human accuracy is a defined target."],"supporting_citations":[{"why":"Supplies synchronized egocentric-exocentric video and action/keystep annotations used across Relation, View Transition, and Temporal Reasoning subtasks.","marker":"[17]"},{"why":"Provides multi-agent multi-task video with action annotations used for Task, Action, and Object Relation plus Action Prediction and Action Ordering.","marker":"[26]"},{"why":"Provides asynchronous ego-exo demonstration videos and skill annotations used for Task Relation, Action Relation, Body Part Action Understanding, and Skill Evaluation.","marker":"[23]"},{"why":"Qwen2.5-32B powers the vision-grounded filtering, consistency checks, and distractor selection that define benchmark difficulty.","marker":"[70]"},{"why":"DeepSeek-V3 generates question-answer pairs and distractors for LLM-constructed subtasks such as Body Part Action Understanding and Sequence Alignment.","marker":"[36]"},{"why":"EgoSchema is the prior single-view egocentric benchmark whose strong model scores contrast with EgoExoBench's cross-view drop.","marker":"[40]"},{"why":"Video-MME is the third-person video QA benchmark used to show that the same models perform well on single-view tasks.","marker":"[14]"},{"why":"EgoGPT's egocentric-pretrained model is the baseline whose marginal gains support the claim that egocentric pretraining alone is insufficient.","marker":"[72]"},{"why":"TF2023 provides synchronized ego-exo frame pairs with person bounding boxes used to build the Egocentric Wearer Identification subtask.","marker":"[76]"},{"why":"CVMHAT supplies synchronized egocentric and top-view outdoor recordings with person annotations used for Person Relation.","marker":"[19]"}],"fun_headline_variants":["AI lags humans by 42 points on cross-view video reasoning","New test: MLLMs can't connect first- and third-person video","EgoExoBench: AI fails to relate egocentric and exocentric views","Cross-view video: AI scores 48%, humans 90%","AI's blind spot: seeing a scene from two vantage points"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The benchmark's validity depends on the assumption that the LLM-based filtering removes every question that can be answered from text alone or from a single view, so that the remaining 7,330 multiple-choice questions genuinely require cross-view visual reasoning.","fun_headline_variants_meta":{"raw":{"variants":["AI lags humans by 42 points on cross-view video reasoning","New test: MLLMs can't connect first- and third-person video","EgoExoBench: AI fails to relate egocentric and exocentric views","Cross-view video: AI scores 48%, humans 90%","AI's blind spot: seeing a scene from two vantage points"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000236,"raw_usage":{"total_tokens":1507,"prompt_tokens":952,"completion_tokens":555,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":460}},"tokens_in":568,"tokens_out":555,"duration_ms":5992,"temperature":1.0,"reasoning_tokens":460,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:14:19.888169+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Give an LLM the text-only version of a random sample of EgoExoBench questions (question and options, with no video frames) under the same standardized answer-extraction protocol; if accuracy on that set rises substantially above the 25 percent random baseline, the vision-grounded filter has leaked text-solvable items and the reported gap is inflated.","supporting_citations":[{"cited_title":"Lemma: A multi-view dataset for le arning m ulti-agent m ulti-task a ctivities","cited_arxiv_id":null,"evidence_quote":"Provides multi-agent multi-task video with action annotations used for Task, Action, and Object Relation plus Action Prediction and Action Ordering."},{"cited_title":"Egoexolearn: A dataset for bridging asynchronous ego-and exo-centric view of procedural activities in real world","cited_arxiv_id":null,"evidence_quote":"Provides asynchronous ego-exo demonstration videos and skill annotations used for Task Relation, Action Relation, Body Part Action Understanding, and Skill Evaluation."},{"cited_title":"Egoschema: A diagnostic benchmark for very long-form video language understanding","cited_arxiv_id":null,"evidence_quote":"EgoSchema is the prior single-view egocentric benchmark whose strong model scores contrast with EgoExoBench's cross-view drop."},{"cited_title":"Fusing personal and environmental cues for identification and segmentation of first-person camera wearers in third-person views","cited_arxiv_id":null,"evidence_quote":"TF2023 provides synchronized ego-exo frame pairs with person bounding boxes used to build the Egocentric Wearer Identification subtask."}],"review_version":2}