Pith. sign in

REVIEW 4 major objections 7 minor 3 cited by

PointArena: Probing Multimodal Grounding Through Language-Guided Pointing

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper introduces PointArena, a three-stage platform of static benchmark, human-preference arena, and real-robot manipulation for evaluating multimodal models' language-guided pointing, and reports that Molmo-72B leads while…

desk verdict Solid new pointing benchmark with overclaimed validation; R²=0.92 rests on three agents and the top-model edge is within noise. read the letter →

arxiv 2505.09990 v2 pith:HXU3LV7T submitted 2025-05-15 cs.CV

Long Cheng , Jiafei Duan , Yi Ru Wang , Haoquan Fang , Boyang Li , Yushan Huang , Elvis Wang , Ainaz Eftekhar
show 7 more authors
Jason Lee Wentao Yuan Rose Hendrix Noah A. Smith Fei Xia Dieter Fox Ranjay Krishna
This is my paper · ORCID
classification cs.CV
keywords multimodallargelanguagemodelslanguage-guidedpointingvisualgroundingbenchmarkhumanpreferenceevaluationroboticmanipulationspatialreasoningreferentiallocalizationsupervision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PointArena's central claim is that language-guided pointing can be measured as a standalone capability, and that a static pointing benchmark can stand in for both human preference and real physical task success. The paper builds three linked evaluation stages: Point-Bench, 982 curated image-query pairs spanning Spatial, Affordance, Counting, Steerable, and Reasoning; Point-Battle, a blind pairwise human-voting arena with more than 4,500 votes; and Point-Act, a real xArm robot that executes pick-and-place from a model's predicted points. Testing 16 multimodal models, it reports that Molmo-72B scores highest on Point-Bench, that explicit pointing supervision lifts accuracy far more than model scale does, and that Point-Bench accuracy correlates with human preference at $R^2=0.85$ and with robot task success at $R^2=0.92$. If these correlations hold, a cheap static test can rank models for embodied and assistive use, and pointing should be treated as its own training target rather than an emergent side effect of general vision-language training.

What carries the argument

The machinery is a single success rule applied across all stages: a predicted point is correct if it falls inside a human-verified binary target mask, and a task succeeds if the predicted set covers every target region. That rule makes Point-Bench automatically scoreable, makes model outputs directly comparable in Point-Battle, and converts naturally into robot actions in Point-Act because each point is just an image coordinate. The benchmark's five categories (Spatial, Affordance, Counting, Steerable, Reasoning) are the sampling mechanism for covering different grounding demands, and a query is admitted to the dataset only if one or fewer current multimodal models answer it correctly, which keeps the benchmark hard while making its composition depend on the failure modes of the models it evaluates.

What would settle it

Run a broader Point-Act validation with at least ten diverse multimodal models and several tabletop scenes, then compare model ordering by Point-Bench score with ordering by physical pick-and-place success rate; if the ranking correlation drops well below the reported $R^2=0.92$, the claim that Point-Bench predicts real-world pointing would be falsified.

Watch

Extended reading notes

Core claim

The paper's discovery is that pointing is a separable, trainable capability with external validity. On the 982-example Point-Bench, Molmo-72B outperforms all other tested models, with Gemini-2.5-Pro statistically tied; open-source models trained on explicit pointing data (Molmo, Qwen2.5-VL with PixMo) match or beat proprietary models, while LLaVA variants without such data land at 4.8–17.4%. The same predicted points drive all three stages, so the authors can compare static accuracy, human preference, and physical execution directly: Point-Bench agrees with Point-Battle at $R^2=0.85$ and with Point-Act at $R^2=0.92$, and Molmo-7B-D beats GPT-4o on the robot by 65% in their user study. The ablation results add a second finding: chain-of-thought prompting hurts pointing accuracy in both GPT-4o and Gemini-2.5-Flash, suggesting that spatial pointing benefits from tight, coordinate-oriented prompts rather than extended verbal reasoning.

Load-bearing premise

The load-bearing premise is that Point-Bench accuracy predicts real-world pointing ability, with the evidence being a linear fit to only three agents—Molmo-7B-D, GPT-4o, and a human reference—tested on one fixed scene with ten participants.

Editorial extensions

If this is right

  • Model releases can be screened cheaply: Point-Bench accuracy on 982 pairs gives a first-pass estimate of how well a model will point in physical pick-and-place settings.
  • Explicit pointing supervision is a stronger lever than parameter count; scaling from 7B to 72B changes accuracy by only a few points in the tested open-source families.
  • Prompt engineering for pointing should stay concise: adding chain-of-thought or verbose user-style phrasing hurts spatial grounding in GPT-4o and Gemini-2.5-Flash.
  • Arena-style human-preference rankings can track progress as static benchmarks saturate, because the two evaluation modes align at $R^2=0.85$.
  • Open-weight pointing models such as Molmo-7B-D can outscore proprietary APIs in human preference and robot usability, making them viable for assistive and embodied applications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: If the $R^2=0.92$ proxy relationship holds beyond the three evaluated agents, Point-Bench could serve as a low-cost screening gate for robot manipulation policies, reserving expensive physical trials for models that already pass the static threshold.
  • Editorial inference: The chain-of-thought finding suggests spatial pointing is more perceptual than deliberative; a testable extension is whether visual backbones pretrained on dense alignment tasks benefit more from additional pointing data than from reasoning prompts.
  • Editorial inference: Because dataset acceptance depends on current model failures, Point-Bench is inherently a moving target; future releases may need a rolling refresh from Point-Battle's user-uploaded images to avoid contamination and saturation.
  • Editorial inference: The benchmark's category structure could be reused to build curriculum data for pointing supervision, with per-category scores identifying which spatial skill a model lacks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces PointArena, a three-part evaluation platform for language-guided pointing in multimodal models: Point-Bench, a curated 982-item static dataset spanning five reasoning categories; Point-Battle, a live pairwise human-vote arena; and Point-Act, a real-world robotic manipulation setup with a user study. The authors benchmark 16 MLLMs, report that Molmo-72B is the top Point-Bench performer, claim that explicit pointing supervision boosts accuracy, and report strong correlations among the three evaluation stages, including an R^2=0.92 claim that Point-Bench accuracy predicts real-world task success.

Significance. If the results hold, PointArena would be a useful community resource: the dataset is curated and publicly available, the three-stage design is thoughtful, the zero-shot evaluation protocol is standard, and the authors report standard deviations over three runs. The inclusion of a live arena and a real-robot evaluation is a strength, and the paper honestly discloses the statistically insignificant Molmo-Gemini margin. However, the load-bearing validation of Point-Bench as a proxy for real-world pointing rests on a three-point regression, and several headline claims are stronger than the evidence supports, so the current version does not fully substantiate the central conclusions.

major comments (4)
  1. [§4.3 (Point-Act validation)] The claim 'Point-Bench accuracy predicts real-world task success' with R^2=0.92 is based on exactly three agents (Molmo-7B-D, GPT-4o, and a human reference) evaluated on one fixed scene. With n=3, a linear regression has one residual degree of freedom, so a high R^2 provides essentially no information about predictive validity. The human reference anchors the high end of both scales and is likely responsible for much of the fit; excluding it leaves only two model points. This does not support the load-bearing conclusion that Point-Bench is a reliable proxy for real-world pointing. Please collect data from additional agents and scenes, or explicitly reframe this as an illustrative pilot rather than a validation.
  2. [§4.2 and Abstract (Molmo-72B ranking)] The text states that Molmo-72B outperforms Gemini-2.5-Pro by 0.43 percentage points and calls this margin 'statistically insignificant (p≈0.29)', yet the Abstract says Molmo-72B 'consistently outperforms' other models and §4.2 says it 'achieves the highest performance on the Point-Bench benchmark'. These statements are internally inconsistent. The headline ranking should be reported as statistically tied at the top, with appropriate multiple-comparison correction if a ranking is still claimed.
  3. [§4.2 (Pointing supervision)] The claim that 'explicit pointing data is a key driver of model accuracy' is supported by comparing Qwen2-VL-7B (17.4%) with Qwen2.5-VL-7B (52.3%). These are different model generations that differ in architecture, pretraining data, and other training choices, not solely in the presence of PixMo pointing data. This confound does not justify the causal attribution of the large gain to pointing supervision. Provide a controlled comparison using the same base architecture with and without the pointing corpus, or substantially weaken the claim.
  4. [§3.2 (Dataset acceptance rule)] The Point-Bench construction accepts a query only if one or fewer of three anonymized MLLMs answer correctly. This makes the dataset composition depend on the failure modes of the filter models, whose identities are not disclosed. Since several evaluated models come from the same families as the likely filter models, the scoring on the resulting subset may be biased for or against particular models. This compromises the neutrality of the benchmark and the generality of conclusions about category difficulty. Please disclose the filter models and analyze the sensitivity of the results to the acceptance threshold and to the choice of filters.
minor comments (7)
  1. [§3.2] 'collected from public sources posted after 20 April 20, 2025' appears to be a typo; the intended date is probably 'April 20, 2025'.
  2. [§4.2 and Figure 5a] The text refers to 'GPT-o3 improved by 21.1 points over GPT-4-Turbo' but Figure 5a labels this as GPT-4.1; the text also mentions 'Gemini-2.5-Flash' while the figure caption says 'Gemini-2.0-Flash'. Please align model names between text and figures.
  3. [References] The Deitke et al. 2024a and 2024b entries appear to be the same paper with slightly different subtitles; please merge or correct the duplicate.
  4. [§3.4] The phrase 'double-blind MLLM' is ambiguous; it likely means that the user (not the model) is blind to the agent identity. Please reword for clarity.
  5. [Figure 5b] The R^2=0.85 regression is computed from only five models; please include confidence intervals and explicitly acknowledge the small sample size in the text.
  6. [§1 and §3.2] The description of Point-Bench as 'the largest benchmark for evaluating language-guided pointing' is asserted without a comparison to other benchmarks; please cite or qualify this claim.
  7. [Throughout] There are inconsistent model-name spellings such as 'LLaV A' (LLaVA) and 'Molmo-7B-O'; please standardize the naming.

Circularity Check

2 steps flagged · score 6.0 of 10

The R²=0.92 proxy claim is an in-sample fit over three agents including a human reference, and the PixMo-sourced Steerable split partly explains the measured supervision gain.

  1. fitted input called prediction [Section 4.3, paragraph 'Point-Bench accuracy predicts real-world task success']
    "We validated Point-Bench as a reliable proxy by testing three agents—Molmo-7B-D, GPT-4o, and a human reference—on Point-Act. Success rates closely aligned with Point-Bench scores, yielding a strong linear correlation (R2 = 0.92). This high correlation indicates that Point-Bench is a reliable proxy for the pointing capability of multimodal LLMs in practical settings."

    The R2 is computed by fitting a line through the very three agents whose real-world success it is said to predict, with no held-out agent or scene. With only three points, estimating slope and intercept leaves one residual degree of freedom, and the human reference anchors the upper end of both the Point-Bench and Point-Act scales, making a near-perfect line essentially forced. Excluding the human leaves two model points for which the correlation is undefined. The claim that Point-Bench is a 'reliable proxy' is therefore a restatement of the in-sample fit rather than an out-of-sample prediction, so the paper's load-bearing proxy conclusion reduces to the fitted input.

  2. other [Section 3.2 (Steerable category) and Section 4.2 ('Pointing supervision significantly boosts performance')]
    "Steerable — Images from the PixMo dataset Deitke et al. [2024b] each include a reference point. ... incorporating the PixMo corpus into Qwen2.5-VL-7B increased performance to 52.3%, a substantial gain over the 17.4% achieved by Qwen2-VL-7B, which did not use such data."

    The supervision benefit is measured on Point-Bench, but Point-Bench's Steerable category is constructed from the same PixMo images used to train Qwen2.5-VL-7B. The 52.3% versus 17.4% comparison therefore partly measures whether the model has seen the exact evaluation images during training, not a general effect of pointing supervision. The benchmark's input data (PixMo images) is also the claimed cause of the performance gain, so the evaluation is circular for that category, and the paper's aggregate number inherits the overlap.

full rationale

PointArena is primarily a benchmark-construction paper rather than a derivation, and most of its content is self-contained: Point-Bench accuracy scores, Point-Battle Elo ratings, and the prompt-format ablations are independently measurable and do not reduce to the paper's own claims. The circularity concerns concentrate in two places. First, the headline validation that 'Point-Bench accuracy predicts real-world task success' is supported only by a linear fit through three agents (Molmo-7B-D, GPT-4o, and a human reference) on one fixed scene; with n=3 and the human reference anchoring the upper end of both axes, R2=0.92 is a property of the fitted line, not an out-of-sample prediction, so the proxy claim reduces to the fit. Second, the claim that pointing supervision boosts performance compares Qwen2.5-VL-7B, trained on PixMo, with Qwen2-VL-7B on Point-Bench, whose Steerable category is built from PixMo images; part of the measured gain is in-distribution recall rather than a general supervision effect. These are genuine partial circularities in two central claims, but the benchmark itself still has substantial independent content: most model rankings, the Point-Battle platform, and the CoT ablations do not depend on those circular steps. The Section 3.2 acceptance rule (keep queries only if one or fewer current MLLMs answer correctly) is self-referential in that dataset composition depends on the failure modes of the models being benchmarked, but it does not by itself force any particular accuracy result, so it is noted here without being counted as a separate circular step.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No fitted constants or invented physical entities appear in the benchmark formulation. The statistical claims depend on annotation-quality and metric assumptions listed above, plus the acceptance rule in Section 3.2. These are domain assumptions rather than free parameters.

assumptions (4)
  • domain assumption Pointing correctness is fully captured by whether a predicted point falls inside any annotated target mask, with exactly K predicted points matching K target regions.
    Defines the success metric in Section 3.1 and excludes partial credit or semantic ambiguity in human interpretation.
  • domain assumption SAM-generated masks refined by annotators are sufficiently accurate ground truth for pixel-level pointing evaluation.
    Section 3.2 uses SAM plus grid refinement; Section 5 admits coarse and imprecise boundaries for fine-grained shapes.
  • domain assumption For non-counting tasks, the first autoregressively generated point is the model's highest-confidence prediction.
    Section 4.1 evaluation protocol; if models order points differently, the metric can misrepresent ability.
  • domain assumption Images sourced post-April 2025 are not part of model training corpora.
    Section 3.2 states collection from public sources posted after April 20, 2025, to avoid contamination, but this cannot be verified from the text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PointArena: Probing Multimodal Grounding Through Language-Guided Pointing." pith.science (2026). https://pith.science/paper/HXU3LV7T

@misc{pith2026250509990,
  author       = {Pith},
  title        = {Pith review of: PointArena: Probing Multimodal Grounding Through Language-Guided Pointing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HXU3LV7T}},
  note         = {Machine review of arXiv:2505.09990}
}
read the original abstract

Pointing serves as a fundamental and intuitive mechanism for grounding language within visual contexts, with applications spanning robotics, assistive technologies, and interactive AI systems. While recent multimodal models have started to support pointing capabilities, existing benchmarks typically focus only on referential object localization tasks. We introduce PointArena, a comprehensive platform for evaluating multimodal pointing across diverse reasoning scenarios. PointArena comprises three components: (1) Point-Bench, a curated dataset containing approximately 1,000 pointing tasks across five reasoning categories; (2) Point-Battle, an interactive, web-based arena facilitating blind, pairwise model comparisons, which has already gathered over 4,500 anonymized votes; and (3) Point-Act, a real-world robotic manipulation system allowing users to directly evaluate multimodal model pointing capabilities in practical settings. We conducted extensive evaluations of both state-of-the-art open-source and proprietary multimodal models. Results indicate that Molmo-72B consistently outperforms other models, though proprietary models increasingly demonstrate comparable performance. Additionally, we find that supervised training specifically targeting pointing tasks significantly enhances model performance. Across our multi-stage evaluation pipeline, we also observe strong correlations, underscoring the critical role of precise pointing capabilities in enabling multimodal models to effectively bridge abstract reasoning with concrete, real-world actions. Project page: https://pointarena.github.io/

Figures

Figures reproduced from arXiv: 2505.09990 by the authors.

Figure 1
Figure 1. Overview of PointArena. PointArena consists of three components: Point-Bench, a curated dataset for evaluating grounded pointing across five reasoning types; Point-Battle, a live platform for blind, pairwise model comparisons with user voting; and Point-Act, real-world task involving manipulation via pointing-based language commands. Recent advances in multimodal models have begun to incorporate more dynamic and spa… view at source ↗
Figure 2
Figure 2. Overview of the five Point-Bench categories and the annotation UI. Point-Bench includes 982 image-query pairs grouped into five categories: Spatial (positional references), Affordance (functional part identification), Counting (attribute-based grouping), Steerable (relative pointing), and Reasoning (open-ended visual inference). Each example shows a representative query and the corresponding target. On the right, we… view at source ↗
Figure 3
Figure 3. Success rates of MLLMs on Point-Bench across six task categories: Spatial, Affordance, Counting, Steerable, Reasoning, and Average. Each bar represents the mean success rate (%) for a given model, with error bars indicating standard deviation across three evaluation runs. The “Human” bar serves as an upper-bound reference. The results demonstrate substantial performance disparities, with top models (e.g., GPT-4o, Ge… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative predictions across Point-Bench categories. Example model predictions are shown for each of the five Point-Bench categories: Spatial, Affordance, Counting, Steerable, and Reasoning. Each colored dot corresponds to a prediction from a different MLLM, labeled …
Figure 5
Figure 5. Figure 5: Insights drawn from Point-Battle and Point-Bench. (a) This figure shows Point-Bench performance (%) of MLLMs over time, grouped by model family. A sharp performance increase is observed in models released after the PixMo dataset or RoboPoint (dashed line, October 2024 …
Figure 6
Figure 6. Figure 6: Performance on Human Preference Evaluation with Point-Battle. We collected over 4,500 votes from more than 100 global participants. Based on the Elo ratings derived from these votes, we observed a clear preference for outputs from open-source models such as Molmo-7B-D …
Figure 7
Figure 7. Figure 7: Overview of the Point-Act system. (a) The Point-Act manipulation setup enables remote control of a real-world xArm 6 Lite robot via language instructions, allowing users to evaluate pointing MLLMs. (b) User-blind evaluations and SUS preference scores collected for each…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Frontier MLLMs remain far from mastering atomic visual perception: none reach 60% on a failure-derived, perception-only benchmark of ten capabilities.

  2. RoboPIN: Grounded Embodied Reasoning via Pinned Chain-of-Thought

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    Training a 4B vision-language model to emit identity-tracked, visually grounded reasoning anchors improves embodied spatial, multi-view, and pointing task performance over 7B baselines.

  3. Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models

    cs.RO 2026-06 unverdicted novelty 5.0 of 10

    Embodied-R1.5 is an 8B EFM achieving SOTA on 16 of 24 embodied VLM benchmarks, fine-tunable to outperform leading VLAs, with claimed zero-shot real-robot generalization.

Reference graph

Works this paper leans on

38 extracted references · 8 canonical work pages · cited by 3 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, and Leonidas J. Guibas. ReferIt3D : Neural listeners for fine-grained 3d object identification in real-world scenes. In 16th European Conference on Computer Vision (ECCV), 2020

    work page 2020

  3. [3]

    The claude 3 model family: Opus, sonnet, haiku

    Anthropic. The claude 3 model family: Opus, sonnet, haiku. https://www.anthropic.com/news/claude-3-family, 2024. Claude-3 Model Card

    work page 2024

  4. [4]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  5. [5]

    Werewolf arena: A case study in llm evaluation via social deduction, 2024

    Suma Bailis, Jane Friedhoff, and Feiyang Chen. Werewolf arena: A case study in llm evaluation via social deduction, 2024. URL https://arxiv.org/abs/2407.13943

    arXiv 2024

  6. [6]

    Vizwiz: nearly real-time answers to visual questions

    Jeffrey P Bigham, Chandrika Jayant, Hanjie Ji, Greg Little, Andrew Miller, Robert C Miller, Robin Miller, Aubrey Tatarowicz, Brandyn White, Samual White, et al. Vizwiz: nearly real-time answers to visual questions. In Proceedings of the 23nd annual ACM symposium on User interface software and technology, pages 333--342, 2010

    work page 2010

  7. [7]

    Chang, and Matthias Nießner

    Dave Zhenyu Chen, Angel X. Chang, and Matthias Nießner. Scanrefer: 3d object localization in rgb-d scans using natural language, 2020. URL https://arxiv.org/abs/1912.08830

    arXiv 2020

  8. [8]

    Gonzalez, and Ion Stoica

    Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. Chatbot arena: An open platform for evaluating llms by human preference, 2024. URL https://arxiv.org/abs/2403.04132

    arXiv 2024

Show all 38 references
  1. [9]

    Guesswhat?! visual object discovery through multi-modal dialogue, 2017

    Harm de Vries, Florian Strub, Sarath Chandar, Olivier Pietquin, Hugo Larochelle, and Aaron Courville. Guesswhat?! visual object discovery through multi-modal dialogue, 2017. URL https://arxiv.org/abs/1611.08481

    2017 arXiv

  2. [11]

    Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, and Aniruddha Kembhavi

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison-Burch,...

    2024 arXiv

  3. [12]

    Ar2-d2: Training a robot without a robot

    Jiafei Duan, Yi Ru Wang, Mohit Shridhar, Dieter Fox, and Ranjay Krishna. Ar2-d2: Training a robot without a robot. arXiv preprint arXiv:2306.13818, 2023

    2023 arXiv

  4. [13]

    Manipulate-anything: Automating real-world robots using vision-language models

    Jiafei Duan, Wentao Yuan, Wilbert Pumacay, Yi Ru Wang, Kiana Ehsani, Dieter Fox, and Ranjay Krishna. Manipulate-anything: Automating real-world robots using vision-language models. arXiv preprint arXiv:2406.18915, 2024

    2024 arXiv

  5. [14]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Petko Georgiev et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  6. [15]

    A real-world webagent with planning, long context understanding, and program synthesis

    Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust. A real-world webagent with planning, long context understanding, and program synthesis. arXiv preprint arXiv:2307.12856, 2023

    2023 arXiv

  7. [16]

    Visual sketchpad: Sketching as a visual chain of thought for multimodal language models

    Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. arXiv preprint arXiv:2406.09403, 2024

    2024 arXiv

  8. [17]

    Referitgame: Referring to objects in photographs of natural scenes

    Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. Referitgame: Referring to objects in photographs of natural scenes. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 787--798, 2014

    2014

  9. [18]

    Tournament evaluation of large language models, 2025

    Richard Kelley and Duncan Wilson. Tournament evaluation of large language models, 2025. URL https://openreview.net/forum?id=5ZpN6W5uRm

    2025

  10. [19]

    Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything, 2023. URL https://arxiv.org/abs/2304.02643

    2023 arXiv

  11. [20]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024

    2024 arXiv

  12. [21]

    am-elo: A stable framework for arena-based llm evaluation, 2025

    Zirui Liu, Jiatong Li, Yan Zhuang, Qi Liu, Shuanghong Shen, Jie Ouyang, Mingyue Cheng, and Shijin Wang. am-elo: A stable framework for arena-based llm evaluation, 2025. URL https://arxiv.org/abs/2505.03475

    2025 arXiv

  13. [22]

    Improving your model ranking on chatbot arena by vote rigging, 2025

    Rui Min, Tianyu Pang, Chao Du, Qian Liu, Minhao Cheng, and Min Lin. Improving your model ranking on chatbot arena by vote rigging, 2025. URL https://arxiv.org/abs/2501.17858

    2025 arXiv

  14. [23]

    Cityrefer: Geography-aware 3d visual grounding dataset on city-scale point cloud data, 2023

    Taiki Miyanishi, Fumiya Kitamori, Shuhei Kurita, Jungdae Lee, Motoaki Kawanabe, and Nakamasa Inoue. Cityrefer: Geography-aware 3d visual grounding dataset on city-scale point cloud data, 2023. URL https://arxiv.org/abs/2310.18773

    2023 arXiv

  15. [24]

    Pivot: Iterative visual prompting elicits actionable knowledge for vlms, 2024

    Soroush Nasiriany, Fei Xia, Wenhao Yu, Ted Xiao, Jacky Liang, Ishita Dasgupta, Annie Xie, Danny Driess, Ayzaan Wahid, Zhuo Xu, Quan Vuong, Tingnan Zhang, Tsang-Wei Edward Lee, Kuang-Huei Lee, Peng Xu, Sean Kirmani, Yuke Zhu, Andy Zeng, Karol Hausman, Nicolas Heess, Chelsea Fin...

    2024 arXiv

  16. [25]

    Multimodal explanations: Justifying decisions and pointing to the evidence, 2018

    Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, and Marcus Rohrbach. Multimodal explanations: Justifying decisions and pointing to the evidence, 2018. URL https://arxiv.org/abs/1802.08129

    2018 arXiv

  17. [26]

    Plummer, Liwei Wang, Chris M

    Bryan A. Plummer, Liwei Wang, Chris M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models, 2016. URL https://arxiv.org/abs/1505.04870

    2016 arXiv

  18. [27]

    Sat: Spatial aptitude training for multimodal language models

    Arijit Ray, Jiafei Duan, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, Kuo-Hao Zeng, et al. Sat: Spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755, 2024

    2024

  19. [28]

    Openarena: An open platform for llm-as-a-judge evaluation

    SYV-AI. Openarena: An open platform for llm-as-a-judge evaluation. https://github.com/syv-ai/OpenArena, 2024. Accessed: 2025-05-09

    2024

  20. [29]

    Alex Hofer, Jan Humplik, Atil Iscen, Mithun George Jacob, Deepali Jain, Ryan Julian, Dmitry Kalashnikov, M

    Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, Steven Bohez, Konstantinos Bousmalis, Anthony Brohan, Thomas Buschmann, Arunkumar Byr...

    2025 arXiv

  21. [30]

    A new look at infant pointing

    Michael Tomasello, Malinda Carpenter, and Ulf Liszkowski. A new look at infant pointing. Child development, 78 0 (3): 0 705--722, 2007

    2007

  22. [31]

    A comprehensive review of multimodal large language models: Performance and challenges across different tasks, 2024

    Jiaqi Wang, Hanqi Jiang, Yiheng Liu, Chong Ma, Xu Zhang, Yi Pan, Mengyuan Liu, Peiran Gu, Sichen Xia, Wenjun Li, Yutong Zhang, Zihao Wu, Zhengliang Liu, Tianyang Zhong, Bao Ge, Tuo Zhang, Ning Qiang, Xintao Hu, Xi Jiang, Xin Zhang, Wei Zhang, Dinggang Shen, Tianming Liu, and S...

    2024 arXiv

  23. [32]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  24. [33]

    Grok-2 model card

    xAI. Grok-2 model card. https://x.ai/news/grok-2, 2024. Large language model

    2024

  25. [34]

    A survey on multimodal large language models

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. A survey on multimodal large language models. National Science Review, 11 0 (12): 0 nwae403, 11 2024. ISSN 2095-5138. doi:10.1093/nsr/nwae403. URL https://doi.org/10.1093/nsr/nwae403

    2024 doi

  26. [36]

    Berg, and Tamara L

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C. Berg, and Tamara L. Berg. Modeling context in referring expressions, 2016 b . URL https://arxiv.org/abs/1608.00272

    2016 arXiv

  27. [37]

    Robopoint: A vision-language model for spatial affordance prediction for robotics

    Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali, Arsalan Mousavian, and Dieter Fox. Robopoint: A vision-language model for spatial affordance prediction for robotics. arXiv preprint arXiv:2406.10721, 2024

    2024 arXiv

  28. [38]

    Robopoint: A vision-language model for spatial affordance prediction in robotics

    Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali, Arsalan Mousavian, and Dieter Fox. Robopoint: A vision-language model for spatial affordance prediction in robotics. In Proc. Conference on Robot Learning (CoRL), volume 270, pages...

    2025

  29. [39]

    Auto-arena: Automating llm evaluations with agent peer battles and committee discussions, 2024

    Ruochen Zhao, Wenxuan Zhang, Yew Ken Chia, Weiwen Xu, Deli Zhao, and Lidong Bing. Auto-arena: Automating llm evaluations with agent peer battles and committee discussions, 2024. URL https://arxiv.org/abs/2405.20267

    2024 arXiv

  30. [40]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena, 2023. URL https://arxiv.org/abs/2306.05685

    2023 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.