Pith. sign in

REVIEW 4 major objections 3 minor 2 cited by

VLM4D: Towards Spatiotemporal Awareness in Vision Language Models

T0 review · 4 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces VLM4D, a benchmark of real and synthetic videos with question-answer pairs designed to test whether vision-language models can track objects through space and time, and reports that current models fall well short of…

desk verdict A promising first-benchmark claim for VLM spatiotemporal reasoning, but the abstract alone doesn't carry the load: the construct-validity controls are missing. read the letter →

arxiv 2508.02095 v2 pith:2DY2LRBD submitted 2025-08-04 cs.CV cs.AI

Shijie Zhou , Alexander Vilesov , Xuehai He , Ziyu Wan , Shuwang Zhang , Aditya Nagachandra , Di Chang , Dongdong Chen
show 2 more authors
Xin Eric Wang Achuta Kadambi
This is my paper
classification cs.CVcs.AI
keywords VLM4Dspatiotemporalreasoningvisionlanguagemodelsvideounderstandingbenchmark4Dfeaturefieldsmotioncontinuityperspectiveawareness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces VLM4D, a benchmark of real and synthetic videos with question-answer pairs designed to test spatiotemporal awareness in vision-language models (VLMs). The authors' central aim is to establish that current state-of-the-art open and closed-source VLMs, despite strong static-image performance, fall well short of human baselines when they must track how objects move, rotate, and shift in perspective over time. Evaluations across the benchmark show the largest failures come from integrating multiple visual cues at once and keeping a coherent story across frames. The paper also presents evidence that two interventions—reconstructing a 4D feature field of the scene and fine-tuning on spatiotemporal question-answer data—improve these abilities. If the benchmark is a fair measure, then dynamic scene understanding is a distinct and still-unsolved capability that standard video benchmarks do not expose.

What carries the argument

The central object is VLM4D, a benchmark whose video-question pairs are designed to separate spatiotemporal reasoning into measurable components: translating objects, rotating objects, viewpoint and perspective changes, and continuity of motion across frames. The curated pairing is what carries the argument, because each question is constructed to require a specific spatial-temporal inference rather than a static-image shortcut. The paper also uses a 4D feature field reconstruction as an intervention: it builds a representation in which every spatial location can be queried over time, giving the model an explicit handle on motion, and combines it with supervised fine-tuning on spatiotemporal question-answer pairs. Human baselines on the same questions provide the reference point against which model gaps are defined.

What would settle it

A single controlled experiment would settle the benchmark's validity: paraphrase every question pair without changing the underlying spatiotemporal inference, and regrade the same models. If the human-model gap shrinks or reverses under paraphrasing, the gap is a wording artifact; if it persists across paraphrases and across a different set of videos, the deficiency claim is supported.

Watch

Extended reading notes

Core claim

The central discovery is that modern VLMs do not genuinely reason about space and time together: on VLM4D they show significant gaps against human baselines, and error analysis identifies two specific failure modes—models have trouble combining multiple visual cues in one inference, and they lose temporal coherence as motion unfolds. The benchmark itself is the load-bearing artifact: each video is paired with curated questions that isolate translational motion, rotational motion, perspective awareness, and motion continuity, so that a score can be attributed to a particular spatiotemporal skill. The paper further reports that feeding the model a reconstructed 4D feature field—a representation that lets a network query any point in space at any time—plus targeted spatiotemporal supervised fine-tuning, measurably improves comprehension on these tasks. In the authors' framing, the gap is not a matter of scale but a fundamental deficiency in how existing models encode dynamic structure.

Load-bearing premise

The load-bearing premise is that VLM4D's curated videos and question pairs measure spatiotemporal reasoning well enough that the models' low scores reflect a genuine inability to reason about dynamic scenes, not quirks of how the questions are phrased or which videos were chosen.

Editorial extensions

If this is right

  • If VLM4D is a valid measure, then state-of-the-art VLMs should be re-evaluated for dynamic scene understanding before being deployed in settings where motion matters, such as robotics and autonomous driving.
  • The reported human-model gap implies that progress on static image benchmarks does not transfer to spatiotemporal reasoning; new evaluation and training signals are needed.
  • The success of 4D feature field reconstruction suggests that explicit time-varying scene representations can supply the structural cues that frame-wise attention misses.
  • Targeted spatiotemporal fine-tuning, as demonstrated, offers a practical route to improve these abilities in existing VLMs without changing the architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the benchmark could be turned into a diagnostic by measuring each model's score per subtask (translation, rotation, perspective, continuity) to map exactly which spatiotemporal skill is missing, rather than reporting one aggregate gap.
  • A testable consequence the authors do not state: if QA paraphrase sets produce the same ranking of models, the conclusion would be robust; if not, the benchmark's wording would need to be treated as part of what is being measured.
  • The 4D feature field result points toward a larger design principle: future VLMs may need an explicit world model queried in space and time, rather than frame-wise attention over a video, to close the gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper introduces VLM4D, a benchmark designed to evaluate spatiotemporal reasoning in vision-language models (VLMs) using real-world and synthetic videos with curated question-answer pairs. The authors report that state-of-the-art open and closed-source VLMs perform significantly worse than humans, especially on tasks requiring integration of multiple visual cues and temporal coherence. They also propose two directions to improve spatiotemporal comprehension: leveraging 4D feature field reconstruction and targeted spatiotemporal supervised fine-tuning. The claims are based on the abstract only, as the full text is not available in the submitted manuscript.

Significance. If the benchmark is valid and the reported gaps are reproducible, VLM4D would fill a genuine gap in dynamic scene understanding evaluation and could catalyze research on temporal grounding in VLMs. The proposed improvement directions are plausible and of interest to the community. However, the significance of the work rests entirely on the construct validity of the benchmark and the rigor of the evaluation, neither of which can be assessed from the abstract alone. The paper's central claim of 'fundamental deficiencies' is potentially impactful but is currently unsupported by the visible evidence.

major comments (4)
  1. [Abstract] The abstract reports 'significant performance gaps' between VLMs and humans, but it provides no evidence that the question-answer pairs actually require dynamic, multi-frame understanding. Without ablations such as static-frame input, shuffled frame order, or question-only inference, the observed gaps could arise from single-frame visual cues or language priors rather than deficiencies in spatiotemporal reasoning. This is the central load-bearing point of the paper, so it must be addressed with concrete control conditions or ablation studies.
  2. [Abstract] The human baseline is described only as 'human baselines,' with no specification of the viewing protocol, such as whether participants were allowed to replay videos, the time limits imposed, the number of attempts, or whether answers were open-ended or multiple-choice. A confounded protocol could explain or exaggerate the reported gaps, so the methods for collecting human performance must be fully described and justified.
  3. [Abstract] The claimed effectiveness of '4D feature field reconstruction and targeted spatiotemporal supervised fine-tuning' is stated without any experimental details, including dataset splits, hyperparameters, or evaluation protocols. There is also a risk of circularity if the supervised fine-tuning data overlaps with the VLM4D test set; the paper must explicitly state how such overlap is avoided or measured.
  4. [Full text] The manuscript made available to the referee contains only the abstract; the main text with the benchmark construction, dataset statistics, model list, evaluation protocol, and full result tables is absent. Consequently, none of the quantitative claims or analyses can be checked, and the paper is not in a reviewable state. This is a blocking incompleteness that must be resolved before any substantive assessment is possible.
minor comments (3)
  1. [Abstract] The description of VLM4D as 'the first benchmark specifically designed to evaluate the spatiotemporal reasoning capabilities of VLMs' should be supported by a brief comparison with existing video question-answering benchmarks such as NExT-QA, EgoSchema, or TGIF-QA to clarify the claimed novelty.
  2. [Abstract] The term '4D feature field reconstruction' is introduced without definition or citation; the abstract should either explain the concept briefly or reference the approach so that readers can understand the proposed direction.
  3. [Abstract] The phrase 'fundamental deficiencies in existing models' is a strong generalization; unless the benchmark demonstrably covers a broad and representative set of spatiotemporal tasks, the authors may wish to qualify it as 'limitations on VLM4D tasks' to avoid overclaiming.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark's evaluation pipeline is independent of its inputs; no derivation reduces to a fit or self-citation in the available text.

full rationale

The paper presents a new benchmark, VLM4D, with curated question–answer pairs over real and synthetic videos, and reports evaluations of external state-of-the-art VLMs against human baselines. The abstract contains no equations, fitted parameters, or predictions that are defined in terms of the outcome being measured. The proposed remedies (4D feature field reconstruction, spatiotemporal supervised fine-tuning) are presented as directions, not as the source of the benchmark's validation, and no passage indicates that these methods were trained on VLM4D test data. Concerns about construct validity—whether static frames or language priors could answer the questions—are substantive evaluation-design issues, not circularity: they question whether the benchmark measures what it claims, but they do not make the derived conclusion logically identical to an input. No self-citations appear in the provided text, and no uniqueness theorem or prior-work ansatz is invoked to force a conclusion. Under the requirement to exhibit a specific reduction and avoid speculation, the available evidence supports a non-circular, primarily empirical contribution.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

No quantitative model is derived in the abstract, so no free parameters are visible. The benchmark assumes that its QA pairs capture the target reasoning ability, and that human performance is the correct reference.

assumptions (2)
  • domain assumption The curated QA pairs in VLM4D are a valid and representative measure of spatiotemporal reasoning.
    The benchmark's core validity depends on this assumption, stated in the abstract as 'carefully curated question-answer pairs emphasizing translational and rotational motions, perspective awareness, and motion continuity.' If the questions do not track the intended ability, the reported model gaps do not measure spatiotemporal reasoning.
  • domain assumption Human baselines are a meaningful reference for VLM spatiotemporal reasoning.
    The abstract compares models to 'human baselines' and interprets the gap as deficiency. This assumes humans are a fair ground-truth for the video QA tasks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VLM4D: Towards Spatiotemporal Awareness in Vision Language Models." pith.science (2026). https://pith.science/paper/2DY2LRBD

@misc{pith2026250802095,
  author       = {Pith},
  title        = {Pith review of: VLM4D: Towards Spatiotemporal Awareness in Vision Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2DY2LRBD}},
  note         = {Machine review of arXiv:2508.02095}
}
read the original abstract

Vision language models (VLMs) have shown remarkable capabilities in integrating linguistic and visual reasoning but remain fundamentally limited in understanding dynamic spatiotemporal interactions. Humans effortlessly track and reason about object movements, rotations, and perspective shifts-abilities essential for robust dynamic real-world understanding yet notably lacking in current VLMs. In this paper, we introduce VLM4D, the first benchmark specifically designed to evaluate the spatiotemporal reasoning capabilities of VLMs. Our benchmark comprises diverse real-world and synthetic videos accompanied by carefully curated question-answer pairs emphasizing translational and rotational motions, perspective awareness, and motion continuity. Through comprehensive evaluations of state-of-the-art open and closed-source VLMs, we identify significant performance gaps compared to human baselines, highlighting fundamental deficiencies in existing models. Extensive analysis reveals that VLMs struggle particularly with integrating multiple visual cues and maintaining temporal coherence. We further explore promising directions, such as leveraging 4D feature field reconstruction and targeted spatiotemporal supervised fine-tuning, demonstrating their effectiveness in enhancing spatiotemporal comprehension. Our work aims to encourage deeper exploration into improving VLMs' spatial and temporal grounding, paving the way towards more capable and reliable visual intelligence for dynamic environments.

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The TIME Machine: On The Power of Motion for Efficient Perception

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    TIME is a motion-based embedding from point tracks, trained only on synthetic data via masked autoencoding, that matches state-of-the-art video model performance with up to 10,000x less training data.

  2. SpaceVista: All-Scale Visual Spatial Reasoning from mm to km

    cs.CV 2025-10 conditional novelty 7.0 of 10

    SpaceVista contributes a 1M-QA, 38K-video all-scale spatial reasoning dataset spanning mm to km, a manually verified benchmark, and a fine-tuned 7B MLLM with scale experts and progressive reward training.

Reference graph

Works this paper leans on

105 extracted references · 28 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Phi-3 technical report: A highly capable language model locally on your phone

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024 a

    arXiv 2024

  3. [3]

    Phi-4 technical report

    Marah Abdin, Jyoti Aneja, Harkirat Behl, S \'e bastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, et al. Phi-4 technical report. arXiv preprint arXiv:2412.08905, 2024 b

    arXiv 2024

  4. [4]

    Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras, 2025

    Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, Dong Chen, Dongdong Chen, Junkun Chen, Weizhu Chen, Yen-Chun Chen, Yi ling Chen, Qi Dai, Xiyang Dai, Ruchao Fan, Mei Gao, Min Gao, Amit Garg, Abhishek Goswami, Junheng Hao, Amr Hendy, Yuxuan Hu, Xin Jin...

    2025

  5. [5]

    Cosmos world foundation model platform for physical ai

    Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.03575, 2025

    arXiv 2025

  6. [6]

    Pixtral 12b

    Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, et al. Pixtral 12b. arXiv preprint arXiv:2410.07073, 2024

    arXiv

  7. [7]

    System card: Claude opus 4 & claude sonnet 4

    Anthropic. System card: Claude opus 4 & claude sonnet 4. Technical report, 2025

    2025

  8. [8]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

Show all 105 references
  1. [9]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024

    2024

  2. [10]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

    1901

  3. [11]

    Spatial memory: how egocentric and allocentric combine

    Neil Burgess. Spatial memory: how egocentric and allocentric combine. Trends in Cognitive Sciences, 10 0 (12): 0 551--557, 2006

    2006

  4. [12]

    On learning mechanical laws of motion from video using neural networks

    Pradyumna Chari, Yunhao Ba, Shijie Zhou, Chinmay Talegaonkar, Shreeram Athreya, and Achuta Kadambi. On learning mechanical laws of motion from video using neural networks. IEEE Access, 11: 0 30129--30145, 2023

    2023

  5. [13]

    Spatialvlm: Endowing vision-language models with spatial reasoning capabilities

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455--144...

    2024

  6. [14]

    Visualgpt: Data-efficient adaptation of pretrained language models for image captioning

    Jun Chen, Han Guo, Kai Yi, Boyang Li, and Mohamed Elhoseiny. Visualgpt: Data-efficient adaptation of pretrained language models for image captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18030--18040, 2022

    2022

  7. [15]

    Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024 b

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024 b

    2024 arXiv

  8. [16]

    Sharegpt4video: Improving video understanding and generation with better captions

    Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang, et al. Sharegpt4video: Improving video understanding and generation with better captions. arXiv preprint arXiv:2406.04325, 2024 c

    2024 arXiv

  9. [17]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024 d

    2024 arXiv

  10. [18]

    Spatialrgpt: Grounded spatial reasoning in vision-language models

    An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatialrgpt: Grounded spatial reasoning in vision-language models. Advances in Neural Information Processing Systems, 37: 0 135062--135093, 2024 a

    2024

  11. [19]

    Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms. arXiv preprint arXiv:2406.07476, 2024 b

    2024 arXiv

  12. [20]

    Sharegpt-4o: Comprehensive multimodal annotations with gpt-4o, 2024

    Erfei Cui, Yinan He, Zheng Ma, Zhe Chen, Hao Tian, Weiyun Wang, Kunchang Li, Yi Wang, Wenhai Wang, Xizhou Zhu, Lewei Lu, Tong Lu, Yali Wang, Limin Wang, Yu Qiao, and Jifeng Dai. Sharegpt-4o: Comprehensive multimodal annotations with gpt-4o, 2024

    2024

  13. [21]

    Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023

    2023

  14. [22]

    Myers, and Anna C

    Julian De Freitas, Nicholas E. Myers, and Anna C. Nobre. Tracking the changing feature of a moving object. Journal of Vision, 16 0 (3): 0 22, 2016

    2016

  15. [23]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human langu...

    2019

  16. [24]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  17. [25]

    Palm-e: An embodied multimodal language model

    Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, et al. Palm-e: An embodied multimodal language model. 2023

    2023

  18. [26]

    Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill? arXiv preprint arXiv:2504.06514, 2025 a

    Chenrui Fan, Ming Li, Lichao Sun, and Tianyi Zhou. Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill? arXiv preprint arXiv:2504.06514, 2025 a

    2025 arXiv

  19. [27]

    Large spatial model: End-to-end unposed images to semantic 3d

    Zhiwen Fan, Jian Zhang, Wenyan Cong, Peihao Wang, Renjie Li, Kairun Wen, Shijie Zhou, Achuta Kadambi, Zhangyang Wang, Danfei Xu, et al. Large spatial model: End-to-end unposed images to semantic 3d. Advances in neural information processing systems, 37: 0 40212--40229, 2024

    2024

  20. [28]

    Vlm-3r: Vision-language models augmented with instruction-aligned 3d reconstruction

    Zhiwen Fan, Jian Zhang, Renjie Li, Junge Zhang, Runjin Chen, Hezhen Hu, Kevin Wang, Huaizhi Qu, Dilin Wang, Zhicheng Yan, et al. Vlm-3r: Vision-language models augmented with instruction-aligned 3d reconstruction. arXiv preprint arXiv:2505.20279, 2025 b

    2025 arXiv

  21. [29]

    Freyd and Ronald A

    Jennifer J. Freyd and Ronald A. Finke. Representational momentum. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10 0 (1): 0 126--132, 1984

    1984

  22. [30]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXiv:2405.21075, 2024

    2024 arXiv

  23. [31]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities

    Google Gemini Team. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. Technical report, 2025

    2025

  24. [32]

    Multimodal-gpt: A vision and language model for dialogue with humans

    Tao Gong, Chengqi Lyu, Shilong Zhang, Yudong Wang, Miao Zheng, Qian Zhao, Kuikun Liu, Wenwei Zhang, Ping Luo, and Kai Chen. Multimodal-gpt: A vision and language model for dialogue with humans. arXiv preprint arXiv:2305.04790, 2023

    2023 arXiv

  25. [33]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF conference on computer vision ...

    2022

  26. [34]

    Mmworld: Towards multi-discipline multi-faceted world model evaluation in videos

    Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, et al. Mmworld: Towards multi-discipline multi-faceted world model evaluation in videos. arXiv preprint arXiv:2406.08407, 2024 a

    2024 arXiv

  27. [35]

    Mojito: Motion trajectory and intensity control for video generation

    Xuehai He, Shuohang Wang, Jianwei Yang, Xiaoxia Wu, Yiping Wang, Kuan Wang, Zheng Zhan, Olatunji Ruwase, Yelong Shen, and Xin Eric Wang. Mojito: Motion trajectory and intensity control for video generation. arXiv preprint arXiv: 2412.08948, 2024 b

    2024 arXiv

  28. [36]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  29. [37]

    Visual perception of biological motion and a model for its analysis

    Gunnar Johansson. Visual perception of biological motion and a model for its analysis. Perception & Psychophysics, 14 0 (2): 0 201--211, 1973

    1973

  30. [38]

    How good is my video lmm? complex video reasoning and robustness evaluation suite for video-lmms

    Muhammad Uzair Khattak, Muhammad Ferjad Naeem, Jameel Hassan, Muzammal Naseer, Federico Tombari, Fahad Shahbaz Khan, and Salman Khan. How good is my video lmm? complex video reasoning and robustness evaluation suite for video-lmms. arXiv preprint arXiv:2405.03690, 2024

    2024 arXiv

  31. [39]

    Openvla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024

    2024 arXiv

  32. [40]

    Decomposing nerf for editing via feature field distillation

    Sosuke Kobayashi, Eiichi Matsumoto, and Vincent Sitzmann. Decomposing nerf for editing via feature field distillation. Advances in neural information processing systems, 35: 0 23311--23330, 2022

    2022

  33. [41]

    On space-time interest points

    Ivan Laptev. On space-time interest points. International Journal of Computer Vision, 64 0 (2-3): 0 107--123, 2005

    2005

  34. [42]

    Denker, Donnie Henderson, Richard E

    Yann LeCun, Bernhard Boser, John S. Denker, Donnie Henderson, Richard E. Howard, Wayne Hubbard, and Lawrence D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1 0 (4): 0 541--551, 1989

    1989

  35. [43]

    Alan M. Leslie. Spatiotemporal continuity and the perception of causality in infants. Perception, 13 0 (3): 0 287--305, 1984

    1984

  36. [44]

    Seed-bench: Benchmarking multimodal large language models

    Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. Seed-bench: Benchmarking multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13299--13308, 2024 a

    2024

  37. [45]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024 b

    2024 arXiv

  38. [46]

    Aria: An open multimodal native mixture-of-experts model

    Dongxu Li, Yudong Liu, Haoning Wu, Yue Wang, Zhiqi Shen, Bowen Qu, Xinyao Niu, Guoyin Wang, Bei Chen, and Junnan Li. Aria: An open multimodal native mixture-of-experts model. arXiv preprint arXiv:2410.05993, 2024 c

    2024 arXiv

  39. [47]

    Videochat: Chat-centric video understanding

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355, 2023 a

    2023 arXiv

  40. [48]

    Mvbench: A comprehensive multi-modal video understanding benchmark, 2023 b

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, and Yu Qiao. Mvbench: A comprehensive multi-modal video understanding benchmark, 2023 b

    2023

  41. [49]

    Mvbench: A comprehensive multi-modal video understanding benchmark

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22195--2...

    2024

  42. [50]

    4k4 DG en: Panoramic 4d generation at 4k resolution

    Renjie Li, Panwang Pan, Bangbang Yang, Dejia Xu, Shijie Zhou, Xuanyang Zhang, Zeming Li, Achuta Kadambi, Zhangyang Wang, Zhengzhong Tu, and Zhiwen Fan. 4k4 DG en: Panoramic 4d generation at 4k resolution. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  43. [51]

    Videoeval: Comprehensive benchmark suite for low-cost evaluation of video foundation model

    Xinhao Li, Zhenpeng Huang, Jing Wang, Kunchang Li, and Limin Wang. Videoeval: Comprehensive benchmark suite for low-cost evaluation of video foundation model. arXiv preprint arXiv:2407.06491, 2024 e

    2024 arXiv

  44. [52]

    Scenethesis: A language and vision agentic framework for 3d scene generation

    Lu Ling, Chen-Hsuan Lin, Tsung-Yi Lin, Yifan Ding, Yu Zeng, Yichen Sheng, Yunhao Ge, Ming-Yu Liu, Aniket Bera, and Zhaoshuo Li. Scenethesis: A language and vision agentic framework for 3d scene generation. arXiv preprint arXiv:2505.02836, 2025

    2025 arXiv

  45. [53]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36: 0 34892--34916, 2023

    2023

  46. [54]

    World model on million-length video and language with ringattention

    Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. World model on million-length video and language with ringattention. arXiv preprint, 2024 a

    2024

  47. [55]

    World model on million-length video and language with blockwise ringattention

    Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. World model on million-length video and language with blockwise ringattention. arXiv preprint arXiv:2402.08268, 2024 b

    2024 arXiv

  48. [56]

    Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216--233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216--233. Springer, 2024 c

    2024

  49. [57]

    Deepseek-vl: towards real-world vision-language understanding

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, et al. Deepseek-vl: towards real-world vision-language understanding. arXiv preprint arXiv:2403.05525, 2024

    2024 arXiv

  50. [58]

    Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks

    Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In Neural Information Processing Systems, 2019

    2019

  51. [59]

    Video-chatgpt: Towards detailed video understanding via large vision and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.05424, 2023

    2023 arXiv

  52. [60]

    Marr and S

    D. Marr and S. Ullman. Directional selectivity and its use in early visual processing. Proceedings of the Royal Society of London. Series B, Biological Sciences, 211 0 (1183): 0 151--180, 1981

    1981

  53. [61]

    The llama 4 herd: The beginning of a new era of natively multimodal ai innovation

    Meta. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation. Technical report, 2025

    2025

  54. [62]

    Video-bench: A comprehensive benchmark and toolkit for evaluating video-based large language models

    Munan Ning, Bin Zhu, Yujia Xie, Bin Lin, Jiaxi Cui, Lu Yuan, Dongdong Chen, and Li Yuan. Video-bench: A comprehensive benchmark and toolkit for evaluating video-based large language models. arXiv preprint arXiv:2311.16103, 2023

    2023 arXiv

  55. [63]

    Sceneteller: Language-to-3d scene generation

    Ba s ak Melis \"O cal, Maxim Tatarchenko, Sezer Karao g lu, and Theo Gevers. Sceneteller: Language-to-3d scene generation. In European Conference on Computer Vision, pages 362--378. Springer, 2024

    2024

  56. [64]

    Hello gpt-4o

    OpenAI. Hello gpt-4o. Technical report, 2024

    2024

  57. [65]

    A real-to-sim-to-real approach to robotic manipulation with vlm-generated iterative keypoint rewards

    Shivansh Patel, Xinchen Yin, Wenlong Huang, Shubham Garg, Hooshang Nayyeri, Li Fei-Fei, Svetlana Lazebnik, and Yunzhu Li. A real-to-sim-to-real approach to robotic manipulation with vlm-generated iterative keypoint rewards. arXiv preprint arXiv:2502.08643, 2025

    2025 arXiv

  58. [66]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, page...

    2016

  59. [67]

    The 2017 davis challenge on video object segmentation

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Arbel \'a ez, Alex Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation. arXiv preprint arXiv:1704.00675, 2017

    2017 arXiv

  60. [68]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. 2018

    2018

  61. [69]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of th...

    2021

  62. [70]

    Learning to localize objects improves spatial reasoning in visual-llms

    Kanchana Ranasinghe, Satya Narayan Shukla, Omid Poursaeed, Michael S Ryoo, and Tsung-Yu Lin. Learning to localize objects improves spatial reasoning in visual-llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12977--12987, 2024

    2024

  63. [71]

    Two-stream convolutional networks for action recognition in videos

    Karen Simonyan and Andrew Zisserman. Two-stream convolutional networks for action recognition in videos. In Advances in Neural Information Processing Systems (NIPS), pages 568--576, 2014

    2014

  64. [72]

    Spelke and Katherine D

    Elizabeth S. Spelke and Katherine D. Kinzler. Core knowledge. Developmental Science, 10 0 (1): 0 89--96, 2007

    2007

  65. [73]

    Alanavlm: A multimodal embodied ai foundation model for egocentric video understanding

    Alessandro Suglia, Claudio Greco, Katie Baker, Jose L Part, Ioannis Papaioannou, Arash Eshghi, Ioannis Konstas, and Oliver Lemon. Alanavlm: A multimodal embodied ai foundation model for egocentric video understanding. arXiv preprint arXiv:2406.13807, 2024

    2024 arXiv

  66. [74]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  67. [75]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  68. [76]

    Wan: Open and advanced large-scale video generative models

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025

    2025 arXiv

  69. [77]

    Vlm see, robot do: Human demo video to robot action plan via vision language model

    Beichen Wang, Juexiao Zhang, Shuwen Dong, Irving Fang, and Chen Feng. Vlm see, robot do: Human demo video to robot action plan via vision language model. arXiv preprint arXiv:2410.08792, 2024 a

    2024

  70. [78]

    Action recognition by dense trajectories

    Heng Wang, Alexander Kl \"a ser, Cordelia Schmid, and Cheng-Lin Liu. Action recognition by dense trajectories. In 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3169--3176, 2011

    2011

  71. [79]

    Grounded-videollm: Sharpening fine-grained temporal grounding in video large language models

    Haibo Wang, Zhiyang Xu, Yu Cheng, Shizhe Diao, Yufan Zhou, Yixin Cao, Qifan Wang, Weifeng Ge, and Lifu Huang. Grounded-videollm: Sharpening fine-grained temporal grounding in video large language models. arXiv preprint arXiv:2410.03290, 2024 b

    arXiv

  72. [80]

    Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024 c

    2024 arXiv

  73. [81]

    Compositional 4d dynamic scenes understanding with physics priors for video question answering

    Xingrui Wang, Wufei Ma, Angtian Wang, Shuo Chen, Adam Kortylewski, and Alan Yuille. Compositional 4d dynamic scenes understanding with physics priors for video question answering. arXiv preprint arXiv:2406.00622, 2024 d

    2024 arXiv

  74. [82]

    Internvideo2: Scaling foundation models for multimodal video understanding

    Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang, Yansong Shi, et al. Internvideo2: Scaling foundation models for multimodal video understanding. In European Conference on Computer Vision, pages 396--416. Springer, 2024 e

    2024

  75. [83]

    Internvideo2

    Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, et al. Internvideo2. 5: Empowering video mllms with long and rich context modeling. arXiv preprint arXiv:2501.12386, 2025

    2025 arXiv

  76. [84]

    Finetuned language models are zero-shot learners

    Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021

    2021 arXiv

  77. [85]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  78. [86]

    Cat4d: Create anything in 4d with multi-view video diffusion models

    Rundi Wu, Ruiqi Gao, Ben Poole, Alex Trevithick, Changxi Zheng, Jonathan T Barron, and Aleksander Holynski. Cat4d: Create anything in 4d with multi-view video diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 26057--26068, 2025

    2025

  79. [87]

    Videollm-mod: Efficient video-language streaming with mixture-of-depths vision computation

    Shiwei Wu, Joya Chen, Kevin Qinghong Lin, Qimeng Wang, Yan Gao, Qianli Xu, Tong Xu, Yao Hu, Enhong Chen, and Mike Zheng Shou. Videollm-mod: Efficient video-language streaming with mixture-of-depths vision computation. Advances in Neural Information Processing Systems, 37: 0 10...

    2024

  80. [88]

    Grok-2 beta release

    xAI. Grok-2 beta release. Technical report, 2024

    2024

  81. [89]

    Youtube-vos: Sequence-to-sequence video object segmentation

    Ning Xu, Linjie Yang, Yuchen Fan, Jianchao Yang, Dingcheng Yue, Yuchen Liang, Brian Price, Scott Cohen, and Thomas Huang. Youtube-vos: Sequence-to-sequence video object segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 585--601, 2018

    2018

  82. [90]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024 a

    2024 arXiv

  83. [91]

    Thinking in space: How multimodal large language models see, remember, and recall spaces

    Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. arXiv preprint arXiv:2412.14171, 2024 b

    2024 arXiv

  84. [92]

    Videorefer suite: Advancing spatial-temporal object understanding with video llm

    Yuqian Yuan, Hang Zhang, Wentong Li, Zesen Cheng, Boqiang Zhang, Long Li, Xin Li, Deli Zhao, Wenqiao Zhang, Yueting Zhuang, et al. Videorefer suite: Advancing spatial-temporal object understanding with video llm. arXiv preprint arXiv:2501.00599, 2024

    2024 arXiv

  85. [93]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on C...

    2024

  86. [94]

    Improving 2d feature representations by 3d-aware fine-tuning

    Yuanwen Yue, Anurag Das, Francis Engelmann, Siyu Tang, and Jan Eric Lenssen. Improving 2d feature representations by 3d-aware fine-tuning. In European Conference on Computer Vision, pages 57--74. Springer, 2024 b

    2024

  87. [95]

    Videollama 3: Frontier multimodal foundation models for image and video understanding

    Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501.13106, 2025

    2025 arXiv

  88. [96]

    Video-llama: An instruction-tuned audio-visual language model for video understanding

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding. arXiv preprint arXiv:2306.02858, 2023

    2023 arXiv

  89. [97]

    Combo: compositional world models for embodied multi-agent cooperation

    Hongxin Zhang, Zeyuan Wang, Qiushi Lyu, Zheyuan Zhang, Sunli Chen, Tianmin Shu, Yilun Du, and Chuang Gan. Combo: compositional world models for embodied multi-agent cooperation. arXiv preprint arXiv:2404.10775, 2024 a

    2024 arXiv

  90. [98]

    Llava-next: A strong zero-shot video understanding model, 2024 b

    Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. Llava-next: A strong zero-shot video understanding model, 2024 b

    2024

  91. [99]

    Mmvu: Measuring expert-level multi-discipline video understanding

    Yilun Zhao, Lujing Xie, Haowei Zhang, Guo Gan, Yitao Long, Zhiyuan Hu, Tongyan Hu, Weiyuan Chen, Chuhan Li, Junyang Song, et al. Mmvu: Measuring expert-level multi-discipline video understanding. arXiv preprint arXiv:2501.12380, 2025

    2025 arXiv

  92. [100]

    Llamafactory: Unified efficient fine-tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System ...

    2024

  93. [101]

    Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields

    Shijie Zhou, Haoran Chang, Sicheng Jiang, Zhiwen Fan, Zehao Zhu, Dejia Xu, Pradyumna Chari, Suya You, Zhangyang Wang, and Achuta Kadambi. Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields. In Proceedings of the IEEE/CVF Conference on Computer...

    2024

  94. [102]

    Dreamscene360: Unconstrained text-to-3d scene generation with panoramic gaussian splatting

    Shijie Zhou, Zhiwen Fan, Dejia Xu, Haoran Chang, Pradyumna Chari, Tejas Bharadwaj, Suya You, Zhangyang Wang, and Achuta Kadambi. Dreamscene360: Unconstrained text-to-3d scene generation with panoramic gaussian splatting. In European Conference on Computer Vision, pages 324--34...

    2024

  95. [103]

    Feature4x: Bridging any monocular video to 4d agentic ai with versatile gaussian feature fields

    Shijie Zhou, Hui Ren, Yijia Weng, Shuwang Zhang, Zhen Wang, Dejia Xu, Zhiwen Fan, Suya You, Zhangyang Wang, Leonidas Guibas, et al. Feature4x: Bridging any monocular video to 4d agentic ai with versatile gaussian feature fields. In Proceedings of the Computer Vision and Patter...

    2025

  96. [104]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023

    2023 arXiv

  97. [105]

    Apollo: An exploration of video understanding in large multimodal models

    Orr Zohar, Xiaohan Wang, Yann Dubois, Nikhil Mehta, Tong Xiao, Philippe Hansen-Estruch, Licheng Yu, Xiaofang Wang, Felix Juefei-Xu, Ning Zhang, et al. Apollo: An exploration of video understanding in large multimodal models. arXiv preprint arXiv:2412.10360, 2024

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.