Pith. sign in

REVIEW 2 major objections 5 minor 43 references

Pixels to Principles: Probing Intuitive Physics Understanding in Multimodal Language Models

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multimodal language models fail at intuitive physics because vision and language are misaligned, not because they cannot see.

desk verdict Useful performance update on MLLM intuitive physics, but the probing evidence for the vision-language misalignment claim doesn't hold up because the probe can exploit scene identity. read the letter →

arxiv 2507.16572 v1 pith:E44EJ52F submitted 2025-07-22 cs.CL

Mohamad Ballout , Serwan Jassim , Elia Bruni This is my paper
classification cs.CL
keywords multimodallargelanguagemodelsintuitivephysicsvision-languagealignmentembeddingprobingGRASPIntPhys2physicalplausibilityrepresentationanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that state-of-the-art multimodal language models still cannot reliably tell physically plausible from implausible videos on the GRASP and IntPhys 2 benchmarks, which use simulated scenes to test principles such as gravity and object permanence; the best models hover near chance while humans reach about 80 percent accuracy. To find out why, the paper probes the models' internal representations and reports that the vision encoder often carries enough information to classify plausibility, with a simple classifier reaching 85 percent accuracy on its features. That information is largely lost by the time it passes through the language model, even when the prompt spells out the relevant physical rules. The paper concludes that the main bottleneck is not visual perception but the alignment between what the model sees and what it reasons about in language.

What carries the argument

The central mechanism is embedding probing at architectural boundaries: the authors extract averaged token embeddings from the last vision transformer block, from after the vision-language projection layer (the connector that maps visual features into the language model's embedding space), and from the final language layer under simple and detailed prompts, then train one-hidden-layer MLP classifiers to read out plausible-versus-implausible labels. The before-versus-after comparison localizes information loss to the projection and alignment step, which is the load-bearing move of the paper.

What would settle it

Hold out all videos from one GRASP Level 2 scene during probe training, train on the other fifteen, and test on the held-out scene; if accuracy drops to chance, the probe was exploiting scene identity rather than physical plausibility.

Watch

Extended reading notes

Core claim

On GRASP Level 2, a probe (a simple classifier trained on the embeddings) trained on vision embeddings extracted before the projection layer classifies video plausibility well above chance, reaching 85.0 percent for InternVL 2.5 (78B), while probes on language-model embeddings under both a simple and a detailed physics prompt lag behind for all nine open-source models tested. The t-SNE visualizations show the vision embeddings form distinct clusters that track the 16 physics scenes, and this structure fades after the projection layer. From this the paper concludes that a vision-language misalignment emerges at intermediate task difficulty: the visual system encodes physical plausibility cues, but the language model fails to exploit them. On the harder IntPhys 2 set, neither vision nor language embeddings support above-chance classification, so the alignment claim is specific to tasks where the visual signal is already present.

Load-bearing premise

The load-bearing premise is that the probe's high accuracy on the averaged visual summary of a video reflects physical plausibility rather than lower-level cues such as which of the sixteen scenes the video belongs to, since videos from the same scene appear in both training and test splits.

Editorial extensions

If this is right

  • Scaling model size alone is unlikely to solve intuitive physics, since the largest variants stay near chance and one 78B model performs slightly worse than its smaller sibling.
  • Improving vision encoders will not close the gap on GRASP Level 2, because the needed information is already present in vision features; progress must come from better vision-language alignment.
  • Explicit rule-based prompting is not a reliable fix: the detailed physics prompt helped InternVL 2.5 and LLaVA-OneVision but hurt all Qwen 2.5 VL models.
  • On harder photorealistic benchmarks such as IntPhys 2, the bottleneck is different: even the vision encoder does not yet encode usable plausibility information.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the diagnosis is right, further model scaling alone should yield only small gains on intuitive physics, while training the projection layer with physics-specific objectives would attack the bottleneck the paper localizes.
  • Because the detailed prompt helped some model families and hurt others, prompt-based fixes are likely brittle; a more robust route is to change the alignment objective itself rather than the instruction text.
  • The same probing procedure could be applied on a frame-wise or layer-wise basis to identify exactly which transformer layer in the language model first loses the plausibility signal, turning the paper's coarse bottleneck into a precise one.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper evaluates four multimodal large language model families (InternVL 2.5, Qwen 2.5 VL, LLaVA-OneVision, and Gemini 2.0 Flash Thinking) on the GRASP and IntPhys 2 intuitive-physics benchmarks, reporting that all models perform near or below chance. To explain this, the authors extract embeddings from vision and language layers and train MLP probes to distinguish plausible from implausible videos. They report that probes on vision embeddings reach 70–85% accuracy on GRASP Level 2, while probes on language embeddings are lower, and they conclude that the vision encoder captures physical-plausibility cues but the language model fails to use them, identifying vision-language misalignment as the primary bottleneck. The main claim is that better vision encoders will not fix intuitive physics reasoning; only better vision-language alignment will.

Significance. If the central claim were established, it would be a valuable and actionable result for the MLLM community: it would redirect effort toward vision-language alignment rather than vision encoder scaling. The end-task accuracy measurements are straightforward, and the paper is honest about its limitations. The probing methodology is appropriate in spirit, and the inclusion of multiple model families and sizes is a strength. However, the central claim rests on the interpretation of probe accuracies, and that interpretation is currently undercut by a serious evaluation confound. The paper's own t-SNE analysis shows that vision embeddings cluster by scene rather than by plausibility, and the train/test split is at the video level, so scene identity is a plausible alternative explanation for the high probe accuracies. Until this confound is controlled, the significance of the paper is limited to a benchmark update rather than a diagnostic discovery.

major comments (2)
  1. [§5.2, §6 Insight 2] The MLP probe evaluation uses a random video-level 80/10/10 split on GRASP Level 2, which contains 16 scenes with 128 plausible and 128 implausible videos each. Because every scene appears in both training and test partitions, an MLP can exploit scene-specific feature-label regularities without learning any transferable notion of physical plausibility. The paper's own Figure 3(a) shows that vision embeddings form 15 distinct clusters corresponding to scenes, not to plausible/implausible classes. The reported 70–85% probe accuracies (e.g., InternVL 2.5 78B at 85.0%) therefore cannot be interpreted as evidence that vision encoders 'capture physical plausibility cues' unless a leave-one-scene-out evaluation, a shuffled-label baseline, or a per-scene analysis is provided. Please add these controls and report whether probe accuracy remains above chance when the test scenes are unseen during training.
  2. [§6 Insight 3] The comparison that supports 'the language model is not able to extract relevant information' is also confounded by the same scene-identity issue. Vision embeddings show strong scene clustering, while language embeddings appear more dispersed in Figure 3. If the vision probe's high accuracy is driven by scene identity, then the lower accuracy of language-embedding probes may simply reflect the loss of scene-level information during projection, not a loss of physical-plausibility information. To support the alignment-bottleneck interpretation, the authors need to demonstrate that the vision probe transfers to held-out scenes and, ideally, that language embeddings also retain scene structure but not plausibility structure. Without such controls, the conclusion that the vision encoder is sufficient and the language model is the bottleneck is not established.
minor comments (5)
  1. [§4.1] The text refers to 'Qwen 2.5 VL (78B)' in the leaderboard comparison, but Qwen 2.5 VL's largest configuration is 72B, not 78B. Please correct the model size reference for consistency with Table 1 and Table 2.
  2. [§5.2] For GRASP Level 1, the authors tested only shape and movement because color and directionality have four classes with only 32 samples per class after splitting. This is reasonable, but the claim in Insight 1 that 'all probed features achieved 100% accuracy' and that models 'effectively encode basic visual properties' should be scoped to the two binary tasks actually evaluated.
  3. [§4.1, Figure 1] The error bars in Figure 1 are described as the standard deviation across all tests of the respective dataset, but they do not reflect variance over random seeds or across runs for the generation-based methods. Reporting confidence intervals or significance tests for the differences between models and prompting methods would strengthen the end-task accuracy comparisons.
  4. [Appendix E] The detailed prompt in Appendix E enumerates intuitive physics concepts (Collision, Gravity, Continuity, Object Permanence, Solidity, Unchangeableness, Gravity Continuity, Gravity Inertia/Support, Inertia), but the paper does not explicitly map these concepts to the 16 GRASP Level 2 scenes mentioned in Section 3. Please provide the mapping so readers can follow which scenes are covered by the prompt.
  5. [§6, Figure 3] The t-SNE figure is a helpful qualitative illustration, but the red/blue color scheme for plausible/implausible may be difficult to distinguish for color-blind readers. Consider adding distinct markers or a grayscale-friendly palette. Also, the caption states '15 distinct clusters, with one larger cluster merging two similar scenes,' which is an interesting observation; please quantify cluster purity or scene-label accuracy to make the claim concrete.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper reports empirical probe measurements and its central claim is an interpretation, not a derivation from fitted inputs.

full rationale

The paper's derivation chain is empirical rather than inferential. It evaluates MLLMs on the published GRASP and IntPhys 2 benchmarks and trains MLP probes on extracted embeddings (Section 5.2), then compares probe accuracies across vision and language embedding positions. The central claim — that vision embeddings carry physical-plausibility information that the language model uses less effectively — is a measured comparison, not a quantity defined in terms of itself. No equation in the paper defines the target conclusion in terms of the probe inputs, and no fitted parameter is renamed as a prediction. The detailed prompt in Appendix E mirrors GRASP's scene definitions, but it is used as a test condition; the language-embedding accuracies are measured rather than derived from the prompt. GRASP was introduced by co-authors Jassim and Bruni, but it is an externally published benchmark and is not fitted to this paper's conclusion, so this self-citation is not load-bearing. Section 8 explicitly concedes that the exact cause of information loss is not pinpointed, which is consistent with an empirical finding rather than a circular derivation. Concerns about scene-identity confounds in the random video-level split are validity threats to the probe measurement, not circularity: they question whether the measurement supports the causal interpretation, not whether the conclusion is assumed by construction. Therefore no circular step is identified and the score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on assumptions about what the probe measures and how embeddings are pooled; no new theoretical entities are introduced.

free parameters (3)
  • MLP probe hyperparameters = hidden_size=512, ReLU, 500 epochs, cross-entropy, 80/10/10 split, grid search
    The probe architecture and training schedule are chosen by hand; reported probe accuracies depend on them and no sensitivity analysis is provided.
  • Generation sampling parameters = temperature=0.6, top_p=0.9, top_k=40, three seeds
    Free generation and chain-of-thought accuracy depend on these decoding choices.
  • Video frame sampling rate = 1 FPS, 10 frames per 10-second video
    Temporal downsampling may remove short physical events, affecting both end-task accuracy and probe content.
assumptions (3)
  • domain assumption Averaging embeddings over the sequence dimension preserves task-relevant spatial and temporal information.
    Section 5.1 averages all token embeddings into one vector per video; if averaging erases location or motion cues, the probe results underestimate or mischaracterize what the model encodes.
  • domain assumption Random 80/10/10 video-level splitting prevents scene-level leakage.
    GRASP Level 2 has 16 scenes with balanced labels; because embeddings cluster by scene (Figure 3a), a random split can let classifiers exploit scene identity rather than physical plausibility.
  • domain assumption A supervised probe's accuracy reflects information the language model can use during generation.
    The interpretation in Insight 3 compares trained probes on embeddings to the model's own free-form answers; a probe can extract linearly separable signals that the generative decoder is not trained or prompted to output.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Pixels to Principles: Probing Intuitive Physics Understanding in Multimodal Language Models." pith.science (2026). https://pith.science/paper/E44EJ52F

@misc{pith2026250716572,
  author       = {Pith},
  title        = {Pith review of: Pixels to Principles: Probing Intuitive Physics Understanding in Multimodal Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E44EJ52F}},
  note         = {Machine review of arXiv:2507.16572}
}
read the original abstract

This paper presents a systematic evaluation of state-of-the-art multimodal large language models (MLLMs) on intuitive physics tasks using the GRASP and IntPhys 2 datasets. We assess the open-source models InternVL 2.5, Qwen 2.5 VL, LLaVA-OneVision, and the proprietary Gemini 2.0 Flash Thinking, finding that even the latest models struggle to reliably distinguish physically plausible from implausible scenarios. To go beyond performance metrics, we conduct a probing analysis of model embeddings, extracting intermediate representations at key processing stages to examine how well task-relevant information is preserved. Our results show that, depending on task difficulty, a critical vision-language misalignment can emerge: vision encoders successfully capture physical plausibility cues, but this information is not effectively utilized by the language model, leading to failures in reasoning. This misalignment suggests that the primary limitation of MLLMs in intuitive physics tasks is not the vision component but the ineffective integration of visual and linguistic information. Our findings highlight vision-language alignment as a key area for improvement, offering insights for future MLLMs development.

Figures

Figures reproduced from arXiv: 2507.16572 by the authors.

Figure 1
Figure 1. Average accuracy (%) across the intuitive physics tests of GRASP and IntPhys 2 for the largest models. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparison of model performance across three model families. Each subplot shows average accuracies [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. 2D t-SNE visualizations of video features (GRASP, level 2) for LLaVA-OneVision-72B across different [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Average accuracy (%) across the intuitive physics tests of GRASP and IntPhys 2 for the smaller sizes [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 11 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. 2022. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716--23736

    2022

  3. [3]

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the IEEE International Conference on Computer Vision, pages 2425--2433

    2015

  4. [4]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

    arXiv 2025

  5. [5]

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432--7439

    2020

  6. [6]

    Florian Bordes, Quentin Garrido, Justine T Kao, Adina Williams, Michael Rabbat, and Emmanuel Dupoux. 2025. Intphys 2: Benchmarking intuitive physics understanding in complex synthetic environments. arXiv preprint arXiv:2506.09849

    arXiv 2025

  7. [7]

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3):1--45

    2024

  8. [8]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. 2024. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271

    arXiv 2024

Show all 43 references
  1. [9]

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023), 2(3):6

    2023

  2. [10]

    Arijit Dasgupta, Jiafei Duan, Marcelo H Ang Jr, Yi Lin, Su-hua Wang, Ren \'e e Baillargeon, and Cheston Tan. 2021 a . A benchmark for modeling violation-of-expectation in physical reasoning across event categories. arXiv preprint arXiv:2111.08826

    2021 arXiv

  3. [11]

    Arijit Dasgupta, Jiafei Duan, Marcelo H Ang Jr, and Cheston Tan. 2021 b . Avoe: a synthetic 3d dataset on understanding violation of expectation for artificial cognition. arXiv preprint arXiv:2110.05836

    2021 arXiv

  4. [12]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human...

    2019

  5. [13]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint...

    2020 arXiv

  6. [14]

    Quentin Garrido, Nicolas Ballas, Mahmoud Assran, Adrien Bardes, Laurent Najman, Michael Rabbat, Emmanuel Dupoux, and Yann LeCun. 2025. https://arxiv.org/abs/2502.11831 Intuitive physics understanding emerges from self-supervised pretraining on natural videos . Preprint, arXiv:...

    2025 arXiv

  7. [15]

    Google Gemini Team . 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530

    2024 arXiv

  8. [16]

    Bin Huang, Xin Wang, Hong Chen, Zihan Song, and Wenwu Zhu. 2024. Vtimellm: Empower llm to grasp video moments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14271--14280

    2024

  9. [17]

    Serwan Jassim, Mario Holubar, Annika Richter, Cornelius Wolff, Xenia Ohmer, and Elia Bruni. 2023. Grasp: A novel benchmark for evaluating language grounding and situated physics understanding in multimodal language models. arXiv preprint arXiv:2311.09048

    2023 arXiv

  10. [18]

    Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. 2017. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Reco...

    2017

  11. [19]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326

    2024 arXiv

  12. [20]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023 a . Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730--19742. PMLR

    2023

  13. [21]

    Lei Li, Jingjing Xu, Qingxiu Dong, Ce Zheng, Qi Liu, Lingpeng Kong, and Xu Sun. 2023 b . Can language models understand physical concepts? arXiv preprint arXiv:2305.14057

    2023 arXiv

  14. [22]

    Fangyu Liu, Guy Emerson, and Nigel Collier. 2023. Visual spatial reasoning. Transactions of the Association for Computational Linguistics, 11:635--651

    2023

  15. [23]

    Xiao Liu, Da Yin, Yansong Feng, and Dongyan Zhao. 2022. Things not written in text: Exploring spatial commonsense from visual signals. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2365--2376

    2022

  16. [24]

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. 2024. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216--233. Springer

    2024

  17. [25]

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2024. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Repre...

    2024

  18. [26]

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. 2023. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.05424

    2023 arXiv

  19. [27]

    Julius Mayer, Mohamad Ballout, Serwan Jassim, Farbod Nosrat Nezami, and Elia Bruni. 2025. ivispar--an interactive visual-spatial reasoning benchmark for vlms. arXiv preprint arXiv:2502.03214

    2025

  20. [28]

    Luis S Piloto, Ari Weinstein, Peter Battaglia, and Matthew Botvinick. 2022. Intuitive physics learning in a deep-learning model inspired by developmental psychology. Nature human behaviour, 6(9):1257--1267

    2022

  21. [29]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  22. [30]

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training

    2018

  23. [31]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  24. [32]

    Ronan Riochet, Mario Ynocente Castro, Mathieu Bernard, Adam Lerer, Rob Fergus, V \'e ronique Izard, and Emmanuel Dupoux. 2021. Intphys 2019: A benchmark for visual intuitive physics understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9):5016--5025

    2021

  25. [33]

    Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, and Deng Cai. 2023. Pandagpt: One model to instruction-follow them all. arXiv preprint arXiv:2305.16355

    2023 arXiv

  26. [34]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  27. [35]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  28. [36]

    Yiqi Wang, Wentao Chen, Xiaotian Han, Xudong Lin, Haiteng Zhao, Yongfei Liu, Bohan Zhai, Jianbo Yuan, Quanzeng You, and Hongxia Yang. 2024. Exploring the reasoning abilities of multimodal large language models (mllms): A comprehensive survey on emerging trends in multimodal re...

    2024 arXiv

  29. [37]

    Luca Weihs, Amanda Yuile, Ren \'e e Baillargeon, Cynthia Fisher, Gary Marcus, Roozbeh Mottaghi, and Aniruddha Kembhavi. 2022. Benchmarking progress to infant-level physical reasoning in ai. Transactions on Machine Learning Research

    2022

  30. [38]

    Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B Tenenbaum. 2020. Clevrer: Collision events for video representation and reasoning. In International Conference on Learning Representations

    2020

  31. [39]

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2024. Mmm...

    2024

  32. [40]

    Chenyu Zhang, Benjamin Van Durme, Zhuowan Li, and Elias Stengel-Eskin. 2022. Visual commonsense in pretrained unimodal and multimodal models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Te...

    2022

  33. [41]

    Hang Zhang, Xin Li, and Lidong Bing. 2023. Video-llama: An instruction-tuned audio-visual language model for video understanding. arXiv preprint arXiv:2306.02858

    2023 arXiv

  34. [42]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  35. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.