Pith. sign in

REVIEW 3 major objections 4 minor 5 cited by

PyVision: Agentic Vision with Dynamic Tooling

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PyVision claims that letting multimodal language models generate, execute, and refine Python code during inference consistently improves visual reasoning, with gains up to 31 points on symbolic-vision puzzles.

desk verdict PyVision is a well-engineered empirical report on letting MLLMs write and run Python during visual reasoning; the gains are real but the paper overclaims by attributing them to dynamic tooling without isolating that variable. read the letter →

arxiv 2507.07998 v3 pith:DJHKDUHR submitted 2025-07-10 cs.CL cs.AIcs.CV

Shitian Zhao , Haoquan Zhang , Shaoheng Lin , Ming Li , Qilong Wu , Kaipeng Zhang , Chen Wei This is my paper
classification cs.CLcs.AIcs.CV
keywords dynamictoolgenerationmultimodalreasoningPythoncodeexecutionagenticvisualquestionansweringmulti-turninteractiontaxonomychain-of-thoughtbaselinecode-augmentedinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PyVision is an interactive framework that lets a multimodal language model write, run, and revise its own Python code while answering visual questions, instead of relying on a fixed set of image tools. The paper claims this dynamic tooling improves results across six benchmarks for two strong backend models, with the largest gains on fine-grained visual search (+7.8% for GPT-4.1 on V*) and on symbolic-vision puzzles (+31.1% for Claude-4.0-Sonnet on VLMsAreBlind-mini). The authors also show that the kinds of tools the model invents fall into a small taxonomy—cropping, contrast enhancement, segmentation, sketching, and numerical analysis—and that task and domain shape which tools get used. If the claim is right, it suggests that giving models a general programming environment is a route to more adaptive and interpretable visual reasoning than predefining tool pipelines.

What carries the argument

The load-bearing mechanism is the multi-turn agentic loop between the MLLM and a Python interpreter. In each turn the model writes a complete script, the runtime executes it in a subprocess, and the executed output—printed numbers, displayed images, or error messages—is appended to the model's context for the next turn. Cross-turn persistence lets the model build on earlier results, converting raw pixels into measurements, masks, or sketches that ground its final answer. The system prompt and a small set of interface conventions (pre-loaded image variables, <code> and <answer> tags, print() and plt.show() for output) make this loop reliable enough for current models to use without crashing.

What would settle it

Run PyVision against a control that matches the multi-turn interaction and output length but replaces executed code with unexecuted code, simulated tool output, or a scratchpad of comparable length; if the gains persist without real execution, the claim that tool execution drives the improvement is falsified.

Watch

Extended reading notes

Core claim

The central claim is that a multimodal large language model (MLLM) can improve its own visual reasoning by acting as an agent that generates Python programs on the fly, executes them in an isolated runtime, inspects the textual and visual outputs, and iterates until it settles on an answer. PyVision provides this loop: images arrive pre-loaded as variables, the model emits code wrapped in tags, the interpreter returns results into context, and the session persists state across turns. On six benchmarks, PyVision reports consistent gains over a plain chain-of-thought baseline for GPT-4.1 and Claude-4.0-Sonnet, including a 7.8-point improvement on the high-resolution visual-search benchmark V* and a 31.1-point jump on the symbolic-vision set VLMsAreBlind-mini. The paper further finds that the gains track the backend model's existing strengths—perception-heavy models gain more on search tasks, reasoning-heavy models gain more on math and puzzles—and that tools fall into recognizable categories such as cropping, enhancement, segmentation, sketching, and numerical analysis. This is presented as evidence that dynamic tooling lets models invent tools rather than merely use them.

Load-bearing premise

The load-bearing premise is that a plain prompt-based reasoning baseline without code execution isolates the effect of letting the model generate tools; if the extra turns, extra output, and execution compute in PyVision—rather than tool generation itself—are what drive the gains, the central claim is overstated.

Editorial extensions

If this is right

  • Fixed visual pipelines that require detection or segmentation models become unnecessary on the evaluated benchmarks, because the model rewrites the needed functionality in Python.
  • The size of the gain depends on the backend model's baseline skills: perception-strong models benefit most on search-heavy tasks, reasoning-strong models on math and symbolic puzzles.
  • Because every intermediate script is inspectable, the framework produces a transparent trace of how a visual answer was computed, from crop coordinates to contour perimeters.
  • The same loop carries over to video by letting the model sample and zoom into selected frames rather than processing all frames.
  • Tool-usage patterns are predictable by benchmark and domain, suggesting that dynamic tooling adapts strategy to task demands rather than applying one workflow everywhere.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported gains may partly reflect extra inference compute and longer outputs, since the PyVision baseline produces more turns and code than the plain chain-of-thought comparison; an ablation that equalizes token budgets would separate the tooling effect from the compute effect.
  • The 31-point jump on VLMsAreBlind-mini suggests that Claude-4.0-Sonnet's low baseline is a specific weakness in parsing abstract visual primitives that deterministic code trivially fixes, so the gain may not transfer to tasks where code has little purchase.
  • A natural next step would be to train or fine-tune the model with reinforcement learning on successful tool-generation trajectories, turning the framework from a prompting wrapper into a learned skill.
  • The framework's reliance on Python's installed libraries means its effectiveness is bounded by what those libraries can express; tasks requiring learned perception (rare object categories, ambiguous segmentation) may resist pure code solutions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces PyVision, an interactive multi-turn framework that lets a multimodal large language model (MLLM) generate, execute, and iteratively refine Python code during visual reasoning. The framework provides a Python runtime, cross-turn state persistence, and a system prompt that encourages the model to write and execute code. The authors evaluate PyVision with GPT-4.1 and Claude-4.0-Sonnet on six benchmarks (MathVista, MathVision-mini, MMMU, VisualPuzzles, VLMsAreBlind-mini, V*), reporting consistent gains over a plain chain-of-thought baseline, including +7.8% on V* for GPT-4.1 and +31.1% on VLMsAreBlind-mini for Claude-4.0-Sonnet. They also present a taxonomy of generated tools and analyze tool-use patterns across tasks and domains.

Significance. If the reported gains are causally attributable to dynamic tooling, PyVision would be a useful and broadly applicable contribution to agentic visual reasoning, and the qualitative material — case studies, tool taxonomy, and cross-benchmark usage patterns — is a valuable resource for the community. The framework is simple, the inference code is released, and the authors are explicit about their evaluation setup. However, the central causal claim is currently undermined by confounded comparisons and the absence of variance estimates, so the quantitative contribution is not yet established at the level claimed.

major comments (3)
  1. [Sec. 4 (Baselines, Table 2) and Appendix A.1/A.2] The comparison between PyVision and the plain chain-of-thought baseline does not isolate the effect of dynamic tooling. The PyVision system prompt (Appendix A.1) instructs the model to 'MUST use the python tool', to 'plan extensively before each function call', to 'keep going until the user’s query is completely resolved', and to 'reflect extensively on the outcomes'; the CoT baseline prompt (Appendix A.2) contains none of these directives. PyVision also adds multi-turn interaction and real execution feedback, so the intervention bundles at least three factors: (1) a code-encouraging prompt, (2) actual code execution, and (3) iterative self-correction across turns. The +7.8% (V*, GPT-4.1) and +31.1% (VLMsAreBlind-mini, Claude-4.0-Sonnet) gains in Table 2 therefore cannot be attributed specifically to dynamic tooling. A no-execution control, in which the model writes code but receives no interpreter output, or a multi-turn CoT control without code, is needed to separate these factors.
  2. [Appendix A.2 and Sec. 4 (Qwen2.5-VL-72B paragraph)] The Qwen2.5-VL-72B evidence for the 'amplifies what the backend MLLM does best' claim is weakened by the authors' own caveat that the CoT baseline may be 'damaged' for this model because it 'was not specifically optimized for CoT reasoning'. The reported +10.0% on V* and +1.7% on MathVision-mini are then differences against a baseline the authors themselves flag as potentially unfair, so these numbers cannot support the inference that PyVision amplifies perceptual strength while yielding little benefit for weaker abstract reasoners. A fairer comparison would use a decoding strategy or prompt format that is not known to disadvantage the backend model.
  3. [Table 2 and the 'PyVision Amplifies...' paragraph] Table 2 reports single point estimates with no error bars, repeated runs, or significance testing. Several deltas are very small — Claude-4.0-Sonnet gains of +0.2 on MMMU and +0.3 on V* are within the noise typical of these benchmarks — and the conclusion that PyVision 'amplifies what the backend MLLM does best' is based on comparing differences of near-zero quantities (+7.8 vs. +0.3 for V*; +3.3 vs. +2.3 for MathVision-mini). Without confidence intervals or at least multiple seeds, the claim of 'consistent performance gains' is not quantitatively supported. I recommend reporting variance estimates, ideally with paired evaluation across the same items, and tempering the amplification claim accordingly.
minor comments (4)
  1. [Abstract and Introduction] The phrase 'invent them' may overstate what the framework does; the system prompt strongly encourages code generation, and the taxonomy shows mostly known image-processing operations. Consider softening this wording to 'dynamically compose' or 'generate'.
  2. [Appendix A.2] The benchmark name appears inconsistently as 'MathVision-mini' in the main text and Table 2 but 'MathVision-testmini' in Appendix A.2; please unify the terminology.
  3. [Sec. 3.1 (Tooling Taxonomy)] The taxonomy is derived by clustering text-embedding-3-large embeddings of code snippets, but the clustering method, number of clusters, and validation of cluster interpretability are not described. Adding these details would improve reproducibility of the qualitative analysis.
  4. [Fig. 10] The figure caption states that the legend 'indicates the percentage of query sessions that include any code generation,' but the main text says Claude 'reaches 100% code coverage.' Clarify whether the percentages are per-benchmark or aggregated, and define 'coverage' precisely.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper reports external-benchmark comparisons with no fitted parameter relabeled as a prediction and no load-bearing self-citation chain.

full rationale

PyVision makes no theoretical derivation that could reduce to its own inputs. Its central claims are measured accuracy improvements on established external benchmarks such as V*, MathVista, MMMU, VisualPuzzles, and VLMsAreBlind-mini, compared against the same backend MLLM with a plain chain-of-thought prompt. There are no equations, no fitted parameters, and no quantity that is defined in terms of the result it is supposed to predict. The tooling taxonomy is a post-hoc descriptive clustering of generated code, not a predictive claim. The only notable weakness is experimental attribution: the PyVision condition bundles a code-encouraging system prompt, multi-turn interaction, and execution feedback, so the baseline does not perfectly isolate the effect of dynamic tooling. That is a validity concern, not circularity, because the reported gains are not equivalent by construction to any model input or fitted value. The paper also does not rest on a self-citation, uniqueness theorem, or ansatz smuggled in via citation; external references are used for benchmarks and prior work, and the authors' own contribution is an open-loop framework applied to those benchmarks. Therefore no circular step can be exhibited, and the appropriate score is 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its central claim rests on empirical assumptions about model code-generation reliability, library availability in the sandbox, and the fairness of the baseline evaluation.

free parameters (1)
  • inference temperature = 0.6
    Set for all evaluation runs in Appendix A.2; no sensitivity analysis or multiple seeds, so results may depend on this choice.
assumptions (4)
  • domain assumption The backend MLLM reliably generates parsable, executable Python code that runs without crashes in the sandbox.
    Section 2 states the two models can reliably generate parsable and executable code blocks that rarely crash; no quantitative crash rate is given, and much of the framework's value depends on this.
  • domain assumption The Python runtime provides the expected scientific libraries, including OpenCV, PIL, NumPy, scikit-image, and EasyOCR.
    Appendix B code snippets call libraries such as EasyOCR and scikit-image; if these are unavailable or behave differently, tool execution and results would change.
  • ad hoc to paper A single evaluation run at temperature 0.6 is a representative estimator of benchmark accuracy.
    Table 2 reports one number per model and dataset without variance or multiple seeds, yet claims consistent gains.
  • ad hoc to paper Plain chain-of-thought prompting is a fair control that attributes gains to dynamic tooling rather than to extra inference-time compute, more tokens, or multi-turn self-correction.
    Section 4 and Appendix A.2 compare PyVision to CoT; no compute-matched baseline is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PyVision: Agentic Vision with Dynamic Tooling." pith.science (2026). https://pith.science/paper/DJHKDUHR

@misc{pith2026250707998,
  author       = {Pith},
  title        = {Pith review of: PyVision: Agentic Vision with Dynamic Tooling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DJHKDUHR}},
  note         = {Machine review of arXiv:2507.07998}
}
read the original abstract

LLMs are increasingly deployed as agents, systems capable of planning, reasoning, and dynamically calling external tools. However, in visual reasoning, prior approaches largely remain limited by predefined workflows and static toolsets. In this report, we present PyVision, an interactive, multi-turn framework that enables MLLMs to autonomously generate, execute, and refine Python-based tools tailored to the task at hand, unlocking flexible and interpretable problem-solving. We develop a taxonomy of the tools created by PyVision and analyze their usage across a diverse set of benchmarks. Quantitatively, PyVision achieves consistent performance gains, boosting GPT-4.1 by +7.8% on V* and Claude-4.0-Sonnet by +31.1% on VLMsAreBlind-mini. These results point to a broader shift: dynamic tooling allows models not just to use tools, but to invent them, advancing toward more agentic visual reasoning.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection

    cs.AI 2025-12 unverdicted novelty 7.0 of 10

    ForenAgent lets MLLMs create and iteratively improve low-level Python tools for image forgery detection via a two-stage training pipeline and a new 100k-image benchmark dataset.

  2. VC-Tooler: Learning Compositional and Adaptive Visual Tool Use

    cs.CV 2026-08 conditional novelty 6.0 of 10

    VC-Tooler trains Qwen3-VL-8B on hierarchically synthesized tool trajectories via SFT then GRPO with a judge-based tool reward, reaching open-source SOTA on V* (95.8) and VTC-Bench (35.3) and transferring to 11 unseen ...

  3. CanvasAgent: Enabling Complex Image Creation and Editing via Visual Tool Orchestration

    cs.CV 2026-07 conditional novelty 6.0 of 10

    SFT+GRPO training on CanvasCraft teaches an MLLM to orchestrate heterogeneous visual tools for long-horizon image creation and editing.

  4. Visual Programmability: A Guide for Code-as-Thought in Chart Understanding

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A vision-language model learns to dynamically switch between code-based and visual reasoning for chart questions, improving average accuracy by about one point over fixed strategies.

  5. Reinforced Visual Perception with Tools

    cs.CV 2025-09 conditional novelty 5.0 of 10

    ReVPT uses GRPO reinforcement learning with a cold-start SFT phase to make Qwen2.5-VL models call visual tools, improving perception benchmarks over SFT and text-only RL baselines.

Reference graph

Works this paper leans on

61 extracted references · 35 canonical work pages · cited by 5 Pith papers

  1. [1]

    Kimi k2: Open agentic intelligence, 2025

    MoonShot AI. Kimi k2: Open agentic intelligence, 2025

    work page 2025

  2. [2]

    Neural Module Networks

    Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Deep compositional question answering with neural module networks. arXiv:1511.02799, 2015

    work page Pith review arXiv 2015

  3. [3]

    Introducing claude 4, 2025

    Anthropic. Introducing claude 4, 2025

    2025

  4. [4]

    Vending-bench: A benchmark for long-term coherence of autonomous agents

    Axel Backlund and Lukas Petersson. Vending-bench: A benchmark for long-term coherence of autonomous agents. arXiv preprint arXiv:2502.15840, 2025

    arXiv 2025

  5. [5]

    Digirl: Training in-the-wild device-control agents with autonomous reinforcement learning

    Hao Bai, Yifei Zhou, Jiayi Pan, Mert Cemri, Alane Suhr, Sergey Levine, and Aviral Kumar. Digirl: Training in-the-wild device-control agents with autonomous reinforcement learning. NeurIPS, 2024

    work page 2024

  6. [6]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report. arXiv:2502.13923, 2025

    arXiv 2025

  7. [7]

    The opencv library

    Gary Bradski. The opencv library. Dr. Dobb’s Journal: Software Tools for the Professional Programmer, 2000

    work page 2000

  8. [8]

    Pillow (pil fork) documentation

    Alex Clark et al. Pillow (pil fork) documentation. readthedocs, 2015

    work page 2015

Show all 61 references
  1. [9]

    Retool: Reinforcement learning for strategic tool use in llms

    Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in llms. arXiv:2504.11536, 2025

    2025 arXiv

  2. [10]

    The parable of the parser, 2024

    Ross Girshick. The parable of the parser, 2024

    2024

  3. [11]

    Tora: A tool-integrated reasoning agent for mathematical problem solving

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, and Weizhu Chen. Tora: A tool-integrated reasoning agent for mathematical problem solving. arXiv preprint arXiv:2309.17452, 2023

    2023 arXiv

  4. [12]

    Visual programming: Compositional visual reasoning without training

    Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. In CVPR, 2023

    2023

  5. [13]

    Array programming with numpy

    Charles R Harris, K Jarrod Millman, Stéfan J Van Der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J Smith, et al. Array programming with numpy. Nature, 2020

    2020

  6. [14]

    Metagpt: Meta programming for multi-agent collaborative framework

    Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352, 2023. 16

    2023 arXiv

  7. [15]

    Cogagent: A visual language model for gui agents

    Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, et al. Cogagent: A visual language model for gui agents. In CVPR, 2024

    2024

  8. [16]

    Owl: Optimized workforce learning for gen- eral multi-agent assistance in real-world task automation

    Mengkang Hu, Yuhang Zhou, Wendong Fan, Yuzhou Nie, Bowei Xia, Tao Sun, Ziyu Ye, Zhaoxuan Jin, Yingru Li, Qiguang Chen, et al. Owl: Optimized workforce learning for gen- eral multi-agent assistance in real-world task automation. arXiv preprint arXiv:2505.23885, 2025

    2025 arXiv

  9. [17]

    Smith, and Ranjay Krishna

    Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. In NeurIPS, 2024

    2024

  10. [18]

    OmniMedVQA: A new large-scale comprehensive evaluation benchmark for medical lvlm

    Yutao Hu, Tianbin Li, Quanfeng Lu, Wenqi Shao, Junjun He, Yu Qiao, and Ping Luo. OmniMedVQA: A new large-scale comprehensive evaluation benchmark for medical lvlm. In CVPR, 2024

    2024

  11. [19]

    The ebbinghaus illusion: New contextual effects and theoretical considerations

    Ted Jaeger and Kyle Klahs. The ebbinghaus illusion: New contextual effects and theoretical considerations. Perceptual and motor skills, 2015

    2015

  12. [20]

    Lawrence Zitnick, and Ross B

    Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Judy Hoffman, Li Fei-Fei, C. Lawrence Zitnick, and Ross B. Girshick. Inferring and executing programs for visual reasoning. In ICCV, 2017

    2017

  13. [21]

    Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything, 2023

    2023

  14. [22]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. NeurIPS, 2022

    2022

  15. [23]

    Geochat: Grounded large vision-language model for remote sensing

    Kartik Kuckreja, Muhammad Sohail Danish, Muzammal Naseer, Abhijit Das, Salman Khan, and Fahad Shahbaz Khan. Geochat: Grounded large vision-language model for remote sensing. In CVPR, 2024

    2024

  16. [24]

    Visual abstract thinking empowers multimodal reasoning

    Dairu Liu, Ziyue Wang, Minyuan Ruan, Fuwen Luo, Chi Chen, Peng Li, and Yang Liu. Visual abstract thinking empowers multimodal reasoning. arXiv:2505.20164, 2025

    2025

  17. [25]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In ECCV, 2024

    2024

  18. [26]

    Mathvista: Evaluating mathe- matical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathe- matical reasoning of foundation models in visual contexts. arXiv:2310.02255, 2023

    arXiv

  19. [27]

    Octo- tools: An agentic framework with extensible tools for complex reasoning

    Pan Lu, Bowen Chen, Sheng Liu, Rahul Thapa, Joseph Boen, and James Zou. Octo- tools: An agentic framework with extensible tools for complex reasoning. arXiv preprint arXiv:2502.11271, 2025

    2025 arXiv

  20. [28]

    Deepswe: Training a state-of- the-art coding agent from scratch by scaling rl, 2025

    Michael Luo, Naman Jain, Jaskirat Singh, Sijun Tan, Ameen Patel, Qingyang Wu, Alpay Ariyak, Colin Cai, Shang Zhu Tarun Venkat, Ben Athiwaratkun, Manan Roongta, Ce Zhang, Li Erran Li, Raluca Ada Popa, Koushik Sen, and Ion Stoica. Deepswe: Training a state-of- the-art coding age...

    2025

  21. [29]

    Meet genspark super agent, 2025

    MainFunc. Meet genspark super agent, 2025. 17

    2025

  22. [30]

    Leave it to manus, 2025

    Manus. Leave it to manus, 2025

    2025

  23. [31]

    pandas: a foundational python library for data analysis and statistics

    Wes McKinney et al. pandas: a foundational python library for data analysis and statistics. Python for high performance and scientific computing, 2011

    2011

  24. [32]

    Minimax-agent, 2025

    MiniMax. Minimax-agent, 2025

    2025

  25. [33]

    Computer-using agent, 2025

    OpenAI. Computer-using agent, 2025

    2025

  26. [34]

    Introducing codex, 2025

    OpenAI. Introducing codex, 2025

    2025

  27. [35]

    Introducing gpt-4.1 in the api, 2025

    OpenAI. Introducing gpt-4.1 in the api, 2025

    2025

  28. [36]

    New embedding models and api updates, 2025

    OpenAI. New embedding models and api updates, 2025

    2025

  29. [37]

    Thinking with images, 2025

    OpenAI. Thinking with images, 2025

    2025

  30. [38]

    Scikit-learn: Machine learning in python

    Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 2011

    2011

  31. [39]

    Cogcom: A visual language model with chain-of- manipulations reasoning

    Ji Qi, Ming Ding, Weihan Wang, Yushi Bai, Qingsong Lv, Wenyi Hong, Bin Xu, Lei Hou, Juanzi Li, Yuxiao Dong, and Jie Tang. Cogcom: A visual language model with chain-of- manipulations reasoning. In ICLR, 2025

    2025

  32. [40]

    Alita: Generalist agent enabling scalable agentic reasoning with minimal predefinition and maximal self-evolution

    Jiahao Qiu, Xuan Qi, Tongcheng Zhang, Xinzhe Juan, Jiacheng Guo, Yifu Lu, Yimin Wang, Zixin Yao, Qihan Ren, Xun Jiang, et al. Alita: Generalist agent enabling scalable agentic reasoning with minimal predefinition and maximal self-evolution. arXiv preprint arXiv:2505.20286, 2025

    2025 arXiv

  33. [41]

    Vision language models are blind

    Pooyan Rahmanzadehgervi, Logan Bolton, Mohammad Reza Taesiri, and Anh Totti Nguyen. Vision language models are blind. In ACCV, 2024

    2024

  34. [42]

    Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face

    Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. In NeurIPS, 2023

    2023

  35. [43]

    Vi- sualpuzzles: Decoupling multimodal reasoning evaluation from domain knowledge

    Yueqi Song, Tianyue Ou, Yibo Kong, Zecheng Li, Graham Neubig, and Xiang Yue. Vi- sualpuzzles: Decoupling multimodal reasoning evaluation from domain knowledge. arXiv:2504.10342, 2025

    2025 arXiv

  36. [44]

    Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning, 2025

    Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning, 2025

    2025

  37. [45]

    Openthinkimg: Learning to think with images via visual tool reinforcement learning

    Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, et al. Openthinkimg: Learning to think with images via visual tool reinforcement learning. arXiv preprint arXiv:2505.08617, 2025

    2025 arXiv

  38. [46]

    Thinking with images for multimodal reasoning: Foundations, methods, and future frontiers

    Zhaochen Su, Peng Xia, Hangyu Guo, Zhenhua Liu, Yan Ma, Xiaoye Qu, Jiaqi Liu, Yanshu Li, Kaide Zeng, Zhengyuan Yang, et al. Thinking with images for multimodal reasoning: Foundations, methods, and future frontiers. arXiv preprint arXiv:2506.23918, 2025

    2025 arXiv

  39. [47]

    Vipergpt: Visual inference via python execution for reasoning

    Dídac Surís, Sachit Menon, and Carl Vondrick. Vipergpt: Visual inference via python execution for reasoning. In ICCV, 2023

    2023

  40. [48]

    scikit-image: image processing in python

    Stefan Van der Walt, Johannes L Schönberger, Juan Nunez-Iglesias, François Boulogne, Joshua D Warner, Neil Yager, Emmanuelle Gouillart, and Tony Yu. scikit-image: image processing in python. PeerJ, 2014. 18

    2014

  41. [49]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. NeurIPS, 2024

    2024

  42. [50]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. NeurIPS, 2022

    2022

  43. [51]

    Visual chatgpt: Talking, drawing and editing with visual foundation models

    Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv:2303.04671, 2023

    2023 arXiv

  44. [52]

    V*: Guided visual search as a core mechanism in multimodal llms

    Penghao Wu and Saining Xie. V*: Guided visual search as a core mechanism in multimodal llms. In CVPR, 2024

    2024

  45. [53]

    Simpletir: End-to-end reinforcement learning for multi-turn tool-integrated reasoning, 2025

    Zhenghai Xue, Longtao Zheng, Qian Liu, Yingru Li, Zejun Ma, and Bo An. Simpletir: End-to-end reinforcement learning for multi-turn tool-integrated reasoning, 2025. Notion Blog

    2025

  46. [54]

    Set-of- mark prompting unleashes extraordinary visual grounding in gpt-4v, 2023

    Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of- mark prompting unleashes extraordinary visual grounding in gpt-4v, 2023

    2023

  47. [55]

    Thinking in Space: How Multimodal Large Language Models See, Remember and Recall Spaces

    Jihan Yang, Shusheng Yang, Anjali Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in Space: How Multimodal Large Language Models See, Remember and Recall Spaces. arXiv:2412.14171, 2024

    2024 arXiv

  48. [56]

    Mm-react: Prompting chatgpt for multimodal reasoning and action

    Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv:2303.11381, 2023

    2023 arXiv

  49. [57]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In CVPR, 2024

    2024

  50. [58]

    Chain-of-focus: Adaptive visual search and zooming for multimodal reasoning via rl, 2025

    Xintong Zhang, Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaowen Zhang, Yang Liu, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, and Qing Li. Chain-of-focus: Adaptive visual search and zooming for multimodal reasoning via rl, 2025

    2025

  51. [59]

    Gpt-4v(ision) is a generalist web agent, if grounded, 2024

    Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. Gpt-4v(ision) is a generalist web agent, if grounded, 2024

    2024

  52. [60]

    thinking with images

    Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing "thinking with images" via reinforcement learning, 2025

    2025

  53. [61]

    image_clue_i

    Qiji Zhou, Ruochen Zhou, Zike Hu, Panzhong Lu, Siyang Gao, and Yue Zhang. Image-of- thought prompting for visual reasoning refinement in multimodal large language models. arXiv:2405.13872, 2024. 19 Appendix Contents A. Additional Evaluation Details . . . . . . . . . . . . . . ...

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.