pith. machine review for the scientific record. sign in

arxiv: 2602.02276 · v1 · submitted 2026-02-02 · 💻 cs.CL · cs.AI· cs.LG

Recognition: 2 theorem links

· Lean Theorem

Kimi K2.5: Visual Agentic Intelligence

Aidi Li, Angang Du, Ao Wang, Bohong Yin, Bo Pang, Bowei Xing, Bowen Qu, Bowen Wang, Boyu Xu, Chao Hong, Chaoran Tian, Cheng Chen, Chengjie Wu, Cheng Li, Cheng Liu, Chenguang Zhao, Chengyang Gong, Chengzhen Yu, Chenjun Xiao, Chensi Wang, Chenyu Liu, Chenzhuang Du, Chuang Wang, Chuan Wen, Chuning Tang, Chu Wei, C. Li, Congcong Wang, Dan Ye, Dazhi Cheng, Dehao Zhang, Dikang Du, Dingkun Wang, Dinglu Wang, Dongliang Wang, Dunyuan Zha, Enming Yuan, Enzhe Lu, Fang Li, Fanqing Meng, Feifan Song, Feifan Zhao, Feng Wang, Flood Sung, Garimugai Fu, Guanduo Chen, Guanghe Li, Guangyao Yang, Guanyu Li, Guokun Lai, Hailong Wang, Haiming Wang, Haitao Li, Haobing Zhan, Hao Ding, Hao Hu, Haoning Wu, Haotian Yao, Hao Yang, Haoyang Li, Haoyu Lu, Hao Zhang, Hengzhi Wang, Heyi Tang, Hongcheng Gao, Hongjin Su, Hongzhang Liu, H.S. Che, Huabin Zheng, Huaqing Wang, Huarong Chen, Hui Wang, Jia Chen, Jiahao Chen, Jiahao Wang, Jialei Cui, Jia Li, Jianfan Xu, Jianlin Su, Jianlong Chen, Jiaqi Deng, Jiawei Lin, Jiawen Tao, Jiaxi Hu, Jiezhong Qiu, Jinguo Zhu, Jingwei Li, Jing Xu, Jingze Zhuang, Jinhong Wang, Jinjing Xu, Jinsong Sun, Jinxiang Zhao, Jin Xie, Jin Zhang, Jiuzheng Wang, Juanfeng Shi, Jun Chen, Junfeng Zhong, Junjie Yan, Junwei Yang, Junxiong Li, Junyan Wu, Junyao Sun, Junyu Luo, Kaixin Wang, Kai Yang, Kefan Chen, Ke Huang, Kelin Fu, Kimi Team: Tongtong Bai, Kun Ouyang, L.H. Xu, Liang Chen, Liang Liu, Lidong Shi, Lincan Li, Lingxiao Du, Linian Wang, Lin Sui, Lin Xu, Liya Zhu, Longguang Zhong, Longhui Yu, Long Ma, Longyu Guan, Mengfan Dong, Mengjie Yuan, Mengnan Dong, Minghan Chu, Ming Wei, Minqing Ni, Mo Li, Muxi Diao, M. Zhou, Ningyuan Yang, Pengfei Tian, Pengwei Song, Puqi Zhang, Qiao Zhang, Qibin Wang, Qiulin Feng, Qizheng Gu, Rucong Wu, Ruihan Yang, Ruihan Zheng, Ruijue Chen, Ruiyuan Huang, Rui Zhang, Runjie Zhou, Ruoyu Qin, Shangyi Geng, Shaoguang Mao, Shaojie Zheng, Shaowei Liu, S.H. Cai, Shengjie Wang, Shengjun Fang, Shengyuan Shi, Shiyuan Teng, Shuai Zhao, Shudong Liu, Shuran Liu, Shuyi Wang, Si Wang, Siyuan Pan, Suting Xu, Tao Jiang, Tao Yu, Tengyang Zheng, Tianhui Song, Tianwei Liu, Tianxiang Yu, Tianxiao Shen, Tianyu Liu, Tong Gao, Tongxu Luo, Tongyu Sun, Weihao Zeng, Weihong Li, Weilong Liao, Weiming Zhong, Weiran He, Wei Wang, Weixiao Huang, Weixin Xu, Weiyu Zhuang, Weizhou Liu, Wenhao Wu, Wenjie Ye, Wentao Li, Wenyang He, Xiangyan Liu, Xiangyu Zhao, Xiaobin Zhang, Xiaochen Gong, Xiaochen Wang, Xiaofei Yang, Xiaohan Lin, Xiaojuan Tang, Xiaokun Yuan, Xiaoru Hao, Xiaotong Xie, Xiaoxi Song, Xinbo Xu, Xinhang Li, Xinhao Chen, Xinhao Li, Xinhao Zhu, Xinlong Yang, Xin Men, Xinran Gu, Xinran Xu, Xinxing Zu, Xinyi Jin, Xinyuan Wang, Xinyu Zhou, Yadong Zhang, Yangchuan Xu, Yangkun Zhang, Yang Li, Yangyang Hu, Yangyang Liu, Yang Yue, Yanhao Li, Yanming Liu, Yanru Chen, Yanxu Chen, Yao Wang, Yashuo Luo, Y. Charles, Yejie Wang, Yibo Liu, Yibo Miao, Yichang Xu, Yichen Feng, Yicheng Gu, Yichi Zhang, Yicun Chen, Yifan Bai, Yifei Xin, Yikai Zhao, Yimin Chen, Yingjiang Chen, Yingwei Ma, Ying Yang, Ying Zou, Yiping Bao, Yipu Wang, Yiqin Wang, Yiwei Li, Yi Yang, Yizhi Zhang, Yongting Zhang, Youbo Shao, Yuan Cao, Yuankun Chen, Yuan Mei, Yuanxin Liu, Yuanying Guo, Yuchao Qian, Yucheng Wang, Yuchong Xie, Yuefeng Wu, Yue Liu, Yuemeng Xu, Yu Fan, Yuhao Dong, Yuhao Wu, Yujie Chen, Yu Jing, Yulun Du, Yunjia He, Yunpeng Tai, Yushun Zhang, Yutao Zhang, Yutian Chen, Yutong Zhang, Yuxiao Li, Yuxin Dong, Yuxin Wu, Yuxuan Zhu, Yuyao Ge, Yu Zhang, Yuzhi Wang, Yuzi Yan, Y. Zhang, Zaida Zhou, Zelai Xu, Zeyu Qin, Zeyu Shang, Zhaochen Su, Zhaoji Wang, Zhaowei Li, Zhaowei Wang, Zhejun Jiang, Zheming Li, Zhengtao Wang, Zhengyang Tang, Zhengying Liu, Zheng Zhang, Zhennan Shen, Zhenxing Hu, Zhen Yang, Zhen Zhu, Zhexu Wang, Zhilin Yang, Zhiqi Huang, Zhirong Chen, Zhishan Lin, Zhiyong Meng, Zhiyuan Lu, Zhongnuo Liu, Zhuoma Gongque, Zhuorui Ye, Zichao Lin, Zichen Wen, Zihan Wang, Zijian Wu, Zijia Zhao, Ziwei Chen, Ziyao Xu, Zizhe Wang, Zonghan Yang

Pith reviewed 2026-05-10 16:02 UTC · model grok-4.3

classification 💻 cs.CL cs.AIcs.LG
keywords multimodal agentic modelAgent Swarmjoint text-vision optimizationstate-of-the-art resultslatency reductionopen-source checkpointparallel agent orchestrationmultimodal reinforcement learning
0
0 comments X

The pith

Kimi K2.5 reaches state-of-the-art results in coding vision reasoning and agentic tasks by jointly optimizing text and vision then adding a parallel Agent Swarm.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Kimi K2.5 as an open-source multimodal model that trains text and vision together through joint pre-training, zero-vision supervised fine-tuning, and joint reinforcement learning so the modalities strengthen each other. It layers on Agent Swarm, a framework that automatically breaks complex tasks into different sub-problems and runs them with multiple agents at once. A sympathetic reader would care because this points to agentic systems that handle real multimodal work more capably and with less delay than single large models. If correct, the approach suggests coordination across agents can deliver both higher performance and practical speed gains.

Core claim

Kimi K2.5 shows that joint optimization of text and vision modalities through pre-training, zero-vision SFT, and reinforcement learning, when combined with the Agent Swarm orchestration framework for dynamic decomposition and concurrent execution of heterogeneous sub-tasks, produces state-of-the-art performance across coding, vision, reasoning, and agentic domains while cutting latency by up to 4.5 times relative to single-agent baselines.

What carries the argument

Agent Swarm, a self-directed parallel agent orchestration framework that decomposes complex tasks into heterogeneous sub-problems and executes them concurrently.

If this is right

  • Complex multimodal tasks can be solved more efficiently by decomposing them into parallel heterogeneous sub-problems rather than sequential single-agent processing.
  • The open-source release of the post-trained checkpoint allows direct experimentation and extension by the research community.
  • Joint text-vision training enables each modality to improve the other, supporting stronger performance on tasks that require both seeing and reasoning.
  • Latency reductions of this magnitude make real-time agentic applications more feasible without sacrificing capability.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The swarm approach may transfer to embodied settings such as robotics where vision, planning, and action must be coordinated under time constraints.
  • Releasing the checkpoint could speed up community development of open multimodal agents that outperform closed single-model systems.
  • Future scaling experiments could test whether the latency advantage holds when the underlying base model is made substantially larger.

Load-bearing premise

The claimed state-of-the-art scores and latency reductions are produced by the joint text-vision techniques and Agent Swarm rather than by model scale, data choices, or evaluation details not described in the paper.

What would settle it

A side-by-side evaluation of Kimi K2.5 with and without the Agent Swarm component or the joint text-vision training steps that shows the full model no longer matches the reported state-of-the-art scores or the 4.5 times latency reduction.

read the original abstract

We introduce Kimi K2.5, an open-source multimodal agentic model designed to advance general agentic intelligence. K2.5 emphasizes the joint optimization of text and vision so that two modalities enhance each other. This includes a series of techniques such as joint text-vision pre-training, zero-vision SFT, and joint text-vision reinforcement learning. Building on this multimodal foundation, K2.5 introduces Agent Swarm, a self-directed parallel agent orchestration framework that dynamically decomposes complex tasks into heterogeneous sub-problems and executes them concurrently. Extensive evaluations show that Kimi K2.5 achieves state-of-the-art results across various domains including coding, vision, reasoning, and agentic tasks. Agent Swarm also reduces latency by up to $4.5\times$ over single-agent baselines. We release the post-trained Kimi K2.5 model checkpoint to facilitate future research and real-world applications of agentic intelligence.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces Kimi K2.5, an open-source multimodal agentic model that jointly optimizes text and vision modalities via pre-training, zero-vision SFT, and joint RL. It further proposes Agent Swarm, a self-directed parallel orchestration framework for decomposing and executing heterogeneous sub-tasks concurrently. The manuscript claims state-of-the-art results across coding, vision, reasoning, and agentic tasks, plus up to 4.5× latency reduction relative to single-agent baselines, and releases the post-trained model checkpoint.

Significance. If the empirical claims hold, the work would advance multimodal agentic systems by demonstrating synergistic text-vision training and a practical parallel-agent framework, with the open checkpoint release providing a concrete resource for reproducibility and follow-on research. The emphasis on joint optimization and dynamic task decomposition addresses real deployment constraints in agentic intelligence.

major comments (2)
  1. [Abstract] Abstract: The manuscript asserts SOTA performance across multiple domains and a 4.5× latency reduction from Agent Swarm, yet supplies no benchmark scores, named baselines, dataset specifications, error bars, or quantitative tables anywhere in the text. This absence prevents verification of the central attribution that the listed techniques (joint pre-training, zero-vision SFT, joint RL, Agent Swarm) causally produce the gains rather than unreported differences in scale or data.
  2. [Evaluation and Methodology sections] Evaluation and Methodology sections: No ablation studies, controls for model size, or comparisons isolating each component are provided. Without these, the causal link between the described joint text-vision methods / Agent Swarm and the headline outcomes cannot be established, rendering the performance claims unverifiable from the manuscript.
minor comments (2)
  1. The description of Agent Swarm would benefit from pseudocode or a formal algorithmic outline to clarify its self-directed decomposition and concurrency mechanics.
  2. The open release of the post-trained checkpoint is a clear strength that supports community follow-up.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below and will revise the manuscript accordingly to strengthen the presentation of our results.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The manuscript asserts SOTA performance across multiple domains and a 4.5× latency reduction from Agent Swarm, yet supplies no benchmark scores, named baselines, dataset specifications, error bars, or quantitative tables anywhere in the text. This absence prevents verification of the central attribution that the listed techniques (joint pre-training, zero-vision SFT, joint RL, Agent Swarm) causally produce the gains rather than unreported differences in scale or data.

    Authors: We agree that the current abstract does not contain specific quantitative results. In the revised manuscript we will expand the abstract to include key benchmark scores, named baselines, dataset references, and the reported latency reduction, with explicit pointers to the evaluation tables. We will also ensure the main text contains complete tables with scores, error bars, and dataset specifications so that the performance claims can be directly verified. revision: yes

  2. Referee: [Evaluation and Methodology sections] Evaluation and Methodology sections: No ablation studies, controls for model size, or comparisons isolating each component are provided. Without these, the causal link between the described joint text-vision methods / Agent Swarm and the headline outcomes cannot be established, rendering the performance claims unverifiable from the manuscript.

    Authors: We acknowledge that the submitted manuscript lacks ablation studies and component-isolation experiments. In the revision we will add a dedicated ablation section that reports results for each training stage (joint pre-training, zero-vision SFT, joint RL) and for Agent Swarm versus single-agent baselines, including controls that hold model size and data scale constant. These additions will directly address the causal attribution of the observed gains. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical performance claims with no derivations or self-referential reductions

full rationale

The paper presents Kimi K2.5 as an empirical multimodal model whose claims rest on reported benchmark performance (SOTA across coding/vision/reasoning/agentic tasks) and a latency reduction (up to 4.5× via Agent Swarm). No equations, first-principles derivations, fitted parameters renamed as predictions, or load-bearing self-citations appear in the abstract or described content. Techniques such as joint text-vision pre-training and Agent Swarm are introduced as design choices whose effects are asserted via external evaluation rather than constructed internally. The central attribution to these techniques is therefore not circular by construction; it is simply an empirical claim whose strength depends on unreported ablations and controls, not on definitional equivalence to the inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 1 invented entities

Only the abstract is available, so the ledger reflects the high-level techniques stated without access to training details, data, or proofs. The central claims rest on the unverified assumption that the listed training stages and orchestration method produce the reported gains.

invented entities (1)
  • Agent Swarm no independent evidence
    purpose: self-directed parallel agent orchestration framework that decomposes tasks into heterogeneous sub-problems executed concurrently
    Presented as a novel component on top of the multimodal model; no independent evidence or external validation is mentioned in the abstract.

pith-pipeline@v0.9.0 · 6778 in / 1286 out tokens · 39232 ms · 2026-05-10T16:02:36.256801+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Agent-ValueBench: A Comprehensive Benchmark for Evaluating Agent Values

    cs.AI 2026-05 unverdicted novelty 8.0

    Agent-ValueBench is the first dedicated benchmark for agent values, showing they diverge from LLM values, form a homogeneous 'Value Tide' across models, and bend under harnesses and skill steering.

  2. Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs

    cs.CL 2026-05 unverdicted novelty 8.0

    Soohak is a new 439-problem mathematician-authored benchmark showing frontier LLMs reach only 30% on research math and fail to exceed 50% on refusing ill-posed questions.

  3. When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds

    cs.LG 2026-05 unverdicted novelty 8.0

    SignSGD provably beats SGD by a factor of d under sparse noise via matched ℓ1-norm upper and lower bounds, with an equivalent result for Muon on matrices, and this predicts faster GPT-2 pretraining.

  4. WildTableBench: Benchmarking Multimodal Foundation Models on Table Understanding In the Wild

    cs.CV 2026-05 unverdicted novelty 8.0

    WildTableBench is the first benchmark for multimodal models on naturally occurring table images, with only one of 21 tested models exceeding 50% accuracy and most ranging from 4.1% to 49.9%.

  5. Can Coding Agents Reproduce Findings in Computational Materials Science?

    cs.SE 2026-05 conditional novelty 8.0

    AutoMat benchmark shows current LLM coding agents achieve at most 54.1% success when reproducing computational materials science claims from papers.

  6. From Context to Skills: Can Language Models Learn from Context Skillfully?

    cs.AI 2026-04 unverdicted novelty 8.0

    Ctx2Skill lets language models autonomously evolve context-specific skills via multi-agent self-play, improving performance on context learning tasks without human supervision.

  7. AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery

    cs.AI 2026-04 accept novelty 8.0

    AutoResearchBench is a new benchmark showing top AI agents achieve under 10% success on complex scientific literature discovery tasks that demand deep comprehension and open-ended search.

  8. HWE-Bench: Benchmarking LLM Agents on Real-World Hardware Bug Repair Tasks

    cs.AI 2026-04 unverdicted novelty 8.0

    HWE-Bench is the first repository-level benchmark for LLM agents on real hardware bug repair, where the best agent fixes 70.7% of 417 tasks but drops below 65% on complex SoC projects.

  9. VoxSafeBench: Not Just What Is Said, but Who, How, and Where

    cs.SD 2026-04 unverdicted novelty 8.0

    VoxSafeBench reveals that speech language models recognize social norms from text but fail to apply them when acoustic cues like speaker or scene determine the appropriate response.

  10. Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillation

    cs.LG 2026-04 unverdicted novelty 8.0

    Lightning OPD enforces teacher consistency by precomputing log-probabilities over SFT rollouts, matching standard OPD performance with bounded gradient discrepancy and achieving 4x speedup on math and code reasoning tasks.

  11. Narrative over Numbers: The Identifiable Victim Effect and its Amplification Under Alignment and Reasoning in Large Language Models

    cs.CL 2026-04 conditional novelty 8.0

    Large language models display the identifiable victim effect at roughly twice the human baseline, strongly amplified by instruction tuning and chain-of-thought prompting but inverted by reasoning-specialized models.

  12. OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language Environment Simulation

    cs.CL 2026-04 unverdicted novelty 8.0

    OccuBench is a new benchmark for AI agents on real-world occupational tasks via LLM-driven simulators, showing no model dominates all industries, implicit faults are hardest, and larger models with more reasoning perf...

  13. FashionMV: Product-Level Composed Image Retrieval with Multi-View Fashion Data

    cs.CV 2026-04 unverdicted novelty 8.0

    FashionMV introduces product-level multi-view CIR, a 127K-product dataset built via automated LMM pipeline, and a 0.8B ProCIR model that beats larger baselines on three fashion benchmarks.

  14. PinpointQA: A Dataset and Benchmark for Small Object-Centric Spatial Understanding in Indoor Videos

    cs.CV 2026-04 unverdicted novelty 8.0

    PinpointQA is the first benchmark dataset for small object-centric spatial understanding in indoor videos, with four tasks showing MLLM capability gaps that improve via supervised fine-tuning.

  15. Learning Agentic Policy from Action Guidance

    cs.CL 2026-05 unverdicted novelty 7.0

    ActGuide-RL uses human action data as plan-style guidance in mixed-policy RL to overcome exploration barriers in LLM agents, matching SFT+RL performance on search benchmarks without cold-start training.

  16. Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters

    cs.CV 2026-05 unverdicted novelty 7.0

    Chronicles-OCR is the first benchmark with 2,800 images across the complete evolutionary trajectory of Chinese characters, defining four tasks to evaluate VLLMs' cross-temporal visual perception.

  17. AssayBench: An Assay-Level Virtual Cell Benchmark for LLMs and Agents

    cs.LG 2026-05 unverdicted novelty 7.0

    AssayBench is a new gene-ranking benchmark for phenotypic CRISPR screens that shows zero-shot generalist LLMs outperform both biology-specific LLMs and trainable baselines on adjusted nDCG.

  18. StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs

    cs.CY 2026-05 accept novelty 7.0

    StereoTales shows that all tested LLMs emit harmful stereotypes in open-ended stories, with associations adapting to prompt language and targeting locally salient groups rather than transferring uniformly across languages.

  19. StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs

    cs.CY 2026-05 unverdicted novelty 7.0

    StereoTales shows that LLMs produce harmful, culturally adapted stereotypes in open-ended multilingual stories, with patterns consistent across providers and aligned human-LLM harm judgments.

  20. Not All Proofs Are Equal: Evaluating LLM Proof Quality Beyond Correctness

    cs.CL 2026-05 unverdicted novelty 7.0

    LLM proofs for hard math problems show large differences in quality metrics like conciseness and cognitive simplicity that correctness-only tests miss, along with trade-offs between quality and correctness.

  21. The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space

    cs.CV 2026-05 unverdicted novelty 7.0

    MLLMs scoring 70-83% on Cartesian visual tasks drop to 31-39% on logically equivalent polar versions, exposing reliance on grid discretization shortcuts instead of topology-invariant reasoning.

  22. Training with Harnesses: On-Policy Harness Self-Distillation for Complex Reasoning

    cs.CL 2026-05 unverdicted novelty 7.0

    OPHSD uses harness-augmented models as teachers to distill reasoning capabilities into base LLMs, yielding strong standalone performance on classification and math tasks.

  23. VibeProteinBench: An Evaluation Benchmark for Language-interfaced Vibe Protein Design

    q-bio.QM 2026-05 unverdicted novelty 7.0

    VibeProteinBench is a three-stage language-interfaced benchmark revealing that no current LLM performs strongly across recognition, engineering, and generation of proteins.

  24. CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios

    cs.CR 2026-05 unverdicted novelty 7.0

    LLM agents exhibit persistent attack-selection biases as fixed traits independent of success rates, with a bias momentum effect that resists steering and yields no performance gain.

  25. HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents

    cs.LG 2026-05 unverdicted novelty 7.0

    HyperEyes uses a dual-grained RL framework with parallel tool actions and efficiency rewards to achieve 9.9% higher accuracy and 5.3x fewer tool calls than prior open-source multimodal agents.

  26. Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key

    cs.AI 2026-05 unverdicted novelty 7.0

    RL training compute for logical reasoning follows a power law in proof depth whose exponent rises with logic expressiveness, and more expressive training yields larger gains on downstream benchmarks.

  27. Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key

    cs.AI 2026-05 unverdicted novelty 7.0

    RL training on more expressive logical tasks follows a steeper power-law scaling with reasoning depth and transfers more efficiently to math and reasoning benchmarks.

  28. Breaking, Stale, or Missing? Benchmarking Coding Agents on Project-Level Test Evolution

    cs.SE 2026-05 unverdicted novelty 7.0

    TEBench is a new project-level benchmark for test evolution showing coding agents achieve only 45-49% F1 on identifying tests needing changes, with stale tests hardest due to reliance on execution failures.

  29. On Time, Within Budget: Constraint-Driven Online Resource Allocation for Agentic Workflows

    cs.AI 2026-05 unverdicted novelty 7.0

    MCPP is a Monte Carlo simulation-based online planner that improves the probability of agentic workflows completing successfully under explicit budget and deadline constraints compared to baselines on CodeFlow and Pro...

  30. AffectSeek: Agentic Affective Understanding in Long Videos under Vague User Queries

    cs.CV 2026-05 unverdicted novelty 7.0

    AffectSeek is an agentic framework that localizes affective moments, classifies emotions, and generates rationales in long videos under vague user queries, backed by the new VQAU-Bench benchmark.

  31. AstroAlertBench: Evaluating the Accuracy, Reasoning, and Honesty of Multimodal LLMs in Astronomical Classification

    astro-ph.IM 2026-05 unverdicted novelty 7.0

    AstroAlertBench evaluates multimodal LLMs on astronomical classification accuracy, reasoning, and honesty using real ZTF alerts, revealing that high accuracy often diverges from self-assessed reasoning quality.

  32. MolViBench: Evaluating LLMs on Molecular Vibe Coding

    cs.CL 2026-05 unverdicted novelty 7.0

    MolViBench is the first benchmark designed to evaluate LLMs on generating executable programs for molecular tasks in drug discovery.

  33. Training Computer Use Agents to Assess the Usability of Graphical User Interfaces

    cs.CL 2026-04 unverdicted novelty 7.0

    uxCUA is a trained computer use agent that assesses GUI usability more accurately than larger models by learning to prioritize and execute important user interactions on labeled interface datasets.

  34. Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses

    cs.CL 2026-04 unverdicted novelty 7.0

    AHE automates coding-agent harness evolution via component, experience, and decision observability, raising Terminal-Bench 2 pass@1 from 69.7% to 77.0% with transfer gains across models and benchmarks.

  35. Benchmarking and Improving GUI Agents in High-Dynamic Environments

    cs.CV 2026-04 unverdicted novelty 7.0

    DynamicUI improves GUI agent performance in high-dynamic environments by processing interaction videos with frame clustering, action-conditioned refinement, and reflection, outperforming prior approaches on the new Dy...

  36. Benchmarking and Improving GUI Agents in High-Dynamic Environments

    cs.CV 2026-04 conditional novelty 7.0

    DynamicUI improves GUI agent performance in high-dynamic environments by using video-based dynamic perception, action-conditioned refinement, and reflection, outperforming prior agents on the new DynamicGUIBench while...

  37. SketchVLM: Vision language models can annotate images to explain thoughts and guide users

    cs.CV 2026-04 unverdicted novelty 7.0

    SketchVLM lets VLMs generate non-destructive SVG annotations on input images to visually explain answers, raising visual reasoning accuracy by up to 28.5 points and annotation quality by 1.48x over baselines.

  38. MathDuels: Evaluating LLMs as Problem Posers and Solvers

    cs.CL 2026-04 unverdicted novelty 7.0

    Self-play between LLMs for problem authoring and solving, scored via Rasch modeling, shows that authoring and solving skills are partially decoupled and that the benchmark difficulty evolves with new models.

  39. Synthesizing Multi-Agent Harnesses for Vulnerability Discovery

    cs.CR 2026-04 unverdicted novelty 7.0

    AgentFlow uses a typed graph DSL covering roles, prompts, tools, topology and protocol plus a runtime-signal feedback loop to optimize multi-agent harnesses, reaching 84.3% on TerminalBench-2 and discovering ten new z...

  40. Single-Language Evidence Is Insufficient for Automated Logging: A Multilingual Benchmark and Empirical Study with LLMs

    cs.SE 2026-04 unverdicted novelty 7.0

    MultiLogBench shows that LLM performance on automated logging varies substantially across programming languages, demonstrating that single-language evidence is insufficient for general claims about model behavior or t...

  41. S-GRPO: Unified Post-Training for Large Vision-Language Models

    cs.LG 2026-04 unverdicted novelty 7.0

    S-GRPO unifies SFT and RL for LVLMs via conditional ground-truth injection that supplies a maximal-reward anchor when group exploration fails completely.

  42. RiskWebWorld: A Realistic Interactive Benchmark for GUI Agents in E-commerce Risk Management

    cs.AI 2026-04 unverdicted novelty 7.0

    RiskWebWorld is the first realistic interactive benchmark for GUI agents in e-commerce risk management, revealing a large gap between generalist and specialized models plus RL gains.

  43. Many-Tier Instruction Hierarchy in LLM Agents

    cs.CL 2026-04 unverdicted novelty 7.0

    ManyIH and ManyIH-Bench address instruction conflicts in LLM agents with up to 12 privilege levels across 853 tasks, revealing frontier models achieve only ~40% accuracy.

  44. ClawBench: Can AI Agents Complete Everyday Online Tasks?

    cs.CL 2026-04 unverdicted novelty 7.0

    ClawBench is a benchmark of 153 live-web tasks where AI agents achieve low success rates, e.g. 33.3% for Claude Sonnet 4.6.

  45. Learning to Interrupt in Language-based Multi-agent Communication

    cs.CL 2026-04 unverdicted novelty 7.0

    HANDRAISER learns optimal interruption points in multi-agent LLM communication using estimated future reward and cost, achieving 32.2% lower communication cost with comparable or better task results across games, sche...

  46. DetailVerifyBench: A Benchmark for Dense Hallucination Localization in Long Image Captions

    cs.CV 2026-04 unverdicted novelty 7.0

    DetailVerifyBench supplies 1,000 images and densely annotated long captions to evaluate precise hallucination localization in multimodal large language models.

  47. EpiBench: Benchmarking Multi-turn Research Workflows for Multimodal Agents

    cs.CL 2026-04 unverdicted novelty 7.0

    EpiBench is a new episodic multi-turn multimodal benchmark where even leading AI agents score only 29.23% on hard tasks requiring cross-paper evidence integration from figures and tables.

  48. MMORF: A Multi-agent Framework for Designing Multi-objective Retrosynthesis Planning Systems

    cs.AI 2026-04 unverdicted novelty 7.0

    MMORF provides a modular multi-agent framework for multi-objective retrosynthesis planning, with MASIL and RFAS systems showing strong safety, cost, and success metrics on a new 218-task benchmark.

  49. Self-Distilled RLVR

    cs.LG 2026-04 unverdicted novelty 7.0

    RLSD mixes self-distillation for token-level policy difference magnitudes with RLVR for reliable update directions from response correctness to reach higher convergence and better training stability.

  50. AgentSZZ: Teaching the LLM Agent to Play Detective with Bug-Inducing Commits

    cs.SE 2026-04 conditional novelty 7.0

    AgentSZZ is an LLM-agent framework that identifies bug-inducing commits with up to 27.2% higher F1 scores than prior methods by enabling adaptive exploration and causal tracing, especially for cross-file and ghost commits.

  51. HumorRank: A Tournament-Based Leaderboard for Evaluating Humor Generation in Large Language Models

    cs.CL 2026-03 unverdicted novelty 7.0

    HumorRank ranks nine LLMs on textual humor using GTVH-grounded pairwise tournaments and Adaptive Swiss aggregation on the SemEval-2026 MWAHAHA dataset, finding that comedic mechanism mastery matters more than scale.

  52. MinT: Managed Infrastructure for Training and Serving Millions of LLMs

    cs.LG 2026-05 unverdicted novelty 6.0

    MinT enables efficient management of million-scale LoRA-adapted LLM policies over shared 1T-parameter base models by moving only small adapters through training and serving pipelines.

  53. MMSkills: Towards Multimodal Skills for General Visual Agents

    cs.AI 2026-05 unverdicted novelty 6.0

    MMSkills turns public interaction trajectories into compact multimodal skill packages that visual agents can consult at runtime to improve decision-making on benchmarks.

  54. ToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agents

    cs.AI 2026-05 unverdicted novelty 6.0

    ToolCUA introduces a trajectory scaling pipeline and staged RL to optimize GUI-tool switching, reaching 46.85% accuracy on OSWorld-MCP for a 66% relative gain over baseline.

  55. Missing Old Logits in Asynchronous Agentic RL: Semantic Mismatch and Repair Methods for Off-Policy Correction

    cs.LG 2026-05 unverdicted novelty 6.0

    Missing old logits in async agentic RL entangle discrepancy and staleness terms in PPO off-policy correction; exact acquisition methods and revised PPO-EWMA restore decoupled updates with reported gains in speed and p...

  56. LLaVA-UHD v4: What Makes Efficient Visual Encoding in MLLMs?

    cs.CV 2026-05 unverdicted novelty 6.0

    LLaVA-UHD v4 reduces visual-encoding FLOPs by 55.8% for high-resolution images in MLLMs via slice-based encoding plus intra-ViT early compression while matching or exceeding baseline performance on document, OCR, and ...

  57. OrScale: Orthogonalised Optimization with Layer-Wise Trust-Ratio Scaling

    cs.LG 2026-05 unverdicted novelty 6.0

    OrScale adds a Frobenius-norm trust-ratio layer-wise scaler to Muon’s orthogonalized updates, with per-layer calibration for language models, yielding higher CIFAR-10 accuracy and better language-model pre-training lo...

  58. From 0-Order Selection to 2-Order Judgment: Combinatorial Hardening Exposes Compositional Failures in Frontier LLMs

    cs.CL 2026-05 unverdicted novelty 6.0

    LogiHard hardens reasoning benchmarks by transforming 0-order selection into 2-order judgment, causing 31-56% accuracy drops in 12 frontier LLMs and a 47% drop on zero-shot MMLU, revealing a combinatorial reasoning ga...

  59. HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents

    cs.LG 2026-05 unverdicted novelty 6.0

    HyperEyes presents a parallel multimodal search agent using dual-grained efficiency-aware RL with a new TRACE reward and IMEB benchmark, claiming 9.9% higher accuracy and 5.3x fewer tool calls than prior open-source agents.

  60. Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less

    cs.LG 2026-05 unverdicted novelty 6.0

    Full finetuning with the pretraining optimizer reduces forgetting compared to other optimizers or LoRA while achieving comparable new-task performance.

Reference graph

Works this paper leans on

112 extracted references · 112 canonical work pages · cited by 99 Pith papers · 26 internal anchors

  1. [1]

    2025.URL:https://moonshotai.github.io/Kimi- K2/thinking.html

    Moonshot AI.Introducing Kimi K2 Thinking. 2025.URL:https://moonshotai.github.io/Kimi- K2/thinking.html

    work page 2025

  2. [2]

    2025.URL: https://moonshotai.github.io/Kimi-Researcher/

    Moonshot AI.Kimi-Researcher End-to-End RL Training for Emerging Agentic Capabilities. 2025.URL: https://moonshotai.github.io/Kimi-Researcher/

    work page 2025

  3. [3]

    Amazon Web Services.Amazon Simple Storage Service (Amazon S3). Web. Available at:https://aws. amazon.com/s3/. 2023.URL:https://aws.amazon.com/s3/(visited on 12/15/2023)

    work page 2023

  4. [4]

    Held on Febru- ary 6, 2025

    Mathematical Association of America.2025 American Invitational Mathematics Examination I. Held on Febru- ary 6, 2025. 2025.URL:https://artofproblemsolving.com/wiki/index.php/2025_AIME_ I

    work page 2025

  5. [5]

    2026.URL:https://claude.com/ blog/building-multi-agent-systems-when-and-how-to-use-them

    Anthropic.Building multi-agent systems: when and how to use them. 2026.URL:https://claude.com/ blog/building-multi-agent-systems-when-and-how-to-use-them

    work page 2026

  6. [6]

    2025.URL:https : / / www - cdn

    Anthropic.Claude Opus 4.5 System Card. 2025.URL:https : / / www - cdn . anthropic . com / bf10f64990cfda0ba858290be7b8cc6317685f47.pdf

    work page 2025

  7. [7]

    2025.URL:https://www.anthropic.com/ engineering/multi-agent-research-system

    Anthropic.How we built our multi-agent research system. 2025.URL:https://www.anthropic.com/ engineering/multi-agent-research-system

    work page 2025

  8. [8]

    Shuai Bai et al.Qwen3-VL Technical Report. 2025. arXiv:2511 . 21631 [cs.CV].URL:https : / / arxiv.org/abs/2511.21631

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  9. [9]

    Yushi Bai et al.LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Mul- titasks. 2025. arXiv:2412.15204 [cs.CL].URL:https://arxiv.org/abs/2412.15204

    work page arXiv 2025

  10. [10]

    Greg Brockman et al.OpenAI Gym. 2016. arXiv:1606.01540 [cs.LG].URL:https://arxiv.org/ abs/1606.01540

    work page internal anchor Pith review arXiv 2016

  11. [11]

    Language Models are Few-Shot Learners

    Tom B. Brown et al.Language Models are Few-Shot Learners. 2020. arXiv:2005.14165 [cs.CL].URL: https://arxiv.org/abs/2005.14165

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  12. [12]

    Liang Chen et al.BabyVision: Visual Reasoning Beyond Language. 2026. arXiv:2601.06521 [cs.CV]. URL:https://arxiv.org/abs/2601.06521

    work page arXiv 2026

  13. [13]

    Xianfu Cheng et al.SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Models

    work page

  14. [14]

    arXiv:2502.13059 [cs.CL].URL:https://arxiv.org/abs/2502.13059

    work page arXiv

  15. [15]

    DeepSeek-AI et al.DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models. 2025. arXiv:2512. 02556 [cs.CL].URL:https://arxiv.org/abs/2512.02556

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  16. [16]

    Mostafa Dehghani et al.Patch n’ Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution. 2023. arXiv:2307.06304 [cs.CV].URL:https://arxiv.org/abs/2307.06304

    work page arXiv 2023

  17. [17]

    SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?

    Xiang Deng et al. “SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?” In: arXiv preprint arXiv:2509.16941(2025)

    work page internal anchor Pith review arXiv 2025

  18. [18]

    Chaoyou Fu et al.Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis. 2025. arXiv:2405.21075 [cs.CV].URL:https://arxiv.org/abs/2405.21075

    work page internal anchor Pith review arXiv 2025

  19. [19]

    Xingyu Fu et al.BLINK: Multimodal Large Language Models Can See but Not Perceive. 2024. arXiv:2404. 12390 [cs.CV].URL:https://arxiv.org/abs/2404.12390

    work page arXiv 2024

  20. [20]

    Datacomp: In search of the next generation of multimodal datasets

    Samir Yitzhak Gadre et al. “Datacomp: In search of the next generation of multimodal datasets”. In:Advances in Neural Information Processing Systems36 (2024)

    work page 2024

  21. [21]

    2025.URL:https://deepmind.google/models/gemini/pro/

    Google.Gemini 3 Pro. 2025.URL:https://deepmind.google/models/gemini/pro/

    work page 2025

  22. [22]

    Dong Guo et al.Seed1.5-VL Technical Report. 2025. arXiv:2505 . 07062 [cs.CV].URL:https : / / arxiv.org/abs/2505.07062

    work page internal anchor Pith review arXiv 2025

  23. [23]

    Lukas Haas et al.SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge

    work page

  24. [24]

    arXiv:2509.07968 [cs.CL].URL:https://arxiv.org/abs/2509.07968

    work page arXiv

  25. [25]

    Yun He et al.AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM In- struction Following. 2025. arXiv:2511.10507 [cs.CL].URL:https://arxiv.org/abs/2511. 10507

    work page arXiv 2025

  26. [26]

    Wenyi Hong et al.MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models. 2025. arXiv:2501 . 02955 [cs.CV].URL:https : / / arxiv . org / abs / 2501.02955

    work page internal anchor Pith review arXiv 2025

  27. [27]

    Kairui Hu et al.Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos

    work page

  28. [28]

    Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos

    arXiv:2501.13826 [cs.CV].URL:https://arxiv.org/abs/2501.13826. 16 Kimi K2.5TECHNICALREPORT

    work page internal anchor Pith review arXiv

  29. [29]

    Liang Hu et al.FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Rea- soning. 2025. arXiv:2509.13160 [cs.CL].URL:https://arxiv.org/abs/2509.13160

    work page arXiv 2025

  30. [30]

    Yanping Huang et al.GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism. 2019. arXiv:1811.06965 [cs.CV].URL:https://arxiv.org/abs/1811.06965

    work page Pith review arXiv 2019

  31. [31]

    LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

    Naman Jain et al. “Livecodebench: Holistic and contamination free evaluation of large language models for code”. In:arXiv preprint arXiv:2403.07974(2024)

    work page internal anchor Pith review arXiv 2024

  32. [32]

    SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

    Carlos E Jimenez et al. “Swe-bench: Can language models resolve real-world github issues?” In:arXiv preprint arXiv:2310.06770(2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  33. [33]

    2024.URL:https : / / kellerjordan.github.io/posts/muon/

    Keller Jordan et al.Muon: An optimizer for hidden layers in neural networks. 2024.URL:https : / / kellerjordan.github.io/posts/muon/

    work page 2024

  34. [34]

    Kimi k1.5: Scaling Reinforcement Learning with LLMs

    Kimi Team. “Kimi k1. 5: Scaling reinforcement learning with llms”. In:arXiv preprint arXiv:2501.12599 (2025)

    work page internal anchor Pith review arXiv 2025

  35. [35]

    Obelics: An open web-scale filtered dataset of interleaved image-text documents

    Hugo Laurençon et al. “Obelics: An open web-scale filtered dataset of interleaved image-text documents”. In: Advances in Neural Information Processing Systems36 (2024)

    work page 2024

  36. [36]

    GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

    Dmitry Lepikhin et al. “Gshard: Scaling giant models with conditional computation and automatic sharding”. In:arXiv preprint arXiv:2006.16668(2020)

    work page internal anchor Pith review arXiv 2006

  37. [37]

    Muon is Scalable for LLM Training

    Jingyuan Liu et al. “Muon is Scalable for LLM Training”. In:arXiv preprint arXiv:2502.16982(2025)

    work page internal anchor Pith review arXiv 2025

  38. [38]

    OCRBench: on the hidden mystery of OCR in large multimodal models

    Yuliang Liu et al. “OCRBench: on the hidden mystery of OCR in large multimodal models”. In:Science China Information Sciences67.12 (Dec. 2024).ISSN: 1869-1919.DOI:10.1007/s11432-024-4235-6.URL: http://dx.doi.org/10.1007/s11432-024-4235-6

    work page doi:10.1007/s11432-024-4235-6.url: 2024

  39. [39]

    Pan Lu et al.MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts. 2024. arXiv:2310.02255 [cs.CV].URL:https://arxiv.org/abs/2310.02255

    work page internal anchor Pith review arXiv 2024

  40. [40]

    Towards Robust Mathematical Reasoning

    Thang Luong et al. “Towards Robust Mathematical Reasoning”. In:Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Ed. by Christos Christodoulopoulos et al. Suzhou, China: Association for Computational Linguistics, Nov. 2025, pp. 35418–35442.ISBN: 979-8-89176-332-6.DOI:10. 18653 / v1 / 2025 . emnlp - main . 1794.URL:htt...

    work page 2025

  41. [41]

    Minesh Mathew et al.InfographicVQA. 2021. arXiv:2104.12756 [cs.CV].URL:https://arxiv. org/abs/2104.12756

    work page arXiv 2021

  42. [42]

    Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

    Mike A Merrill et al. “Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces”. In:arXiv preprint arXiv:2601.11868(2026)

    work page internal anchor Pith review arXiv 2026

  43. [43]

    Deepak Narayanan et al.Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron- LM. 2021. arXiv:2104.04473 [cs.CL].URL:https://arxiv.org/abs/2104.04473

    work page arXiv 2021

  44. [44]

    2025.URL:https://openai.com/index/introducing- gpt- 5- 2/

    OpenAI.Introducing GPT 5.2. 2025.URL:https://openai.com/index/introducing- gpt- 5- 2/

    work page 2025

  45. [45]

    Linke Ouyang et al.OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive An- notations. 2025. arXiv:2412.07626 [cs.CV].URL:https://arxiv.org/abs/2412.07626

    work page arXiv 2025

  46. [46]

    Tejal Patwardhan et al.GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. 2025. arXiv:2510.04374 [cs.LG].URL:https://arxiv.org/abs/2510.04374

    work page arXiv 2025

  47. [47]

    YaRN: Efficient Context Window Extension of Large Language Models

    Bowen Peng et al. “Yarn: Efficient context window extension of large language models”. In:arXiv preprint arXiv:2309.00071(2023)

    work page internal anchor Pith review arXiv 2023

  48. [48]

    SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models

    Thinh Pham et al.SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models. Seal-0 is the main subset of this benchmark. 2025. arXiv:2506.01062 [cs.CL].URL:https://arxiv.org/ abs/2506.01062

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  49. [49]

    Long Phan et al.Humanity’s Last Exam. 2025. arXiv:2501.14249 [cs.LG].URL:https://arxiv. org/abs/2501.14249

    work page internal anchor Pith review arXiv 2025

  50. [50]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein et al. “Gpqa: A graduate-level google-proof q&a benchmark”. In:First Conference on Language Modeling. 2024

    work page 2024

  51. [51]

    Jonathan Roberts et al.ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Mod- els. 2025. arXiv:2502.09696 [cs.CV].URL:https://arxiv.org/abs/2502.09696

    work page arXiv 2025

  52. [52]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann et al. “Laion-5b: An open large-scale dataset for training next generation image-text models”. In:Advances in Neural Information Processing Systems35 (2022), pp. 25278–25294

    work page 2022

  53. [53]

    Proximal Policy Optimization Algorithms

    John Schulman et al. “Proximal Policy Optimization Algorithms”. In:arXiv preprint arXiv:1707.06347(2017). URL:https://arxiv.org/abs/1707.06347. 17 Kimi K2.5TECHNICALREPORT

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  54. [54]

    Tianhui Song et al.Towards Pixel-Level VLM Perception via Simple Points Prediction. 2026. arXiv:2601. 19228 [cs.CV].URL:https://arxiv.org/abs/2601.19228

    work page arXiv 2026

  55. [55]

    Paperbench: Evaluating ai’s ability to replicate ai research, 2025

    Giulio Starace et al. “PaperBench: Evaluating AI’s Ability to Replicate AI Research”. In:arXiv preprint arXiv:2504.01848(2025)

    work page arXiv 2025

  56. [56]

    Kimi K2: Open Agentic Intelligence

    Kimi Team et al. “Kimi k2: Open agentic intelligence”. In:arXiv preprint arXiv:2507.20534(2025)

    work page internal anchor Pith review arXiv 2025

  57. [57]

    Kimi-VL Technical Report

    Kimi Team et al. “Kimi-vl technical report”. In:arXiv preprint arXiv:2504.07491(2025)

    work page internal anchor Pith review arXiv 2025

  58. [58]

    Longcat-flash-omni technical report.ArXiv, abs/2511.00279,

    Meituan LongCat Team et al. “Longcat-flash-omni technical report”. In:arXiv preprint arXiv:2511.00279 (2025)

    work page arXiv 2025

  59. [59]

    Scicode: A research coding benchmark curated by scientists

    Minyang Tian et al. “Scicode: A research coding benchmark curated by scientists”. In:Advances in Neural Information Processing Systems37 (2024), pp. 30624–30650

    work page 2024

  60. [60]

    Shengbang Tong et al.Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs. 2024. arXiv: 2401.06209 [cs.CV].URL:https://arxiv.org/abs/2401.06209

    work page arXiv 2024

  61. [61]

    Held on February 15, 2025

    Harvard-MIT Mathematics Tournament.Harvard-MIT Mathematics Tournament, February 2025. Held on February 15, 2025. 2025.URL:https://www.hmmt.org/www/archive/282

    work page 2025

  62. [62]

    Attention is All you Need

    Ashish Vaswani et al. “Attention is All you Need”. In:Advances in Neural Information Processing Systems. Ed. by I. Guyon et al. V ol. 30. Curran Associates, Inc., 2017.URL:https://proceedings.neurips. cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

    work page 2017

  63. [63]

    2025.URL:https : / / storage

    Nikhita Vedula et al.DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agents. 2025.URL:https : / / storage . googleapis . com / deepmind - media / DeepSearchQA / DeepSearchQA_benchmark_paper.pdf

    work page 2025

  64. [64]

    Ke Wang et al.Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset. 2024. arXiv: 2402.14804 [cs.CV].URL:https://arxiv.org/abs/2402.14804

    work page arXiv 2024

  65. [65]

    Weihan Wang et al.LVBench: An Extreme Long Video Understanding Benchmark. 2025. arXiv:2406.08035 [cs.CV].URL:https://arxiv.org/abs/2406.08035

    work page arXiv 2025

  66. [66]

    Xinyuan Wang et al.OpenCUA: Open Foundations for Computer-Use Agents. 2025. arXiv:2508 . 09123 [cs.AI].URL:https://arxiv.org/abs/2508.09123

    work page arXiv 2025

  67. [67]

    Yubo Wang et al.MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Bench- mark. 2024. arXiv:2406.01574 [cs.CL].URL:https://arxiv.org/abs/2406.01574

    work page internal anchor Pith review arXiv 2024

  68. [68]

    OJBench: A Competition Level Code Benchmark For Large Language Models

    Zhexu Wang et al. “OJBench: A Competition Level Code Benchmark For Large Language Models”. In:arXiv preprint arXiv:2506.16395(2025)

    work page arXiv 2025

  69. [69]

    Cy- berGym: Evaluating AI agents’ real-world cybersecurity capabilities at scale.arXiv preprint arXiv:2506.02548, 2025

    Zhun Wang et al. “CyberGym: Evaluating AI Agents’ Cybersecurity Capabilities with Real-World Vulnerabili- ties at Scale”. In:arXiv preprint arXiv:2506.02548(2025)

    work page arXiv 2025

  70. [70]

    Zirui Wang et al.CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs. 2024. arXiv: 2406.18521 [cs.CL].URL:https://arxiv.org/abs/2406.18521

    work page arXiv 2024

  71. [71]

    Jason Wei et al.BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents. 2025. arXiv:2504. 12516 [cs.CL].URL:https://arxiv.org/abs/2504.12516

    work page internal anchor Pith review arXiv 2025

  72. [72]

    Ryan Wong et al.WideSearch: Benchmarking Agentic Broad Info-Seeking. 2025. arXiv:2508 . 07999 [cs.CL].URL:https://arxiv.org/abs/2508.07999

    work page arXiv 2025

  73. [73]

    Haoning Wu et al.LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understand- ing. 2024. arXiv:2407.15754 [cs.CV].URL:https://arxiv.org/abs/2407.15754

    work page arXiv 2024

  74. [74]

    Xixi Wu et al.ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization. 2025. arXiv: 2509.13313 [cs.CL].URL:https://arxiv.org/abs/2509.13313

    work page arXiv 2025

  75. [75]

    Introducing OSWorld-Verified

    Tianbao Xie et al. “Introducing OSWorld-Verified”. In:xlang.ai(July 2025).URL:https://xlang.ai/ blog/osworld-verified

    work page 2025

  76. [76]

    Tianbao Xie et al.OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Envi- ronments. 2024. arXiv:2404.07972 [cs.AI]

    work page internal anchor Pith review arXiv 2024

  77. [77]

    Feng Yao et al.Your Efficient RL Framework Secretly Brings You Off-Policy RL Training. Aug. 2025.URL: https://fengyao.notion.site/off-policy-rl

    work page 2025

  78. [78]

    Xiang Yue et al.MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark. 2025. arXiv:2409.02813 [cs.CL].URL:https://arxiv.org/abs/2409.02813

    work page internal anchor Pith review arXiv 2025

  79. [79]

    MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI

    Xiang Yue et al. “MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI”. In:Proceedings of CVPR. 2024

    work page 2024

  80. [80]

    Xiaohua Zhai et al.Sigmoid Loss for Language Image Pre-Training. 2023. arXiv:2303.15343 [cs.CV]. URL:https://arxiv.org/abs/2303.15343. 18 Kimi K2.5TECHNICALREPORT

    work page arXiv 2023

Showing first 80 references.