Pith. sign in

REVIEW 4 major objections 5 minor 30 cited by

Keye-VL-1.5 claims state-of-the-art video understanding for an 8-billion-parameter model by spending high resolution on changing frames and many cheap frames on static ones.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Keye-VL-1.5 combines similarity-based Slow-Fast video token allocation with progressive context extension and iterative RL, reporting leading video-understanding results among 8B-scale multimodal models.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A well-engineered industrial tech report with a genuinely new Slow-Fast video token idea, but the headline SOTA video numbers rest on dedup and evaluation details that are thinner than they look. the 4 major comments →

arxiv 2509.01563 v3 pith:5GRWYWP5 submitted 2025-09-01 cs.CV

Kwai Keye-VL 1.5 Technical Report

Biao Yang , Bin Wen , Boyang Ding , Changyi Liu , Chenglong Chu , Chengru Song , Chongling Rao , Chuan Yi
show 53 more authors
Da Li Dunju Zang Fan Yang Guorui Zhou Guowang Zhang Han Shen Hao Peng Haojie Ding Hao Wang Haonan Fan Hengrui Ju Jiaming Huang Jiangxia Cao Jiankang Chen Jingyun Hua Kaibing Chen Kaiyu Jiang Kaiyu Tang Kun Gai Muhao Wei Qiang Wang Ruitao Wang Sen Na Shengnan Zhang Siyang Mao Sui Huang Tianke Zhang Tingting Gao Wei Chen Wei Yuan Xiangyu Wu Xiao Hu Xingyu Lu Yi-Fan Zhang Yiping Yang Yulong Chen Zeyi Lu Zhenhua Wu Zhixin Ling Zhuoran Yang Ziming Li Di Xu Haixuan Gao Hang Li Jing Wang Lejian Ren Qigen Hu Qianqian Wang Shiyao Wang Xinchen Luo Yan Li Yuhang Hu Zixing Zhang
This is my paper
classification cs.CV
keywords video understandingmultimodal large language modelSlow-Fast video encodinglong-context pre-trainingchain-of-thought reasoningreinforcement learning with verifiable rewardsmodel mergingvision-language benchmarks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents Keye-VL-1.5, an 8-billion-parameter multimodal model aimed at video understanding without sacrificing general vision-language ability. Its central claim is that the usual trade-off between spatial resolution and temporal coverage can be broken by a Slow-Fast encoding: frames that change a lot get high-resolution processing, while near-static frames get low resolution but wider temporal coverage. Around this, the report builds a four-stage pre-training routine that stretches context from 8K to 128K tokens, and a post-training stack that adds chain-of-thought cold-start data, rule-reward reinforcement learning, and preference alignment. On public benchmarks the paper reports state-of-the-art results among open models of similar scale on several video tasks, with the largest margins on long and professional video understanding. The reader should care because the design is a concrete recipe for making video input affordable within a fixed token budget.

Core claim

The paper's core claim is that allocating visual tokens by measured inter-frame change, rather than uniform sampling, lets an 8B model see both fine detail and long temporal structure. A patch-based similarity function labels a frame as slow if it differs from the last key frame by more than roughly five percent; slow frames receive high resolution, and the remaining fast frames receive about 30 percent of a slow frame's token budget, selected by binary search under a fixed total budget. The report argues this is why Keye-VL-1.5 outperforms prior open 8B models on video benchmarks while staying competitive on images, math, and OCR, and why it can answer time-grounded questions such as when a

What carries the argument

Slow-Fast video encoding is the load-bearing mechanism. It first marks the first frame as slow; every later frame whose patch similarity to the latest slow frame is below a 95% threshold becomes a new slow frame, everything else is fast. Slow frames are encoded at full resolution, fast frames at a reduced token budget (about 30% of a slow frame), with a binary search fixing per-frame tokens under the total video budget (about 75,000 tokens). Special start/end tokens and absolute timestamps mark slow/fast boundaries so the decoder knows temporal positions. The surrounding machinery—native-resolution vision encoder with 2D rotary positions, 128K context extension, GSPO reinforcement learning,

Load-bearing premise

The reported video gains assume that the training corpus was actually scrubbed of near-duplicates of the evaluation videos; the report describes this deduplication as rigorous but gives no method, threshold, or coverage numbers.

What would settle it

Search the deduplicated training corpus for near-duplicate frames of the public video benchmarks; substantial overlap would inflate the reported video scores.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • At a fixed token budget, a video model can cover much longer videos without losing detail on motion-critical moments, so long-video QA and temporal grounding become cheaper.
  • The same four-stage pre-training with context extension from 8K to 128K can be reused to train other backbones for long multimodal inputs.
  • The iterative post-training loop—cold-start CoT, reward-based RL with progressive hints, then rejection sampling to refresh SFT data—yields reported gains on math, OCR, and instruction following as well as video.
  • If the public-benchmark numbers hold, open 8B models close much of the gap to much larger proprietary systems on video understanding.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the paper's mechanism would ablate the 95% similarity threshold and the 30% fast-frame budget: if performance barely moves, the gain may come from the larger context or post-training instead.
  • Because the deduplication step is described without parameters, an independent near-duplicate audit of the training corpus against the evaluation benchmarks would settle whether the reported video scores are contaminated.
  • The Slow-Fast idea transfers naturally to other temporally redundant signals, such as audio or depth streams, where a similar change-based allocation could cut compute.
  • The internal human evaluation rests on only 200 cases; a larger blind comparison could test whether the +0.51 gain over the preview model is real or within annotation noise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents Keye-VL-1.5, an 8B-parameter multimodal model built on Qwen3-8B and SigLIP-400M. Its main contributions are a Slow-Fast video encoding strategy that assigns high-resolution tokens to visually changing frames and low-resolution tokens to static frames; a four-stage pre-training schedule that extends context length from 8K to 128K tokens; and a post-training pipeline combining SFT, MPO, a learned reward model, LongCoT cold-start data, iterative GSPO-based RL with progressive hint sampling, and alignment RL. The central claim is state-of-the-art video understanding at the 8B scale, supported by Table 3 with Video-MME 73.0, Video-MMMU 66.0, TempCompass 75.5, LongVideoBench 66.0, and MMVU 68.3, plus competitive results on general and mathematical benchmarks. The paper also reports an internal 200-case human evaluation and several ablations of training-data and post-training choices.

Significance. If the reported numbers are trustworthy, this is a practically valuable technical result: the public benchmark gains over Qwen2.5-VL-7B, InternVL3-8B, and MiMo-VL-7B-RL are large, especially on Video-MMMU (+18.6 over Qwen2.5-VL), and the Slow-Fast encoding idea is simple and well-motivated. The authors provide broad benchmark coverage, several controlled ablations, and they release model links, which makes the central claim externally checkable. The paper is also honest about limitations of public benchmarks and makes an effort to construct an internal evaluation suite. However, the central claim is empirical, and the current reporting leaves two load-bearing gaps: video-frame contamination is not ruled out with evidence, and the Table 6 ablations do not consistently support the described training pipeline. These issues should be resolved before the SOTA claim can be endorsed.

major comments (4)
  1. [§3.1, §3.1.5, Table 3] The central SOTA-on-video claim depends on the absence of training/evaluation overlap, but the paper only says "we also conduct rigorous image-based deduplication operation" (§3.1) and gives no threshold, hashing method, coverage, or video-frame-level procedure. §3.1.5 says video data come from "diverse open-source datasets (ShareGPT4V, Pandas and others)" plus in-house data, with no deduplication of video frames against Video-MME/Video-MMMU. Since Table 3 reports very large margins over Qwen2.5-VL (73.0 vs 65.1 on Video-MME; 66.0 vs 47.4 on Video-MMMU), even modest near-duplicate leakage could inflate the central result. Please report the exact deduplication protocol and, ideally, a contamination analysis measuring frame-level similarity between the training corpus and the evaluation benchmarks, or an evaluation on a fresh held-out set.
  2. [§6.6.1, Table 6] The ablation table is difficult to reconcile with the text. The SFT+MPO rows are far below the Keye-VL-7B-Preview baseline (e.g., OpenCompass 70.34 vs 77.43, MMBEN 83.44 vs 92.03, MMVet 62.34 vs 68.62, MMMU 58.33 vs 71.67), while the surrounding text says "increasing the volume of SFT training data consistently enhances model performance" and "MPO consistently yields additional performance improvements." Since SFT+MPO is listed as a component of the final pipeline (§4.1), the reader needs to know what dataset sizes the final model used and why the SFT+MPO ablation is so much weaker than the earlier preview model. As printed, the table neither validates the non-reasoning stage nor supports the monotonic-data-volume claim.
  3. [§6.4, §6.5, Tables 4–5] The headline internal improvements (+0.51 vs Keye-VL-Preview, +0.13 vs MiMoVL-7B-RL-2508) are based on only 200 test cases, with per-dimension sample sizes of 11–35. No confidence intervals, standard errors, or inter-annotator agreement are reported, and the evaluation uses a mixture of 1–5 scoring and GSB preference. With this sample size, several dimension-level differences (e.g., +0.25 on Reasoning Ability in Table 5) may not be statistically distinguishable from noise. Please report uncertainty bounds and annotator agreement, at least for the composite score and for the dimensions with the smallest N.
  4. [§2.2, §6.2] The Slow-Fast scheme is the paper's named architectural novelty, but its two key hyperparameters — the 95% patch-similarity threshold for labeling a frame as fast, and the 30% token budget for fast frames — are presented as fixed choices with no sensitivity analysis. Figure 7 compares the full scheme against Qwen2.5-VL, so it does not isolate the effect of these thresholds. A small ablation over the threshold and budget would substantiate the claim that the mechanism, not merely the added token budget, drives the video gains.
minor comments (5)
  1. [§3.1] The citation for deduplication, Dixit et al. (2021), is a paper on silent data corruptions, not on deduplication or contamination filtering. Please replace it with an appropriate reference or remove it.
  2. [§6.3, Table 3] The text says Keye-VL-1.5 obtains 86.7 on AI2D, but Table 3 lists 89.5 for Keye-VL-1.5 and 86.7 for Keye-VL-Preview. Please correct the text/table inconsistency.
  3. [§6.3] The paper refers to "Qwen2.5-VL 7B" in Table 3 and in the first paragraph of §6.3, but later says "Qwen2.5-VL 8B." Please standardize the model designation.
  4. [§3.1.5] The dataset name "Pandas" appears to be a typo for "Panda" (or another named dataset); please verify the exact dataset name and add a reference.
  5. [§6.2] Figure 7 reports VideoMME scores under many frame/FPS settings, but the exact scoring protocol and whether subtitles were used are not stated in that section. Please clarify, since Table 3 explicitly distinguishes "Video-MME w/o sub.".

Circularity Check

0 steps flagged

No significant circularity: headline results are measured on independent public benchmarks; internal self-training loops are not used as the evidence for the central claims.

full rationale

The paper's central quantitative claims are public-benchmark scores in Table 3 (e.g., Video-MME 73.0, Video-MMMU 66.0, TempCompass 75.5, LongVideoBench 66.0, MMVU 68.3), obtained on external, independently maintained benchmarks (Video-MME, Video-MMMU, TempCompass, LongVideoBench, MMVU, OpenCompass, MMMU, AI2D, MMBench, MathVista, etc.). These evaluation suites are outside the paper's training pipeline, so the headline 'state-of-the-art' claim does not reduce to a fitted parameter, a self-citation, or the model's own reward model. The paper does contain self-referential training loops: Section 4.2 states 'We take our reward model to evaluate the quality of Keye-VL’s sampling results, which are applied to update the training data and provide reward signals,' and Section 4.4.3 updates cold-start data using the RL model's own rejection-sampled outputs. However, these loops are used to construct training data and RL rewards, not to derive or validate the reported benchmark improvements; the validation is the external benchmark evaluation. Self-citations such as Keye-VL-Preview (used as a baseline and as the reward-model base) and TaskGalaxy (used as an SFT data-categorization tool) are not load-bearing proofs of the central claim and do not import an unverified uniqueness theorem. The internal evaluation in Section 6.4 is small (200 cases, three annotators) and could be underpowered, and Section 6.4 itself acknowledges data-contamination risks for public benchmarks; the deduplication description in Section 3.1 is image-based and gives no thresholds or coverage. These are legitimate correctness and robustness concerns (possible leakage, limited statistical power), but they are not cases where a claimed prediction is equivalent to its inputs by construction. No specific equation-level or citation-level reduction can be exhibited, so the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The paper is an empirical systems report. It introduces no new physical or mathematical entities. The free parameters are engineering choices in the encoding and training recipe; they are tuned by experiment rather than derived. The key assumptions are that the base models (SigLIP, Qwen3), the public benchmarks, and the internal reward model are each reliable enough to support the reported performance claims.

free parameters (5)
  • Slow/Fast similarity threshold = 95% patch similarity
    Hand-chosen threshold that determines which frames are classified as slow vs. fast; directly controls the token distribution in the Slow-Fast encoding (Section 2.2).
  • Fast frame token budget ratio = 30% of a slow frame's budget
    Hand-chosen ratio balancing temporal coverage against per-frame detail; not derived from data (Section 2.2).
  • Total video token budget = 75,000 tokens
    A fixed budget applied to video inputs; chosen by engineering considerations rather than derived (Section 2.2).
  • Data mixture ratios in annealing stage = 24% video, 50% image, 26% text
    Selected after controlled experiments; these ratios affect the balance of video, image, and text capabilities (Section 3.2).
  • RoPE inverse frequency reset = 8,000,000
    Chosen value for extending context length from 8K to 128K; set by hand rather than derived (Section 3.2).
axioms (4)
  • domain assumption SigLIP-400M-384-14 provides a strong visual backbone for high-resolution image and video understanding.
    The entire vision encoder is initialized from SigLIP; if this base representation were weak, the model's downstream visual capabilities would be limited. Invoked in Section 2.
  • domain assumption Qwen3-8B provides a strong language base for instruction following and reasoning.
    The language decoder is initialized from Qwen3-8B; the report relies on its world knowledge and reasoning abilities. Invoked in Section 2.
  • domain assumption Public video and multimodal benchmarks (Video-MME, Video-MMMU, etc.) measure the capabilities the paper claims.
    The central SOTA claim is based on these benchmarks; if they are contaminated or poorly designed, the claim weakens. Invoked in Section 6.3.
  • domain assumption The in-house reward model (trained from Keye-VL-Preview) produces preference judgments that align with human quality.
    The reward model is used for data filtering, rejection sampling, and alignment RL; if it is biased, the training loop may reinforce model-specific errors. Invoked in Sections 4.2 and 4.4.3.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Kwai Keye-VL 1.5 Technical Report." pith.science (2026). https://pith.science/paper/5GRWYWP5

@misc{pith2026250901563,
  author       = {Pith},
  title        = {Pith review of: Kwai Keye-VL 1.5 Technical Report},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5GRWYWP5}},
  note         = {Machine review of arXiv:2509.01563}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

In recent years, the development of Large Language Models (LLMs) has significantly advanced, extending their capabilities to multimodal tasks through Multimodal Large Language Models (MLLMs). However, video understanding remains a challenging area due to the dynamic and information-dense nature of videos. Existing models struggle with the trade-off between spatial resolution and temporal coverage when processing video content. We present Keye-VL-1.5, which addresses fundamental challenges in video comprehension through three key innovations. First, we introduce a novel Slow-Fast video encoding strategy that dynamically allocates computational resources based on inter-frame similarity, processing key frames with significant visual changes at higher resolution (Slow pathway) while handling relatively static frames with increased temporal coverage at lower resolution (Fast pathway). Second, we implement a progressive four-stage pre-training methodology that systematically extends the model's context length from 8K to 128K tokens, enabling processing of longer videos and more complex visual content. Third, we develop a comprehensive post-training pipeline focusing on reasoning enhancement and human preference alignment, incorporating a 5-step chain-of-thought data construction process, iterative GSPO-based reinforcement learning with progressive prompt hinting for difficult cases, and alignment training. Through extensive evaluation on public benchmarks and rigorous internal human assessment, Keye-VL-1.5 demonstrates significant improvements over existing models, particularly excelling in video understanding tasks while maintaining competitive performance on general multimodal benchmarks.

Figures

Figures reproduced from arXiv: 2509.01563 by Biao Yang, Bin Wen, Boyang Ding, Changyi Liu, Chenglong Chu, Chengru Song, Chongling Rao, Chuan Yi, Da Li, Di Xu, Dunju Zang, Fan Yang, Guorui Zhou, Guowang Zhang, Haixuan Gao, Hang Li, Han Shen, Haojie Ding, Haonan Fan, Hao Peng, Hao Wang, Hengrui Ju, Jiaming Huang, Jiangxia Cao, Jiankang Chen, Jing Wang, Jingyun Hua, Kaibing Chen, Kaiyu Jiang, Kaiyu Tang, Kun Gai, Lejian Ren, Muhao Wei, Qiang Wang, Qianqian Wang, Qigen Hu, Ruitao Wang, Sen Na, Shengnan Zhang, Shiyao Wang, Siyang Mao, Sui Huang, Tianke Zhang, Tingting Gao, Wei Chen, Wei Yuan, Xiangyu Wu, Xiao Hu, Xinchen Luo, Xingyu Lu, Yan Li, Yi-Fan Zhang, Yiping Yang, Yuhang Hu, Yulong Chen, Zeyi Lu, Zhenhua Wu, Zhixin Ling, Zhuoran Yang, Ziming Li, Zixing Zhang.

Figure 1
Figure 1. Figure 1: Benchmark Performance of Kwai Keye-VL-1.5. Keye-VL-1.5-8B establishes new state-of-the-art performance among models of similar scale, demonstrating superior results on video-centric benchmarks while maintaining competitive performance on general multimodal and reasoning tasks. Compared to Keye-VL-Preview, this version shows significant improvements across all three evaluation dimensions, validating the eff… view at source ↗
Figure 2
Figure 2. Figure 2: The Kwai Keye-VL-1.5 model architecture is based on the Qwen3-8B language model and incorporates a vision encoder initialized from the open-source SigLIP. It supports SlowFast video encoding and native dynamic resolution, preserving the original aspect ratio of images by dividing each into a 14x14 patch sequence. A simple MLP layer then maps and merges the visual tokens. The model uses 3D RoPE for unified … view at source ↗
Figure 3
Figure 3. Figure 3: A SlowFast video encoding demonstration: the Slow processes a smaller number of frames at higher resolution, while the Fast handles more frames at lower resolution. the spatial information. According to it, we first employ interpolation techniques to extend fixed￾length learnable position embeddings into resolution-adaptive position embeddings, enabling our basic native-resolution modeling while preserving… view at source ↗
Figure 4
Figure 4. Figure 4: The Kwai Keye-VL-1.5 pre-training pipeline, featuring a four-stage progressive strategy: Image-Text Matching, ViT-LLM Alignment, Multi-task Pre-training, and Annealing with model merging. ⋄ Multiple video matching: Provided with a group of related videos and a set of candidate videos, our model is required to identify the most contextually relevant candidate, which refines its understanding of semantic con… view at source ↗
Figure 5
Figure 5. Figure 5: Post-Training Pipeline: The post-training process includes non-reasoning stage and reasoning stage. The non-reasoning stage is composed of SFT and MPO training. The reasoning stage consists of three key steps: CoT Cold Start (we construct a five-step construction pipeline to generate high￾quality CoT Cold-Start Dataset and apply model merging to refine model performance), General RL (we concentrate on impr… view at source ↗
Figure 6
Figure 6. Figure 6: Overview of our five-step automated LongCoT data generation pipeline. The pipeline begins with (a) sampling from data and prompt pools using MLLMs to generate thinking processes and logit information, followed by (b) quality assessment using MLLM as judge to evaluate both outcomes and reasoning processes with step-wise scoring, (c) categorization into three quality tiers (A: high quality, B: middle quality… view at source ↗
Figure 7
Figure 7. Figure 7: SlowFast (Keye-VL-1.5-Base) and 2D convolution (Qwen-2.5 VL) video encoding strategies [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Benefits of rejection sampling in the RL Phase. Starting from Keye-VL-8B-Preview, we compare the performance of direct RL and RFT-RL strategies. across other benchmarks. Based on these findings, we adopt the SFT-RL-(RFT-SFT)-(RFT-RL) iterative model to further enhance performance. 7 Conclusion and Discussion In this work, we presented Keye-VL-1.5, an advanced multimodal model that significantly enhances vi… view at source ↗
Figure 9
Figure 9. Figure 9: In this 26-second video, the creator provides a detailed introduction to the performance, [PITH_FULL_IMAGE:figures/full_fig_p030_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: In this video, the caption shows that the dog dad prevents the little dog from eating the [PITH_FULL_IMAGE:figures/full_fig_p031_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: This video describes a very beautiful yet rare phenomenon: hail falling in the forest. Without [PITH_FULL_IMAGE:figures/full_fig_p032_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 30 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding

    cs.CV 2026-01 unverdicted novelty 8.0

    Molmo2 delivers state-of-the-art open-weight video VLMs with new grounding datasets and training methods that outperform prior open models and match or exceed some proprietary ones on pointing and tracking tasks.

  2. MuseBench: Benchmarking Intent-Level Audiovisual Arts Understanding in MLLMs

    cs.CV 2026-06 unverdicted novelty 7.0

    MuseBench shows state-of-the-art MLLMs achieve only 48.29% accuracy on intent-level audiovisual arts understanding versus 87.18% for human experts.

  3. Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events

    cs.CV 2026-06 conditional novelty 7.0

    Moment-Video benchmark shows top video MLLM achieves only 39.6% accuracy on momentary visual event tasks, with most open-source models below 25%.

  4. Towards Open-World Referring Expression Comprehension: A Benchmark with Training-free Multi-task Consistency Checker

    cs.CV 2026-05 unverdicted novelty 7.0

    OpenRef benchmark for open-world REC with F1 and N3R metrics and training-free MCC to improve existing models in complex scenarios.

  5. Can MLLMs Reason Beyond Language? VisReason: A Comprehensive Benchmark for Vision-Centric Reasoning

    cs.CV 2026-05 unverdicted novelty 7.0

    VisReason is a new benchmark exposing substantial gaps in vision-centric reasoning for MLLMs versus humans, with limited gains from test-time reasoning strategies.

  6. SciVQR: A Multidisciplinary Multimodal Benchmark for Advanced Scientific Reasoning Evaluation

    cs.CV 2026-05 unverdicted novelty 7.0

    SciVQR is a new benchmark dataset for evaluating multimodal AI models on complex scientific reasoning tasks across six disciplines, including expert solutions for nearly half the items.

  7. Perception Without Engagement: Dissecting the Causal Discovery Deficit in LMMs

    cs.CL 2026-05 unverdicted novelty 7.0

    LMMs perceive videos but underexploit visual content for causal reasoning due to textual shortcuts; ProCauEval diagnoses this and ADPO training reduces reliance on priors.

  8. Visual Preference Optimization with Rubric Rewards

    cs.CV 2026-04 unverdicted novelty 7.0

    rDPO uses offline-built rubrics to generate on-policy preference data for DPO, raising benchmark scores in visual tasks over outcome-based filtering and style baselines.

  9. ESOM: Efficiently Understanding Streaming Video Anomalies with Open-world Dynamic Definitions

    cs.CV 2026-04 unverdicted novelty 7.0

    ESOM is a training-free streaming model for open-world video anomaly detection with dynamic definitions that achieves real-time single-GPU efficiency and state-of-the-art results on a new benchmark.

  10. DocPO: Advancing Document Policy Optimization via Tailored Step-Aware Rewards

    cs.CV 2026-08 conditional novelty 6.0

    A reward-annealing trick that progressively sharpens edit-distance rewards improves GRPO-style RL for document parsing across text, tables, and formulas.

  11. RefCaptioner: Multi-Reference Image-Grounded Video Captioning

    cs.CV 2026-07 conditional novelty 6.0

    Mixed-data SFT plus Hierarchical Coverage-Discounted GRPO yields open-source SOTA multi-reference image-grounded video captions on MRVBench without hurting general captioning.

  12. ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering

    cs.CV 2026-06 unverdicted novelty 6.0

    ViTexQA is a dataset forcing multi-frame text fusion for all questions, with FrameThinker achieving 6.3% ROUGE-L gain over baselines via CoT SFT and temporally-grounded RL.

  13. AdaCodec: A Predictive Visual Code for Video MLLMs

    cs.CV 2026-06 unverdicted novelty 6.0

    AdaCodec introduces a predictive visual code that cuts visual token use in video MLLMs by sending full frames only on high predictive cost and otherwise encoding inter-frame changes as P-tokens, yielding better benchm...

  14. LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding

    cs.CV 2026-05 unverdicted novelty 6.0

    LocateAnything proposes Parallel Box Decoding for unified generative visual grounding and detection, paired with a 138M-sample dataset, to raise both speed and high-IoU accuracy.

  15. IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams

    cs.CV 2026-05 unverdicted novelty 6.0

    IPIBench evaluates MLLMs on interactive proactive intelligence in streaming videos, identifies unstable triggering and poor coordination, and proposes the training-free IPI-Agent framework to improve performance acros...

  16. Building a Precise Video Language with Human-AI Oversight

    cs.CV 2026-04 unverdicted novelty 6.0

    CHAI framework pairs AI pre-captions with expert human critiques to produce precise video descriptions, enabling open models to outperform closed ones like Gemini-3.1-Pro and improve fine-grained control in video gene...

  17. POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs

    cs.CV 2026-04 unverdicted novelty 6.0

    POINTS-Long is a dual-mode multimodal large language model that uses dynamic visual token scaling to retain 97.7-99.7% accuracy on long-form tasks with 1/40 to 1/10th the tokens and supports streaming via detachable KV-cache.

  18. Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding

    cs.CV 2026-04 unverdicted novelty 6.0

    Video-MME-v2 is a new benchmark that applies progressive visual-to-reasoning levels and non-linear group scoring to expose gaps in video MLLM capabilities.

  19. Joint Reward Modeling: Internalizing Chain-of-Thought for Efficient Visual Reward Models

    cs.AI 2026-02 conditional novelty 6.0

    JRM jointly optimizes preference ranking and language modeling on a shared vision-language backbone, yielding an efficient discriminative reward model that outperforms both discriminative and generative baselines on i...

  20. Streaming Video Instruction Tuning

    cs.CV 2025-12 unverdicted novelty 6.0

    Streamo is a streaming video LLM trained end-to-end on the new Streamo-Instruct-465K dataset that unifies multiple real-time video tasks with claimed strong temporal reasoning and generalization.

  21. Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model

    cs.CV 2026-07 conditional novelty 5.0

    Codec-guided sparse patch selection plus a lightweight speak/silent gate yields a 4B streaming VLM that is competitive on static tasks, stronger on video/spatial benchmarks, and much cheaper at inference.

  22. Towards Open-World Referring Expression Comprehension: A Benchmark with Training-free Multi-task Consistency Checker

    cs.CV 2026-05 conditional novelty 5.0

    OpenRef benchmarks open-world REC with multi/none-target and hard scenes, and MCC is a training-free consistency plug-in that improves existing models.

  23. SciVQR: A Multidisciplinary Multimodal Benchmark for Advanced Scientific Reasoning Evaluation

    cs.CV 2026-05 unverdicted novelty 5.0

    SciVQR is a new multimodal benchmark covering 54 scientific subfields that evaluates MLLMs on visual comprehension and multi-step reasoning, revealing significant limitations in leading models.

  24. Scaling Video Understanding via Compact Latent Multi-Agent Collaboration

    cs.CV 2026-05 unverdicted novelty 5.0

    MACF decouples agent perception budgets from overall video length using latent token collaboration to scale video understanding in MLLMs beyond current limits.

  25. UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameters

    cs.CV 2025-12 conditional novelty 5.0

    A 0.1B-parameter text/formula recognition model trained on a new 40M-sample dataset matches or beats much larger OCR models and runs 2-9× faster.

  26. InternVideo3: Agentify Foundation Models with Multimodal Contextual Reasoning

    cs.CV 2026-06 unverdicted novelty 4.0

    InternVideo3 introduces Multimodal Contextual Reasoning and M^2LA attention to enable closed-loop evidence accumulation in long-video understanding and agentic tool use, reporting strong benchmark results.

  27. Kwai Keye-VL-2.0 Technical Report

    cs.CV 2026-06 unverdicted novelty 4.0

    Kwai Keye-VL-2.0-30B-A3B is a 30B MoE model with 3B active parameters using DSA adaptation and MOPD distillation that reports SOTA results on video understanding and agent benchmarks.

  28. Watch, Remember, Reason: Human-View Video Understanding with MLLMs

    cs.CV 2026-06 unverdicted novelty 4.0

    This is a survey that frames video MLLM research via a human-view formulation of perceptual representations, memory states, reasoning traces, and predictions, then reviews methods, datasets, benchmarks, and open problems.

  29. LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence

    cs.CV 2026-05 unverdicted novelty 4.0

    LLaVA-OV-2 uses codec-stream tokenization and a shared 3D RoPE to improve video, spatial, and tracking performance over Qwen3-VL-8B, while introducing the JumpScore benchmark for fine-grained motion localization.

  30. Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities

    cs.LG 2024-08 accept novelty 4.0

    The paper introduces a new taxonomy for model merging methods and reviews their applications in LLMs, MLLMs, continual learning, multi-task learning, and other subfields while outlining open challenges.

Reference graph

Works this paper leans on

44 extracted references · 3 canonical work pages · cited by 28 Pith papers

  1. [1]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

    Pith/arXiv arXiv

  2. [3]

    Ernie 4.5 technical report, 2025a

    Baidu ERNIE Team. Ernie 4.5 technical report, 2025a. Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024a. FaceBook. The llama 4 herd: The beginning of a new era of natively multimodal ai innov...

    Pith/arXiv arXiv

  3. [5]

    ByteDance Seed, Jiaze Chen, Tiantian Fan, Xin Liu, Lingjun Liu, Zhiqi Lin, Mingxuan Wang, Chengyi Wang, Xiangpeng Wei, Wenyuan Xu, et al. Seed1. 5-thinking: Advancing superb reasoning models with reinforcement learning. arXiv preprint arXiv:2504.13914,

    arXiv

  4. [6]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025a. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Z...

    Pith/arXiv arXiv

  5. [7]

    Mmict: Boosting multi-modal fine-tuning with in-context examples.ACM Transactions on Multimedia Computing, Communications and Applications, 2024a

    Tao Chen, Enwei Zhang, Yuting Gao, Ke Li, Xing Sun, Yan Zhang, Hui Li, and Rongrong Ji. Mmict: Boosting multi-modal fine-tuning with in-context examples.ACM Transactions on Multimedia Computing, Communications and Applications, 2024a. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, e...

    Pith/arXiv arXiv

  6. [8]

    Kimi-vl technical report

    Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, et al. Kimi-vl technical report. arXiv preprint arXiv:2504.07491, 2025a. Qianhan Feng, Wenshuo Li, Tong Lin, and Xinghao Chen. Align-kd: Distilling cross-modal alignment knowledge for mobile vision-language model. arXiv preprint arXiv:2412.01282,

    Pith/arXiv arXiv

  7. [9]

    Vita-1.5: Towards gpt-4o level real-time vision and speech interaction

    Chaoyou Fu, Haojia Lin, Xiong Wang, Yi-Fan Zhang, Yunhang Shen, Xiaoyu Liu, Haoyu Cao, Zuwei Long, Heting Gao, Ke Li, et al. Vita-1.5: Towards gpt-4o level real-time vision and speech interaction. arXiv preprint arXiv:2501.01957, 2025a. Kai Han, Jianyuan Guo, Yehui Tang, Wei He, Enhua Wu, and Yunhe Wang. Free video-llm: Prompt-guided visual perception for...

    Pith/arXiv arXiv

  8. [10]

    Rain: Your language models can align themselves without finetuning

    Yuhui Li, Fangyun Wei, Jinjing Zhao, Chao Zhang, and Hongyang Zhang. Rain: Your language models can align themselves without finetuning. arXiv preprint arXiv:2309.07124,

    Pith/arXiv arXiv

  9. [12]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 24185–24198, 2024c. Gen Luo, Yiyi Zhou, Yuxin...

    Pith/arXiv arXiv

  10. [13]

    Denseworld-1m: Towards detailed dense grounded caption in the real world, 2025a

    Xiangtai Li, Tao Zhang, Yanwei Li, Haobo Yuan, Shihao Chen, Yikang Zhou, Jiahao Meng, Yueyi Sun, Shilin Xu, Lu Qi, Tianheng Cheng, Yi Lin, Zilong Huang, Wenhao Huang, Jiashi Feng, and Guang Shi. Denseworld-1m: Towards detailed dense grounded caption in the real world, 2025a. URL https://arxiv.org/abs/2506.24102. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin ...

    Pith/arXiv arXiv

  11. [14]

    Mllm-selector: Necessity and diversity-driven high-value data selection for enhanced visual instruction tuning

    25 Yiwei Ma, Guohai Xu, Xiaoshuai Sun, Jiayi Ji, Jie Lou, Debing Zhang, and Rongrong Ji. Mllm-selector: Necessity and diversity-driven high-value data selection for enhanced visual instruction tuning. arXiv preprint arXiv:2503.20502,

    Pith/arXiv arXiv

  12. [15]

    Openthinkimg: Learning to think with images via visual tool reinforcement learning

    Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, et al. Openthinkimg: Learning to think with images via visual tool reinforcement learning. arXiv preprint arXiv:2505.08617,

    Pith/arXiv arXiv

  13. [16]

    Open- reasoner-zero: An open source approach to scaling up reinforcement learning on the base model

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open- reasoner-zero: An open source approach to scaling up reinforcement learning on the base model. arXiv preprint arXiv:2503.24290, 2025a. Yunhang Shen, Chaoyou Fu, Shaoqi Dong, Xiong Wang, Peixian Chen, Mengdan Zhang, Haoyu Cao, Ke Li, Xiawu Zheng, Yan Zhang, et al. L...

    Pith/arXiv arXiv

  14. [17]

    Video-llava: Learning united visual representation by alignment before projection

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.10122,

    Pith/arXiv arXiv

  15. [18]

    Video-rag: Visually-aligned retrieval-augmented long video comprehension

    Yongdong Luo, Xiawu Zheng, Xiao Yang, Guilin Li, Haojia Lin, Jinfa Huang, Jiayi Ji, Fei Chao, Jiebo Luo, and Rongrong Ji. Video-rag: Visually-aligned retrieval-augmented long video comprehension. arXiv preprint arXiv:2411.13093, 2024b. Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Arenas, Travis Armstr...

    arXiv

  16. [20]

    Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al

    URL https: //arxiv.org/abs/2408.01800. Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. Datacomp: In search of the next generation of multimodal datasets. Advances in Neural Information Processing Systems, 36:27092–27112,

    Pith/arXiv arXiv

  17. [21]

    Public domain 12m: A highly aesthetic image-text dataset with novel governance mechanisms

    Jordan Meyer, Nick Padgett, Cullen Miller, and Laura Exline. Public domain 12m: A highly aesthetic image-text dataset with novel governance mechanisms. arXiv preprint arXiv:2410.23144,

    Pith/arXiv arXiv

  18. [22]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13, pages 740–755. Springer,

    work page 2014

  19. [24]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al

    URL https://arxiv.org/abs/2501.07888. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805,

    Pith/arXiv arXiv

  20. [25]

    ReferItGame: Referring to objects in photographs of natural scenes

    Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. ReferItGame: Referring to objects in photographs of natural scenes. In Alessandro Moschitti, Bo Pang, and Walter Daelemans, editors, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 787– 798, Doha, Qatar, October

    work page 2014

  21. [26]

    doi: 10.3115/v1/D14-1086

    Association for Computational Linguistics. doi: 10.3115/v1/D14-1086. URL https://aclanthology.org/D14-1086. Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. Visual genome: Connecting language and vision using crowdsourced...

    doi:10.3115/v1/d14-1086

  22. [28]

    Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models. arXiv preprint arXiv:2409.17146,

    Pith/arXiv arXiv

  23. [29]

    Tempura: Temporal event masked prediction and understanding for reasoning in action

    Jen-Hao Cheng, Vivian Wang, Huayu Wang, Huapeng Zhou, Yi-Hao Peng, Hou-I Liu, Hsiang-Wei Huang, Kuang-Ming Chen, Cheng-Yen Yang, Wenhao Chai, et al. Tempura: Temporal event masked prediction and understanding for reasoning in action. arXiv preprint arXiv:2505.01583, 2025a. Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jiali...

    Pith/arXiv arXiv 2010

  24. [30]

    Taskgalaxy: Scaling multi-modal instruction fine-tuning with tens of thousands vision task types

    Jiankang Chen, Tianke Zhang, Changyi Liu, Haojie Ding, Yaya Shi, Feng Cheng, Huihui Xiao, Bin Wen, Fan Yang, Tingting Gao, et al. Taskgalaxy: Scaling multi-modal instruction fine-tuning with tens of thousands vision task types. arXiv preprint arXiv:2502.09925,

    Pith/arXiv arXiv

  25. [31]

    Enhancing the reasoning ability of multimodal large language models via mixed preference optimization

    Weiyun Wang, Zhe Chen, Wenhai Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu, Yu Qiao, and Jifeng Dai. Enhancing the reasoning ability of multimodal large language models via mixed preference optimization. arXiv preprint arXiv:2411.10442, 2024b. Yi-Fan Zhang, Tao Yu, Haochen Tian, Chaoyou Fu, Peiyan Li, Jianshu Zeng, Wulin Xie...

    Pith/arXiv arXiv

  26. [32]

    Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin

    URL https://arxiv.org/abs/2505.19892. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. Group sequence policy optimization,

    arXiv

  27. [33]

    OpenCompass Contributors

    URL https://arxiv.org/abs/2507.18071. OpenCompass Contributors. Opencompass: A universal evaluation platform for foundation models. https://github.com/open-compass/opencompass,

    Pith/arXiv arXiv

  28. [34]

    A diagram is worth a dozen images

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 235–251. Springer,

    work page 2016

  29. [35]

    Zerobench: An impossible visual benchmark for contemporary large multimodal models

    Jonathan Roberts, Mohammad Reza Taesiri, Ansh Sharma, Akash Gupta, Samuel Roberts, Ioana Croitoru, Simion-Vlad Bogolin, Jialu Tang, Florian Langer, Vyas Raina, et al. Zerobench: An impossible visual benchmark for contemporary large multimodal models. arXiv preprint arXiv:2502.09696,

    Pith/arXiv arXiv

  30. [36]

    Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models

    Weiye Xu, Jiahao Wang, Weiyun Wang, Zhe Chen, Wengang Zhou, Aijun Yang, Lewei Lu, Houqiang Li, Xiaohua Wang, Xizhou Zhu, et al. Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models. arXiv preprint arXiv:2504.15279, 2025b. X. Real world qa benchmark. https://huggingface.co/datasets/xai-org/RealworldQA,

    Pith/arXiv arXiv

  31. [37]

    Simplevqa: Multimodal factuality evaluation for multimodal large language models

    Xianfu Cheng, Wei Zhang, Shiwei Zhang, Jian Yang, Xiangyuan Guan, Xianjie Wu, Xiang Li, Ge Zhang, Jiaheng Liu, Yuying Mai, et al. Simplevqa: Multimodal factuality evaluation for multimodal large language models. arXiv preprint arXiv:2502.13059, 2025b. Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao,...

    Pith/arXiv arXiv

  32. [38]

    Ocrbench: on the hidden mystery of ocr in large multimodal models

    Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: on the hidden mystery of ocr in large multimodal models. Science China Information Sciences, 67(12):220102, 2024c. Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunha...

    Pith/arXiv arXiv

  33. [39]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095–95169, 2024c. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Miche...

    Pith/arXiv arXiv

  34. [40]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008,

    Pith/arXiv arXiv

  35. [41]

    We-math: Does your large multimodal model achieve human-like mathematical reasoning? arXiv preprint arXiv:2407.01284,

    Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, et al. We-math: Does your large multimodal model achieve human-like mathematical reasoning? arXiv preprint arXiv:2407.01284,

    Pith/arXiv arXiv

  36. [42]

    Logicvista: Multimodal llm logical reasoning benchmark in visual contexts

    Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logicvista: Multimodal llm logical reasoning benchmark in visual contexts. arXiv preprint arXiv:2407.04973,

    Pith/arXiv arXiv

  37. [43]

    Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models

    Chengke Zou, Xingang Guo, Rui Yang, Junyu Zhang, Bin Hu, and Huan Zhang. Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models. arXiv preprint arXiv:2411.00836,

    Pith/arXiv arXiv

  38. [44]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479,

    Pith/arXiv arXiv

  39. [45]

    dad") lightly bites the little dog’s ear as a way of correcting the little dog’s improper behavior. This gentle bite is a common

    URL https://arxiv.org/abs/2506.03569. 29 A Case Study Video Case I: Grounding Input Video Prompts: What is the time period during which the handbag appears in this video? Model Responses: The time period during which the bag appears in the video is from 22.3 seconds to 23.8 seconds, i.e., 22.3 s − 23.8s . Figure 9: In this 26-second video, the creator pro...

    Pith/arXiv arXiv

  40. [2014]

    Silent data corruptions at scale

    26 Harish Dattatraya Dixit, Sneha Pendharkar, Matt Beadon, Chris Mason, Tejasvi Chakravarthy, Bharath Muthiah, and Sriram Sankar. Silent data corruptions at scale. arXiv preprint arXiv:2102.11245,

    Pith/arXiv arXiv

  41. [2017]

    URL https://doi.org/10.1007/ s11263-016-0981-7

    doi: 10.1007/s11263-016-0981-7. URL https://doi.org/10.1007/ s11263-016-0981-7 . Dmitry Ustalov, Nikita Pavlichenko, Sergey Koshelev, Daniil Likhobaba, and Alisa Smirnova. Toloka visual question answering benchmark. arXiv preprint arXiv:2309.16511,

    Pith/arXiv arXiv

  42. [2023]

    Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, et al. Seed1. 5-vl technical report. arXiv preprint arXiv:2505.07062, 2025b. Kwai Keye Team, Biao Yang, Bin Wen, Changyi Liu, Chenglong Chu, Chengru Song, Chongling Rao, Chuan Yi, Da Li, Dunju Zang, et al. Kwai keye-vl technical report. ar...

    Pith/arXiv arXiv

  43. [2024]

    Phi-3 technical report: A highly capable language model locally on your phone

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219,

    Pith/arXiv arXiv

  44. [2025]

    Robobrain 2.0 technical report

    BAAI RoboBrain Team. Robobrain 2.0 technical report. arXiv preprint arXiv:TODO, 2025b. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388,

    Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.