Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

EgoVLM: Policy Optimization for Egocentric Video Understanding

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a 3B vision-language model, trained with GRPO on non-chain-of-thought egocentric QA data, outperforms its base model and a 7B variant by roughly 14 accuracy points on EgoSchema, and that this shows domain-specific…

desk verdict A clean, honest R1-Zero-style GRPO recipe for egocentric video QA, but the headline EgoSchema gain is unverified because the training corpus may overlap the test set. read the letter →

arxiv 2506.03097 v1 pith:OGQWIBWM submitted 2025-06-03 cs.CV cs.AI

Ashwin Vinod , Shrey Pandit , Aditya Vavre , Linshen Liu This is my paper
classification cs.CVcs.AI
keywords egocentricvideounderstandingvision-languagemodelGRPOreinforcementlearningquestionansweringEgoSchemakeyframerewardchain-of-thought-freereasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EgoVLM tries to show that a small vision-language model (3 billion parameters) can become much better at answering questions about first-person video by reinforcement learning alone, without any step-by-step reasoning examples. The authors fine-tune Qwen2.5-VL-3B with GRPO, a reinforcement-learning objective that compares a group of sampled answers to set a training signal, using only a format reward and an answer-correctness reward. On the EgoSchema benchmark their model scores 73.7%, up from 59.4% for the base 3B model and 59.8% for the 7B variant, and it also edges out the base models on the EgoPlan benchmark. This matters because embodied applications like wearables need models that reason about intent and actions in first-person video, and the paper argues such reasoning does not require expensive chain-of-thought supervision. The paper's own supervised fine-tuning baseline reaches 74.0% on EgoSchema, so the cleanest reading is that domain-specific training helps a great deal, with GRPO's advantage showing up on goal-directed EgoPlan reasoning.

What carries the argument

The load-bearing mechanism is the GRPO objective combined with a rule-based reward that has no learned reward model. For a question q, the model samples G completions; each gets a reward, the rewards are centered and scaled within the group, and the policy is updated with a clipped importance-ratio objective plus a KL penalty to the reference policy. The reward is the sum of a format reward (forcing <think>...</think> and <answer>...</answer> structure) and an accuracy reward (exact string match for multiple-choice and yes/no, ROUGE-L for open-ended). Training is done with LoRA rank 1024 on a frozen vision encoder, 16 frames per video at reduced resolution, and a detailed prompt that asks for self-reflection. The paper also introduces a keyframe reward—an F1 score between predicted salient frame indices and proxy ground truth from CLIP or model-likelihood scoring—but reports it does not improve accuracy and can lead the model to output fixed frame indices.

What would settle it

Run an overlap analysis between the roughly 7,000 training video clips and the EgoSchema test set: exact or near-duplicate frames appearing in both would mean the reported +14.33 point gain over the base model partly reflects memorized test content. A cleaner test is to train the same recipe on EgoIT99K and evaluate on an unseen egocentric benchmark; if the accuracy jump largely disappears, the paper's conclusion that RL imparts general egocentric reasoning is not supported.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that EgoVLM-3B—Qwen2.5-VL-3B-Instruct trained with GRPO on about 27,000 non-CoT egocentric QA pairs from EgoIT99K—achieves state-of-the-art accuracy on EgoSchema, beating the base 3B model by +14.33 points and the 7B model by +13.87 points, and surpassing all open-source baselines on the EgoPlan validation set. The training uses a rule-based reward that sums a format reward (thinking in <think> tags, answer in <answer> tags) and an accuracy reward (exact match for multiple choice, ROUGE-L for open-ended). GRPO, unlike PPO, needs no separate value network: for each question it samples a group of completions, normalizes their rewards, and optimizes the policy with a clipped KL-regularized objective. The paper also reports that Dr. GRPO, a variant that removes length and variance normalization, performs worse at group size 6, and that a supervised fine-tuning baseline scores slightly higher on EgoSchema (74.0) but lower on EgoPlan (32.1 versus 33.8). The authors interpret the EgoPlan result as evidence that GRPO adds goal-directed reasoning that SFT lacks.

Load-bearing premise

The load-bearing premise is that the roughly 27,000 QA pairs from EgoIT99K are a clean training source—representative of egocentric video QA and disjoint from the EgoSchema test videos—because the paper reports no overlap or de-duplication analysis between them.

Editorial extensions

If this is right

  • A 3B egocentric VLM trained with RL alone can exceed general-purpose video VLMs twice its size on egocentric QA benchmarks, suggesting model scale is not the main bottleneck for this task.
  • Explicit reasoning traces can be elicited without any chain-of-thought training data, which lowers the cost of building interpretable egocentric QA systems.
  • GRPO's advantage over SFT appears specifically on goal-directed next-action reasoning (EgoPlan), while SFT matches or slightly beats it on EgoSchema; the right training objective may depend on the benchmark.
  • Detailed prompts and high LoRA rank are important for GRPO to converge on video QA, so RL recipe details can matter as much as the algorithm.
  • The proposed keyframe-based temporal grounding reward does not pay off with proxy keyframe labels; reliable frame supervision is an open problem.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper does not report overlap analysis between EgoIT99K training clips and EgoSchema test videos, the +14.33 point gain should be read with caution until train/test separation is verified.
  • The SFT baseline's 74.0 EgoSchema score suggests the strongest supported claim is that domain-specific training helps, not that GRPO beats SFT; a direct GRPO-versus-SFT comparison on a fresh egocentric benchmark would settle which objective is better.
  • The keyframe reward's failure mode—identical frame predictions across videos—implies proxy scoring is too weak a signal; human keyframe annotations or attention-based grounding would be a natural next test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces EgoVLM, a 3B-parameter vision-language model fine-tuned from Qwen2.5-VL-3B-Instruct using Group Relative Policy Optimization (GRPO) on roughly 27,000 non-chain-of-thought egocentric QA pairs drawn from the EgoIT99K collection. The authors report that EgoVLM-3B improves over the base model by +14.33 accuracy points on EgoSchema (73.7 vs. 59.4), surpasses listed open-source and proprietary MLLMs on that benchmark, and produces explicit reasoning traces. They also report a supervised fine-tuning (SFT) baseline (74.0 on EgoSchema), a Dr. GRPO variant, ablations on LoRA rank and prompt complexity, and a proposed keyframe-based reward that they themselves find does not yet induce video-dependent keyframe selection. The central claim is that domain-specific RL without CoT supervision substantially improves egocentric video QA and yields interpretable reasoning.

Significance. If the EgoSchema gain survives a train/test overlap check, the result is significant: a 3B model trained without CoT supervision and without unfreezing the vision encoder beats substantially larger general-purpose VLMs on a standard egocentric benchmark, and the reported reasoning traces add practical interpretability. The paper deserves credit for honest self-assessment: it reports failure cases (Figures 6-7), states that the keyframe reward produces no immediate gains and that the model predicts near-identical keyframe indices regardless of input, and shows that an SFT baseline actually exceeds GRPO on EgoSchema. The ablations of prompt format and LoRA capacity are useful. The key unverified premise is data-level: the training corpus and the EgoSchema test set may share Ego4D-derived content, so the headline improvement could reflect memorization rather than acquired reasoning. The evidence pattern (large gain on EgoSchema, small gain on EgoPlan, negative gain on MVBench) makes this risk concrete and requires an explicit overlap analysis before the claim can be considered verified.

major comments (4)
  1. [Section 4, Training Dataset and Evaluation Benchmarks] The manuscript provides no train/test overlap analysis. The training corpus (~27,000 QA pairs from ~7,000 clips of EgoIT99K, a curation of nine egocentric datasets per [30]) is used to train on EgoSchema-style MCQA, and the headline result is measured on the EgoSchema test set, which is constructed from Ego4D. If EgoIT99K contains Ego4D-derived clips (which the citation to [30] makes plausible but the paper does not verify), the +14.33-point gain over Qwen2.5-VL-3B could reflect near-duplicate test content rather than improved reasoning. The small EgoPlan gain (+0.8 over the 7B base) and negative MVBench gain (-0.4) are consistent with this concern. I request (i) a video-level de-duplication or near-duplicate analysis between the training clips and the EgoSchema test set, (ii) overlap statistics for the question/answer content, and (iii) at least one evaluation on an egocentric benchmark that is provably disjoint from the training corpus.
  2. [Table 2 vs. Tables 4, 5, 6] The EgoVLM (GRPO) row is reported inconsistently across tables. Table 2 gives EgoSchema 73.7, EgoPlan 33.8, MVBench 66.5, Average 58.0, while Tables 4-6 give the same model as EgoSchema 73.7, EgoPlan 33.3, MVBench 65.9, Average 57.7. The average in Tables 4-6 (57.7) is also inconsistent with its own components under the stated formula (73.7+33.3+65.9)/3 = 57.6. Since Table 2 is the paper's central results table and Section 5 uses the EgoPlan row to claim EgoVLM surpasses all open-source baselines, the discrepancy between 33.8 and 33.3 (and 66.5 and 65.9) must be resolved, and the same checkpoint must be used consistently in all tables.
  3. [Section 5, Table 2 and Abstract] The paper's own SFT baseline (74.0) exceeds the GRPO model (73.7) on EgoSchema. Both start from the same base model and the same data source, so the benchmark evidence supports the claim 'domain-specific training substantially improves performance' but does not support the attribution of the EgoSchema state-of-the-art result specifically to GRPO. The currently worded claims that 'GRPO enables our model to achieve superior performance' (Section 1) and that EgoVLM 'achieves state-of-the-art accuracy on the EgoSchema benchmark' (Section 5) should be rephrased to state precisely which result is attributable to GRPO as opposed to domain-specific training, with SFT as the comparison point. The EgoPlan difference between GRPO and SFT (33.8 vs. 32.1) is the main evidence for the GRPO-specific mechanism and needs to be presented as such.
  4. [Section 5, Table 2] All results are from single runs with no error bars or multiple-seed reporting. This matters because the EgoPlan differences that carry the GRPO-vs-SFT and GRPO-vs-base arguments are small (+0.8 over the 7B base, +1.7 over SFT), and the EgoPlan evaluation is on the validation set rather than the official test set. Please report the number of runs, variance across seeds, or a statistical comparison for at least the main rows of Table 2; without this, the EgoPlan-based conclusions are within plausible run-to-run noise.
minor comments (6)
  1. [Section 4] The phrase 'input resolution to 64 × 28 × 28 per frame' is unclear: the paper states that 16 frames are sampled per video, so the reported dimensions (whether tokens, patches, or spatial resolution) need a precise definition.
  2. [Section 3, Dr. GRPO paragraph] The text states that the removed normalization terms are 'highlighted in red in equations 1 and 2,' but the printed equations contain no highlights; please state explicitly in prose that the removed terms are the per-response length division and the within-group standard deviation scaling.
  3. [Section 6] The paper honestly reports that the keyframe-trained model 'frequently predicts the same set of keyframe indices regardless of the input video.' Given this admission, the abstract and contribution list should state more explicitly that the keyframe reward is a preliminary, non-validated mechanism rather than a demonstrated contribution.
  4. [Figure 1 caption] The caption claims EgoVLM outperforms MM-Ego on EgoSchema and EgoLife on EgoPlan, and claims in the text that this is shown in Table 2, but neither model appears in Table 2; please add the comparison numbers or a clear citation source.
  5. [Tables 4, 5, 6] The repeated note 'these results denote performance after only 1000 training steps' should clarify whether the EgoVLM reference row is the final 3200-step model or the 1000-step checkpoint; as printed, EgoSchema 73.7 appears in both the 1000-step and 3200-step settings, which needs explanation.
  6. [Throughout] There are several typos and formatting issues: 'Perfromance' in Section 5, 'capcbility' in the Figure 5 caption, 'atleast' in Section 6, and references [23] and [24] appear to cite the same DeepSeekMath paper twice.

Circularity Check

0 steps flagged · score 1.0 of 10

No equation-level circularity: GRPO reward is rule-based and evaluation uses external benchmarks; the only residual concern is possible EgoIT99K/EgoSchema overlap, a data-hygiene issue rather than a circular derivation.

full rationale

The paper's central derivation chain is: define a rule-based reward (format + exact-match/ROUGE accuracy) in Section 3, optimize the Qwen2.5-VL policy with GRPO in Section 4, and measure accuracy on the external EgoSchema, EgoPlan, and MVBench benchmarks in Section 5. None of these steps defines the evaluation quantity in terms of the training objective: the training reward is computed on the ~27,000 training QA pairs and the reported numbers are computed on held-out benchmark test/validation sets. The paper does not fit any parameter to the benchmark scores and then rename that fit a prediction; the +14.33 EgoSchema gain is an empirical result, not a consequence of the equations. There is no load-bearing self-citation chain: EgoIT99K is cited to EgoLife [30], whose authors do not overlap with the present authors, and no 'uniqueness theorem' is imported to force the choice of GRPO. The keyframe reward in Section 6 uses CLIP/Qwen2.5-VL-derived proxy labels as supervision and computes F1 against those same proxies, but the paper explicitly labels these as proxies and does not claim to predict external keyframe annotations, so this is a training objective rather than a circular prediction. The legitimate concern raised by the paper's setup is data-level: EgoIT99K is 'a 9 classic egocentric video datasets curated by [30]' and EgoSchema is built from Ego4D, and the paper reports no de-duplication or overlap analysis between training and test corpora. That is a soundness/contamination risk that could invalidate the headline number, but it is not a derivation-level circularity under the definitions used here. Accordingly, the circularity score is set to 1: effectively no circular derivation, with a minor data-hygiene caveat attached to the central empirical claim.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim depends on standard GRPO hyperparameters and on a data assumption that EgoIT99K is disjoint from and representative of the EgoSchema test set. No new physical entities are introduced. The keyframe reward uses proxy supervision but is honestly reported as not improving accuracy.

free parameters (6)
  • GRPO group size G = 6
    Number of sampled completions per question; the paper attributes Dr. GRPO's worse performance to this small group size.
  • KL coefficient beta = 0.04
    Penalty strength toward the reference policy in the GRPO objective.
  • LoRA rank = 1024
    Adapter capacity; ablation shows rank 256 converges slower and performs worse.
  • Sampling temperature = 0.9
    Controls exploration during GRPO response sampling.
  • Training steps = 3200
    Stopping point for the main run; ablated runs stop at 1000 steps.
  • Frame sampling schedule = 16 frames at 64x28x28 tokens (train), 32 frames at 128x28x28 tokens (inference)
    Input representation chosen under compute constraints; sensitivity not studied.
assumptions (5)
  • domain assumption GRPO with rule-based format and accuracy rewards is a suitable training objective for video QA.
    The paper assumes binary string-match accuracy on training QA pairs transfers to improved reasoning on held-out benchmarks without a learned reward model (Section 3).
  • domain assumption EgoIT99K is representative of the egocentric QA test distribution and disjoint from EgoSchema test videos.
    Section 4 builds the training corpus from EgoIT99K and evaluates on EgoSchema test, but never reports overlap or de-duplication.
  • domain assumption Uniform sampling of 16 frames (training) and 32 frames (inference) at reduced token resolution preserves answer-relevant video content.
    Section 4 sets the input representation; no ablation tests frame count or resolution sensitivity.
  • domain assumption Keeping the vision encoder frozen is sufficient for egocentric video QA.
    Section 4 fixes the vision backbone; Section 5 limitations acknowledge unfreezing might improve fine-grained detection.
  • domain assumption CLIP cosine similarity and Qwen2.5-VL-7B answer probabilities are valid proxies for keyframe importance.
    Section 6 uses these as pseudo ground truth for the keyframe reward and later reports proxy noise and poor temporal awareness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EgoVLM: Policy Optimization for Egocentric Video Understanding." pith.science (2026). https://pith.science/paper/OGQWIBWM

@misc{pith2026250603097,
  author       = {Pith},
  title        = {Pith review of: EgoVLM: Policy Optimization for Egocentric Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OGQWIBWM}},
  note         = {Machine review of arXiv:2506.03097}
}
read the original abstract

Emerging embodied AI applications, such as wearable cameras and autonomous agents, have underscored the need for robust reasoning from first person video streams. We introduce EgoVLM, a vision-language model specifically designed to integrate visual comprehension and spatial-temporal reasoning within egocentric video contexts. EgoVLM is fine-tuned via Group Relative Policy Optimization (GRPO), a reinforcement learning method adapted to align model outputs with human-like reasoning steps. Following DeepSeek R1-Zero's approach, we directly tune using RL without any supervised fine-tuning phase on chain-of-thought (CoT) data. We evaluate EgoVLM on egocentric video question answering benchmarks and show that domain-specific training substantially improves performance over general-purpose VLMs. Our EgoVLM-3B, trained exclusively on non-CoT egocentric data, outperforms the base Qwen2.5-VL 3B and 7B models by 14.33 and 13.87 accuracy points on the EgoSchema benchmark, respectively. By explicitly generating reasoning traces, EgoVLM enhances interpretability, making it well-suited for downstream applications. Furthermore, we introduce a novel keyframe-based reward that incorporates salient frame selection to guide reinforcement learning optimization. This reward formulation opens a promising avenue for future exploration in temporally grounded egocentric reasoning.

Figures

Figures reproduced from arXiv: 2506.03097 by the authors.

Figure 1
Figure 1. We significantly enhance the reasoning ability of the base Qwen2.5-VL (3B) model, surpassing even its 7B variant on two popular [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An example illustrating the reasoning capability of our EgoVLM model. The model successfully interprets the person’s actions [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Training visualization of EgoVLM with the GRPO training objective. Completion length refers to the length of generated output. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: An example illustrating the reasoning capability of our EgoVLM model. EgoVLM is able to reason about a person’s intent, [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: An example illustrating the reasoning capcbility of our EgoVLM model. EgoVLM reasons and disambiguates between two [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: A failure case of our EgoVLM model. EgoVLM fails to identify a mop in the video. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: A failure case of our EgoVLM model. EgoVLM produces answers that are inconsistent with its reasoning traces. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Accuracy reward with different LoRA ranks in GRPO [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Pro$^2$Assist: Continuous Step-aware Proactive Assistance with Multi-modal Egocentric Perception for Long-horizon Procedural Tasks

    cs.AI 2026-05 unverdicted novelty 7.0 of 10

    Pro²Assist uses multimodal egocentric perception from AR glasses to track fine-grained progress in long-horizon procedural tasks and deliver timely proactive assistance, outperforming baselines by over 21% in action u...

  2. EgoVITA: Learning to Plan and Verify for Egocentric Video Reasoning

    cs.CV 2025-11 conditional novelty 7.0 of 10

    EgoVITA, a GRPO-based plan-then-verify framework with dense visual-grounding rewards, improves egocentric video reasoning by up to +7.7 points and keeps exocentric video performance intact.

  3. EgoIntent: A Pre-Outcome Micro-Step Benchmark for Understanding What, Why, and Next

    cs.CV 2026-03 conditional novelty 6.0 of 10

    A step-level egocentric-video benchmark for What/Why/Next intent shows current multimodal models score only about 33/100, though some supporting experiments are missing from the paper.

Reference graph

Works this paper leans on

35 extracted references · 15 canonical work pages · cited by 3 Pith papers

  1. [30]

    Egolife: Towards egocentric life assistant

    Jingkang Yang, Shuai Liu, Hongming Guo, Yuhao Dong, Xi- amengwei Zhang, Sicheng Zhang, Pengyun Wang, Zitang Zhou, Binzhu Xie, Ziyue Wang, Bei Ouyang, Zhengyu Lin, Marco Cominelli, Zhongang Cai, Yuanhan Zhang, Peiyuan Zhang, Fangzhou Hong, Joerg Widmer, Francesco Gringoli, Lei Yang, Bo Li, and Ziwei Liu. Egolife: Towards egocentric life assistant. In The I...

    work page 2025

  2. [1]

    Gpt-4v(ision) system card. 2023. 6

    work page 2023

  3. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  4. [3]

    Back to basics: Revisiting reinforce style op- timization for learning from human feedback in llms

    Arash Ahmadian, Chris Cremer, Matthias Gall ´e, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet ¨Ust¨un, and Sara Hooker. Back to basics: Revisiting reinforce style op- timization for learning from human feedback in llms. arXiv preprint arXiv:2402.14740, 2024. 3

    arXiv 2024

  5. [4]

    Qwen2.5-vl technical report, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025. 6

    2025

  6. [5]

    Egoplan- bench: Benchmarking multimodal large language models for human-level planning

    Yi Chen, Yuying Ge, Yixiao Ge, Mingyu Ding, Bohao Li, Rui Wang, Ruifeng Xu, Ying Shan, and Xihui Liu. Egoplan- bench: Benchmarking multimodal large language models for human-level planning. arXiv preprint arXiv:2312.06722,

    arXiv

  7. [6]

    Vide- gothink: Assessing egocentric video understanding capabili- ties for embodied ai.arXiv preprint arXiv:2410.11623, 2024

    Sijie Cheng, Kechen Fang, Yangyang Yu, Sicheng Zhou, Bo- hao Li, Ye Tian, Tingguang Li, Lei Han, and Yang Liu. Vide- gothink: Assessing egocentric video understanding capabili- ties for embodied ai.arXiv preprint arXiv:2410.11623, 2024. 1

    arXiv

  8. [7]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shi- rong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai...

    work page 2025

Show all 35 references
  1. [8]

    Chatglm: A family of large language models from glm-130b to glm-4 all tools

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793, 2024. 3

    2024 arXiv

  2. [9]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 6

    2024 arXiv

  3. [10]

    Mvbench: A comprehensive multi- modal video understanding benchmark, 2024

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, and Yu Qiao. Mvbench: A comprehensive multi- modal video understanding benchmark, 2024. 4

    2024

  4. [11]

    2d-curri-dpo: Two- dimensional curriculum learning for direct preference opti- mization

    Mengyang Li and Zhong Zhang. 2d-curri-dpo: Two- dimensional curriculum learning for direct preference opti- mization. arXiv preprint arXiv:2504.07856, 2025. 3

    2025 arXiv

  5. [12]

    ROUGE: A package for automatic evaluation of summaries

    Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain, 2004. Association for Computa- tional Linguistics. 4

    2004

  6. [13]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceeding...

    2014

  7. [14]

    Understanding r1-zero-like training: A critical perspective

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025. 4, 5

    2025 arXiv

  8. [15]

    Visual- rft: Visual reinforcement fine-tuning

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual- rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.01785, 2025. 3

    2025 arXiv

  9. [16]

    Reasoning models can be effective without thinking, 2025

    Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking, 2025. 9

    2025

  10. [17]

    Egoschema: A diagnostic benchmark for very long- form video language understanding

    Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic benchmark for very long- form video language understanding. Advances in Neural In- formation Processing Systems, 36:46212–46244, 2023. 4

    2023

  11. [18]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Ad- vances in neural information processing systems, 35:...

    2022

  12. [19]

    Medvlm-r1: Incentivizing medical reasoning ca- pability of vision-language models (vlms) via reinforcement learning, 2025

    Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. Medvlm-r1: Incentivizing medical reasoning ca- pability of vision-language models (vlms) via reinforcement learning, 2025. 1, 3

    2025

  13. [20]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of th...

    2021

  14. [21]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023. 3

    2023

  15. [22]

    Proximal policy optimization algo- rithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. Proximal policy optimization algo- rithms. arXiv preprint arXiv:1707.06347, 2017. 3

    2017 arXiv

  16. [23]

    Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. 3

    2024 arXiv

  17. [24]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junx- iao Song, Mingchuan Zhang, Y .K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reason- ing in open language models, 2024. 1

    2024

  18. [25]

    Vlm- r1: A stable and generalizable r1-style large vision-language model

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. Vlm- r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615, 2025. 4

    2025 arXiv

  19. [26]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. 6

    2023 arXiv

  20. [27]

    Internvideo2.5: Empowering video mllms with long and rich context modeling, 2025

    Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xi- angyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, Min Dou, Kai Chen, Wenhai Wang, Yu Qiao, Yali Wang, and Limin Wang. Internvideo2.5: Empowering video mllms with long and rich context modeling, 2025. 6

    2025

  21. [28]

    Wizardlm 2, 2024

    WizardLM Team. Wizardlm 2, 2024. 3

    2024

  22. [29]

    St-think: How multimodal large language mod- els reason about 4d worlds from ego-centric videos

    Peiran Wu, Yunze Liu, Chonghan Liu, Miao Liu, and Junx- iao Shen. St-think: How multimodal large language mod- els reason about 4d worlds from ego-centric videos. arXiv preprint arXiv:2503.12542, 2025. 3

    2025 arXiv

  23. [31]

    Mm-ego: Towards build- ing egocentric multimodal llms

    Hanrong Ye, Haotian Zhang, Erik Daxberger, Lin Chen, Zongyu Lin, Bowen Zhang, Haoxuan You, Dan Xu†, Zhe Gan, Jiasen Lu, and Yinfei Yang. Mm-ego: Towards build- ing egocentric multimodal llms. In ICLR, 2025. 2, 3

    2025

  24. [32]

    Video instruction tuning with synthetic data, 2024

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Zi- wei Liu, and Chunyuan Li. Video instruction tuning with synthetic data, 2024. 6

    2024

  25. [33]

    Llamafac- tory: Unified efficient fine-tuning of 100+ language mod- els

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafac- tory: Unified efficient fine-tuning of 100+ language mod- els. In Proceedings of the 62nd Annual Meeting of the As- sociation for Computational Linguistics (Volume 3: S...

    2024

  26. [34]

    R1-zero’s ”aha moment” in visual reasoning on a 2b non-sft model, 2025

    Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1-zero’s ”aha moment” in visual reasoning on a 2b non-sft model, 2025. 5

    2025

  27. [35]

    Egotextvqa: Towards egocentric scene-text aware video question answering, 2025

    Sheng Zhou, Junbin Xiao, Qingyun Li, Yicong Li, Xun Yang, Dan Guo, Meng Wang, Tat-Seng Chua, and Angela Yao. Egotextvqa: Towards egocentric scene-text aware video question answering, 2025. 1, 9

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.