REVIEW 4 major objections 5 minor 1 cited by
PixelThink: Towards Efficient Chain-of-Pixel Reasoning
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read PixelThink claims that multiplying the Seg-Zero reward by a soft, difficulty- and uncertainty-aware token penalty teaches multimodal LLMs to reason concisely and segment more accurately.
desk verdict Useful efficiency recipe and benchmark for reasoning segmentation, but the difficulty-adaptive mechanism isn't yet supported by the experiments. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the soft length penalty $s(L_{\text{used}}, L_{\text{budget}})$ (Eq. 4), a multiplicative reward modifier that returns $1$ for chains inside their budget and decays linearly once the chain exceeds it. The budget $L_{\text{budget}}$ is a piecewise function (Eq. 3) of $D$, an external difficulty score in $[1,10]$ averaged from scene complexity, segmentation challenge, and linguistic ambiguity, and of $U$, an internal uncertainty defined as $1$ minus the average top-2 token-probability margin. The final reward $R_{\text{final}} = R_{\text{original}} \cdot s(L_{\text{used}}, L_{\text{budget}})$ is optimized with GRPO, giving the model a smooth incentive to stay concise while leaving medium-difficulty samples unconstrained.
What would settle it
A shuffle test would settle it: retrain PixelThink with the difficulty labels randomly permuted across samples, preserving the same budget distribution. If token counts and gIoU stay essentially the same, the gains come from the soft penalty's overall pressure rather than from difficulty-adaptive allocation; the paper does not report this control.
Extended reading notes
Core claim
The paper's central claim is that a reward signal which gently taxes token overuse can teach a reasoning-segmentation model when to stop thinking. PixelThink multiplies the Seg-Zero reward $R_{\text{original}}$ by a soft penalty $s(L_{\text{used}}, L_{\text{budget}})$: the penalty is $1$ when the chain stays within budget and $1 - \beta(L_{\text{used}} - L_{\text{budget}})$ when it exceeds it, where the budget itself is $L_{\text{base}} + \alpha U$ for hard examples, $L_{\text{low}}$ for easy examples, and unconstrained for medium ones. Because $L_{\text{budget}}$ depends on an externally estimated difficulty $D$ and an internally measured uncertainty $U$, the model learns to allocate reasoning effort to cases that need it. On the paper's ReasonSeg-DIFF test benchmark this yields 47.66 tokens per chain versus Seg-Zero's 90.58, with gIoU rising from 58.20 to 60.17.
Load-bearing premise
The mechanism depends on the outside difficulty rating actually tracking how much reasoning each image needs; if that rating is noisy or biased, the token budgets and the difficulty-stratified benchmark both rest on an unreliable ruler.
Editorial extensions
If this is right
- On ReasonSeg-DIFF test, average reasoning tokens drop from 90.58 to 47.66 while gIoU rises from 58.20 to 60.17, so the same architecture becomes cheaper and more accurate at inference.
- Trained only on RefCOCOg, PixelThink improves over Seg-Zero on ReasonSeg (val gIoU 63.8 vs 61.6; test 60.2 vs 58.2) and remains competitive on RefCOCO, RefCOCO+, and RefCOCOg, indicating the efficiency gain is not bought by losing generalization.
- The 3B variant of PixelThink performs on par with prior 7B counterparts on ReasonSeg, suggesting length control can substitute for some model-capacity scaling in this task.
- No-thinking variants that remove reasoning entirely score lower on segmentation, supporting the paper's conclusion that concise reasoning, not reasoning omission, carries the benefit.
- The new metrics SAT, RST, and URSS make reasoning quality and segmentation accuracy comparable per token, giving future work a common yardstick for efficiency-aware evaluation.
Reading between the lines
- Because the uncertainty term $U$ is computed from token-level margins at no extra annotation cost, one could deploy the same budget rule at inference time without the external difficulty scorer, asking the model to stop when its own confidence is high; the paper does not test this adaptive-compute extension.
- The benchmark's RScore rewards completeness regardless of length, so PixelThink's modest RScore drop (6.92 vs 7.67) may partly reflect the reference being verbose; a length-controlled judge or length-matched references would isolate genuine reasoning-quality loss from brevity.
- The same multiplicative soft-penalty recipe could be transplanted to other GRPO-trained multimodal tasks such as visual question answering or grounding wherever overthinking is diagnosed, though the paper only demonstrates it for segmentation.
- If the difficulty score $D$ were replaced by a cheap proxy such as expression length or scene-object count, the reward's benefit might be reproducible without a large 72B annotator; the paper's difficulty-granularity ablation suggests sensitivity to how $D$ is defined.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. PixelThink proposes an RL fine-tuning scheme for reasoning segmentation that modulates the Seg-Zero GRPO reward with a soft length penalty s(L_used, L_budget), where L_budget depends on an external difficulty score D (from Qwen2.5-VL-72B) and an internal uncertainty estimate U (based on top-2 token probabilities). Easy samples are assigned a small budget, hard samples a larger budget scaled by U, and medium samples are left unconstrained. The paper also introduces ReasonSeg-DIFF, a benchmark with difficulty scores and short/long reference reasoning chains, plus metrics RScore/RST, SAT, and URSS. On ReasonSeg-DIFF, PixelThink reports reduced reasoning token counts (e.g., 47.66 vs 90.58 test tokens) with improved gIoU (60.17 vs 58.20) over Seg-Zero, and presents ablations of the D/U components, difficulty splits, and budget settings.
Significance. If the difficulty-adaptive compression claim is established, the reward design would be a simple and useful recipe for controlling reasoning length in multimodal LLMs, and ReasonSeg-DIFF would be a valuable resource for jointly evaluating reasoning quality, segmentation accuracy, and efficiency. The paper is clearly written, includes extensive ablations and qualitative analyses, and the planned release of code and benchmark is welcome. However, the central mechanism is under-supported: the missing constant-budget control and the circular use of D in training and evaluation leave room for alternative explanations of the reported token reduction, so the main claims require additional evidence.
major comments (4)
- [§5.2, Table 4; Eq. (3)–(5)] The ablation in Table 4 compares no length penalty to penalties modulated by D and/or U, but it never trains a control with a fixed, non-adaptive L_budget. Without this control, the token reduction from 86.92 to 39.95–46.98 could be caused by the soft penalty alone, with the D/U modulation in Eq. (3) making little contribution. This concern is supported by the paper's own observations: Appendix D.2 states final reasoning lengths converge across difficulty levels, and Table 2 shows easy/medium/hard tokens of 44.73/47.00/51.79 despite budgets of 96/unconstrained/approximately 256–281, a gradient no larger than Seg-Zero's own difficulty gradient (84.97/90.73/95.37). Please add a training run with a constant budget (e.g., 96 or 128 tokens for all samples) under the identical pipeline and report per-difficulty token counts and gIoU; without it, the central claim that the model 'learns to compress reasoning length in accordance with scene complexity' is not demonstrated. The non-monotonicity in Table 4 (39.95 tokens with D only, 42.41 with U only, 46.98 with both) also requires explanation.
- [§4.1, §4.2; Eq. (3)] The difficulty score D is computed by Qwen2.5-VL-72B (Section 4.1) and used both to set L_budget in training (Eq. 3) and to stratify the evaluation and select short versus long reference chains for RScore (Section 4.2); the RScore judge is also a Qwen2.5 model. This creates a closed loop between the training signal and the evaluation signal. The only guard is the statement that D is 'cross-validated against human annotations' (Section 4.1), but no quantitative agreement is reported. Please report human-model agreement statistics (e.g., correlation or Cohen's kappa on a held-out set), and evaluate with an independent difficulty estimator or with D held fixed across all methods to show that the difficulty-aware results are not an artifact of aligning to the same estimator.
- [§5.1, Tables 1–3] No seed or error-bar information is reported for any experiment. The headline claims on the ReasonSeg-DIFF test set (60.17 vs 58.20 gIoU; 55.77 vs 52.37 cIoU) are small enough that RL training noise and LLM-judge variance could alter the conclusions, and RScore is an LLM-based scalar. Please report means and standard deviations over at least three runs (or confidence intervals) for the main tables, including RScore, so the reader can assess whether the reported improvements are statistically meaningful.
- [§5.1 vs Appendix D.3] The text states that PixelThink 'preserves reasoning quality' (Section 5.1) and 'maintaining reasoning quality' (Introduction), but Table 1 shows RScore dropping from 7.67 to 6.92 on both validation and test splits. Appendix D.3 acknowledges this drop and attributes it to lower completeness in concise chains. As written, the paper's own metric contradicts the 'maintains reasoning quality' claim. Please either soften the claim or provide statistical evidence that the RScore drop is within noise; otherwise the reader cannot conclude that reasoning quality is maintained.
minor comments (5)
- [Eq. (3) and §5 Implementation Details] The thresholds τ1 and τ2 are first used in Eq. (3) but defined only later in Section 5; please define them at first use and state whether the easy condition is D < τ2 or D ≤ τ2.
- [Table 1 and Appendix A.1] The L1-Exact rows report 65.66 and 65.24 tokens despite a '64-token upper limit' in Appendix A.1; please clarify whether L1-Exact is a hard cap or an exact-length target and explain the discrepancy.
- [§4.2] The SAT and RST formulas use arbitrary constants (100 and 10) and P in billions; please state whether the resulting dimensionless values are comparable across model sizes and provide a sensitivity analysis or justification for the default γ=0.7 in URSS.
- [References and figures] There are minor typos: 'Empirical Eethods' in reference [30] should be 'Empirical Methods', and the token counts in Figure 1 (44 vs 47.7) should be reconciled with Table 1.
- [Appendix D.1] The explanation that the soft penalty 'implicitly encourages' shorter outputs is informal; a sentence describing how the multiplicative penalty interacts with GRPO group-relative normalization would help readers understand the mechanism.
Circularity Check
No material circularity: the difficulty-estimator training/evaluation overlap is a benchmark-validity concern, not an equation-level reduction, and token counts and gIoU/cIoU are externally measured.
full rationale
PixelThink's core mechanism (Eqs. 3-5) is an explicit reward modulation, R_final = R_original * s(L_used, L_budget), with L_budget set by an external difficulty estimate D and the model's own uncertainty U. The reported efficiency and segmentation numbers are measured from actual model outputs (token counts, gIoU, cIoU) and are not equal to any input by construction. In fact, Appendix D.2 concedes that final reasoning lengths converge across difficulty levels, and Table 2 shows the model does not reproduce L_budget (easy: 44.73 tokens vs L_low=96; hard: 51.79 vs ~256+25U), so the difficulty-aware part of the claim is empirically weak but not definitionally forced. The same Qwen2.5-VL-72B estimate D is used in Eq. 3 to set the training budget and in Section 4.2 to select short/long reference chains for RScore; this creates an evaluation-dependence loop for the 'difficulty-aware' RScore comparison, but RScore content and segmentation accuracy are not equal to the reward input, so this is a validation-validity concern rather than a circular derivation. The paper's self-citations (e.g., the authors' own survey [26], CoT-Valve [56], TokenPacker [63]) appear only in related work and are not load-bearing for the method. Section E.1 explicitly acknowledges reliance on coarse-grained difficulty scores and manual budget rules, which limits the strength of the central claim but does not make it circular. Overall, no 'prediction' reduces by construction; the main weaknesses are missing controls (no constant-budget ablation) and unquantified human cross-validation of D, which are empirical omissions, not circularity.
Assumptions & free parameters
free parameters (7)
- Lbase =
256
- Llow =
96
- alpha =
25
- beta =
2e-3
- tau1 and tau2 =
5.0 and 3.5
- gamma =
0.7
- RScore reference selection rule =
short for easy/medium, long for hard
assumptions (5)
- domain assumption GRPO is a stable and valid optimizer for the non-differentiable reward R_final.
- domain assumption The average top-2 token-probability margin in Eq. 1 is a valid proxy for model uncertainty in reward shaping.
- domain assumption Qwen2.5-VL-72B difficulty scores are accurate and human-aligned.
- domain assumption LLM-based RScore with Qwen-generated short/long references is a valid measure of reasoning quality.
- ad hoc to paper The re-implemented L1 and prompt baselines are faithful adaptations of L1 to segmentation.
Cite this review
Pith. "Pith review of PixelThink: Towards Efficient Chain-of-Pixel Reasoning." pith.science (2026). https://pith.science/paper/DXBGCOOX
@misc{pith2026250523727,
author = {Pith},
title = {Pith review of: PixelThink: Towards Efficient Chain-of-Pixel Reasoning},
year = {2026},
howpublished = {\url{https://pith.science/paper/DXBGCOOX}},
note = {Machine review of arXiv:2505.23727}
}
read the original abstract
Existing reasoning segmentation approaches typically fine-tune multimodal large language models (MLLMs) using image-text pairs and corresponding mask labels. However, they exhibit limited generalization to out-of-distribution scenarios without an explicit reasoning process. Although recent efforts leverage reinforcement learning through group-relative policy optimization (GRPO) to enhance reasoning ability, they often suffer from overthinking - producing uniformly verbose reasoning chains irrespective of task complexity. This results in elevated computational costs and limited control over reasoning quality. To address this problem, we propose PixelThink, a simple yet effective scheme that integrates externally estimated task difficulty and internally measured model uncertainty to regulate reasoning generation within a reinforcement learning paradigm. The model learns to compress reasoning length in accordance with scene complexity and predictive confidence. To support comprehensive evaluation, we introduce ReasonSeg-Diff, an extended benchmark with annotated reasoning references and difficulty scores, along with a suite of metrics designed to assess segmentation accuracy, reasoning quality, and efficiency jointly. Experimental results demonstrate that the proposed approach improves both reasoning efficiency and overall segmentation performance. Our work contributes novel perspectives towards efficient and interpretable multimodal understanding. The code and model will be publicly available.
Figures
Figures from the paper (1 more)
Forward citations
Cited by 1 Pith paper
-
DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation
Separate semantic-spatial MLLM cues plus a quality-supervised dynamic gate yield 69.6/67.3 gIoU on ReasonSeg val/test, beating strong MLLM+SAM baselines.
Reference graph
Works this paper leans on
-
[1]
Modeling context in referring expressions
Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expressions. InEuropean Conference on Computer Vision, pages 69–85. Springer, 2016
2016
-
[2]
Lisa: Reasoning segmentation via large language model
Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9579–9589, 2024
work page 2024
-
[3]
POPEN: Preference-Based Optimization and Ensemble for LVLM-Based Reasoning Segmentation
Lanyun Zhu, Tianrun Chen, Qianxiong Xu, Xuanyi Liu, Deyi Ji, Haiyang Wu, De Wen Soh, and Jun Liu. Popen: Preference-based optimization and ensemble for lvlm-based reasoning segmentation.arXiv preprint arXiv:2504.00640, 2025
work page Pith review arXiv 2025
-
[4]
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4):834–848, 2017
work page 2017
-
[5]
Masked- attention mask transformer for universal image segmentation
Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked- attention mask transformer for universal image segmentation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1290–1299, 2022
work page 2022
-
[6]
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. InIEEE International Conference on Computer Vision, pages 2961–2969, 2017
work page 2017
-
[7]
Zhenfei Yin, Jiong Wang, Jianjian Cao, Zhelun Shi, Dingning Liu, Mukai Li, Xiaoshui Huang, Zhiyong Wang, Lu Sheng, Lei Bai, et al. Lamm: Language-assisted multi-modal instruction-tuning dataset, framework, and benchmark.Advances in Neural Information Processing Systems, 36:26650–26685, 2023
work page 2023
-
[8]
Hanxun Yu, Wentong Li, Song Wang, Junbo Chen, and Jianke Zhu. Inst3d-lmm: Instance-aware 3d scene understanding with multi-modal instruction tuning.arXiv preprint arXiv:2503.00513, 2025
arXiv 2025
Show all 81 references
-
[9]
Drivevlm: The convergence of autonomous driving and large vision-language models.arXiv preprint arXiv:2402.12289, 2024
Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models.arXiv preprint arXiv:2402.12289, 2024
2024 arXiv
-
[10]
Are vlms ready for autonomous driving? an empirical study from the reliability, data, and metric perspectives.arXiv preprint arXiv:2501.04003, 2025
Shaoyuan Xie, Lingdong Kong, Yuhao Dong, Chonghao Sima, Wenwei Zhang, Qi Alfred Chen, Ziwei Liu, and Liang Pan. Are vlms ready for autonomous driving? an empirical study from the reliability, data, and metric perspectives.arXiv preprint arXiv:2501.04003, 2025
2025 arXiv
-
[11]
Visual instruction tuning.Advances in Neural Information Processing Systems, 36:34892–34916, 2023
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in Neural Information Processing Systems, 36:34892–34916, 2023
2023
-
[12]
Improved baselines with visual instruction tuning
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024
2024
-
[13]
Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024
2024 arXiv
-
[14]
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025
2025 arXiv
-
[15]
Pixellm: Pixel reasoning with large multimodal model
Zhongwei Ren, Zhicheng Huang, Yunchao Wei, Yao Zhao, Dongmei Fu, Jiashi Feng, and Xiaojie Jin. Pixellm: Pixel reasoning with large multimodal model. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26374–26383, 2024
2024
-
[16]
Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding
Tao Zhang, Xiangtai Li, Hao Fei, Haobo Yuan, Shengqiong Wu, Shunping Ji, Chen Change Loy, and Shuicheng Yan. Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding. InAdvances in Neural Information Processing Systems, volume 37, pages 71737–71767, 2024
2024
-
[17]
One token to seg them all: Language instructed reasoning segmentation in videos.Advances in Neural Information Processing Systems, 37:6833–6859, 2024
Zechen Bai, Tong He, Haiyang Mei, Pichao Wang, Ziteng Gao, Joya Chen, Zheng Zhang, and Mike Zheng Shou. One token to seg them all: Language instructed reasoning segmentation in videos.Advances in Neural Information Processing Systems, 37:6833–6859, 2024
2024
-
[18]
Seg-zero: Reasoning- chain guided segmentation via cognitive reinforcement.arXiv preprint arXiv:2503.06520, 2025
Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. Seg-zero: Reasoning- chain guided segmentation via cognitive reinforcement.arXiv preprint arXiv:2503.06520, 2025
2025 arXiv
-
[19]
Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025
2025 arXiv
-
[20]
Towards reasoning era: A survey of long chain-of-thought for reasoning large language models.arXiv preprint arXiv:2503.09567, 2025
Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models.arXiv preprint arXiv:2503.09567, 2025
2025 arXiv
-
[21]
From system 1 to system 2: A survey of reasoning large language models.arXiv preprint arXiv:2502.17419, 2025
Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. From system 1 to system 2: A survey of reasoning large language models.arXiv preprint arXiv:2502.17419, 2025. 20
2025 arXiv
-
[22]
Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024
2024 arXiv
-
[23]
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025
2025 arXiv
-
[24]
R1-V.https://github.com/Deep-Agent/R1-V?tab=readme-ov-file, 2025
R1-V Team. R1-V.https://github.com/Deep-Agent/R1-V?tab=readme-ov-file, 2025
2025
-
[25]
Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025
2025 arXiv
-
[26]
Efficient reasoning models: A survey.arXiv preprint arXiv:2504.10903, 2025
Sicheng Feng, Gongfan Fang, Xinyin Ma, and Xinchao Wang. Efficient reasoning models: A survey.arXiv preprint arXiv:2504.10903, 2025
2025
-
[27]
Stop overthinking: A survey on efficient reasoning for large language models.arXiv preprint arXiv:2503.16419, 2025
Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Hanjie Chen, Xia Hu, et al. Stop overthinking: A survey on efficient reasoning for large language models.arXiv preprint arXiv:2503.16419, 2025
2025 arXiv
-
[28]
Sam4mllm: Enhance multi-modal large language model for referring expression segmentation
Yi-Chia Chen, Wei-Hua Li, Cheng Sun, Yu-Chiang Frank Wang, and Chu-Song Chen. Sam4mllm: Enhance multi-modal large language model for referring expression segmentation. InEuropean Conference on Computer Vision, pages 323–340. Springer, 2024
2024
-
[29]
L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025
Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697, 2025
2025 arXiv
-
[30]
Referitgame: Referring to objects in photographs of natural scenes
Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. Referitgame: Referring to objects in photographs of natural scenes. InConference on Empirical Eethods in Natural Language Processing, pages 787–798, 2014
2014
-
[31]
Towards robust referring image segmentation.IEEE Transactions on Image Processing, 2024
Jianzong Wu, Xiangtai Li, Xia Li, Henghui Ding, Yunhai Tong, and Dacheng Tao. Towards robust referring image segmentation.IEEE Transactions on Image Processing, 2024
2024
-
[32]
Remamber: Referring image segmentation with mamba twister
Yuhuan Yang, Chaofan Ma, Jiangchao Yao, Zhun Zhong, Ya Zhang, and Yanfeng Wang. Remamber: Referring image segmentation with mamba twister. InEuropean Conference on Computer Vision, pages 108–126. Springer, 2024
2024
-
[33]
Mask grounding for referring image segmentation
Yong Xien Chng, Henry Zheng, Yizeng Han, Xuchong Qiu, and Gao Huang. Mask grounding for referring image segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26573–26583, 2024
2024
-
[34]
A mutual supervision framework for referring expression segmentation and generation.International Journal of Computer Vision, pages 1–16, 2025
Shijia Huang, Feng Li, Hao Zhang, Shilong Liu, Lei Zhang, and Liwei Wang. A mutual supervision framework for referring expression segmentation and generation.International Journal of Computer Vision, pages 1–16, 2025
2025
-
[35]
Pixel-sail: Single transformer for pixel-grounded understanding.arXiv, 2025
Tao Zhang, Xiangtai Li, Zilong Huang, Yanwei Li, Weixian Lei, Xueqing Deng, Shihao Chen, Shunping Ji, , and Jiashi Feng. Pixel-sail: Single transformer for pixel-grounded understanding.arXiv, 2025
2025
-
[36]
U-net: Convolutional networks for biomedical image segmentation
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, pages 234–241. Springer, 2015
2015
-
[37]
Segment anything
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. InIEEE/CVF International Conference on Computer Vision, pages 4015–4026, 2023
2023
-
[38]
Lisa++: An improved baseline for reasoning segmentation with large language model.arXiv preprint arXiv:2312.17240, 2023
Senqiao Yang, Tianyuan Qu, Xin Lai, Zhuotao Tian, Bohao Peng, Shu Liu, and Jiaya Jia. Lisa++: An improved baseline for reasoning segmentation with large language model.arXiv preprint arXiv:2312.17240, 2023
2023 arXiv
-
[39]
Sa2va: Marrying sam2 with llava for dense grounded understanding of images and videos.arXiv, 2025
Haobo Yuan, Xiangtai Li, Tao Zhang, Zilong Huang, Shilin Xu, Shunping Ji, Yunhai Tong, Lu Qi, Jiashi Feng, and Ming-Hsuan Yang. Sa2va: Marrying sam2 with llava for dense grounded understanding of images and videos.arXiv, 2025
2025
-
[40]
Chain-of-thought prompting elicits reasoning in large language models.Advances in Neural Information Processing Systems, 35:24824–24837, 2022
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in Neural Information Processing Systems, 35:24824–24837, 2022
2022
-
[41]
Automatic chain of thought prompting in large language models.arXiv preprint arXiv:2210.03493, 2022
Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models.arXiv preprint arXiv:2210.03493, 2022
2022 arXiv
-
[42]
Towards better chain-of-thought prompting strategies: A survey.arXiv preprint arXiv:2310.04959, 2023
Zihan Yu, Liang He, Zhen Wu, Xinyu Dai, and Jiajun Chen. Towards better chain-of-thought prompting strategies: A survey.arXiv preprint arXiv:2310.04959, 2023
2023 arXiv
-
[43]
Math-shepherd: Verify and reinforce llms step-by-step without human annotations.arXiv preprint arXiv:2312.08935, 2023
Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce llms step-by-step without human annotations.arXiv preprint arXiv:2312.08935, 2023. 21
2023 arXiv
-
[44]
Visualprm: An effective process reward model for multimodal reasoning.arXiv preprint arXiv:2503.10291, 2025
Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, et al. Visualprm: An effective process reward model for multimodal reasoning.arXiv preprint arXiv:2503.10291, 2025
2025 arXiv
-
[45]
Prmbench: A fine-grained and challenging benchmark for process-level reward models.arXiv preprint arXiv:2501.03124, 2025
Mingyang Song, Zhaochen Su, Xiaoye Qu, Jiawei Zhou, and Yu Cheng. Prmbench: A fine-grained and challenging benchmark for process-level reward models.arXiv preprint arXiv:2501.03124, 2025
2025 arXiv
-
[46]
OpenAI o1.https://openai.com/o1/, 2024
OpenAI. OpenAI o1.https://openai.com/o1/, 2024
2024
-
[47]
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025
2025 arXiv
-
[48]
s1: Simple test-time scaling
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025
2025 arXiv
-
[49]
Can 1b llm surpass 405b llm? rethinking compute-optimal test-time scaling.arXiv preprint arXiv:2502.06703, 2025
Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, and Bowen Zhou. Can 1b llm surpass 405b llm? rethinking compute-optimal test-time scaling.arXiv preprint arXiv:2502.06703, 2025
2025 arXiv
-
[50]
Ttrl: Test-time reinforcement learning.arXiv preprint arXiv:2504.16084, 2025
Yuxin Zuo, Kaiyan Zhang, Shang Qu, Li Sheng, Xuekai Zhu, Biqing Qi, Youbang Sun, Ganqu Cui, Ning Ding, and Bowen Zhou. Ttrl: Test-time reinforcement learning.arXiv preprint arXiv:2504.16084, 2025
2025 arXiv
-
[51]
Open R1 Multimodal
EvolvingLMMs Lab. Open R1 Multimodal. https://github.com/EvolvingLMMs-Lab/ open-r1-multimodal, 2025
2025
-
[52]
Reason-rft: Reinforcement fine-tuning for visual reasoning.arXiv preprint arXiv:2503.20752, 2025
Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Reason-rft: Reinforcement fine-tuning for visual reasoning.arXiv preprint arXiv:2503.20752, 2025
2025
-
[53]
Efficient inference for large reasoning models: A survey.arXiv preprint arXiv:2503.23077, 2025
Yue Liu, Jiaying Wu, Yufei He, Hongcheng Gao, Hongyu Chen, Baolong Bi, Jiaheng Zhang, Zhiqi Huang, and Bryan Hooi. Efficient inference for large reasoning models: A survey.arXiv preprint arXiv:2503.23077, 2025
2025 arXiv
-
[54]
A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond.arXiv preprint arXiv:2503.21614, 2025
Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, et al. A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond.arXiv preprint arXiv:2503.21614, 2025
2025
-
[55]
Token-budget- aware llm reasoning.arXiv preprint arXiv:2412.18547, 2024
Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token-budget- aware llm reasoning.arXiv preprint arXiv:2412.18547, 2024
2024 arXiv
-
[56]
Cot-valve: Length- compressible chain-of-thought tuning.arXiv preprint arXiv:2502.09601, 2025
Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. Cot-valve: Length- compressible chain-of-thought tuning.arXiv preprint arXiv:2502.09601, 2025
2025 arXiv
-
[57]
O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning.arXiv preprint arXiv:2501.12570, 2025
Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning.arXiv preprint arXiv:2501.12570, 2025
2025 arXiv
-
[58]
Chain of draft: Thinking faster by writing less
Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600, 2025
2025 arXiv
-
[59]
Tokenskip: Controllable chain-of-thought compression in llms.arXiv preprint arXiv:2502.12067, 2025
Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable chain-of-thought compression in llms.arXiv preprint arXiv:2502.12067, 2025
2025
-
[60]
Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning.arXiv preprint arXiv:2504.01296, 2025
Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang. Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning.arXiv preprint arXiv:2504.01296, 2025
2025 arXiv
-
[61]
Self-training elicits concise reasoning in large language models.arXiv preprint arXiv:2502.20122, 2025
Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang, Yujin Kim, and Se-Young Yun. Self-training elicits concise reasoning in large language models.arXiv preprint arXiv:2502.20122, 2025
2025 arXiv
-
[62]
Llava-prumerge: Adaptive token reduction for efficient large multimodal models.arXiv preprint arXiv:2403.15388, 2024
Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. Llava-prumerge: Adaptive token reduction for efficient large multimodal models.arXiv preprint arXiv:2403.15388, 2024
2024
-
[63]
Tokenpacker: Efficient visual projector for multimodal llm.arXiv preprint arXiv:2407.02392, 2024
Wentong Li, Yuqian Yuan, Jian Liu, Dongqi Tang, Song Wang, Jie Qin, Jianke Zhu, and Lei Zhang. Tokenpacker: Efficient visual projector for multimodal llm.arXiv preprint arXiv:2407.02392, 2024
2024 arXiv
-
[64]
An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models
Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. InEuropean Conference on Computer Vision, pages 19–35. Springer, 2024
2024
-
[65]
Visionzip: Longer is better but not necessary in vision language models.arXiv preprint arXiv:2412.04467, 2024
Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. Visionzip: Longer is better but not necessary in vision language models.arXiv preprint arXiv:2412.04467, 2024
2024
-
[66]
Learning to inference adaptively for multimodal large language models.arXiv preprint arXiv:2503.10905, 2025
Zhuoyan Xu, Khoi Duc Nguyen, Preeti Mukherjee, Saurabh Bagchi, Somali Chaterji, Yingyu Liang, and Yin Li. Learning to inference adaptively for multimodal large language models.arXiv preprint arXiv:2503.10905, 2025. 22
2025 arXiv
-
[67]
Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024
2024 arXiv
-
[68]
Minimum-margin active learning.arXiv preprint arXiv:1906.00025, 2019
Heinrich Jiang and Maya Gupta. Minimum-margin active learning.arXiv preprint arXiv:1906.00025, 2019
1906 arXiv
-
[69]
Chain-of-thought reasoning without prompting.arXiv preprint arXiv:2402.10200, 2024
Xuezhi Wang and Denny Zhou. Chain-of-thought reasoning without prompting.arXiv preprint arXiv:2402.10200, 2024
2024 arXiv
-
[70]
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024
2024 arXiv
-
[71]
Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters
Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. InACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 3505–3506, 2020
2020
-
[72]
Open-vocabulary semantic segmentation with mask-adapted clip
Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Diana Marculescu. Open-vocabulary semantic segmentation with mask-adapted clip. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7061–7070, 2023
2023
-
[73]
Gres: Generalized referring expression segmentation
Chang Liu, Henghui Ding, and Xudong Jiang. Gres: Generalized referring expression segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23592–23601, 2023
2023
-
[74]
Grounded sam: Assembling open-world models for diverse visual tasks.arXiv preprint arXiv:2401.14159, 2024
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks.arXiv preprint arXiv:2401.14159, 2024
2024 arXiv
-
[75]
Lavt: Language- aware vision transformer for referring image segmentation
Zhao Yang, Jiaqi Wang, Yansong Tang, Kai Chen, Hengshuang Zhao, and Philip HS Torr. Lavt: Language- aware vision transformer for referring image segmentation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18155–18165, 2022
2022
-
[76]
Perceptiongpt: Effectively fusing visual perception into llm
Renjie Pi, Lewei Yao, Jiahui Gao, Jipeng Zhang, and Tong Zhang. Perceptiongpt: Effectively fusing visual perception into llm. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27124–27133, 2024
2024
-
[77]
Think or not think: A study of explicit thinking in rule-based visual reinforcement fine-tuning.arXiv preprint arXiv:2503.16188, 2025
Ming Li, Shitian Zhao, Jike Zhong, Yuxiang Lai, and Kaipeng Zhang. Think or not think: A study of explicit thinking in rule-based visual reinforcement fine-tuning.arXiv preprint arXiv:2503.16188, 2025
2025
-
[78]
Reasoning models can be effective without thinking.arXiv preprint arXiv:2504.09858, 2025
Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking.arXiv preprint arXiv:2504.09858, 2025
2025 arXiv
-
[79]
Deepscaler: Surpassing o1-preview with a 1.5 b model by scaling rl.Notion Blog, 2025
Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, et al. Deepscaler: Surpassing o1-preview with a 1.5 b model by scaling rl.Notion Blog, 2025
2025
-
[80]
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InEuropean Conference on Computer Vision, pages 740–755, 2014
2014
-
[81]
Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024. 23
2024 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.