Pith. sign in

REVIEW 4 major objections 5 minor 54 references

Reasoning-free reinforcement learning aligns document QA models directly to grounded outputs and outperforms reasoning-enabled training at the 4-billion-parameter scale.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 01:21 UTC pith:ZX7ZCWZX

load-bearing objection A useful applied result with an overstated reasoning conclusion; worth a look if you work on document grounding or RL for perception tasks. the 4 major comments →

arxiv 2607.14682 v1 pith:ZX7ZCWZX submitted 2026-07-16 cs.AI cs.CLcs.LG

Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment

Harikrishnan P M , Goutham Vignesh , Ganesh Parab , Saisubramaniam Gopalakrishnan , Vishal Vaddina , Varun V , Rohit Agrawal This is my paper
classification cs.AI cs.CLcs.LG
keywords document visual groundingreinforcement learningGRPOmultimodal question answeringreasoning compressionperception alignmentSFT-to-RL transfergrounding divergence
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Document visual grounding – locating the exact region that supports an answer – is usually trained either with large supervised datasets or with reinforcement learning that encourages long reasoning traces. This paper argues that the task is perception-dominant: RL should bypass intermediate reasoning and align visual features directly to structured answer-and-bounding-box outputs. Introducing Perception-RFT, a GRPO-based framework with a gated dense reward, the authors find that when reasoning is enabled, models initially write reasoning traces but progressively compress and drop them, converging to direct-perception policies. At the 4B scale, reasoning-enabled RL underperforms perception-only training and adds over 60% extra inference tokens. The paper also documents SFT saturation, cold-start RL instability, and a newly named Grounding Divergence in which localization generalizes out-of-distribution while semantic robustness degrades.

Core claim

Perception-RFT applies Group Relative Policy Optimization to document visual grounding under a 'Direct Perception' constraint that forbids reasoning tokens, with a three-part gated reward: binary JSON format, quadratically scaled ANLS for semantics, and a thresholded IoU reward that shapes toward tight boxes. Under identical rewards, the reasoning-enabled variant collapses its reasoning traces during training: average reasoning length falls from about 191 to 72 tokens (62% reduction) while performance stays level or drops below the perception-only variant. The paper's central discovery is that explicit reasoning is neither retained nor beneficial for this grounding task; the optimal policy i

What carries the argument

The central object is Perception-RFT, a training framework built on GRPO (group-relative policy optimization) with a Gated Dense Reward. The Direct Perception system prompt suppresses <think> tokens and forces immediate JSON output {answer, bbox 2d}; the gated reward splits the optimization surface into a shaping state (λ·IoU below threshold 0.6) and a success state (1.0 above), preventing degenerate oversized boxes. A reasoning variant with identical rewards is used as a controlled comparison.

Load-bearing premise

The reasoning-enabled variant is tested only in cold-start RL with a reward that never rewards producing or retaining reasoning traces, so it may not capture what a warm-started, reasoning-supervised pipeline would do; if such a model kept its traces and improved grounding, the central 'reasoning is unnecessary' claim would collapse.

What would settle it

Train a reasoning-enabled model with supervised fine-tuning on reasoning-annotated grounding data, then apply RL with a reward that explicitly encourages retaining reasoning; if it outperforms perception-only training and keeps its traces, the paper's conclusion is overturned. Alternatively, run the same GRPO setup with a small reward bonus for reasoning tokens and observe whether performance exceeds the perception-only baseline.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • RL post-training lifts joint semantic-geometric accuracy beyond the supervised fine-tuning plateau on in-distribution documents (F1 all from 0.668 to 0.718 for SFT→RL).
  • SFT initialization is required for stable geometric learning under RL; cold-start RL is substantially worse, confirming that text-domain cold-start instability carries over to multimodal grounding.
  • Reasoning-enabled RL converges to a policy with minimal reasoning tokens, cutting per-query inference tokens by more than 60% while underperforming perception-only training.
  • An early SFT→RL transition (300 of 1,113 SFT steps) matches or exceeds full SFT localization with about 65% less training data.
  • Under distribution shift, localization improves consistently while semantic extraction often regresses (Grounding Divergence), so joint grounding still improves but the trade-off is asymmetric.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If this finding holds at larger scales, reasoning-centric RL pipelines for perception-heavy document tasks may be paying an inference cost without a grounding benefit; testing at 7B+ would clarify.
  • The reasoning ablation is cold-start only; a warm-started reasoning policy trained on reasoning-annotated supervision might retain traces and improve grounding, which the paper itself flags as an open question.
  • The gated reward design (shaping below threshold, cliff-edge success) could transfer to other structured-output tasks where degenerate solutions are common, such as table extraction or layout parsing.
  • Grounding Divergence could be reduced by decoupling semantic and geometric rewards, e.g., weighting ANLS more heavily on OOD splits.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Perception-RFT, a GRPO-based post-training method for document visual grounding (DVG) that directly optimizes structured JSON outputs (answer text and bounding box) without intermediate reasoning tokens. Using Qwen3-VL-4B trained on a 23,696-sample financial document corpus, it compares SFT, cold-start RL (RFTb), and SFT-then-RL (RFTs) on in-distribution and two OOD benchmarks (DOGR-Bench and MMDocBench, 4,828 samples). The central empirical claims are: (i) SFT saturates while RL continues to improve geometric precision; (ii) SFT initialization is needed for stable RL; (iii) an early SFT-to-RL transition achieves comparable precision with ~65% less training data; (iv) a reasoning-enabled cold-start RL variant (Reasoning-RFTb) underperforms the perception-only variant, suppresses its reasoning traces during training, and reduces inference token length by ~62% on ID data; and (v) a new 'Grounding Divergence' phenomenon describes selective trade-offs between semantic robustness and geometric precision under distribution shift. The paper introduces a Gated Dense Reward mechanism with a success threshold and a shaping scale.

Significance. If the central finding holds, the paper makes a practically relevant contribution: it shows at the 4B scale that reasoning traces are not necessary for DVG under cold-start RL, yielding faster and cheaper inference, and that RL can improve geometric grounding beyond SFT. The controlled comparison under identical reward settings is a genuine strength, as is the use of a strict joint success metric (F1 all) and the analysis of optimization dynamics. The paper is also transparent about several limitations. However, the headline claim that 'reasoning-enabled RL does not improve performance' rests on a single cold-start RL ablation in which the reward (Eq. 3–5) never scores reasoning tokens, so the observed trace suppression is partly a reward-design artifact. The authors acknowledge this limitation but the abstract and conclusions still overgeneralize. The paper is a promising empirical study that needs additional evidence or a more carefully scoped claim before its main conclusion is fully accepted.

major comments (4)
  1. [§3.4, §5.4, Eqs. (3)–(5)] The reasoning ablation is not a fair test of the value of reasoning. The reward is computed exclusively from the final JSON: format validity, ANLS of the answer string, and gated IoU (Eq. 3–5). Reasoning tokens inside <think> tags are never scored, so GRPO has no incentive to retain them; the observed compression in Figure 5 is a direct consequence of this reward design. The paper acknowledges in §6 that reasoning-annotated SFT data are unavailable, but this is load-bearing: the central claim that 'optimal grounding policies rely on direct perception rather than explicit reasoning' is supported only for cold-start RL with no reward signal for reasoning. To support the stronger claim, the authors would need a warm-started SFT→RL reasoning pipeline (e.g., with DocThinker-style reasoning-annotated data) or a reward that explicitly rewards reasoning quality. Absent that, the conclusion must
  2. [Table 1, Figures 2–5] No error bars, confidence intervals, or significance tests are reported even though the text repeatedly says 'averaged over 3 independent runs.' Several key comparisons are small in magnitude: on DOGR-Bench, SFT semantic F1 drops from the base 0.743 to 0.722, and on MMDocBench SFT (0.616) and RFTs (0.620) are within 0.02 of the base (0.636). Without variance estimates, the reader cannot distinguish genuine effects from run-to-run noise. This is particularly important for the claim that 'all trained variants fall below the zero-shot base' on DOGR and for the 'Grounding Divergence' phenomenon. Please report standard deviations or at least error bars.
  3. [§5.2, Table 1] The 'Grounding Divergence' is presented as a consistent pattern, but the results are actually inconsistent across the two OOD benchmarks. On DOGR-Bench, all trained variants drop below zero-shot on semantic F1, matching the stated divergence. On MMDocBench, by contrast, RFTb improves semantic F1 from 0.636 to 0.702, while SFT and RFTs drop slightly. This is not a consistent selective trade-off but a task-dependent result. The paper acknowledges in §5.2 that 'the severity of the Grounding Divergence varies with visual domain distance,' but the abstract and §5.1 describe it as a general phenomenon. The claim needs to be substantially nuanced or supported by a mechanism that explains the differential behavior.
  4. [§5.4, Table 2, Fig. 4] The reasoning ablation is only run in a cold-start RL setting, which limits the generality of the conclusion stated in the introduction and abstract: 'reasoning-enabled RL does not improve performance and can underperform direct perception-based training.' Because the reward never incentivizes retaining reasoning traces, a warm-started reasoning model might behave differently. The authors explicitly note this limitation in §6, but the title and abstract still imply a general conclusion about reasoning's non-necessity. The recommendation should be either to add the warm-start experiment when data become available, or to clearly state in the abstract that the finding is specific to cold-start RL without reasoning supervision.
minor comments (5)
  1. [§1, §5.1] Typo in the introduction: 'We confirm that, the two dynamics' should read 'We confirm that the two dynamics.' Also the phrase 'the, the two dynamics' in §5.1 is garbled.
  2. [§3.4 vs Appendix A.1] The reasoning variant is described as 'the model may generate intermediate <think> traces' in §3.4, but the reasoning system prompt in Appendix A.1 says 'You MUST first formulate your extraction plan internally' and requires <think> tags. This inconsistency should be resolved; the implementation appears to mandate reasoning, which strengthens the ablation but should be stated precisely.
  3. [Figure 5 and §5.4] The text claims a '>60% inference-token reduction' based on ID data (from ~191 to ~72 tokens), but Figure 5 shows that on OOD data the reasoning length slightly increases at later checkpoints. The claim should be scoped to the ID setting or averaged over both settings with the reported variance.
  4. [Table 1 and §4.3] The table note 'DOGR SFT is evaluated on DOGR-Bench only' is confusing, since the SFT row reports ID, DOGR, and MMDocBench numbers. Clarify what this note means. Also, Gemini 3.0 Flash is described as 'evaluated zero-shot' but no details are given about API version, sampling, or whether the same system prompts are used.
  5. [§3.3, Eq. (5)] The success threshold τ=0.6 and shaping scale λ=0.3 are set 'empirically' with no sensitivity analysis. The paper should at least report the stability of results to reasonable variations of these hyperparameters, since the 'cliff-edge' design is a core methodological contribution.

Circularity Check

0 steps flagged

No significant circularity: the paper's claims are empirical comparisons on held-out benchmarks, and the reasoning-ablation limitation is acknowledged rather than built into the derivation.

full rationale

The paper's central claims are empirical results, not derivations: Perception-RFT is compared against SFT, cold-start RL, and a reasoning-enabled variant on held-out ID and OOD benchmarks (DOGR-Bench, MMDocBench). The reasoning ablation is run under the same reward function, and the observed compression of <think> tokens is reported as a measured training trajectory (Figure 5), not as a consequence of the reward equations by construction. Although the reward (Eq. 3-5) does not score reasoning tokens, the paper explicitly scopes the conclusion to cold-start RL under identical reward settings, and the Limitations section acknowledges that a warm-started SFT-to-RL reasoning pipeline remains untested. The SFT saturation, cold-start instability, and Grounding Divergence findings are supported by external benchmarks and are not fitted parameters renamed as predictions; hyperparameters τ and λ are presented as empirical tuning choices, not as outputs of the method. There is no load-bearing self-citation chain. The paper is self-contained against external evaluation, so no circular step is exhibited.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 1 invented entities

The central empirical comparison uses two hand-tuned reward constants (τ, λ), several domain assumptions about benchmark compatibility and token-length-as-reasoning, and one named phenomenon (Grounding Divergence) that lacks independent evidence. These are standard for an empirical ML paper but should be stated explicitly.

free parameters (2)
  • IoU success threshold τ = 0.6
    Set empirically in Eq. 5, positioned slightly above the standard IoU≥0.5 evaluation threshold; this shapes the reward cliff-edge and the localization improvements.
  • Shaping scale λ = 0.3
    Set empirically in Eq. 5; scales the partial-overlap shaping reward and determines how much gradient low-IoU outputs receive.
axioms (5)
  • domain assumption GRPO with group-relative advantage (Eq. 2) is a valid and stable optimizer for a 4B multimodal policy.
    The paper relies on GRPO as introduced by Shao et al. and DeepSeek-R1 without proving convergence; cold-start instability is treated as an empirical finding.
  • domain assumption Output-token length is a valid proxy for the amount of reasoning the model performs.
    Section 5.4 and Fig. 5 measure reasoning by token count; this assumes reasoning is externalized and that no latent or covert reasoning occurs. If false, 'reasoning suppression' is only output suppression.
  • domain assumption DocILE/FormNLU KIE key-value annotations can be faithfully converted into document visual QA grounding targets with valid bounding boxes.
    Section 3.5's task-adaptation protocol assumes the mapping preserves the geometric ground truth needed to train and evaluate IoU.
  • domain assumption DOGR-Bench and MMDocBench labels and coordinate formats are compatible with the model's normalized output range [0,1000].
    Section 4.1 uses these as OOD evaluation sets without reporting coordinate-system alignment or label-format reconciliation.
  • domain assumption Three SFT epochs represent the saturation limit of supervised fine-tuning.
    Section 5.2 concludes SFT saturates from 3-epoch training on 23,696 samples; this assumes further SFT would not continue to improve precision.
invented entities (1)
  • Grounding Divergence no independent evidence
    purpose: Labels the observed asymmetric OOD generalization where localization improves but semantic extraction degrades under joint RL optimization.
    It is a post-hoc name for results on two benchmarks; on MMDocBench the semantic component improves for RFTb, so the pattern is not uniform, and no falsifiable prediction beyond the reported runs is offered.

pith-pipeline@v1.3.0-alltime-deepseek · 12712 in / 12628 out tokens · 122602 ms · 2026-08-02T01:21:14.910427+00:00 · methodology

0 comments
read the original abstract

Efficient multimodal document question answering with explicit visual grounding, locating the precise document region that supports each answer remains an open challenge. Current approaches bifurcate into Supervised Fine-Tuning (SFT), which requires large annotated datasets and reaches optimization plateaus, and reasoning-centric Reinforcement Learning (RL), which depends on verbose intermediate traces that inflate inference token cost without clear benefit. We introduce Perception-RFT, a training framework that applies Group Relative Policy Optimization (GRPO) to multimodal document QA, bypassing intermediate reasoning tokens to directly align visual features with structured grounding outputs. To rigorously evaluate the necessity of reasoning, we construct a reasoning variant under identical reward settings. We find that reasoning-enabled models suppress their reasoning traces during training, converging to direct perception-based policies at the 4B parameter scale, reducing per-query inference token length by more than 60%, while reasoning-enabled RL underperforms perception-only training. Through a fine-grained analysis of Qwen3-VL-4B optimization dynamics, we confirm that SFT saturation and cold-start RL instability established in text-domain post-training extend to multimodal, and identify a previously uncharacterized Grounding Divergence: a selective trade-off between semantic robustness and geometric precision on two out of distribution (OOD) benchmarks (4,828 samples) under joint RL optimization. We further show that an early SFT$\rightarrow$RL transition achieves comparable precision with 65% less training data.

Figures

Figures reproduced from arXiv: 2607.14682 by Ganesh Parab, Goutham Vignesh, Harikrishnan P M, Rohit Agrawal, Saisubramaniam Gopalakrishnan, Varun V, Vishal Vaddina.

Figure 1
Figure 1. Figure 1: The Perception-RFT Framework. Left: System prompting enforces the Direct Perception constraint. Right: The Gated Dense Reward mechanism stabilizes GRPO by defining separate shaping and success regions. (Best viewed zoomed in.) steps; Output ONLY the final valid JSON result.” This forces the optimization process to bypass verbose thought chains and align visual features directly to coordinate regression out… view at source ↗
Figure 2
Figure 2. Figure 2: ID Optimization Dynamics. SFT (Blue) plateaus early across all metrics. RFTs (Orange) breaks the ceiling. Notably, in (b), RFTs-Early (Green) eventually matches or slightly exceeds the localization performance of Full SFT, proving that early-exit policy optimization is more data-efficient than prolonged supervised learning. (Best viewed zoomed in.) 1 2 3 4 5 6 Training Checkpoint 0.68 0.69 0.70 0.71 0.72 0… view at source ↗
Figure 3
Figure 3. Figure 3: OOD Dynamics. (a) RFTb (Red) retains the highest reading ability among trained variants, minimizing the Grounding Divergence. (b) RFTs (Orange) breaks the SFT Stagnation (Blue), achieving a distinct geometric surge where the supervised baseline fails to improve. (Best viewed zoomed in.) [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Checkpoint-wise comparison of cold-start RL vari￾ants. (a) ID F1all across checkpoints for RFTb and Reasoning￾RFTb. (b) OOD F1all across checkpoints for the same variants. RFTb training shows smoother and stronger convergence, while reasoning-enabled training exhibits higher variance and inferior final performance. Final performance. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Reasoning token-length dynamics under cold-start RL. Both ID (solid) and OOD (dashed) settings show a strong reduction in reasoning length during training. While ID exhibits stable compression, OOD shows a slight increase at later stages, indicating higher variability under distribution shift. Optimization behavior [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative Analysis: SFT vs. Perception-RFT (ID). On dense financial documents, SFT often drifts into nearby words. RFT corrects this by snapping to the pixel edge. Question: “According to the text, what value is given for the ship-to name?” Ground Truth: WEBER SHANDWICK (BSMG-CG) SFT: WEBER SHANDWICK (BSMG-CG) 676 NORTH ST CLAIR SUITE 1000 CHICAGO, IL, 60611 (IoU 0.21 - Loose Box) Perception-RFT: WEBER S… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 19 linked inside Pith

  1. [1]

    Qwen3-vl technical report, 2025

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y., Tang, ...

    Pith/arXiv arXiv 2025

  2. [2]

    F., Tito, R., Mafla, A., Gomez, L., Rusinol, M., Valveny, E., Jawahar, C., and Karatzas, D

    Biten, A. F., Tito, R., Mafla, A., Gomez, L., Rusinol, M., Valveny, E., Jawahar, C., and Karatzas, D. Scene text visual question answering. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 4291--4301, 2019

    2019

  3. [6]

    Ding, Y., Long, S., Huang, J., Ren, K., Luo, X., Chung, H., and Han, S. C. Form-nlu: dataset for the form natural language understanding. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.\ 2807--2816, 2023

    2023

  4. [8]

    Boundingdocs: a unified dataset for document question answering with spatial annotations: S

    Giovannini, S., Coppini, F., Gemelli, A., and Marinai, S. Boundingdocs: a unified dataset for document question answering with spatial annotations: S. giovannini et al. International Journal on Document Analysis and Recognition (IJDAR), pp.\ 1--16, 2025

    2025

  5. [10]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models, 2021. URL https://arxiv.org/abs/2106.09685

    Pith/arXiv arXiv 2021

  6. [12]

    Layoutlmv3: Pre-training for document ai with unified text and image masking

    Huang, Y., Lv, T., Cui, L., Lu, Y., and Wei, F. Layoutlmv3: Pre-training for document ai with unified text and image masking. In Proceedings of the 30th ACM international conference on multimedia, pp.\ 4083--4091, 2022

    2022

  7. [15]

    B., and Zhang, K

    Li, M., Zhong, J., Zhao, S., Lai, Y., Zhang, H., Zhu, W. B., and Zhang, K. To think or not to think: A study of thinking in rule-based visual reinforcement fine-tuning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025 b

    2025

  8. [19]

    4v (ision) system card https://cdn

    OpenAI, G. 4v (ision) system card https://cdn. openai. com/papers. GPTV\_System\_Card. pdf, 2023

    2023

  9. [23]

    Docile benchmark for document information localization and extraction

    S imsa, S ., S ulc, M., U r i c \'a r , M., Patel, Y., Hamdi, A., Koci \'a n, M., Skalick \`y , M., Matas, J., Doucet, A., Coustaty, M., et al. Docile benchmark for document information localization and extraction. In International Conference on Document Analysis and Recognition, pp.\ 147--166. Springer, 2023

    2023

  10. [27]

    Unsloth: Fast fine-tuning and training of llms

    Team, U. Unsloth: Fast fine-tuning and training of llms. https://github.com/unslothai/unsloth, 2024

    2024

  11. [28]

    A., Jung, K., J \"a lk \"o , J., D’Andecy, V

    Tito, R., Nguyen, K., Tobaben, M., Kerkouche, R., Souibgui, M. A., Jung, K., J \"a lk \"o , J., D’Andecy, V. P., Joseph, A., Kang, L., et al. Privacy-aware document visual question answering. In International Conference on Document Analysis and Recognition, pp.\ 199--218. Springer, 2024

    2024

  12. [29]

    Drishtikon: Multi-granular visual grounding for text-rich document images

    Vishal Kasuba, B., Chaudhuri, P., and Ramakrishnan, G. Drishtikon: Multi-granular visual grounding for text-rich document images. arXiv e-prints, pp.\ arXiv--2506, 2025

    2025

  13. [30]

    Towards visual grounding: A survey

    Xiao, L., Yang, X., Lan, X., Wang, Y., and Xu, C. Towards visual grounding: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp.\ 1--20, 2025. doi:10.1109/TPAMI.2025.3630635

    arXiv 2025

  14. [33]

    Docthinker: Explainable multimodal large language models with rule-based reinforcement learning for document understanding

    Yu, W., Yang, Z., Liu, Y., and Bai, X. Docthinker: Explainable multimodal large language models with rule-based reinforcement learning for document understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 837--847, 2025 b

    2025

  15. [34]

    Dogr: Towards versatile visual document grounding and referring

    Zhou, Y., Chen, Y., Lin, H., Wu, Y., Yang, S., Qi, Z., Ma, C., and Zhu, L. Dogr: Towards versatile visual document grounding and referring. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 3596--3606, 2025

    2025

  16. [35]

    X., Wu, H., Wang, W., Feng, F., Wang, C., Luan, H., and Chua, T.-S

    Zhu, F., Liu, Z., Yao, N. X., Wu, H., Wang, W., Feng, F., Wang, C., Luan, H., and Chua, T.-S. Mmdocbench: Benchmarking large vision-language models for fine-grained visual document understanding and grounding. In Loko c , J., Pe s ka, L., Zah \'a lka, J., Rudinac, S., Kastner, M., Chen, J., Hu, M.-C., Wu, J., and Sharma, U. (eds.), MultiMedia Modeling, pp...

    2026

  17. [36]

    arXiv preprint arXiv:2412.14056 , year=

    A review of multimodal explainable artificial intelligence: Past, present and future , author=. arXiv preprint arXiv:2412.14056 , year=

    Pith/arXiv arXiv

  18. [37]

    arXiv preprint arXiv:2501.00321 , year=

    Ocrbench v2: An improved benchmark for evaluating large multimodal models on visual text localization and reasoning , author=. arXiv preprint arXiv:2501.00321 , year=

    Pith/arXiv arXiv

  19. [38]

    arXiv preprint arXiv:2504.04974 , year=

    Towards visual text grounding of multimodal large language model , author=. arXiv preprint arXiv:2504.04974 , year=

    arXiv

  20. [39]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Dogr: Towards versatile visual document grounding and referring , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  21. [40]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Docthinker: Explainable multimodal large language models with rule-based reinforcement learning for document understanding , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  22. [41]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    Pith/arXiv arXiv

  23. [42]

    arXiv preprint arXiv:2501.17161 , year=

    Sft memorizes, rl generalizes: A comparative study of foundation model post-training , author=. arXiv preprint arXiv:2501.17161 , year=

    Pith/arXiv arXiv

  24. [43]

    arXiv preprint arXiv:2509.12235 , year=

    Rl fine-tuning heals ood forgetting in sft , author=. arXiv preprint arXiv:2509.12235 , year=

    Pith/arXiv arXiv

  25. [44]

    5-vl technical report , author=

    Qwen2. 5-vl technical report , author=. arXiv preprint arXiv:2502.13923 , year=

    Pith/arXiv arXiv

  26. [45]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  27. [46]

    Proceedings of the 30th ACM international conference on multimedia , pages=

    Layoutlmv3: Pre-training for document ai with unified text and image masking , author=. Proceedings of the 30th ACM international conference on multimedia , pages=

  28. [47]

    arXiv preprint arXiv:2507.05595 , year=

    Paddleocr 3.0 technical report , author=. arXiv preprint arXiv:2507.05595 , year=

    Pith/arXiv arXiv

  29. [48]

    arXiv preprint arXiv:2403.05530 , year=

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context , author=. arXiv preprint arXiv:2403.05530 , year=

    Pith/arXiv arXiv

  30. [49]

    4V (ision) System Card https://cdn. openai. com/papers , author=. GPTV\_System\_Card. pdf , year=

  31. [50]

    arXiv preprint arXiv:2412.00151 , year=

    DLaVA: Document Language and Vision Assistant for Answer Localization with Enhanced Interpretability and Trustworthiness , author=. arXiv preprint arXiv:2412.00151 , year=

    Pith/arXiv arXiv

  32. [51]

    arXiv preprint arXiv:2306.15195 , year=

    Shikra: Unleashing multimodal llm's referential dialogue magic , author=. arXiv preprint arXiv:2306.15195 , year=

    Pith/arXiv arXiv

  33. [52]

    arXiv preprint arXiv:2310.07704 , year=

    Ferret: Refer and ground anything anywhere at any granularity , author=. arXiv preprint arXiv:2310.07704 , year=

    Pith/arXiv arXiv

  34. [53]

    arXiv preprint arXiv:2309.11419 , year=

    Kosmos-2.5: A multimodal literate model , author=. arXiv preprint arXiv:2309.11419 , year=

    Pith/arXiv arXiv

  35. [54]

    Farrar, Straus and Giroux , year=

    Thinking, fast and slow , author=. Farrar, Straus and Giroux , year=

  36. [55]

    arXiv preprint arXiv:2504.07954 , year=

    Perception-r1: Pioneering perception policy with reinforcement learning , author=. arXiv preprint arXiv:2504.07954 , year=

    Pith/arXiv arXiv

  37. [56]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    To think or not to think: A study of thinking in rule-based visual reinforcement fine-tuning , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  38. [57]

    arXiv preprint arXiv:2503.20752 , year=

    Reason-rft: Reinforcement fine-tuning for visual reasoning , author=. arXiv preprint arXiv:2503.20752 , year=

    arXiv

  39. [58]

    2021 , eprint=

    Key Information Extraction From Documents: Evaluation And Generator , author=. 2021 , eprint=

    2021

  40. [59]

    International Conference on Document Analysis and Recognition , pages=

    Privacy-aware document visual question answering , author=. International Conference on Document Analysis and Recognition , pages=. 2024 , organization=

    2024

  41. [60]

    International Conference on Document Analysis and Recognition , pages=

    Docile benchmark for document information localization and extraction , author=. International Conference on Document Analysis and Recognition , pages=. 2023 , organization=

    2023

  42. [61]

    Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    Form-NLU: dataset for the form natural language understanding , author=. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

  43. [62]

    Giovannini et al

    BoundingDocs: a Unified Dataset for Document Question Answering with Spatial Annotations: S. Giovannini et al. , author=. International Journal on Document Analysis and Recognition (IJDAR) , pages=. 2025 , publisher=

    2025

  44. [63]

    arXiv e-prints , pages=

    DrishtiKon: Multi-Granular Visual Grounding for Text-Rich Document Images , author=. arXiv e-prints , pages=

  45. [64]

    arXiv preprint arXiv:2503.01785 , year=

    Visual-rft: Visual reinforcement fine-tuning , author=. arXiv preprint arXiv:2503.01785 , year=

    Pith/arXiv arXiv

  46. [65]

    arXiv preprint arXiv:2503.06749 , year=

    Vision-r1: Incentivizing reasoning capability in multimodal large language models , author=. arXiv preprint arXiv:2503.06749 , year=

    Pith/arXiv arXiv

  47. [66]

    arXiv preprint arXiv:2504.07615 , year=

    Vlm-r1: A stable and generalizable r1-style large vision-language model , author=. arXiv preprint arXiv:2504.07615 , year=

    Pith/arXiv arXiv

  48. [67]

    2021 , eprint=

    LoRA: Low-Rank Adaptation of Large Language Models , author=. 2021 , eprint=

    2021

  49. [68]

    GitHub repository , howpublished =

    Unsloth Team , title =. GitHub repository , howpublished =. 2024 , publisher =

    2024

  50. [69]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Scene text visual question answering , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  51. [70]

    Towards Visual Grounding: A Survey , year=

    Xiao, Linhui and Yang, Xiaoshan and Lan, Xiangyuan and Wang, Yaowei and Xu, Changsheng , journal=. Towards Visual Grounding: A Survey , year=

  52. [71]

    arXiv preprint arXiv:2509.10345 , year=

    Towards understanding visual grounding in visual language models , author=. arXiv preprint arXiv:2509.10345 , year=

    arXiv

  53. [72]

    MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understanding and Grounding

    Zhu, Fengbin and Liu, Ziyang and Yao, NG Xiang and Wu, Haohui and Wang, Wenjie and Feng, Fuli and Wang, Chao and Luan, Huanbo and Chua, Tat-Seng. MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understanding and Grounding. MultiMedia Modeling. 2026

    2026

  54. [73]

    2025 , eprint=

    Qwen3-VL Technical Report , author=. 2025 , eprint=

    2025