Pith. sign in

REVIEW 3 major objections 6 minor 30 references

After frames are chosen, long-video models still need a compact latent channel that binds query-relevant cues across moments before they answer.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 04:53 UTC pith:5EPRZFV2

load-bearing objection Solid systems paper: post-selection latent aggregation with real matched-frame gains and tiny overhead; Adaptive Invocation is the softest piece, not the whole claim. the 3 major comments →

arxiv 2607.28516 v1 pith:5EPRZFV2 submitted 2026-07-30 cs.CV

Beyond Frame Selection: Generative Latent Evidence Aggregation for Long-Video Understanding

Bowen Liu , Shuning Wang , Xinpeng Ding , Zhiheng Wu , Bodong Du , Xiaomeng Li This is my paper
classification cs.CV
keywords long-video understandingVideo-MLLMlatent evidence aggregationframe selectionadaptive evidence invocationquery-conditioned allocationcompact visual context
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Long-video systems usually spend their budget on picking a few frames and feeding them as explicit pixels or tokens. This paper argues that availability is not the same as use: complementary cues can sit in those frames and still fail to shape the answer. It inserts a post-selection stage, a latent evidence interface, that turns the chosen frames into a small set of query-conditioned latent tokens that cross-link what each frame contributes. GenEvA builds that channel with a learned allocation of latent budget, aggregates frame-specific features under a shared query-visual context, and decides from the same allocation’s dispersion whether to insert the latent complement or stay on the explicit path. On four long-video benchmarks and two frozen backbones, the method raises matched-frame accuracy—about +5.2 points on average for LLaVA-Video at 8 frames, and up to +10.1 on LVBench for Qwen2.5-VL—while adding only a fraction of a percent in video tokens. A sympathetic reader cares because the gain comes from organizing evidence the model already has, not from longer contexts or heavier retrieval.

Core claim

The authors claim that compact long-video understanding has a missing post-selection stage: a latent evidence interface that organizes selected frames into query-relevant cross-frame latent evidence before generation. Instantiated as GenEvA, a query-conditioned evidence distribution both allocates a fixed latent-token budget across frames and, via residual allocation dispersion, chooses whether to insert that latent complement. Matched-frame experiments show consistent accuracy gains at 8 and 16 frames on two Video-MLLM backbones with at most 32 latent tokens and 0.11%–0.40% average video-token overhead.

What carries the argument

GenEvA (Generative Latent Evidence Aggregation): a distribution-guided adapter that (1) allocates integer latent slots per selected frame from a query-conditioned distribution, (2) fills those slots by conditioning a shared latent basis on frame features and a query-visual context, then refines them in the frozen LLM’s hidden space, and (3) invokes the latent route only when residual allocation dispersion exceeds 1/2.

Load-bearing premise

That how spread residual latent budget is across frames, cut at a fixed halfway threshold, is a trustworthy built-in signal for when cross-frame latent integration helps versus when the plain explicit path is better.

What would settle it

Hold the selector and backbone fixed at 8 frames with B=32, replace Adaptive Evidence Invocation with always-on latent insertion and with random insertion, and check whether adaptive dispersion routing still beats both on the four-benchmark average; if it does not, or if dispersion no longer tracks localized versus distributed task groups, the central invocation claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Matched-frame long-video pipelines can raise accuracy without adding frames or large explicit context by inserting a small latent evidence channel after selection.
  • The same query-conditioned allocation can both size per-frame latent capacity and decide whether cross-frame integration is needed, avoiding a separate trained router.
  • Task structure shows up in allocation geometry: localized questions concentrate residual budget; distributed or long-span questions spread it.
  • Decoder-side video-token overhead can stay under half a percent while still delivering multi-point gains on standard long-video benchmarks.
  • Frame-selection methods and reasoning-oriented multi-turn systems can be complemented, rather than replaced, by a lightweight post-selection latent interface.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the bottleneck is evidence use rather than evidence arrival, similar latent-interface adapters may help image and document MLLMs that already receive the right regions but still lean on language priors.
  • A fixed dispersion threshold may be replaceable by a cheap calibration per backbone or domain without abandoning the idea that allocation geometry is the routing signal.
  • Training the allocator jointly with answer likelihood suggests future work could supervise allocation directly against human time-span annotations when those exist.
  • The method’s dependence on a frozen selector implies that better selectors and GenEvA-style aggregation could stack rather than compete.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper identifies a post-selection evidence-use bottleneck in compact long-video QA: retained frames need not be converted into answer-relevant representations. It proposes a latent evidence interface instantiated by GenEvA, which (i) allocates a fixed latent-token budget across frozen selected frames via a query-conditioned distribution, (ii) builds frame-wise query-aware latent bundles refined through a LoRA integrator on the frozen LLM, and (iii) at inference inserts that latent complement only when residual allocation dispersion exceeds 1/2 (Adaptive Evidence Invocation). On Video-MME, MLVU, LongVideoBench, and LVBench with LLaVA-Video and Qwen2.5-VL under matched 8/16-frame budgets, GenEvA improves the bases (e.g., +5.2 four-benchmark average for LLaVA-Video at 8 frames; +10.1 LVBench for Qwen2.5-VL at 8 frames) at 0.11%–0.40% average video-token overhead. Ablations separate selection, uniform vs learned allocation, and Full/Random/Adaptive invocation; analyses link dispersion to task evidence scope and show route complementarity.

Significance. If the matched-frame gains hold, the work cleanly separates “what frames reach the model” from “how their cues are organized for generation,” a useful interface-level contribution relative to selection-, compression-, and long-context-centric lines. Strengths include consistent two-backbone, four-benchmark matched-frame evaluation (Table 1), staged ablations (Table 3), low and accounted token overhead (Fig. 4, App. Table 4), latency reporting (Table 2), and qualitative/quantitative links between allocation geometry and localized vs distributed evidence (Figs. 6, 8–9). The Adaptive claim is secondary to the aggregation gains but, if better secured, would support a single distribution governing both capacity and use without a learned router. The result is incremental rather than paradigm-shifting, but practically relevant for budgeted Video-MLLMs.

major comments (3)
  1. [§3.5, Eqs. (8)–(9); Table 3] §3.4–3.5 and Eqs. (7)–(9): Adaptive Evidence Invocation is load-bearing for the claim that “the same distribution determines whether to insert” latent evidence, yet training never optimizes the dispersion rule or the threshold s_disp=1/2. The adapter is supervised under sampled always-on latent paths plus an explicit baseline; Adaptive is applied only at inference with a fixed hand-set cut. Table 3 shows Adaptive only modestly above Full (+0.4/+1.2 Avg at 8/16 frames). Please (i) report threshold sensitivity (and/or Neff) on a held-out split, (ii) discuss train–inference mismatch explicitly, and (iii) clarify whether Adaptive is a primary mechanism or a mild optional regularizer. Without this, the route-selection story is only partially secured even though the aggregation gains remain credible.
  2. [§3.3 Eq. (5); Table 3] §3.3 Eq. (5) and Table 3: Gains could partly reflect extra trainable path capacity (latent basis, conditioning MLP, LoRA δ) rather than genuine cross-frame evidence organization. Full invocation always inserts B latent tokens refined in the LLM hidden space; the Explicit route disables δ. A control that inserts non-informative or shuffled latent slots (or freezes aggregation while keeping LoRA/token count) would isolate “integration of complementary cues” from “extra conditioned tokens help decoding.” The staged Uniform→learned allocation results help but do not fully close this alternative.
  3. [§4.1; Table 1] §4.1 Implementation and Table 1: The adapter is trained on 8,000 LLaVA-Video-178K samples with a frozen TSPO selector; headline numbers use that same selector family and, for one backbone, the LLaVA-Video stack. Please state data hygiene relative to the four eval benchmarks (no overlap / no near-duplicate video IDs), and report at least one result with an alternate frozen selector or uniform frames under GenEvA aggregation so improvements are not tied to a single selector–adapter pairing. Matched-frame Base vs +GenEvA is the right protocol; selector coupling still needs a short stress check.
minor comments (6)
  1. [Figure 1; §1] Fig. 1(b) caption claims “100% evidence hit, yet >50% wrong answers” but the main text describes overlap with annotated evidence without defining the hit metric or subset size. Add the exact criterion and N.
  2. [§3.3 Eq. (4)] Eq. (4) notation is dense (broadcast 1_ni, Z_i basis shared across examples). A one-line dimensional check and whether Z is frame-indexed or globally shared would help reproducibility.
  3. [§3.2–3.4; Fig. 5] Hyperparameters τ_a, λ_m, m, G=8, N_max appear in free form with little sensitivity (only B in Fig. 5). A short appendix sweep or default table would strengthen reproducibility.
  4. [Table 1] Table 1 mixes proprietary, open, and reasoning models under heterogeneous frame budgets; keep the matched-frame block as the primary claim and mark non-comparable rows more clearly to avoid over-reading cross-method ranks.
  5. [Abstract; §1] Typos/style: “alatent evidence interface”, “+10 .1”, “0 .11%”, and occasional missing spaces around citations; normalize throughout.
  6. [Appendix B.3 Table 4] App. Table 4 activation rates imply ρ differs sharply by backbone (latent avg ~9.6 vs ~24.8 at 8 frames). Briefly discuss why Adaptive fires more often on Qwen2.5-VL.

Circularity Check

0 steps flagged

No significant circularity: empirical method trained on answer likelihood and evaluated on external benchmarks; Adaptive threshold is a design choice, not a tautology.

full rationale

GenEvA is a standard empirical Video-MLLM adapter paper. The load-bearing claims are matched-frame accuracy gains on Video-MME, MLVU, LongVideoBench, and LVBench (Table 1), low token overhead (Fig. 4 / Table 4), and ablation gains of Adaptive over Full/Random invocation (Table 3). Training (Eq. 7) maximizes teacher-forced answer log-likelihood via a score-function allocator update plus a margin against the explicit route on LLaVA-Video-178K samples; those objectives do not algebraically force the reported benchmark deltas. Allocation α and integer budgets n are learned inputs to aggregation, not definitions of accuracy. Adaptive Evidence Invocation (Eqs. 8–9) applies a fixed hand-set dispersion threshold only at inference; that is a methodological choice (and a possible train/infer mismatch), but s_disp is not defined to equal correctness, and the paper reports empirical route-complementarity and ablation numbers rather than a by-construction identity. Task-aware dispersion analyses (Fig. 6, RQ3) are post-hoc correlations with independently defined task groups, not predictions fitted to those groups. Appendix self-citations (MedHorizon, Divide-then-Diagnose, etc.) appear only as extended related work and do not underwrite the main results. No self-definitional loop, fitted-input-as-prediction, uniqueness import, or renaming of a known closed-form result is present.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 3 invented entities

This is an empirical systems paper. The claim rests on standard frozen Video-MLLM and selector components, a small set of hand-chosen interface hyperparameters (latent budget, dispersion threshold, sampling count, margin), and the modeling postulate that a compact latent channel organized by a query-conditioned allocation can complement explicit frames. No physical constants; free parameters are design knobs and training hyperparameters. Invented entities are interface abstractions, not new physical objects.

free parameters (7)
  • latent token budget B = 32
    Fixed capacity of the latent evidence sequence; main results use B=32, with limited sensitivity in Fig. 5.
  • dispersion invocation threshold = 0.5
    Fixed boundary s_disp=1/2 (Neff > 2K/(K+1)) chooses Explicit vs Latent route at inference (§3.5).
  • allocation temperature τ_a
    Softmax temperature in evidence-budget scores α_i (Eq. 1); value not reported in the main text.
  • margin weight λ_m and target margin m
    Hinge-style term in the joint objective forcing latent likelihood above explicit by m (Eq. 7); numeric values not stated.
  • allocation samples G per example = 8
    Number of multinomial residual-budget samples for score-function allocator training.
  • per-frame latent-basis capacity N_max
    Caps n_i and must satisfy N_max ≥ B−K+1; exact setting not tabulated beyond the feasibility assumption.
  • training subset size / mixture = 8000 samples
    Adapter trained on 8,000 samples from LLaVA-Video-178K; selection of the 8k and optimization hyperparameters are free experimental choices.
axioms (6)
  • domain assumption Frozen pretrained temporal selector L_ψ★ returns a sufficient candidate set of K frames for the latent interface to organize (problem setup §3.1).
    All GenEvA gains are conditional on selected frames already containing usable evidence; the method does not recover missing frames.
  • domain assumption Frozen Video-MLLM vision tower, projector, and LLM backbone remain fixed; only allocator ϕ and evidence adapter η (including LoRA δ) are trained (§3.3–3.4).
    Standard PEFT assumption that a small adapter can reshape evidence use without updating base weights.
  • domain assumption Answer teacher-forced log-likelihood is a valid training proxy for constructing and allocating latent evidence (Eq. 7).
    Allocator advantages and adapter updates are driven entirely by likelihood, not by explicit grounding labels.
  • ad hoc to paper Residual allocation geometry (dispersion after removing the one-token floor) indicates need for cross-frame integration (Eqs. 8–9).
    Core Adaptive Evidence Invocation design choice; justified interpretively via Neff, not derived from a proven decision-theoretic optimum.
  • ad hoc to paper Integer budget assignment with one-token floor plus residual proportional allocation is a valid discretization of α (Eq. 2).
    Engineering rule ensuring coverage and exact total B; alternatives not extensively compared beyond uniform allocation ablation.
  • standard math Standard multinomial score-function / REINFORCE-style estimator with within-group advantage is a valid gradient surrogate for discrete allocations (Eqs. 6–7).
    Common stochastic computation graph technique; used as-is with stop-gradient advantages.
invented entities (3)
  • latent evidence interface (post-selection) no independent evidence
    purpose: Name the missing stage between frame selection and generation that preserves explicit frames while adding a compact latent complement.
    Conceptual framing introduced in §1 and §3.1; operationalized only through GenEvA, not independently measured outside this pipeline.
  • query-conditioned latent evidence bundles E / eE_i no independent evidence
    purpose: Variable-length frame-wise latent slots conditioned on local frame features and shared query-visual context, then integrated via LoRA.
    Constructed artifacts of the aggregation head (Eqs. 3–5); usefulness is shown only via downstream QA accuracy in this paper.
  • Adaptive Evidence Invocation via allocation dispersion s_disp no independent evidence
    purpose: Reuse the same allocation distribution as a router without a separately trained gating module.
    Defined in §3.5; ablations compare Full/Random/Adaptive but provide no external validation that s_disp measures true cross-frame necessity beyond these benchmarks.

pith-pipeline@v1.2.0-daily-grok45 · 21572 in / 4370 out tokens · 77378 ms · 2026-07-31T04:53:16.175010+00:00 · methodology

0 comments
read the original abstract

Long-video understanding commonly compresses videos into a small set of frames or visual tokens for answer generation. Existing compact pipelines focus on retaining relevant visual content as explicit evidence. Yet making evidence available does not ensure that complementary cues across moments are integrated for answering. Our key idea is to organize selected frames into query-relevant cross-frame evidence before generation. We formulate this post-selection stage as a latent evidence interface and instantiate it with GenEvA ($\textbf{Gen}erative$ $Latent$ $\textbf{Ev}idence$ $\textbf{A}ggregation$), a distribution-guided latent evidence aggregation framework. Specifically, GenEvA uses a query-conditioned evidence distribution to focus aggregation on relevant frames, forming compact cross-frame latent evidence from their frame-specific information. Since cross-frame integration is not always needed, the same distribution determines whether to insert this latent complement. Across four benchmarks and two Video-MLLM backbones, GenEvA consistently improves matched-frame baselines. At 8 frames, it raises the four-benchmark LLaVA-Video average by $+5.2$ points and Qwen2.5-VL accuracy on LVBench by $+10.1$ points. These gains require only $0.11\%$--$0.40\%$ average video-token overhead; analyses further show task-aware allocation and benefits from Adaptive Evidence Invocation.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 21 linked inside Pith

  1. [1]

    EVIDENT: Routing mllm adaptation through entity-grounded visual evidence for cross-domain video temporal grounding.arXiv preprint arXiv:2605.26104,

    Geo Ahn, Jiwook Han, Youngrae Kim, Joonseok Lee, and Jinwoo Choi. EVIDENT: Routing mllm adaptation through entity-grounded visual evidence for cross-domain video temporal grounding.arXiv preprint arXiv:2605.26104,

    Pith/arXiv arXiv

  2. [3]

    VISTAQA: Benchmarking joint visual question answering and pixel-level evidence.arXiv preprint arXiv:2605.20676,

    Mozhgan Nasr Azadani, Yimu Wang, Yongpeng Zhu, Lihong Chen, Milan Ganai, Sean Sedwards, Marco Pavone, and Krzysztof Czarnecki. VISTAQA: Benchmarking joint visual question answering and pixel-level evidence.arXiv preprint arXiv:2605.20676,

    Pith/arXiv arXiv

  3. [4]

    Roush, Ravid Shwartz-Ziv, and Hod Lipson

    Daniel Ben-Levi, Judah Goldfeder, Weiliang Zhao, Raz Lapid, Amit LeVi, Allen G. Roush, Ravid Shwartz-Ziv, and Hod Lipson. Mirage probes: How vision models fake visual understanding.arXiv preprint arXiv:2606.13870,

    arXiv

  4. [5]

    Shikra: Unleashing multimodal llm’s referential dialogue magic.arXiv preprint arXiv:2306.15195,

    Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multimodal llm’s referential dialogue magic.arXiv preprint arXiv:2306.15195,

    Pith/arXiv arXiv

  5. [6]

    Longvila: Scaling long-context visual language models for long videos

    Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, et al. Longvila: Scaling long-context visual language models for long videos. InInternational Conference on Learning Representations, volume 2025, pages 18227–18246,

    2025

  6. [9]

    Alessandro Favero, Luca Zancato, Matthew Trager, Siddharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto

    https://arxiv.org/abs/2605.06537. Alessandro Favero, Luca Zancato, Matthew Trager, Siddharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. Multi-modal hallucination control by visual information grounding. In 10 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14303–14312,

    Pith/arXiv arXiv

  7. [10]

    Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue

    doi: 10.1109/CVPR52733.2024.01356. Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue. Video-R1: Reinforcing video reasoning in MLLMs. InAdvances in Neural Information Processing Systems, volume 38, pages 99114–99137. Curran Associates, Inc., 2025.https://proceeding s.neur...

    arXiv 2024

  8. [11]

    PixelEyes: Decoupling perception and reasoning for pinpoint visual evidence seeking.arXiv preprint arXiv:2607.00115,

    Dengxian Gong, Yuanzheng Wu, Haobo Yuan, Zhengdong Hu, Tao Zhang, Yikang Zhou, Shihao Chen, Quanzhu Niu, Kai Wang, Jason Li, Haochen Wang, Lu Qi, Shunping Ji, and Ming-Hsuan Yang. PixelEyes: Decoupling perception and reasoning for pinpoint visual evidence seeking.arXiv preprint arXiv:2607.00115,

    Pith/arXiv arXiv

  9. [12]

    Decoding by perturbation: Mitigating mllm hallucinations via dynamic textual perturbation.arXiv preprint arXiv:2604.12424,

    Sihang Jia, Shuliang Liu, Songbo Yang, Yibo Yan, Xin Zou, and Xuming Hu. Decoding by perturbation: Mitigating mllm hallucinations via dynamic textual perturbation.arXiv preprint arXiv:2604.12424,

    Pith/arXiv arXiv

  10. [13]

    Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024a

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024a. Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training ...

    Pith/arXiv arXiv 2023

  11. [14]

    Divide-then-diagnose: Weaving clinician-inspired contexts for ultra-long capsule endoscopy videos, 2026a.https://arxiv.org/abs/2604.21814

    Bowen Liu, Li Yang, Shanshan Song, Mingyu Tang, Zhifang Gao, Qifeng Chen, Yangqiu Song, Huimin Chen, and Xiaomeng Li. Divide-then-diagnose: Weaving clinician-inspired contexts for ultra-long capsule endoscopy videos, 2026a.https://arxiv.org/abs/2604.21814. Xiangrui Liu, Yan Shu, Zheng Liu, Ao Li, Yang Tian, and Bo Zhao. Video-xl-pro: Reconstructive token ...

    Pith/arXiv arXiv

  12. [15]

    Moment-video: Diagnosing temporal fidelity of video mllms on momentary visual events.arXiv preprint arXiv:2606.02522, 2026b

    Xiaolin Liu, Yilun Zhu, Xiangyu Zhao, Xuehui Wang, Yan Li, Xin Li, Haoyu Cao, Xing Sun, Shaofeng Zhang, Xu Yang, Zhihang Zhong, and Xue Yang. Moment-video: Diagnosing temporal fidelity of video mllms on momentary visual events.arXiv preprint arXiv:2606.02522, 2026b. 11 Dongsheng Ma, Jiayu Li, Zhengren Wang, Yijie Wang, Jiahao Kong, Weijun Zeng, Jutao Xiao...

    Pith/arXiv arXiv

  13. [17]

    Liu, Yuan Zhu, Wei Fan, and Jindong Han

    Frank Nie, Ethan B. Liu, Yuan Zhu, Wei Fan, and Jindong Han. A cost-effective multimodal llm reasoning framework for question answering over irregular clinical time series, 2026a.https://arxiv.org/abs/2607.25947. Frank Nie, Ethan B. Liu, Yuan Zhu, Loe Yan, Wei Fan, and Jindong Han. Clir-bench: Benchmarking multimodal question answering over irregular clin...

    Pith/arXiv arXiv

  14. [18]

    Kosmos-2: Grounding multimodal large language models to the world.arXiv preprint arXiv:2306.14824,

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world.arXiv preprint arXiv:2306.14824,

    Pith/arXiv arXiv

  15. [19]

    SeViCES: Unifying semantic-visual evidence consensus for long video understanding.arXiv preprint arXiv:2510.20622,

    Yuan Sheng, Yanbin Hao, Chenxu Li, Shuo Wang, and Xiangnan He. SeViCES: Unifying semantic-visual evidence consensus for long video understanding.arXiv preprint arXiv:2510.20622,

    arXiv

  16. [20]

    TSPO: Temporal sampling policy optimization for long-form video language understanding

    Canhui Tang, Zifan Han, Hongbo Sun, Sanping Zhou, Xuchong Zhang, Xin Wei, Ye Yuan, Jinglin Xu, and Hao Sun. TSPO: Temporal sampling policy optimization for long-form video language understanding. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 9368–9376, 2026.https://arxiv.org/abs/2508.04369. Guankun Wang, Long Bai, Wan J...

    arXiv 2026

  17. [21]

    Postalign: Multimodal grounding as a corrective lens for mllms.arXiv preprint arXiv:2506.17901,

    Yixuan Wu, Yang Zhang, Jian Wu, Philip Torr, and Jindong Gu. Postalign: Multimodal grounding as a corrective lens for mllms.arXiv preprint arXiv:2506.17901,

    Pith/arXiv arXiv

  18. [22]

    Staying VIGILant: Mitigating visual laziness via counterfactual visual alignment in mllms.arXiv preprint arXiv:2606.26387,

    Xi Xiao, Chen Liu, Chih-Ting Liao, Yunbei Zhang, Qizhen Lan, Yuxiang Wei, Lin Zhao, Janet Wang, Jianyang Gu, Muchao Ye, Tianyang Wang, and Hao Xu. Staying VIGILant: Mitigating visual laziness via counterfactual visual alignment in mllms.arXiv preprint arXiv:2606.26387,

    Pith/arXiv arXiv

  19. [23]

    Video-MTR: Reinforced multi-turn reasoning for long video understanding

    Yuan Xie, Tianshui Chen, Zheng Ge, and Lionel Ni. Video-MTR: Reinforced multi-turn reasoning for long video understanding. InForty-third International Conference on Machine Learning, 2026.https://openreview.net/for um?id=UhPwL6LYOc. Cilin Yan, Haochen Wang, Shilin Yan, Xiaolong Jiang, Yao Hu, Guoliang Kang, Weidi Xie, and Efstratios Gavves. VISA: Reasonin...

    Pith/arXiv arXiv 2026

  20. [24]

    Ferret: Refer and ground anything anywhere at any granularity.arXiv preprint arXiv:2310.07704,

    Haoxuan You, Haotian Zhang, Zhe Gan, Xianzhi Du, Bowen Zhang, Zirui Wang, Liangliang Cao, Shih-Fu Chang, and Yinfei Yang. Ferret: Refer and ground anything anywhere at any granularity.arXiv preprint arXiv:2310.07704,

    Pith/arXiv arXiv

  21. [25]

    Timesuite: Improving mllms for long video understanding via grounded tuning

    Xiangyu Zeng, Kunchang Li, Chenting Wang, Xinhao Li, Tianxiang Jiang, Ziang Yan, Songze Li, Yansong Shi, Zhengrong Yue, Yi Wang, et al. Timesuite: Improving mllms for long video understanding via grounded tuning. In International Conference on Learning Representations, volume 2025, pages 38057–38081,

    2025

  22. [26]

    Long context transfer from language to vision.arXiv preprint arXiv:2406.16852, 2024a

    Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision.arXiv preprint arXiv:2406.16852, 2024a. Shufan Zhang, Ziyue Lin, Bairun Wang, Lei Jin, Xuanding Ding, Xinzhu Ma, and Kunlin Yang. VTI-CoT: Visual- textual interleaved chain of t...

    Pith/arXiv arXiv

  23. [27]

    Llava-next: A strong zero-shot video understanding model, April 2024b.https://llava-vl.github.io/blog/2024-0 4-30-llava-next-video/

    Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. Llava-next: A strong zero-shot video understanding model, April 2024b.https://llava-vl.github.io/blog/2024-0 4-30-llava-next-video/. Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zha...

    2024

  24. [28]

    VisualFLIP: Do predictions depend on task-critical visual evidence in multimodal reasoning?arXiv preprint arXiv:2606.07872,

    Didi Zhu, Changrui Chen, Stefanos Zafeiriou, and Jiankang Deng. VisualFLIP: Do predictions depend on task-critical visual evidence in multimodal reasoning?arXiv preprint arXiv:2606.07872,

    Pith/arXiv arXiv

  25. [29]

    Videobrain: Learning adaptive frame sampling for long video understanding.arXiv preprint arXiv:2602.04094,

    Junbo Zou, Ziheng Huang, Shengjie Zhang, Liwen Zhang, and Weining Shen. Videobrain: Learning adaptive frame sampling for long video understanding.arXiv preprint arXiv:2602.04094,

    Pith/arXiv arXiv

  26. [30]

    (2024); Chen et al

    13 Appendix A Extended Related Work Long-video understanding.General video instruction-tuned models such as Video-LLaVA, ShareGPT4Video, LLaVA-NeXT-Video, VideoLLaMA2, VideoChat2, LLaVA-OneVision, and InternVL2 extend image-language alignment and instruction tuning to video inputs Lin et al. (2024); Chen et al. (2024a); Zhang et al. (2024b); Cheng et al. ...

    2024

  27. [2023]

    Marco Morini, Sara Sarto, Marcella Cornia, and Lorenzo Baraldi

    https: //arxiv.org/abs/2307.15189. Marco Morini, Sara Sarto, Marcella Cornia, and Lorenzo Baraldi. Look twice: Training-free evidence highlighting in multimodal large language models.arXiv preprint arXiv:2604.01280,

    Pith/arXiv arXiv

  28. [2024]

    Retrieve, integrate, and synthesize: Spatial-semantic grounded latent visual reasoning.arXiv preprint arXiv:2605.07106,

    Jin Cui, Xinyue Long, Xunyong Zhang, Yadong Zhang, Chuanchang Su, Jingye Gan, Boran Zhao, and Pengju Ren. Retrieve, integrate, and synthesize: Spatial-semantic grounded latent visual reasoning.arXiv preprint arXiv:2605.07106,

    Pith/arXiv arXiv

  29. [2025]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024b

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024b. Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zh...

    Pith/arXiv arXiv

  30. [2026]

    O’Sullivan, Fang Cao, Tahoura Nedaee, Kamyar Rajabalifardi, Fei-Fei Li, Ehsan Adeli, and Euan Ashley

    Mohammad Asadi, Jack W. O’Sullivan, Fang Cao, Tahoura Nedaee, Kamyar Rajabalifardi, Fei-Fei Li, Ehsan Adeli, and Euan Ashley. MIRAGE: The illusion of visual understanding.arXiv preprint arXiv:2603.21687,

    arXiv