Pith. sign in

REVIEW 3 major objections 5 minor 62 references

ReasVQA: Advancing VideoQA with Imperfect Reasoning Process

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read VideoQA can be improved by training a small model to regenerate a larger model's cleaned reasoning traces, even when those traces end in a wrong answer.

desk verdict Useful, mostly honest training-recipe paper for VideoQA; the empirical core holds up, but the refinement 'decoupling' story is contradicted by the paper's own example and the abstract oversells the gains. read the letter →

arxiv 2501.13536 v1 pith:PYI7MSHD submitted 2025-01-23 cs.CV cs.CL

Jianxin Liang , Xiaojun Meng , Huishuai Zhang , Yueqian Wang , Jiansheng Wei , Dongyan Zhao This is my paper · ORCID
classification cs.CVcs.CL
keywords videoquestionansweringprocesssupervisionMLLM-generatedreasoningrefinementmulti-tasklearningNExT-QASTARIntentQA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that a small video-question-answering model can be improved by supervising it with reasoning traces produced by a much larger multimodal model, even when many of those traces end in the wrong answer. The method, ReasVQA, strips conclusion sentences and leaked ground-truth words out of the teacher's reasoning, then trains the student on two objectives at once: answer correctly, and regenerate the cleaned reasoning. On NExT-QA, STAR, and IntentQA this raises accuracy to 77.0%, 74.5%, and 77.0%, exceeding the prior best numbers on all three benchmarks. If the claim holds, it means reasoning ability can be transferred cheaply and at scale without human-written rationales.

What carries the argument

The load-bearing mechanism is a three-phase pipeline. Reasoning Generation queries an off-the-shelf MLLM for a free-form answer with reasoning; Reasoning Refinement removes sentences matching a fixed list of conclusion patterns (e.g. 'The correct answer is...') and, for incorrect reasoning, removes words appearing in the ground-truth answer; Learning from Reasoning trains the VideoQA model with the multi-task loss $L_{\mathrm{mt}} = \alpha\,C_{\mathrm{qa}}(f(v,q), a) + \beta\,C_{\mathrm{rea}}(f(v,q), \hat{r})$, where $C_{\mathrm{qa}}$ is answer cross-entropy and $C_{\mathrm{rea}}$ is reasoned-text reconstruction cross-entropy. The refinement step is what lets the method decouple process from prediction, and the multi-task form is what lets imperfect reasoning help rather than hurt.

What would settle it

Train ReasVQA with a control set in which each refined reasoning trace has nouns referring to video objects randomly replaced by objects from a different video. If accuracy stays within one point of the reported 77.0/74.5/77.0, the improvement would be coming from the auxiliary language-modelling task rather than from the reasoning content. If accuracy drops sharply, the reasoning content is carrying the gain.

Watch

Extended reading notes

Core claim

The central discovery is that process supervision transfers from mathematical reasoning to video question answering. A teacher MLLM (InternVL) is asked to answer each question and explain itself; because the teacher is only about 67-69% accurate, a large share of the explanations end with wrong conclusions. The paper's refinement step removes the sentences that state conclusions and, for the wrong-answer cases, deletes words from the ground-truth answer, leaving an imperfect chain of observations and eliminations. Multi-task training on ground-truth answers plus reconstruction of those cleaned chains improves the student model consistently, and using all refined data, correct and incorrect, outperforms using only correct reasoning, whereas single-task training on unrefined reasoning degrades performance.

Load-bearing premise

The load-bearing premise is that after removing conclusion sentences and ground-truth words, the leftover reasoning text teaches the student true facts about the video instead of quietly reinforcing the teacher's wrong conclusions.

Editorial extensions

If this is right

  • A 3B FlanT5 student trained with ReasVQA beats 7B-parameter LLM-based systems on NExT-QA, STAR, and IntentQA, suggesting reasoning supervision can compensate for model scale.
  • After refinement, adding all MLLM reasoning, including incorrect traces, helps more than using only correct reasoning, implying that useful process information survives wrong final answers.
  • The gain transfers across architectures: ReasVQA raises LLaVA-OV 0.5B and 7B by +2.5 and +2.3 average accuracy over three tasks.
  • On the ATP-hard subset of NExT-QA, the method reaches 67.7% total accuracy, 9.3 points above VideoAgent, indicating that multi-frame causal and temporal reasoning improves rather than a single-frame shortcut.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The refinement step uses a short hand-crafted list of conclusion phrases; a natural extension is learned or LLM-based conclusion detection, which would likely catch paraphrases the fixed list misses.
  • The paper's beta-tuning curve plateaus between 0.5 and 0.8; making the reasoning weight sample-adaptive, for instance lower when the teacher is less confident, is a testable way to push the method further.
  • Because teacher accuracy caps at about 69%, the method's ceiling looks tied to teacher coverage; ensembling multiple MLLMs as generators is a direct way to test whether richer correct-reasoning pools yield larger gains.
  • The same recipe could be applied to other multimodal tasks with cheap rationales, such as captioning or embodied question answering, where process supervision is currently rare.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes ReasVQA, a three-phase method that uses MLLM-generated reasoning as auxiliary supervision for a smaller VideoQA model. Phase RG generates free-text reasoning plus a predicted answer from InternVL; Phase RR filters the reasoning by removing sentences that match a fixed list of conclusion patterns and, for incorrect reasoning, removes words containing the ground-truth answer; Phase LR trains the model with a weighted sum of a QA loss and a reasoning-generation loss. Experiments on NExT-QA, STAR, and IntentQA report state-of-the-art accuracies of 77.0%, 74.5%, and 77.0%, respectively, and ablations compare refined vs. original reasoning, single-task vs. multi-task learning, different reasoning generators, and different backbone models.

Significance. If the central claim were fully supported, the paper would offer a simple and transferable recipe for using imperfect MLLM reasoning as process supervision in VideoQA. The ablations are well designed: they isolate the effect of refinement, compare correct vs. all reasoning data, and test the method across two backbone families and two MLLM generators. The multi-task setup with a reasoning-generation auxiliary loss is clean and likely useful in practice. However, the paper's stated mechanism for refinement is contradicted by its own example in Figure 2, and the abstract overstates the margins over prior state of the art. The core empirical finding may survive, but the interpretative claim needs substantial revision.

major comments (3)
  1. [Section 3.2, Appendix A.4, Figure 2] The refinement step is described as retaining "only the essential steps that do not include any conclusion" and removing "the sentences containing answers," but the paper's own Refined Response in Figure 2 still begins with "In the video, the person is seen tidying up a closet or cabinet," which asserts the incorrect predicted option C. None of the ten fixed patterns in Appendix A.4 (e.g., "The correct answer ...", "Therefore, the correct answer is ...") matches this sentence, because the wrong conclusion is stated without the meta-phrase "correct answer." This is not a rare paraphrase; it is the paper's own illustration of the method. Consequently, the claimed decoupling of reasoning steps from erroneous conclusions is not established by the implementation as described. The original-vs-refined comparisons in Figure 3 and Table 10 cannot be read as evidence for "removing conclusions"; they may reflect changes in text length, removal of the final meta-text, or deletion of the ground-truth word. Please either implement a genuinely conclusion-removing filter (for example, removing option letters and option texts and detecting answer assertions beyond the fixed patterns), or reframe the method and the experimental claims as partial filtering and reconsider what the refinement comparison demonstrates.
  2. [Abstract and Section 4.2] The headline gains "+2.9 on NExT-QA, +7.3 on STAR" are presented as establishing new state-of-the-art performance, but the text in Section 4.2 clarifies that these gains are computed against other 3B FlanT5 methods. Against the best prior models in Tables 1 and 2, the actual margins are +1.0 on NExT-QA (77.0 vs. MotionEpic 76.0), +3.5 on STAR (74.5 vs. MotionEpic 71.0), and +5.9 on IntentQA (77.0 vs. LVNet 71.1). The abstract should state the comparison class explicitly, or the claims should be revised so that the reported improvements are not attributable to a weaker baseline class.
  3. [Section 3.2 and Limitations] The paper acknowledges in the Limitations section that "incorrect or biased reasoning produced by these MLLMs can still negatively affect the overall performance," but it never quantifies how often the refinement actually fails to remove a wrong conclusion. Given that Figure 2 shows a failure on the paper's own example, the reader has no basis to estimate the prevalence of retained wrong conclusions across the training set. Please add a post-refinement audit (for example, the fraction of refined responses that still assert an answer option) and report how this correlates with the observed gains. Without such an audit, the central mechanism that imperfect reasoning can be converted into conclusion-free process supervision remains unsupported.
minor comments (5)
  1. [Section 4.1 and Section 4.2] There are typos in section headings and body text: "Implemententation" in Section 4.1, "Reasults" in Section 4.2, "abltion" near the end of Section 4.4, and "Anylysis" in Appendix B.
  2. [Section 4.4] The text refers to "Equation 3.3" when discussing the multi-task loss; this should be Eq. (2).
  3. [Table 5 discussion] The sentence "performance drops significantly by +1.6 (73.2% vs. 71.6%)" uses a plus sign for a drop; it should be -1.6 or "drops by 1.6".
  4. [Appendix A.2] The paper states that code "will be later open sourced" but provides no link or release of the generated reasoning data. Given that the reasoning data is a core contribution, please provide at least the generated reasoning data or a detailed implementation of the refinement filter to enable replication.
  5. [Table 3] The table renders the model name as "LLaV A-OV" with a space; this should be consistent with the text's "LLaVA-OV".

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: reasoning supervision is generated by an external MLLM and evaluated on held-out benchmarks.

full rationale

The paper's derivation chain is empirically self-contained. The auxiliary supervision signal r is produced by InternVL, an off-the-shelf external MLLM, under a zero-shot prompt (Appendix A.3), and is refined by keyword-pattern deletion plus removal of ground-truth words (Section 3.2, Appendix A.4). The refined target is therefore not defined in terms of the trained model f or the reported evaluation accuracies. Training uses a weighted sum of two cross-entropy losses (Eq. 2): one against the dataset's ground-truth answer a and one against the refined reasoning target. Neither loss contains the benchmark accuracy being reported; the claims in Tables 1-3 are measured on held-out validation splits of NExT-QA, STAR, and IntentQA against external baselines. No equation reduces by construction to a fitted parameter, and no load-bearing premise is justified by a self-citation. The citations to the authors' prior work (Liang et al. 2024; Wang et al. 2024) appear only as related-work context or as baselines that ReasVQA outperforms, so they do not carry the argument. The limitation statement that incorrect or biased MLLM reasoning can still hurt performance is an empirical caveat, not a circular step. The concern that the refinement implementation may fail to remove all conclusion-like sentences (as in Figure 2, where the refined response retains 'tidying up a closet or cabinet') is a correctness or validity problem with the filtering procedure, not a circularity: the target is still an external, independently generated text, not the model's own prediction or the evaluation metric. A score of 0 is therefore appropriate.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities. It relies on a few free parameters (loss weights, frame count, keyword list) and on domain assumptions about the usefulness of imperfect reasoning and the sufficiency of keyword-based filtering. The main contributor of inductive bias is the keyword refinement list, which is ad hoc to this paper.

free parameters (4)
  • alpha (α) = 1 - β, default 0.5
    Weight for the QA loss in the multi-task objective (Eq. 2). Set by convention and varied only through β.
  • beta (β) = 0.5 default; 0.8 reported as peak in Figure 4
    Weight for the reasoning-generation loss in Eq. 2. Tuned on the benchmarks and directly affects the reported accuracy.
  • N sampled frames for reasoning generation = 4
    Number of uniformly sampled frames fed to InternVL. Chosen for faster generation, not otherwise motivated.
  • Keyword list for reasoning refinement = 10 fixed patterns (Appendix A.4)
    Hand-crafted patterns used to remove conclusion sentences. The exact list is an ad hoc design choice that determines the quality of the refined supervision.
assumptions (4)
  • domain assumption MLLM-generated reasoning text remains informative even when the final predicted answer is wrong.
    Central premise of the paper (Section 3.2). The paper asserts that incorrect reasoning still contains valuable partial observations, supported only indirectly by ablations in Table 5 and Figure 3.
  • ad hoc to paper Keyword-based removal of conclusion sentences and ground-truth words is sufficient to decouple reasoning steps from erroneous conclusions.
    The refinement relies on a fixed list of patterns (Appendix A.4). If a reasoning text states its conclusion in a way that does not match these patterns, the auxiliary target could leak the wrong answer or the correct answer.
  • domain assumption The ground-truth answer labels in the training sets are correct and the benchmarks' accuracy metric is a valid measure of VideoQA ability.
    Standard supervised-learning assumption; the labels are used both for the QA loss and for classifying and filtering the generated reasoning.
  • domain assumption Finetuning only the projection layers and LoRA weights of a 3B FlanT5, with a frozen ViT-G encoder, is sufficient to benefit from the auxiliary reasoning task.
    The paper does not experiment with unfreezing the visual encoder or full finetuning; all conclusions are scoped to this training setup.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReasVQA: Advancing VideoQA with Imperfect Reasoning Process." pith.science (2026). https://pith.science/paper/PYI7MSHD

@misc{pith2026250113536,
  author       = {Pith},
  title        = {Pith review of: ReasVQA: Advancing VideoQA with Imperfect Reasoning Process},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PYI7MSHD}},
  note         = {Machine review of arXiv:2501.13536}
}
read the original abstract

Video Question Answering (VideoQA) is a challenging task that requires understanding complex visual and temporal relationships within videos to answer questions accurately. In this work, we introduce \textbf{ReasVQA} (Reasoning-enhanced Video Question Answering), a novel approach that leverages reasoning processes generated by Multimodal Large Language Models (MLLMs) to improve the performance of VideoQA models. Our approach consists of three phases: reasoning generation, reasoning refinement, and learning from reasoning. First, we generate detailed reasoning processes using additional MLLMs, and second refine them via a filtering step to ensure data quality. Finally, we use the reasoning data, which might be in an imperfect form, to guide the VideoQA model via multi-task learning, on how to interpret and answer questions based on a given video. We evaluate ReasVQA on three popular benchmarks, and our results establish new state-of-the-art performance with significant improvements of +2.9 on NExT-QA, +7.3 on STAR, and +5.9 on IntentQA. Our findings demonstrate the supervising benefits of integrating reasoning processes into VideoQA. Further studies validate each component of our method, also with different backbones and MLLMs, and again highlight the advantages of this simple but effective method. We offer a new perspective on enhancing VideoQA performance by utilizing advanced reasoning techniques, setting a new benchmark in this research field.

Figures

Figures reproduced from arXiv: 2501.13536 by the authors.

Figure 1
Figure 1. Overview of our method ReasVQA. ⃝1 Reasoning Generation: a SOTA MLLM is prompted to solve complex questions by generating detailed reasoning explanations. ⃝2 Reasoning Refinement: we process and refine the reasoning steps to alleviate the conflict with true answers. ⃝3 Learning from Reasoning (Multi-task Training): the refined reasoning steps are used to guide a model to improve its performance on the VideoQA tasks.… view at source ↗
Figure 2
Figure 2. An example of a response generated by an MLLM: the predicted [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 4
Figure 4. Hyper-parameter tuning for the weight β of the Reason￾ing Generation Loss Crea(·), with the corresponding α = 1 − β. Setting NExT-QA STAR IntentQA Avg. ↑ STLQA 75.4 71.0 73.3 73.2 STLCR 74.2 71.9 74.4 73.5 STLAll 73.3 70.6 70.8 71.6 MTLCR 76.2 72.9 76.1 75.1 MTLAll 76.8 73.2 77.0 75.7 [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 21 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Ziyi Bai, Ruiping Wang, and Xilin Chen. 2024. Glance and focus: Memory prompting for multi-event video question answering. Advances in Neural Information Processing Systems, 36

    work page 2024

  5. [5]

    Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165

    arXiv 2020

  6. [6]

    Shyamal Buch, Cristobal Eyzaguirre, Adrien Gaidon, Jiajun Wu, Li Fei-Fei, and Juan Carlos Niebles. 2022. Revisiting the ``Video'' in Video-Language Understanding . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    work page 2022

  7. [7]

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. 2024 a . How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821

    arXiv 2024

  8. [8]

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. 2024 b . Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185--24198

    2024

Show all 62 references
  1. [9]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1--113

    2023

  2. [10]

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416

    2022 arXiv

  3. [11]

    Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2023. Eva: Exploring the limits of masked visual representation learning at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2023

  4. [12]

    Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Meishan Zhang, Mong-Li Lee, and Wynne Hsu. 2024. Video-of-thought: Step-by-step video reasoning from perception to cognition. In Forty-first International Conference on Machine Learning

    2024

  5. [13]

    Difei Gao, Luowei Zhou, Lei Ji, Linchao Zhu, Yi Yang, and Mike Zheng Shou. 2023. Mist: Multi-modal iterative spatial-temporal transformer for long-form video question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14773--14783

    2023

  6. [14]

    Jiaxian Guo, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Boyang Li, Dacheng Tao, and Steven Hoi. 2023. From images to textual prompts: Zero-shot visual question answering with frozen large language models. In Proceedings of the IEEE/CVF conference on computer vision and pat...

    2023

  7. [15]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  8. [16]

    Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim. 2017. Tgif-qa: Toward spatio-temporal reasoning in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2758--2766

    2017

  9. [17]

    Kumara Kahatapitiya, Kanchana Ranasinghe, Jongwoo Park, and Michael S Ryoo. 2024. Language repository for long video understanding. arXiv preprint arXiv:2403.14622

    2024 arXiv

  10. [18]

    Aisha Urooj Khan, Amir Mazaheri, Niels Da Vitoria Lobo, and Mubarak Shah. 2020. Mmft-bert: Multimodal fusion transformer with bert encodings for visual question answering. arXiv preprint arXiv:2010.14095

    2020 arXiv

  11. [19]

    Dohwan Ko, Ji Lee, Woo-Young Kang, Byungseok Roh, and Hyunwoo Kim. 2023. Large language models are temporal and causal reasoners for video question answering. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4300--4316

    2023

  12. [20]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199--22213

    2022

  13. [21]

    Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L Berg, Mohit Bansal, and Jingjing Liu. 2021. Less is more: Clipbert for video-and-language learning via sparse sampling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7331--7341

    2021

  14. [22]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326

    2024 arXiv

  15. [23]

    Jiapeng Li, Ping Wei, Wenjuan Han, and Lifeng Fan. 2023 a . Intentqa: Context-aware video intent reasoning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11963--11974

    2023

  16. [24]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023 b . Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597

    2023 arXiv

  17. [25]

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. 2023 c . Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355

    2023 arXiv

  18. [26]

    Jianxin Liang, Xiaojun Meng, Yueqian Wang, Chang Liu, Qun Liu, and Dongyan Zhao. 2024. End-to-end video question answering with frame scoring mechanisms and adaptive sampling. arXiv preprint arXiv:2407.15047

    2024 arXiv

  19. [27]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. arXiv preprint arXiv:2305.20050

    2023 arXiv

  20. [28]

    Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. 2023. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.10122

    2023 arXiv

  21. [29]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024. Visual instruction tuning. Advances in neural information processing systems, 36

    2024

  22. [30]

    Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. 2023. Egoschema: A diagnostic benchmark for very long-form video language understanding. Advances in Neural Information Processing Systems, 36:46212--46244

    2023

  23. [31]

    OpenAI. 2024 a . https://openai.com/research/gpt-4v Gpt-4o system card. https://cdn.openai.com/gpt-4o-system-card.pdf

    2024

  24. [32]

    OpenAI. 2024 b . https://openai.com/research/gpt-4v Gpt-4v(ision) system card . https://cdn.openai.com/papers/GPTV_System_Card.pdf

    2024

  25. [33]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

    2022

  26. [34]

    Junting Pan, Ziyi Lin, Yuying Ge, Xiatian Zhu, Renrui Zhang, Yi Wang, Yu Qiao, and Hongsheng Li. 2023. Retrieving-to-answer: Zero-shot video question answering with frozen large language models. arXiv preprint arXiv:2306.11732

    2023 arXiv

  27. [35]

    Jongwoo Park, Kanchana Ranasinghe, Kumara Kahatapitiya, Wonjeong Ryoo, Donghyun Kim, and Michael S Ryoo. 2024. Too many frames, not all useful: Efficient strategies for long-form video qa. arXiv preprint arXiv:2406.09396

    2024

  28. [36]

    Devshree Patel, Ratnam Parikh, and Yesha Shastri. 2021. Recent advances in video question answering: A review of datasets and methods. In Pattern Recognition. ICPR International Workshops and Challenges: Virtual Event, January 10--15, 2021, Proceedings, Part II, pages 339--356...

    2021

  29. [37]

    David Romero and Thamar Solorio. 2024. Question-instructed visual descriptions for zero-shot video question answering. arXiv preprint arXiv:2402.10698

    2024 arXiv

  30. [38]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805

    2023 arXiv

  31. [39]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  32. [40]

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275

    2022 arXiv

  33. [41]

    Jinpeng Wang, Yixiao Ge, Rui Yan, Yuying Ge, Kevin Qinghong Lin, Satoshi Tsutsui, Xudong Lin, Guanyu Cai, Jianping Wu, Ying Shan, et al. 2023 a . All in one: Exploring unified video-language pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2023

  34. [42]

    Shijie Wang, Qi Zhao, Minh Quan Do, Nakul Agarwal, Kwonjoon Lee, and Chen Sun. 2023 b . Vamos: Versatile action models for video understanding. arXiv preprint arXiv:2311.13627

    2023 arXiv

  35. [43]

    Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. 2025. Videoagent: Long-form video understanding with large language model as agent. In European Conference on Computer Vision, pages 58--76. Springer

    2025

  36. [44]

    Xijun Wang, Junbang Liang, Chun-Kai Wang, Kenan Deng, Yu Lou, Ming Lin, and Shan Yang. 2023 c . Vlap: Efficient video-language alignment via frame prompting and distilling for video question answering. arXiv preprint arXiv:2312.08367

    2023 arXiv

  37. [45]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022 a . Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171

    2022 arXiv

  38. [46]

    Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et al. 2022 b . Internvideo: General video foundation models via generative and discriminative learning. arXiv preprint arXiv:2212.03191

    2022 arXiv

  39. [47]

    Yuxuan Wang, Yueqian Wang, Pengfei Wu, Jianxin Liang, Dongyan Zhao, and Zilong Zheng. 2024. Lstp: Language-guided spatial-temporal prompt learning for long-form video-text understanding. arXiv preprint arXiv:2402.16050

    2024 arXiv

  40. [48]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  41. [49]

    Bo Wu, Shoubin Yu, Zhenfang Chen, Joshua B Tenenbaum, and Chuang Gan. 2021. Star: A benchmark for situated reasoning in real-world videos. In Thirty-fifth conference on neural information processing systems datasets and benchmarks track (Round 2)

    2021

  42. [50]

    Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. 2021. Next-qa: Next phase of question-answering to explaining temporal actions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9777--9786

    2021

  43. [51]

    Junbin Xiao, Angela Yao, Zhiyuan Liu, Yicong Li, Wei Ji, and Tat-Seng Chua. 2022 a . Video as conditional graph hierarchy for multi-granular question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 2804--2812

    2022

  44. [52]

    Junbin Xiao, Pan Zhou, Tat-Seng Chua, and Shuicheng Yan. 2022 b . Video graph transformer for video question answering. In European Conference on Computer Vision, pages 39--58. Springer

    2022

  45. [53]

    Dejing Xu, Zhou Zhao, Jun Xiao, Fei Wu, Hanwang Zhang, Xiangnan He, and Yueting Zhuang. 2017. Video question answering via gradually refined attention over appearance and motion. In Proceedings of the 25th ACM international conference on Multimedia, pages 1645--1653

    2017

  46. [54]

    Zenan Xu, Xiaojun Meng, Yasheng Wang, Qinliang Su, Zexuan Qiu, Xin Jiang, and Qun Liu. 2023. https://doi.org/10.24963/ijcai.2023/582 Learning summary-worthy visual representation for abstractive summarization in video . In Proceedings of the Thirty-Second International Joint C...

    2023 doi

  47. [55]

    Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, and Cordelia Schmid. 2021. Just ask: Learning to answer questions from millions of narrated videos. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1686--1697

    2021

  48. [56]

    Zekun Yang, Noa Garcia, Chenhui Chu, Mayu Otani, Yuta Nakashima, and Haruo Takemura. 2020. Bert representations for video question answering. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1556--1565

    2020

  49. [57]

    Qinghao Ye, Guohai Xu, Ming Yan, Haiyang Xu, Qi Qian, Ji Zhang, and Fei Huang. 2023. Hitea: Hierarchical temporal-aware video-language pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15405--15416

    2023

  50. [58]

    Shoubin Yu, Jaemin Cho, Prateek Yadav, and Mohit Bansal. 2023. Self-chained image-language model for video localization and question answering. arXiv preprint arXiv:2305.06988

    2023 arXiv

  51. [59]

    Andy Zeng, Maria Attarian, Brian Ichter, Krzysztof Choromanski, Adrian Wong, Stefan Welker, Federico Tombari, Aveek Purohit, Michael Ryoo, Vikas Sindhwani, et al. 2022. Socratic models: Composing zero-shot multimodal reasoning with language. arXiv preprint arXiv:2204.00598

    2022 arXiv

  52. [60]

    Ce Zhang, Taixi Lu, Md Mohaiminul Islam, Ziyang Wang, Shoubin Yu, Mohit Bansal, and Gedas Bertasius. 2023 a . A simple llm framework for long-range video question-answering. arXiv preprint arXiv:2312.17235

    2023 arXiv

  53. [61]

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2023 b . Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923

    2023 arXiv

  54. [62]

    Yaoyao Zhong, Junbin Xiao, Wei Ji, Yicong Li, Weihong Deng, and Tat-Seng Chua. 2022. Video question answering: Datasets, algorithms and challenges. In The 2022 Conference on Empirical Methods in Natural Language Processing

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.