REVIEW 3 major objections 126 references
Incentivizing Vision Language Models to Search for Long Video Question Answering
T0 review · 3 major / 0 minor · reviewed 2026-07-12 · grok-4.5
Pith's one-line read Vision-language models answer long-video questions better when RL rewards them for retrieving a checklist of verifiable visual primitives, not just for the final answer.
desk verdict Agentic NL search + RL is the real lift; VETL is a sensible but modest dense-reward add-on whose causal role the abstract overstates. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
VETL (Visual Evidence via Temporal Logic): an offline generate–judge loop that turns a question–answer pair into a temporal-logic formula over atomic visual primitives; the RL reward is then simply the fraction of those primitives that appear at least once in the agent’s retrieved clips (order ignored).
What would settle it
Train an otherwise identical agent with only exact-match answer reward on the same long-video suites; if Pass@1 and Pass@4 gains disappear (or reverse) on videos longer than ten minutes, the claim that the VETL evidence reward is what drives the improvement collapses.
Extended reading notes
Core claim
Post-training a vision-language agent with dense rewards derived from temporal-logic decompositions of the question (rather than answer accuracy alone) produces a policy that actively searches for the right visual evidence and therefore answers long-video questions more accurately and more sample-efficiently than outcome-only RL or passive frame sampling.
Load-bearing premise
That an LLM-written temporal-logic checklist plus imperfect visual detection of its atomic primitives is a faithful enough stand-in for “correct context” even though the reward discards ordering and the detector can be wrong.
Editorial extensions
If this is right
- Agentic long-video systems can be improved by shaping intermediate search steps rather than waiting for final-answer supervision.
- Natural-language search interfaces become learnable once a verifiable primitive checklist supplies dense feedback.
- Frame budgets can drop well below the usual fixed 64-frame sampling while accuracy still rises.
- The same neuro-symbolic reward recipe can be reused across GRPO, RLOO and related RL algorithms with only modest hyper-parameter retuning.
Reading between the lines
- If the primitive detector itself is weak, the reward becomes noisy; stronger open-vocabulary detectors or multi-modal verifiers would tighten the training signal without changing the overall architecture.
- Discarding temporal order inside the reward may be fine for non-chronological search, but questions that truly hinge on sequence could benefit from a later stage that re-introduces order constraints.
- The same checklist idea could be applied to other open-ended multimodal tools (audio, 3-D, long documents) wherever unit-test-style verification is currently missing.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents VSeek, an agentic multi-turn framework that equips a VLM with natural-language search, subtitle search, and summary tools over a ViCLIP-indexed long video, then post-trains the policy with GRPO (and variants) so that the model jointly issues targeted queries and answers. To supply denser rewards than exact-match answer accuracy alone, the authors introduce VETL: an offline LLM+judge loop that compiles each question–answer pair into a temporal-logic formula over atomic visual primitives; the online reward (Eq. 6) is the fraction of those primitives detected by max-pooling ViCLIP scores over the retrieved clips (temporal operators are discarded). On LongVideoBench, Video-MME and MLVU the resulting VSeek-VETL model improves Pass@1 by up to ~9 % and Pass@4 by ~13 % relative to the Qwen3-VL-4B-Thinking base, with further gains on longer videos and a substantial reduction in frames processed.
Significance. If the claimed gains hold under stricter controls, the work supplies a practical recipe for turning long-video QA into an RL-search problem and demonstrates that a neuro-symbolic checklist can serve as a dense, offline-verifiable reward. The open-sourced code, multi-algorithm ablations (GRPO/RLOO/GSPO), frame-efficiency numbers, and qualitative traces are concrete assets for the community. The core engineering insight—that natural-language search plus a bag-of-primitives reward already yields large lifts—is useful even if the temporal-logic machinery contributes only marginally.
major comments (3)
- Abstract and §1 attribute the headline “up to 8 % Pass@1 / 15 % Pass@4” gains to “explicitly optimizing for this verifiable evidence-seeking behavior” via VETL. Table 1 shows that the bulk of the lift already appears in VSeek-EM (exact-match reward only): +8.8 % / +8.7 % Pass@1 on LongVideoBench and MLVU versus the same base model. VSeek-VETL adds only +0.4 / +1.2 / +2.1 points on the three ID sets and is mixed or negative on OOD. The causal contribution of the neuro-symbolic reward is therefore overstated; the paper should either rephrase the claim or supply a controlled ablation that isolates VETL’s incremental effect after agentic search is already present.
- §4.3 and Eq. (6) explicitly discard the temporal operators that Algorithm 1 and the abstract emphasize (“along with their temporal ordering”). The reward reduces to a bag-of-primitives max-detection score. Table 5 (VSeek-Direct) shows that even a non-TL decomposition yields nearly identical numbers, confirming that the formal temporal structure is not load-bearing. Either restore order-sensitive verification or remove the repeated claim that temporal logic is essential for the reward.
- No error bars, statistical tests, or multiple random seeds are reported for any table. Given that the VETL–EM deltas are often <2 points, it is impossible to judge whether they are reliable. At minimum the main ID comparisons (Table 1) and the RL-algorithm sweep (Table 4) need standard deviations or bootstrap intervals.
Circularity Check
No derivation-level circularity: VETL rewards are computed offline from gold QA pairs via external LLM+judge and frozen ViCLIP detection; policy rollouts never observe gold answers or the reward model itself.
full rationale
The paper is an empirical methods contribution whose central results (Pass@k lifts on public LVQA benchmarks) are obtained by GRPO/RLOO/GSPO post-training of a VLM policy against a fixed, offline reward. Phase-1 TL specifications are generated once from (q, a_gold) by an external LLM+judge loop (Algorithm 1) and never updated by the policy; Phase-2 RVETL (Eq. 6) is a bag-of-primitives max-detection score under a frozen ViCLIP detector, deliberately discarding temporal order. The final scalar R = 0.7 Rem + 0.3 RVETL therefore supplies an independent training signal, exactly analogous to unit-test rewards in math/code RL. Empirical tables compare against the same base model under exact-match-only RL and against non-agentic baselines; no parameter is fitted on a subset and then re-presented as a prediction, no uniqueness theorem is imported from the authors to force the architecture, and no equation reduces to its own input by construction. Self-citations to the authors’ earlier neuro-symbolic video papers appear in Related Work and for the TL-generation prompt style, but they are not load-bearing for the measured accuracy numbers. The derivation chain is therefore self-contained against external benchmarks.
Assumptions & free parameters
free parameters (3)
- λ1, λ2 (reward mix) =
0.7 / 0.3
- κ (judge acceptance threshold) =
5
- max agent steps B, retriever top-k, clip length =
4 / 4 / 8s / 16
assumptions (3)
- domain assumption ViCLIP cosine similarity is a reliable binary detector Dp(ot) for the atomic visual primitives extracted from TL.
- ad hoc to paper Temporal order of evidence retrieval can be ignored; only presence of primitives matters.
- domain assumption GRPO (and RLOO/GSPO) with token masking of tool outputs yields stable policy improvement for multi-turn video search.
invented entities (2)
-
VETL (Visual Evidence via Temporal Logic) reward pipeline
-
VSeek multi-tool agent loop
Cite this review
Pith. "Pith review of Incentivizing Vision Language Models to Search for Long Video Question Answering." pith.science (2026). https://pith.science/paper/EOIPWEHJ
@misc{pith2026260702959,
author = {Pith},
title = {Pith review of: Incentivizing Vision Language Models to Search for Long Video Question Answering},
year = {2026},
howpublished = {\url{https://pith.science/paper/EOIPWEHJ}},
note = {Machine review of arXiv:2607.02959}
}
read the original abstract
We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process. VSeek utilizes a natural language-driven search to identify relevant context within long videos and is post-trained with reinforcement learning (RL) to jointly formulate targeted search queries and reason over retrieved clips for LVQA. While RL post-training has revolutionized reasoning in symbolic domains such as mathematics and code, its application to long-video understanding remains hindered by a lack of verified rewards. To ensure that the retrieved context is relevant, we propose a novel neuro-symbolic approach that bridges open-ended natural language with discrete visual verification. Specifically, complex user queries are compiled into formal temporal logic specifications for systematically decomposing natural language questions into a definitive checklist of required atomic visual primitives, such as key objects and activities, along with their temporal ordering. These systematically derived grounding events provide the critical feedback signal for RL post-training, enabling dense, verifiable rewards based on the successful retrieval of these specific visual elements rather than relying entirely on outcome-only answer accuracy. By explicitly optimizing for this verifiable evidence-seeking behavior, VSeek improves Pass@1 scores by up to 8% and Pass@4 scores by 15% on long-video understanding benchmarks compared to base models. We open-source our code at https://utaustin-swarmlab.github.io/VSeek.
Figures
Reference graph
Works this paper leans on
-
[1]
arXiv preprint arXiv:2303.08774 (2023)
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)
arXiv 2023
-
[2]
In: Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers)
Ahmadian, A., Cremer, C., Gall´ e, M., Fadaee, M., Kreutzer, J., Pietquin, O., ¨Ust¨ un, A., Hooker, S.: Back to basics: Revisiting reinforce-style optimization for learning from human feedback in llms. In: Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 12248–12267 (2024)
2024
-
[3]
arXiv preprint arXiv:1606.06565 (2016)
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Man´ e, D.: Con- crete problems in ai safety. arXiv preprint arXiv:1606.06565 (2016)
arXiv 2016
-
[4]
In: Proceedings of the IEEE International Con- ference on Computer Vision
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C.L., Parikh, D.: Vqa: Visual question answering. In: Proceedings of the IEEE International Con- ference on Computer Vision. pp. 2425–2433 (2015)
2015
-
[5]
arXiv preprint arXiv:2511.21631 (2025) 16 H
Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al.: Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 (2025) 16 H. Goel et al
arXiv 2025
-
[6]
Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., Zhong, H., Zhu, Y., Yang, M., Li, Z., Wan, J., Wang, P., Ding, W., Fu, Z., Xu, Y., Ye, J., Zhang, X., Xie, T., Cheng, Z., Zhang, H., Yang, Z., Xu, H., Lin, J.: Qwen2.5-vl technical report (2025),https://arxiv.org/abs/2502.13923
arXiv 2025
-
[7]
The MIT Press (2008)
Baier, C., Katoen, J.P.: Principles of Model Checking. The MIT Press (2008)
2008
-
[8]
Bertasius, G., Wang, H., Torresani, L.: Is space-time attention all you need for video understanding? (2021),https://arxiv.org/abs/2102.05095
arXiv 2021
Show all 126 references
-
[9]
Advances in Neural Information Processing Systems37, 53168–53197 (2025)
Chandrasegaran, K., Gupta, A., Hadzic, L.M., Kota, T., He, J., Eyzaguirre, C., Durante, Z., Li, M., Wu, J., Li, F.F.: Hourvideo: 1-hour video-language under- standing. Advances in Neural Information Processing Systems37, 53168–53197 (2025)
2025
-
[10]
arXiv preprint arXiv:2412.12075 (2024)
Chen, G., Liu, Y., Huang, Y., He, Y., Pei, B., Xu, J., Wang, Y., Lu, T., Wang, L.: Cg-bench: Clue-grounded question answering benchmark for long video under- standing. arXiv preprint arXiv:2412.12075 (2024)
2024 arXiv
-
[11]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., et al.: Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern reco...
2024
-
[12]
arXiv preprint arXiv:2205.01089 (2022)
Chen, Z., Yi, K., Li, Y., Ding, M., Torralba, A., Tenenbaum, J.B., Gan, C.: Com- phy: Compositional physical reasoning of objects and events from videos. arXiv preprint arXiv:2205.01089 (2022)
2022 arXiv
-
[13]
In: European Conference on Computer Vision
Choi, M., Goel, H., Omama, M., Yang, Y., Shah, S., Chinchali, S.: Towards neuro- symbolic video understanding. In: European Conference on Computer Vision. pp. 220–236. Springer (2024)
2024
-
[14]
arXiv preprint arXiv:2504.17180 (2025)
Choi, M., Sharan, S., Goel, H., Shah, S., Chinchali, S.: We’ll fix it in post: Im- proving text-to-video generation with neuro-symbolic feedback. arXiv preprint arXiv:2504.17180 (2025)
2025 arXiv
-
[15]
arXiv preprint arXiv:2110.14168 (2021)
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al.: Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021)
2021 arXiv
-
[16]
In: Handbook of Theoretical Com- puter Science, Volume B: Formal Models and Sematics (1991),https://api
Emerson, E.A.: Temporal and modal logic. In: Handbook of Theoretical Com- puter Science, Volume B: Formal Models and Sematics (1991),https://api. semanticscholar.org/CorpusID:6062082
1991
-
[17]
In: Proceedings of the IEEE/CVF international conference on computer vision
Feichtenhofer, C., Fan, H., Malik, J., He, K.: Slowfast networks for video recog- nition. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 6202–6211 (2019)
2019
-
[18]
In: IEEE Conference on Computer Vision and Pattern Recognition
Feichtenhofer, C., Pinz, A., Zisserman, A.: Convolutional two-stream network fu- sion for video action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 1933–1941. IEEE Computer Society, Las Vegas, NV, USA (2016)
1933
-
[19]
arXiv preprint arXiv:2503.21776 (2025)
Feng, K., Gong, K., Li, B., Guo, Z., Wang, Y., Peng, T., Wu, J., Zhang, X., Wang, B., Yue, X.: Video-r1: Reinforcing video reasoning in mllms. arXiv preprint arXiv:2503.21776 (2025)
2025 arXiv
-
[20]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Fu, C., Dai, Y., Luo, Y., Li, L., Ren, S., Zhang, R., Wang, Z., Zhou, C., Shen, Y., Zhang, M., et al.: Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern ...
2025
-
[21]
arXiv preprint arXiv:2605.01248 (2026) VSeek 17
Goel, H., Udathu, A., Jabireddy, S., Kalkar, P., Parulekar, A.: Sˆ 3-r1: Learn- ing to retrieve and answer step-by-step with synthetic data. arXiv preprint arXiv:2605.01248 (2026) VSeek 17
2026 arXiv
-
[22]
arXiv preprint arXiv:2501.12948 (2025)
Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al.: Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025)
2025 arXiv
-
[23]
arXiv preprint arXiv:2401.14196 (2024)
Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y., et al.: Deepseek-coder: when the large language model meets programming– the rise of code intelligence. arXiv preprint arXiv:2401.14196 (2024)
2024 arXiv
-
[24]
In: 2019 IEEE 58th conference on decision and control (CDC)
Hasanbeig, M., Kantaros, Y., Abate, A., Kroening, D., Pappas, G.J., Lee, I.: Re- inforcement learning for temporal logic control synthesis with probabilistic satis- faction guarantees. In: 2019 IEEE 58th conference on decision and control (CDC). pp. 5338–5343. IEEE (2019)
2019
-
[25]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
He, B., Li, H., Jang, Y.K., Jia, M., Cao, X., Shah, A., Shrivastava, A., Lim, S.N.: Ma-lmm: Memory-augmented large multimodal model for long-term video under- standing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 13504–13514 (2024)
2024
-
[26]
arXiv preprint arXiv:2509.24304 (2025)
He, Z., Qu, X., Li, Y., Huang, S., Liu, D., Cheng, Y.: Framethinker: Learn- ing to think with long videos via multi-turn frame spotlighting. arXiv preprint arXiv:2509.24304 (2025)
2025
-
[27]
URL https://www.trychroma.com/research/context-rot, retrieved October20, 2025 (2025)
Hong, K., Troynikov, A., Huber, J.: Context rot: How increasing input tokens impacts llm performance. URL https://www.trychroma.com/research/context-rot, retrieved October20, 2025 (2025)
2025
-
[28]
In: European conference on computer vision
Huang, Q., Xiong, Y., Rao, A., Wang, J., Lin, D.: Movienet: A holistic dataset for movie understanding. In: European conference on computer vision. pp. 709–727. Springer (2020)
2020
-
[29]
In: Proceed- ings of the Computer Vision and Pattern Recognition Conference
Islam, M.M., Nagarajan, T., Wang, H., Bertasius, G., Torresani, L.: Bimba: Selective-scan compression for long-range video question answering. In: Proceed- ings of the Computer Vision and Pattern Recognition Conference. pp. 29096–29107 (2025)
2025
-
[30]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Jang, Y., Song, Y., Yu, Y., Kim, Y., Kim, G.: Tgif-qa: Toward spatio-temporal reasoning in visual question answering. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2758–2766 (2017)
2017
-
[31]
arXiv preprint arXiv:2410.20252 (2024)
Jeoung, S., Huybrechts, G., Ganesh, B., Galstyan, A., Bodapati, S.: Adaptive video understanding agent: Enhancing efficiency with dynamic frame sampling and feedback-driven reasoning. arXiv preprint arXiv:2410.20252 (2024)
2024 arXiv
-
[32]
Journal of Automated Rea- soning60, 43–62 (2018)
Jha, S., Raman, V., Sadigh, D., Seshia, S.A.: Safe autonomy under perception uncertainty using chance-constrained temporal logic. Journal of Automated Rea- soning60, 43–62 (2018)
2018
-
[33]
Jiang, F., Yuan, J., Tsaftaris, S.A., Katsaggelos, A.K.: Anomalous video event detection using spatiotemporal context. Comput. Vis. Image Underst.115(3), 323– 333 (2011).https://doi.org/10.1016/J.CVIU.2010.10.008,https://doi.org/ 10.1016/j.cviu.2010.10.008
2011 doi
-
[34]
arXiv preprint arXiv:2503.09516 (2025)
Jin, B., Zeng, H., Yue, Z., Yoon, J., Arik, S., Wang, D., Zamani, H., Han, J.: Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516 (2025)
2025 arXiv
-
[35]
IEEE Transactions on Robotics25(6), 1370–1381 (2009).https://doi.org/10.1109/TRO.2009.2030225
Kress-Gazit, H., Fainekos, G.E., Pappas, G.J.: Temporal-logic-based reactive mis- sion and motion planning. IEEE Transactions on Robotics25(6), 1370–1381 (2009).https://doi.org/10.1109/TRO.2009.2030225
2009 doi
-
[36]
In: Golenkov, V., Krasno- proshin, V., Golovko, V., Shunkevich, D
Kroshchanka, A., Golovko, V., Mikhno, E., Kovalev, M., Zahariev, V., Zagorskij, A.: A neural-symbolic approach to computer vision. In: Golenkov, V., Krasno- proshin, V., Golovko, V., Shunkevich, D. (eds.) Open Semantic Technologies for Intelligent Systems. pp. 282–309. Springe...
2022
-
[37]
Advances in neural information processing systems 33, 9459–9474 (2020)
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K¨ uttler, H., Lewis, M., Yih, W.t., Rockt¨ aschel, T., et al.: Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33, 9459–9474 (2020)
2020
-
[38]
arXiv preprint arXiv:2408.03326 (2024)
Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Zhang, P., Li, Y., Liu, Z., et al.: Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326 (2024)
2024 arXiv
-
[39]
Neurocomputing490, 482–494 (2022)
Li, N., Chang, F., Liu, C.: Human-related anomalous event detection via spatial- temporal graph convolutional autoencoder with embedded long short-term memory network. Neurocomputing490, 482–494 (2022)
2022
-
[40]
arXiv preprint arXiv:2504.06958 (2025)
Li, X., Yan, Z., Meng, D., Dong, L., Zeng, X., He, Y., Wang, Y., Qiao, Y., Wang, Y., Wang, L.: Videochat-r1: Enhancing spatio-temporal perception via reinforcement fine-tuning. arXiv preprint arXiv:2504.06958 (2025)
2025 arXiv
-
[41]
arXiv preprint arXiv:2602.23553 (2026)
Liang, S., Shah, S., Zhou, C., Sharan, S., Goel, H., Sanyal, A., Chinchali, S., Datta, G.: Le-neus: Latency-efficient neuro-symbolic video understanding via adaptive temporal verification. arXiv preprint arXiv:2602.23553 (2026)
2026
-
[42]
In: European conference on computer vision
Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Jiang, Q., Li, C., Yang, J., Su, H., et al.: Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In: European conference on computer vision. pp. 38–55. Springer (2024)
2024
-
[43]
IEEE Transactions on Pattern Analysis and Machine Intelligence23(8), 873–889 (2001).https://doi.org/10.1109/34.946990
Medioni, G., Cohen, I., Bremond, F., Hongeng, S., Nevatia, R.: Event detection and analysis from video streams. IEEE Transactions on Pattern Analysis and Machine Intelligence23(8), 873–889 (2001).https://doi.org/10.1109/34.946990
2001 doi
-
[44]
Automatica152, 110692 (2023).https://doi.org/https://doi.org/10.1016/j
Mehdipour, N., Althoff, M., Tebbens, R.D., Belta, C.: Formal methods to comply with rules of the road in autonomous driving: State of the art and grand challenges. Automatica152, 110692 (2023).https://doi.org/https://doi.org/10.1016/j. automatica.2022.110692,https://www.scienc...
2023 doi
-
[45]
arXiv preprint arXiv:2511.18701 (2025)
Munir, M., Goel, H., Wei, X., Choi, M., Shah, S., Bhardwaj, K., Whatmough, P., Chinchali, S., Marculescu, R.: Objectalign: Neuro-symbolic object consistency verification and correction. arXiv preprint arXiv:2511.18701 (2025)
2025
-
[46]
arXiv preprint arXiv:2406.07400 (2024)
Murphy, W., Holzer, N., Koenig, N., Cui, L., Rothkopf, R., Qiao, F., Santolucito, M.: Guiding llm temporal logic generation with explicit separation of data and control. arXiv preprint arXiv:2406.07400 (2024)
2024 arXiv
-
[47]
arXiv preprint arXiv:2511.05489 (2025)
Pan, J., Zhang, Q., Zhang, R., Lu, M., Wan, X., Zhang, Y., Liu, C., She, Q.: Timesearch-r: Adaptive temporal search for long-form video understanding via self-verification reinforcement learning. arXiv preprint arXiv:2511.05489 (2025)
2025
-
[48]
18th Annual Symposium on Founda- tions of Computer Science (FOCS) pp
Pnueli, A.: The temporal logic of programs. 18th Annual Symposium on Founda- tions of Computer Science (FOCS) pp. 46–57 (1977)
1977
-
[49]
In: International conference on machine learning
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PMLR (2021)
2021
-
[50]
arXiv preprint arXiv:2308.12950 (2023)
Roziere, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X.E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al.: Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950 (2023)
2023 arXiv
-
[51]
In: CoCo@ NIPS (2015)
Sarkar, S., Lore, K.G., Sarkar, S.: Early detection of combustion instability by neural-symbolic analysis on hi-speed video. In: CoCo@ NIPS (2015)
2015
-
[52]
Advances in neural information processing systems36, 68539–68551 (2023)
Schick, T., Dwivedi-Yu, J., Dess` ı, R., Raileanu, R., Lomeli, M., Hambro, E., Zettle- moyer, L., Cancedda, N., Scialom, T.: Toolformer: Language models can teach VSeek 19 themselves to use tools. Advances in neural information processing systems36, 68539–68551 (2023)
2023
-
[53]
arXiv preprint arXiv:1707.06347 (2017)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)
2017 arXiv
-
[54]
arXiv preprint arXiv:2505.13851 (2025)
Shah, S., Goel, H., Narasimhan, S.S., Choi, M., Sharan, S., Akcin, O., Chinchali, S.: A challenge to build neuro-symbolic video agents. arXiv preprint arXiv:2505.13851 (2025)
2025 arXiv
-
[55]
Proceedings of the AAAI Conference on Artifi- cial Intelligence40(11), 8805–8813 (Mar 2026).https://doi.org/10.1609/aaai
Shah, S., Sharan, S.P., Goel, H., Choi, M., Munir, M., Pasula, M., Marculescu, R., Chinchali, S.: Neus-qa: Grounding long-form video understanding in temporal logic and neuro-symbolic reasoning. Proceedings of the AAAI Conference on Artifi- cial Intelligence40(11), 8805–8813 (...
2026 doi
-
[56]
arXiv preprint arXiv:2511.22570 (2025)
Shao, Z., Luo, Y., Lu, C., Ren, Z., Hu, J., Ye, T., Gou, Z., Ma, S., Zhang, X.: Deepseekmath-v2: Towards self-verifiable mathematical reasoning. arXiv preprint arXiv:2511.22570 (2025)
2025
-
[57]
arXiv preprint arXiv:2402.03300 (2024)
Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al.: Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024)
2024 arXiv
-
[58]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Sharan, S., Choi, M., Shah, S., Goel, H., Omama, M., Chinchali, S.: Neuro-symbolic evaluation of text-to-video models using formal verification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 8395– 8405 (2025)
2025
-
[59]
IEEE Transactions on Pattern Analysis and Machine Intelligence (2025)
Song, E., Chai, W., Ye, T., Hwang, J.N., Li, X., Wang, G.: Moviechat+: Question- aware sparse memory for long video question answering. IEEE Transactions on Pattern Analysis and Machine Intelligence (2025)
2025
-
[60]
arXiv preprint arXiv:2503.05592 (2025)
Song, H., Jiang, J., Min, Y., Chen, J., Chen, Z., Zhao, W.X., Fang, L., Wen, J.R.: R1-searcher: Incentivizing the search capability in llms via reinforcement learning. arXiv preprint arXiv:2503.05592 (2025)
2025 arXiv
-
[61]
Tan, X., Luo, Y., Ye, Y., Liu, F., Cai, Z.: Allvb: All-in-one long video understanding benchmark (2025),https://arxiv.org/abs/2503.07298
2025 arXiv
-
[62]
In: Pro- ceedings of the IEEE conference on computer vision and pattern recognition
Tapaswi, M., Zhu, Y., Stiefelhagen, R., Torralba, A., Urtasun, R., Fidler, S.: Movieqa: Understanding stories in movies through question-answering. In: Pro- ceedings of the IEEE conference on computer vision and pattern recognition. pp. 4631–4640 (2016)
2016
-
[63]
arXiv preprint arXiv:2406.11409 (2024)
Team, C., Zhao, H., Hui, J., Howland, J., Nguyen, N., Zuo, S., Hu, A., Choquette- Choo, C.A., Shen, J., Kelley, J., et al.: Codegemma: Open code models based on gemma. arXiv preprint arXiv:2406.11409 (2024)
2024 arXiv
-
[64]
In: Proceedings of the IEEE/CVF international conference on computer vision
Tian, Y., Pang, G., Chen, Y., Singh, R., Verjans, J.W., Carneiro, G.: Weakly- supervised video anomaly detection with robust temporal feature magnitude learn- ing. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4975–4986 (2021)
2021
-
[65]
In: Proceedings of the IEEE/CVF international conference on computer vision
Tran, D., Wang, H., Torresani, L., Feiszli, M.: Video classification with channel- separated convolutional networks. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 5552–5561 (2019)
2019
-
[66]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Wang, W., He, Z., Hong, W., Cheng, Y., Zhang, X., Qi, J., Ding, M., Gu, X., Huang, S., Xu, B., et al.: Lvbench: An extreme long video understanding benchmark. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 22958–22967 (2025)
2025
-
[67]
In: European Conference on Computer Vision
Wang, X., Zhang, Y., Zohar, O., Yeung-Levy, S.: Videoagent: Long-form video understanding with large language model as agent. In: European Conference on Computer Vision. pp. 58–76. Springer (2024) 20 H. Goel et al
2024
-
[68]
arXiv preprint arXiv:2307.06942 (2023)
Wang, Y., He, Y., Li, Y., Li, K., Yu, J., Ma, X., Li, X., Chen, G., Chen, X., Wang, Y., et al.: Internvid: A large-scale video-text dataset for multimodal understanding and generation. arXiv preprint arXiv:2307.06942 (2023)
2023 arXiv
-
[69]
In: Proceedings of the Computer Vision and Pattern Recognition Confer- ence
Wang, Z., Yu, S., Stengel-Eskin, E., Yoon, J., Cheng, F., Bertasius, G., Bansal, M.: Videotree: Adaptive tree-based video representation for llm reasoning on long videos. In: Proceedings of the Computer Vision and Pattern Recognition Confer- ence. pp. 3272–3283 (2025)
2025
-
[70]
Advances in Neural Information Pro- cessing Systems37, 28828–28857 (2024)
Wu, H., Li, D., Chen, B., Li, J.: Longvideobench: A benchmark for long-context interleaved video-language understanding. Advances in Neural Information Pro- cessing Systems37, 28828–28857 (2024)
2024
-
[71]
Advances in Neural Information Processing Systems38, 90925–90955 (2026)
Xue, Z.S., Luo, R., Grauman, K.: Seeing the arrow of time in large multimodal models. Advances in Neural Information Processing Systems38, 90925–90955 (2026)
2026
-
[72]
arXiv preprint arXiv:2309.10171 (2023)
Yang, Y., Gaglione, J.R., Chinchali, S., Topcu, U.: Specification-driven video search via foundation models and formal verification. arXiv preprint arXiv:2309.10171 (2023)
2023 arXiv
-
[73]
In: The eleventh interna- tional conference on learning representations (2022)
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K.R., Cao, Y.: React: Synergizing reasoning and acting in language models. In: The eleventh interna- tional conference on learning representations (2022)
2022
-
[74]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Ye, J., Wang, Z., Sun, H., Chandrasegaran, K., Durante, Z., Eyzaguirre, C., Bisk, Y., Niebles, J.C., Adeli, E., Fei-Fei, L., et al.: Re-thinking temporal search for long-form video understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...
2025
-
[75]
In: Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval
Ye, Y., Zhao, Z., Li, Y., Chen, L., Xiao, J., Zhuang, Y.: Video question answering via attribute-augmented attention network learning. In: Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. pp. 829–832 (2017)
2017
-
[76]
In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Yu, D., Yang, B., Wei, Q., Li, A., Pan, S.: A probabilistic graphical model based on neural-symbolic reasoning for visual relationship detection. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 10599– 10608 (2022).https://doi.org/10.1109/CVP...
2022 doi
-
[77]
arXiv preprint arXiv:2506.10821 (2025)
Yuan, H., Liu, Z., Zhou, J., Qian, H., Shu, Y., Sebe, N., Wen, J.R., Dou, Z.: Video- explorer: Think with videos for agentic long-video understanding. arXiv preprint arXiv:2506.10821 (2025)
2025
-
[78]
arXiv preprint arXiv:2507.18071 (2025)
Zheng, C., Liu, S., Li, M., Chen, X.H., Yu, B., Gao, C., Dang, K., Liu, Y., Men, R., Yang, A., et al.: Group sequence policy optimization. arXiv preprint arXiv:2507.18071 (2025)
2025 arXiv
-
[79]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Zhou, J., Shu, Y., Zhao, B., Wu, B., Liang, Z., Xiao, S., Qin, M., Yang, X., Xiong, Y., Zhang, B., et al.: Mlvu: Benchmarking multi-task long video understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13691–13701 (2025)
2025
-
[80]
arXiv preprint arXiv:2310.01852 (2023)
Zhu, B., Lin, B., Ning, M., Yan, Y., Cui, J., Wang, H., Pang, Y., Jiang, W., Zhang, J., Li, Z., et al.: Languagebind: Extending video-language pretraining to n-modality by language-based semantic alignment. arXiv preprint arXiv:2310.01852 (2023)
2023 arXiv
-
[81]
Correct Answer
Zhu, Y., Groth, O., Bernstein, M., Fei-Fei, L.: Visual7w: Grounded question an- swering in images. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4995–5004 (2016) VSeek 21 A Appendix: Extra Results and Discussions A.1 Performance of Rewa...
2016
-
[82]
,→ ,→ ,→
Reason: You must write your concise reasoning of 100-200 words within <think>...</think> that first summarizes and analyzes the frames to answer the question or to decide if you need to write a search query for more information. ,→ ,→ ,→
-
[83]
Decide: Based on your reasoning, decide if you have enough information in the frames obtained so far to either search for more information or answer the question. ,→ ,→
-
[84]
Act: At each turn you must choose ONE of the following actions: - If you need to search for more information using language based search to find objects or actions, output <search>...</search>,→ - If you need to search for frames that contain a specific subtitle, output <searc...
-
[85]
thinking
Pouring flour 2. Adding ingredients 3. Pouring flour,→ <think>I should first locate where the chef uses a mixing bowl.</think>,→ <search>a chef with a large mixing bowl</search> tool (After the search tool call, the agent receives frames of the chef placing an empty bowl on th...
-
[86]
First think very concisely about the question and the provided images within the <think> and </think> tags.,→
-
[87]
You must provide the final answer the question with the correct option after ###.,→
-
[88]
Do not provide empty fields and you must provide an answer to the best of your ability.,→
-
[89]
<think>I have found the mixing bowl, but no ingredients have been added yet
For example. <think>I have found the mixing bowl, but no ingredients have been added yet. I need to find the next action where something is put into the bowl. I found it, based on the question, the action is mixing, hence the answer option is 3.</think> ,→ ,→ ,→ ### 3 C Append...
-
[90]
A person in a blue swimsuit jumps into the water
-
[91]
A person in a black swimsuit jumps into the water
-
[92]
Two fully armed people walk on a forest path
-
[93]
The question is about what happens next after a man is surfing on a light blue surfboard with white waves
A person in a swimsuit walks on the beach Continued on next page VSeek 31 T urn / Field Content Original Video Turn 1Reasoning/output:<think>Okay, let’s try to figure out this problem. The question is about what happens next after a man is surfing on a light blue surfboard wit...
-
[94]
Alternatively, maybe the person is wearing a black swimsuit. So the correct answer would be 1.</think> <search>person in black swimsuit</search> Video Search T ool Response: Turn 3Reasoning/output:<think>Now that we’ve confirmed the person is in a black swimsuit, which matches...
-
[95]
The chef adds eggplant to the wok
-
[96]
The chef transfers the vegetables from the wok into a rectangular white plate containing bread
-
[97]
The chef adds scallions to the wok
-
[98]
The chef transfers the vegetables from the wok into a rectangular white plate containing rice
-
[99]
Original Video Turn 1Reasoning/output:<think>Okay, let’s try to figure out this problem
The chef transfers the vegetables from the wok into a round white plate containing rice. Original Video Turn 1Reasoning/output:<think>Okay, let’s try to figure out this problem. The question is about what happens after a chef pours a yellow liquid into a wok with stir-fried ve...
-
[100]
So the correct answer is either 3 or 4
Let me check the options again. So the correct answer is either 3 or 4. Based on the possible frames, the correct one is 3 or 4. So the answer should be 3 or 4. Lets summarize. Reason: After pouring the yellow liquid, the next step is transferring the vegetables to a plate. Th...
-
[101]
He has both hands raised above his head
-
[102]
He has both hands crossed in front of his chest
-
[103]
He is doing the V-sign with both hands
-
[104]
So the question is about a man in a room with a bookshelf, wallpaper, and he’s wearing a red short-sleeve shirt sitting on a gaming chair
He is doing the V-sign with one hand and clenching a fist with the other Original Video Turn 1Reasoning/output:<think>Okay, let’s try to figure out this problem. So the question is about a man in a room with a bookshelf, wallpaper, and he’s wearing a red short-sleeve shirt sit...
-
[105]
The options are given as 0, 1, 2, 3, each with different sequences
2→4→1→3 Original Video Continued on next page VSeek 35 T urn / Field Content Turn 1Reasoning/output:<think>Okay, let’s try to figure out the correct chrono- logical order of the events from the video. The options are given as 0, 1, 2, 3, each with different sequences. First, l...
-
[106]
So 4,2,1,3→option 2
Alternatively, maybe the people are standing (4) first, then the man talks (2), then he’s on the lake (1), then kayaking (3). So 4,2,1,3→option 2. So which is correct? Let me think again. If the man is talking (2), then he’s on the lake side (1), which is part of the same scen...
-
[107]
cooking sausages→baking cookies→milking cow→making jewelry
-
[108]
making jewelry→cooking sausages→milking cow→baking cookies
-
[109]
cooking sausages→making jewelry→baking cookies→milking cow
-
[110]
The user is asking which option correctly matches the sequence of actions from the video
baking cookies→making jewelry→milking cow→cooking sausages Original Video Continued on next page VSeek 37 T urn / Field Content Turn 1Reasoning/output:<think>Okay, let’s try to figure out this problem. The user is asking which option correctly matches the sequence of actions f...
-
[111]
The black-haired woman in the white jacket
-
[112]
The black-haired woman in the olive-green jacket
-
[113]
The man in the black shirt
-
[114]
The woman in the olive-green trench coat
-
[115]
So, the question is asking which person in the scene has their head slightly bowed and is smiling
The woman in the black dress Original Video Turn 1Reasoning/output:<think>Okay, let’s try to figure out this problem. So, the question is asking which person in the scene has their head slightly bowed and is smiling. The options are 0 to 4. First, let’s break down the given in...
-
[116]
Continued on next page VSeek 39 T urn / Field Content Alternatively, the buyer (option 3) could be smiling
is wearing black, but maybe she’s the one. Continued on next page VSeek 39 T urn / Field Content Alternatively, the buyer (option 3) could be smiling. But the description says the woman buying coffee is in the olive-green trench coat, which is option
-
[117]
Wait, but the user says the woman at the register is wearing black clothes, so that’s option
Alternatively, maybe the woman in the black dress (option 4) is the one at the register, and she’s the one with the bowed head. Wait, but the user says the woman at the register is wearing black clothes, so that’s option
-
[118]
Let me check again
Alternatively, perhaps the black-haired woman in the olive-green jacket (option 1) is the one. Let me check again. Alternatively, maybe the person is the one in the white jacket (option 0), but the description says the woman at the register is in black, and the buyer is in oli...
-
[119]
A doll wearing purple clothes
-
[120]
A yellow incense burner
-
[121]
A white chrysanthemum
-
[122]
The user is asking which object is present in the scene where there’s white text and a man in a black suit with a triangular shelf and a basketball
A white dress Original Video Turn 1Reasoning/output:<think>Okay, let’s try to figure out this problem. The user is asking which object is present in the scene where there’s white text and a man in a black suit with a triangular shelf and a basketball. First, the options are 0 ...
-
[123]
The user is asking for the type of movie the scene is in, with options 0 (Science Fiction), 1 (Comedy), 2 (Horror), 3 (Documentary)
Documentary Original Video Turn 1Reasoning/output:<think>Okay, let’s try to figure out this problem. The user is asking for the type of movie the scene is in, with options 0 (Science Fiction), 1 (Comedy), 2 (Horror), 3 (Documentary). First, I need to determine which category t...
-
[124]
Besides the swimming pool, there is also a wooden dining table and some chairs, appears later
Next to the swimming pool, there are white sunshades and trees, and there are some people inside the pool, appears first; A man wearing a blue short- sleeve shirt is taking pictures in front of the outdoor swimming pool. Besides the swimming pool, there is also a wooden dining...
-
[125]
Besides the swimming pool, there is also a wooden dining table and some chairs, appears later
Next to the swimming pool, there are white sunshades and trees, and there are some people inside the pool, appears first; A man wearing a gray short- sleeve shirt is taking pictures in front of the outdoor swimming pool. Besides the swimming pool, there is also a wooden dining...
-
[126]
Besides the swimming pool, there is also a wooden dining table and some chairs, appears first
A man wearing a gray short-sleeve shirt is taking pictures in front of the outdoor swimming pool. Besides the swimming pool, there is also a wooden dining table and some chairs, appears first. Next to the swimming pool, there are white sunshades and trees, and there are some p...
Reviewed July 12, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.