Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

A Challenge to Build Neuro-Symbolic Video Agents

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Proactive video agents will need deep learning plus temporal logic, not deep learning alone.

desk verdict A clearly written position paper that usefully packages the authors' prior neuro-symbolic video benchmarks as a community challenge, but the claim that video agents *need* neuro-symbolic methods is a research hypothesis, not an established result, and one internal contradiction about video generation needs correcting. read the letter →

arxiv 2505.13851 v1 pith:TCVDCB53 submitted 2025-05-20 cs.AI

Sahil Shah , Harsh Goel , Sai Shankar Narasimhan , Minkyu Choi , S P Sharan , Oguzhan Akcin , Sandeep Chinchali This is my paper · ORCID
classification cs.AI
keywords neuro-symbolicAIvideoagentstemporalreasoninglogicunderstandingtoolcallingtext-to-videogenerationformalverification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that proactive video agents—systems that watch a video stream, reason about what is happening, and act on it—cannot be built from deep learning alone. The obstacle is temporal reasoning: neural models recognize objects and short activities well, but they lose track of sequencing and dependencies across long videos. The authors' proposed route is neuro-symbolic: use neural perception to detect atomic events, then express the user's query as a temporal-logic specification and verify it against a state-machine model of the video. If the argument holds, future video systems will combine learned perception with formal reasoning, supported by new benchmarks that annotate not just events but also tool actions and their timing.

What carries the argument

The load-bearing mechanism is the decomposition of a video query into atomic events plus a temporal-logic specification, with the video represented as a probabilistic automaton whose states come from neural perception. Search then becomes a verification problem: does the automaton satisfy the user's temporal-logic query, and which frames correspond to the satisfying run? The same machinery is applied twice in the paper—NSVS-TL for search and NeuS-V for evaluating generated video—and it is the reason the approach can offer interpretability and formal guarantees while neural modules handle spatial perception.

What would settle it

A direct test would be a TLV-style benchmark of temporally structured search and tool-timing queries on which a purely learned video-language model—trained at scale and allowed long-context or memory mechanisms—matches the verification-based agent's precision and recall on queries with four or more ordered events. Alternatively, measuring whether NSVS-TL's performance degrades when its neural perception module outputs noisy atomic events would reveal how much of the claimed advantage comes from verification rather than perception.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that the next generation of video systems will be neuro-symbolic video agents built on three pillars: video search and understanding, integration of understanding with real-world action through tool calling, and video generation for testing and editing. For each pillar, the paper argues that purely learned foundation models fall short—NSVS-TL shows they struggle with temporally complex search queries, and NeuS-V shows text-to-video generation loses temporal fidelity as prompts grow more complex. The proposed remedy is to decompose a query into atomic events, translate it into temporal logic, map the video into a probabilistic automaton using off-the-shelf neural perception, and then treat search and evaluation as formal verification problems. The paper therefore issues a community challenge to build datasets, metrics, and agents that combine these neural and symbolic components.

Load-bearing premise

The load-bearing premise is that temporal reasoning over long videos is a bottleneck that scaling current deep learning cannot overcome; if learned models solve long-horizon temporal dependencies, the case for a neuro-symbolic architecture loses its necessity.

Editorial extensions

If this is right

  • Temporally structured video search can be treated as verification over perception-derived automata rather than as end-to-end pattern matching, which is how the paper's NSVS-TL demonstration outperforms foundation models on complex queries.
  • Text-to-video generation must be judged not only by visual quality but by temporal fidelity, with prompts converted to atomic-event specifications and satisfaction probabilities as in NeuS-V.
  • Video-agent datasets need frame-level event annotations plus the tools to be invoked and the temporal order of invocations; the TLV dataset supplies the first part and leaves the second as an open challenge.
  • Tool calling by a video agent must be evaluated for event-specific selection, temporal alignment with the prompt, and correctness of the invoked inputs, not just whether a plausible tool was chosen.
  • Video generation will serve dual roles: synthetic edge-case testing for agents and privacy-preserving editing for post-hoc analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the necessity claim holds, the reliability gap between learned and verified video agents should widen as query length and event count grow; the paper's Figure 3 data already suggest this for generation.
  • The same pipeline could extend beyond vision to audio triggers, multi-camera viewpoints, and multiple agents, but those settings would require distributed temporal-logic reasoning that the paper raises but does not solve.
  • A purely scaled learned model that matches neuro-symbolic performance on TLV-style temporal search and tool-timing would refute the necessity claim, so the challenge is falsifiable in principle.
  • Formal verification at the agent level could eventually give safety guarantees for home-security and autonomous-driving actions, but only if neural perception errors are tightly bounded; the paper's open questions acknowledge this dependency.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This position paper argues that the next generation of 'video agents'—systems that search and understand video, act in the real world, and generate synthetic video—must be built with neuro-symbolic architectures that combine deep perception with formal temporal-logic representations. The authors support this necessity claim primarily with two strands of evidence drawn from their own prior systems: NSVS-TL outperforms multimodal foundation models on complex video search (Figure 3a), and text-to-video models degrade in temporal fidelity as query complexity increases on the NeuS-V benchmark (Figure 3b). The paper then formalizes a grand challenge, specifying inputs, outputs, and metrics for video agents, and offers TLV as a preliminary dataset, F1 for event search, tool-calling accuracy, and VBench/NeuS-V for generation evaluation. It closes with open questions on multimodal triggers, choice of formal language, and multi-camera multi-agent settings.

Significance. If the central claim is accepted, the paper provides a useful community roadmap: it clearly names three pillars of video-agent capability, proposes concrete evaluation axes, and identifies temporal reasoning as a likely bottleneck. Its most valuable concrete contributions are the articulation of an evaluation protocol for tool-calling agents and the suggestion of combining visual-quality and temporal-fidelity metrics for generative video. The paper is also honest about the preliminary nature of the TLV dataset and explicitly invites the community to build better benchmarks. However, the significance is limited by the fact that the load-bearing necessity claim—that deep learning alone cannot achieve the needed temporal reasoning—is not established in this manuscript; the supporting evidence is drawn entirely from the authors' own benchmarks with no re-analysis, no learned temporal baselines, and no scaling analysis. The paper is best read as a research hypothesis and call to action rather than a demonstrated result.

major comments (4)
  1. [Section 2, final paragraph] The sentence 'video agents would need to be built via neuro-symbolic methods' is a categorical necessity claim, but the support in Figure 3 is a comparison of the authors' NSVS-TL and NeuS-V systems against general-purpose foundation models on benchmarks introduced by the same authors. There is no fine-tuned video-language temporal-grounding baseline, no compute-matched comparison, no scaling curve showing that the gap persists as model size or training data grows, and no error bars or significance testing. As written, the evidence supports 'currently outperform on these benchmarks,' not 'must be neuro-symbolic.' I recommend either softening the claim throughout Section 2 and the abstract or substantially expanding the evidence with learned baselines and scaling analysis.
  2. [Section 3, Video Generation paragraph] The sentence 'to our knowledge, neuro-symbolic methods for solving this problem have not been explored' is immediately followed by a citation to the authors' own Choi et al. 2025a, 'We'll fix it in post: improving text-to-video generation with neuro-symbolic feedback.' This is an internal inconsistency on a point that is partially load-bearing for the claimed gap. If the intended meaning is that no prior work has combined formal verification with the generative pipeline at training time, that distinction needs to be stated explicitly.
  3. [Figures 3(a) and 3(b), Section 6] The quantitative evidence that foundation models 'struggle' and 'deteriorate' is presented as two small panels without the evaluation protocol: which specific models were tested, how many prompts, how queries were varied in complexity, and what variance exists across runs. Because NeuS-V is both the proposed evaluation metric and part of the evidence for the neuro-symbolic approach, this creates a circularity concern. The manuscript would be strengthened by a short description of the evaluation setup or a reference to a version of the underlying benchmarks with this information, and by reporting confidence intervals.
  4. [Section 7] The architecture discussion assumes that neural perception can reliably produce atomic events for formal verification, but this reliability assumption is the soft underbelly of the entire approach. If perception is noisy at the atomic-event level, formal guarantees over the abstracted event sequence do not directly transfer to the video. The paper acknowledges the orchestration challenge only obliquely; a challenge paper on this topic should state the perception-verification gap as a first-class open problem, since it bears directly on the promised 'guarantees on system behavior.'
minor comments (5)
  1. [Section 5] The description of the TLV dataset as compiled from 'static images' from Waymo and NuScenes is confusing for a video-search benchmark; please clarify whether the evaluation actually exercises temporal video dynamics or whether temporal structure is imposed by the specification-pairing procedure. This will affect how readers interpret the claimed temporal-reasoning bottleneck.
  2. [Section 3, Video Agents paragraph] Typo: 'long-from video understanding' should be 'long-form video understanding.'
  3. [Section 4, Inputs/Tools] The text uses 'open-sourced API's' with an apostrophe; use 'APIs.'
  4. [References, Sharan et al. 2024] The reference title contains a typo: 'formalf verification' should be 'formal verification.'
  5. [Section 4, final paragraph] The paper states that data, benchmarks, and example agents are provided on GitHub, but no URL is given in the manuscript; please include the repository link.

Circularity Check

2 steps flagged · score 6.0 of 10

The Section 2 necessity claim is inferred from the authors' own TL-labeled benchmark (TLV) and TL-based metric (NeuS-V), so the neuro-symbolic conclusion is partly built into the evaluation rather than independently established.

  1. self definitional [Section 2 (Figure 3a) and Section 5 / Figure 4 (TLV dataset)]
    "This limitation is demonstrated with Neuro-Symbolic Video Search with Temporal Logic (NSVS-TL) (Choi et al., 2024). ... Consequently, NSVS-TL converts the video search problem into a verification problem and extracts the relevant clips that satisfy the TL specifications corresponding to the user query. ... this approach significantly outperforms multimodal foundation models when faced with elaborate user queries, as shown in Figure 3(a)."

    The benchmark behind Figure 3(a) is the TLV dataset, presented in Section 5 / Figure 4 as a 'TLV Dataset Specification-Video Pairing' where the ground truth relates videos to TL sequences. The success criterion for a video-search query is therefore satisfaction of the query's TL specification, and NSVS-TL's retrieval procedure is exactly to extract clips satisfying that TL specification. The claimed deep-learning deficit is measured with labels and a metric that already encode the neuro-symbolic TL representation; the conclusion that video agents 'would need to be built via neuro-symbolic methods' restates the benchmark's definition of correctness rather than an independent empirical result.

  2. ansatz smuggled in via citation [Section 2 (Figure 3b) and Section 6 (NeuS-V metric)]
    "evaluations on benchmarks such as NeuS-V (Sharan et al., 2024) ... demonstrate a deficiency in temporal fidelity ... Finally, NeuS-V computes the satisfaction probability by formally verifying the video automaton against the TL specification to produce the final score. ... To mitigate this issue, we posit that generative models need to be co-designed with temporal understanding frameworks."

    NeuS-V is a neuro-symbolic metric from the same group: it parses the prompt into TL, scores atomic propositions with a VLM, and reports a formal TL satisfaction probability. Measuring text-to-video models with this TL-satisfaction yardstick and then concluding that the observed deficiency requires 'co-design with temporal understanding frameworks' imports the neuro-symbolic TL representation into the evidence itself. The failure mode is defined by the authors' prior TL-based metric, so the conclusion is partly an artifact of the evaluation construction rather than an independent fact about generative models.

full rationale

This is a position/challenge paper, not a formal derivation, and much of its content is a reasonable research agenda with independent components (tool calling, multimodal triggers, multi-camera coordination, VBench for visual quality). However, the load-bearing empirical support for the strongest claim—that video agents 'would need to be built via neuro-symbolic methods'—is self-referential. Figure 3(a) uses the authors' TLV benchmark, whose specification-video pairing defines temporal correctness in terms of TL satisfaction, and NSVS-TL's objective is exactly TL satisfaction; Figure 3(b) uses the authors' NeuS-V metric, which itself converts prompts to TL and formally verifies satisfaction. In both cases, the neuro-symbolic conclusion is partially built into the benchmark and metric rather than established by an independent test against learned temporal baselines, scaling analysis, or externally defined temporal fidelity. The paper also contains an internal inconsistency in Section 3, stating that neuro-symbolic methods for video generation 'have not been explored' immediately after citing the same authors' 2025a neuro-symbolic text-to-video feedback paper; this further weakens the evidence narrative, though it is a correctness issue rather than a circular step. On balance, the central claim reduces in part to the authors' own TL-based evaluation instruments, so a partial-circularity score of 6 is warranted; the remaining challenge content is independent and prevents a higher score.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No new entities or fitted parameters are introduced. The proposal is assembled from prior published artifacts, and the central claim rests on several domain assumptions, chiefly that neural perception is reliable enough for formal verification and that temporal logic is the right specification language.

assumptions (4)
  • domain assumption Off-the-shelf neural perception models can map video frames to reliable atomic propositions such as 'person detected' or 'package present'.
    The NSVS-TL pipeline described in Section 2 converts frames into states using an off-the-shelf perception model; Section 7 asks how to orchestrate neural models for multimodal triggers. The entire verification approach depends on this reliability.
  • domain assumption Temporal logic is expressive enough to capture the temporal constraints in natural language queries.
    Section 1 proposes TL as the symbolic representation for user queries, and Section 7 acknowledges alternatives such as LTL, STL, and state machines. The suitability is asserted, not derived.
  • domain assumption The NeuS-V metric and VBench together provide a valid evaluation of generated videos, with NeuS-V measuring temporal fidelity.
    Section 6 recommends VBench for visual quality and NeuS-V for temporal coherence as the evaluation standards for the challenge, with no validation of these metrics in this paper.
  • domain assumption The TLV dataset is representative of temporally complex video queries that require agentic responses.
    Section 5 presents TLV as the starting dataset for video search, while admitting it lacks tool-invocation annotations, so representativeness for the full challenge is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Challenge to Build Neuro-Symbolic Video Agents." pith.science (2026). https://pith.science/paper/TCVDCB53

@misc{pith2026250513851,
  author       = {Pith},
  title        = {Pith review of: A Challenge to Build Neuro-Symbolic Video Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TCVDCB53}},
  note         = {Machine review of arXiv:2505.13851}
}
read the original abstract

Modern video understanding systems excel at tasks such as scene classification, object detection, and short video retrieval. However, as video analysis becomes increasingly central to real-world applications, there is a growing need for proactive video agents for the systems that not only interpret video streams but also reason about events and take informed actions. A key obstacle in this direction is temporal reasoning: while deep learning models have made remarkable progress in recognizing patterns within individual frames or short clips, they struggle to understand the sequencing and dependencies of events over time, which is critical for action-driven decision-making. Addressing this limitation demands moving beyond conventional deep learning approaches. We posit that tackling this challenge requires a neuro-symbolic perspective, where video queries are decomposed into atomic events, structured into coherent sequences, and validated against temporal constraints. Such an approach can enhance interpretability, enable structured reasoning, and provide stronger guarantees on system behavior, all key properties for advancing trustworthy video agents. To this end, we present a grand challenge to the research community: developing the next generation of intelligent video agents that integrate three core capabilities: (1) autonomous video search and analysis, (2) seamless real-world interaction, and (3) advanced content generation. By addressing these pillars, we can transition from passive perception to intelligent video agents that reason, predict, and act, pushing the boundaries of video understanding.

Figures

Figures reproduced from arXiv: 2505.13851 by the authors.

Figure 1
Figure 1. An Efficient Neuro-Symbolic Approach to Video Agents. We argue for a neuro￾symbolic approach to develop video agents that combines the per-frame or short-horizon reasoning capabilities of neural perception models with the long-term reasoning abilities of symbolic frame￾works such as temporal logic tools. Here, we show one such example from a home security system, where the agent is required to identify the presence … view at source ↗
Figure 2
Figure 2. NSVS-TL and NeuS-V System Diagrams. In (a), when given video feed from a security system, NSVS-TL demonstrates its capability in identifying exactly when a delivery driver walks up the stairs and drops a package off. Similarly, in (b), a video generated by a foundation model describing a delivery scenario is evaluated for temporal fidelity through NeuS-V. significantly outperforms multimodal foundation models when f… view at source ↗
Figure 3
Figure 3. Foundation models struggle to perform video search and generation with increasing complexity of user queries. However, neuro-symbolic approaches (NSVS-TL) effectively decou￾ple spatial and temporal reasoning using perception modules for spatial reasoning and temporal logic (TL) to model long-term temporal dependencies. As a result, NSVS-TL outperforms foun￾dation models in complex video search tasks (a). Similarly, … view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: TLV Dataset Specification-Video Pairing. An excerpt of the TLV dataset is shown here, demonstrating the efficacy of the TLV dataset in showing the relationships between videos and TL sequences. This figure was taken with permission from Choi et al. (2024). The TLV data…
Figure 5
Figure 5. Figure 5: What is the Correct Action for the Agent? At a first glance, both agents summarize the video nearly identically. However, upon closer inspection, Agent 1, although more vague, correctly identifies the parked delivery truck and notifies the homeowner. In contrast, Agent…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Incentivizing Vision Language Models to Search for Long Video Question Answering

    cs.CV 2026-07 conditional novelty 7.0 of 10

    RL post-training of a VLM agent with neuro-symbolic temporal-logic rewards for evidence retrieval raises Pass@1 by up to 8% and Pass@4 by 15% on long-video QA.

  2. SGA: Plug&Play Geometric Verification for Educational Video Synthesis

    cs.AI 2026-07 conditional novelty 6.0 of 10

    An intercept-and-refine agent that symbolically checks Manim animation code for geometric collisions improves the authors' rendering-free MVQS layout score in 7 of 8 LLM×pipeline configurations, with the metric unvali...

Reference graph

Works this paper leans on

71 extracted references · 46 canonical work pages · cited by 2 Pith papers

  1. [1]

    In 2019 IEEE 58th conference on decision and control (CDC), pages 5338--5343

    Reinforcement learning for temporal logic control synthesis with probabilistic satisfaction guarantees. In 2019 IEEE 58th conference on decision and control (CDC), pages 5338--5343. IEEE, 2019

    work page 2019

  2. [3]

    Principles of Model Checking

    Christel Baier and Joost-Pieter Katoen. Principles of Model Checking. The MIT Press, 2008

    work page 2008

  3. [4]

    Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is space-time attention all you need for video understanding? In Marina Meila and Tong Zhang, editors, International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 813--824. PMLR , 2021

    work page 2021

  4. [5]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22563--22575, 2023

    work page 2023

  5. [6]

    Hourvideo: 1-hour video-language understanding

    Keshigeyan Chandrasegaran, Agrim Gupta, Lea M Hadzic, Taran Kota, Jimming He, Crist \'o bal Eyzaguirre, Zane Durante, Manling Li, Jiajun Wu, and Fei-Fei Li. Hourvideo: 1-hour video-language understanding. Advances in Neural Information Processing Systems, 37: 0 53168--53197, 2025

    work page 2025

  6. [7]

    Langchain, 2022

    Harrison Chase. Langchain, 2022. URL https://github.com/langchain-ai/langchain

    work page 2022

  7. [8]

    Comphy: Compositional physical reasoning of objects and events from videos

    Zhenfang Chen, Kexin Yi, Yunzhu Li, Mingyu Ding, Antonio Torralba, Joshua B Tenenbaum, and Chuang Gan. Comphy: Compositional physical reasoning of objects and events from videos. arXiv preprint arXiv:2205.01089, 2022

    arXiv 2022

  8. [9]

    Choudhary

    Yu Cheng, Quanfu Fan, Sharath Pankanti, and Alok N. Choudhary. Temporal sequence modeling for video event detection. In IEEE Conference on Computer Vision and Pattern Recognition , pages 2235--2242, Columbus, OH, USA, 2014. IEEE Computer Society

    work page 2014

Show all 71 references
  1. [10]

    Sora as an agi world model? a complete survey on text-to-video generation

    Joseph Cho, Fachrina Dewi Puspitasari, Sheng Zheng, Jingyao Zheng, Lik-Hang Lee, Tae-Ho Kim, Choong Seon Hong, and Chaoning Zhang. Sora as an agi world model? a complete survey on text-to-video generation. arXiv preprint arXiv:2403.05131, 2024

    2024

  2. [11]

    Towards neuro-symbolic video understanding

    Minkyu Choi, Harsh Goel, Mohammad Omama, Yunhao Yang, Sahil Shah, and Sandeep Chinchali. Towards neuro-symbolic video understanding. In European Conference on Computer Vision, pages 220--236. Springer, 2024

    2024

  3. [12]

    We'll fix it in post: Improving text-to-video generation with neuro-symbolic feedback

    Minkyu Choi, SP Sharan, Harsh Goel, Sahil Shah, and Sandeep Chinchali. We'll fix it in post: Improving text-to-video generation with neuro-symbolic feedback. arXiv preprint arXiv:2504.17180, 2025 a

    2025 arXiv

  4. [13]

    Real-time privacy preservation for robot visual perception

    Minkyu Choi, Yunhao Yang, Neel P Bhatt, Kushagra Gupta, Sahil Shah, Aditya Rai, David Fridovich-Keil, Ufuk Topcu, and Sandeep P Chinchali. Real-time privacy preservation for robot visual perception. arXiv preprint arXiv:2505.05519, 2025 b

    2025 arXiv

  5. [14]

    Parks research shows consumers are after integrated smart locks and security cameras

    Zachary Comeau. Parks research shows consumers are after integrated smart locks and security cameras. CEPro, September 2024. URL https://www.cepro.com/news/consumers-want-integrated-smart-locks-and-security-cameras/142262/

    2024

  6. [15]

    Towards interpretable video anomaly detection

    Keval Doshi and Yasin Yilmaz. Towards interpretable video anomaly detection. In IEEE/CVF Winter Conference on Applications of Computer Vision , pages 2654--2663, Waikoloa, HI, USA, 2023. IEEE

    2023

  7. [16]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7346--7356, 2023

    2023

  8. [17]

    Convolutional two-stream network fusion for video action recognition

    Christoph Feichtenhofer, Axel Pinz, and Andrew Zisserman. Convolutional two-stream network fusion for video action recognition. In IEEE Conference on Computer Vision and Pattern Recognition , pages 1933--1941, Las Vegas, NV, USA, 2016. IEEE Computer Society

    1933

  9. [18]

    Christoph Feichtenhofer, Axel Pinz, and Richard P. Wildes. Spatiotemporal multiplier networks for video action recognition. In IEEE Conference on Computer Vision and Pattern Recognition , pages 7445--7454, Honolulu, HI, USA, 2017. IEEE Computer Society

    2017

  10. [19]

    Slowfast networks for video recognition

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In IEEE/CVF International Conference on Computer Vision , pages 6201--6210. IEEE , 2019

    2019

  11. [20]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF conference on computer vision ...

    2022

  12. [21]

    The efficacy of neural planning metrics: A meta-analysis of pkl on nuscenes

    Yiluan Guo, Holger Caesar, Beijbom Oscar, Jonah Philion, and Sanja Fidler. The efficacy of neural planning metrics: A meta-analysis of pkl on nuscenes. In IROS 2020 Workshop on Benchmarking Progress in Autonomous Driving, 2020

    2020

  13. [22]

    Stabletoolbench: Towards stable large-scale benchmarking on tool learning of large language models, 2024

    Zhicheng Guo, Sijie Cheng, Hao Wang, Shihao Liang, Yujia Qin, Peng Li, Zhiyuan Liu, Maosong Sun, and Yang Liu. Stabletoolbench: Towards stable large-scale benchmarking on tool learning of large language models, 2024

    2024

  14. [23]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 0 6840--6851, 2020

    2020

  15. [24]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868, 2022

    2022 arXiv

  16. [25]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat...

    2024

  17. [26]

    Adaptive video understanding agent: Enhancing efficiency with dynamic frame sampling and feedback-driven reasoning

    Sullam Jeoung, Goeric Huybrechts, Bhavana Ganesh, Aram Galstyan, and Sravan Bodapati. Adaptive video understanding agent: Enhancing efficiency with dynamic frame sampling and feedback-driven reasoning. arXiv preprint arXiv:2410.20252, 2024

    arXiv

  18. [27]

    Safe autonomy under perception uncertainty using chance-constrained temporal logic

    Susmit Jha, Vasumathi Raman, Dorsa Sadigh, and Sanjit A Seshia. Safe autonomy under perception uncertainty using chance-constrained temporal logic. Journal of Automated Reasoning, 60: 0 43--62, 2018

    2018

  19. [28]

    Tsaftaris, and Aggelos K

    Fan Jiang, Junsong Yuan, Sotirios A. Tsaftaris, and Aggelos K. Katsaggelos. Anomalous video event detection using spatiotemporal context. Comput. Vis. Image Underst., 115 0 (3): 0 323--333, 2011. doi:10.1016/J.CVIU.2010.10.008. URL https://doi.org/10.1016/j.cviu.2010.10.008

    2011 doi

  20. [29]

    Temporal-logic-based reactive mission and motion planning

    Hadas Kress-Gazit, Georgios E Fainekos, and George J Pappas. Temporal-logic-based reactive mission and motion planning. IEEE transactions on robotics, 25 0 (6): 0 1370--1381, 2009

    2009

  21. [30]

    A neural-symbolic approach to computer vision

    Aliaksandr Kroshchanka, Vladimir Golovko, Egor Mikhno, Mikhail Kovalev, Vadim Zahariev, and Aleksandr Zagorskij. A neural-symbolic approach to computer vision. In International Conference on Open Semantic Technologies for Intelligent Systems, pages 282--309. Springer, 2021

    2021

  22. [31]

    Pika ai: Free video generator with scene ingredients, 2024

    Pika Labs. Pika ai: Free video generator with scene ingredients, 2024. URL https://pikartai.com. Pika 2.1 documentation

    2024

  23. [32]

    Human-related anomalous event detection via spatial-temporal graph convolutional autoencoder with embedded long short-term memory network

    Nanjun Li, Faliang Chang, and Chunsheng Liu. Human-related anomalous event detection via spatial-temporal graph convolutional autoencoder with embedded long short-term memory network. Neurocomputing, 490: 0 482--494, 2022

    2022

  24. [33]

    Video-chatgpt: Towards detailed video understanding via large vision and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.05424, 2023

    2023 arXiv

  25. [34]

    Representation learning on visual-symbolic graphs for video understanding

    Effrosyni Mavroudi, Benjam \' n B \' e jar Haro, and Ren \' e Vidal. Representation learning on visual-symbolic graphs for video understanding. In European Conference on Computer Vision, volume 12374 of Lecture Notes in Computer Science, pages 71--90. Springer, 2020

    2020

  26. [35]

    Medioni, Isaac Cohen, Fran c ois Br \' e mond, Somboon Hongeng, and Ramakant Nevatia

    G \' e rard G. Medioni, Isaac Cohen, Fran c ois Br \' e mond, Somboon Hongeng, and Ramakant Nevatia. Event detection and analysis from video streams. IEEE Trans. Pattern Anal. Mach. Intell. , 23 0 (8): 0 873--889, 2001

    2001

  27. [36]

    Formal methods to comply with rules of the road in autonomous driving: State of the art and grand challenges

    Noushin Mehdipour, Matthias Althoff, Radboud Duintjer Tebbens, and Calin Belta. Formal methods to comply with rules of the road in autonomous driving: State of the art and grand challenges. Automatica, 152: 0 110692, 2023

    2023

  28. [37]

    8 must-follow social listening trends for 2025, January 2025

    Metricom . 8 must-follow social listening trends for 2025, January 2025. URL https://www.metricom.io/blog/social-listening-trends-2025

    2025

  29. [38]

    Learning audio-video modalities from image captions

    Arsha Nagrani, Paul Hongsuck Seo, Bryan Seybold, Anja Hauth, Santiago Manen, Chen Sun, and Cordelia Schmid. Learning audio-video modalities from image captions. In European Conference on Computer Vision, pages 407--426. Springer, 2022

    2022

  30. [39]

    Federal motor vehicle safety standards; automatic emergency braking systems for light vehicles

    National Highway Traffic Safety Administration . Federal motor vehicle safety standards; automatic emergency braking systems for light vehicles. Technical Report Docket No. NHTSA-2023-0021, U.S. Department of Transportation, April 2024. URL https://www.nhtsa.gov/sites/nhtsa.go...

    2023

  31. [40]

    Gpt-4 technical report

    OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  32. [41]

    Video generation models as world simulators, 2024

    OpenAI. Video generation models as world simulators, 2024. URL https://openai.com/sora/. Sora technical report

    2024

  33. [42]

    Patil, Tianjun Zhang, Xin Wang, and Joseph E

    Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:2305.15334, 2023

    2023 arXiv

  34. [43]

    Toolllm: Facilitating large language models to master 16000+ real-world apis, 2023

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. Toolllm: Facilitating large language models to master 16000+ ...

    2023

  35. [44]

    Rapidapi hub

    RapidAPI. Rapidapi hub. https://rapidapi.com. Accessed: 2025-03-02

    2025

  36. [45]

    Introducing gen-3 alpha: A new frontier for video generation, 2024

    Runway Research. Introducing gen-3 alpha: A new frontier for video generation, 2024. URL https://runwayml.com/research/introducing-gen-3-alpha. Runway Gen-3 technical report

    2024

  37. [46]

    Early detection of combustion instability by neural-symbolic analysis on hi-speed video

    Soumalya Sarkar, Kin Gwn Lore, and Soumik Sarkar. Early detection of combustion instability by neural-symbolic analysis on hi-speed video. In Proceedings of the NIPS Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches co-located with the 29th Annual C...

    2015

  38. [47]

    Neuro-symbolic evaluation of text-to-video models using formalf verification

    SP Sharan, Minkyu Choi, Sahil Shah, Harsh Goel, Mohammad Omama, and Sandeep Chinchali. Neuro-symbolic evaluation of text-to-video models using formalf verification. arXiv preprint arXiv:2411.16718, 2024

    2024 arXiv

  39. [48]

    Linear temporal logic motion planning for teams of underactuated robots using satisfiability modulo convex programming

    Yasser Shoukry, Pierluigi Nuzzo, Ayca Balkan, Indranil Saha, Alberto L Sangiovanni-Vincentelli, Sanjit A Seshia, George J Pappas, and Paulo Tabuada. Linear temporal logic motion planning for teams of underactuated robots using satisfiability modulo convex programming. In 2017 ...

    2017

  40. [49]

    Shultz and Richard D

    Richard H. Shultz and Richard D. Clarke. Big data at war: Special operations forces, project maven, and twenty-first-century warfare, August 2020. URL https://mwi.westpoint.edu/big-data-at-war-special-operations-forces-project-maven-and-twenty-first-century-warfare/

    2020

  41. [50]

    Videoagent: Self-improving video generation

    Achint Soni, Sreyas Venkataraman, Abhranil Chandra, Sebastian Fischmeister, Percy Liang, Bo Dai, and Sherry Yang. Videoagent: Self-improving video generation. arXiv preprint arXiv:2410.10076, 2024

    2024 arXiv

  42. [51]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF conference on computer vi...

    2020

  43. [52]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  44. [53]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  45. [54]

    Video classification with channel-separated convolutional networks

    Du Tran, Heng Wang, Matt Feiszli, and Lorenzo Torresani. Video classification with channel-separated convolutional networks. In IEEE/CVF International Conference on Computer Vision , pages 5551--5560. IEEE , 2019

    2019

  46. [55]

    Twilio: Cloud communications platform, 2025

    Twilio Inc. Twilio: Cloud communications platform, 2025. URL https://www.twilio.com. Accessed: 2025-03-01

    2025

  47. [56]

    Phenaki: Variable length video generation from open domain textual description

    Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kindermans, Hernan Moraldo, Han Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. Phenaki: Variable length video generation from open domain textual description. arXiv preprint arXiv:2210.02399, 2022

    arXiv

  48. [57]

    Lave: Llm-powered agent assistance and language augmentation for video editing

    Bryan Wang, Yuliang Li, Zhaoyang Lv, Haijun Xia, Yan Xu, and Raj Sodhi. Lave: Llm-powered agent assistance and language augmentation for video editing. In Proceedings of the 29th International Conference on Intelligent User Interfaces, pages 699--714, 2024 a

    2024

  49. [58]

    Videoagent: Long-form video understanding with large language model as agent

    Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Videoagent: Long-form video understanding with large language model as agent. In European Conference on Computer Vision, pages 58--76. Springer, 2024 b

    2024

  50. [59]

    Internvid: A large-scale video-text dataset for multimodal understanding and generation

    Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-text dataset for multimodal understanding and generation. arXiv preprint arXiv:2307.06942, 2023

    2023 arXiv

  51. [60]

    Discovqa: Temporal distortion-content transformers for video quality assessment

    Haoning Wu, Chaofeng Chen, Liang Liao, Jingwen Hou, Wenxiu Sun, Qiong Yan, and Weisi Lin. Discovqa: Temporal distortion-content transformers for video quality assessment. IEEE Transactions on Circuits and Systems for Video Technology, 33 0 (9): 0 4840--4854, 2023 a

    2023

  52. [61]

    Exploring video quality assessment on user generated contents from aesthetic and technical perspectives

    Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. In Proceedings of the IEEE/CVF International Conference on...

    2023

  53. [62]

    A graph-based framework to bridge movies and synopses

    Yu Xiong, Qingqiu Huang, Lingfeng Guo, Hang Zhou, Bolei Zhou, and Dahua Lin. A graph-based framework to bridge movies and synopses. In IEEE/CVF International Conference on Computer Vision , pages 4591--4600. IEEE , 2019

    2019

  54. [63]

    Msr-vtt: A large video description dataset for bridging video and language

    Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5288--5296, 2016

    2016

  55. [64]

    Hauptmann

    Zhongwen Xu, Yi Yang, and Alexander G. Hauptmann. A discriminative CNN video representation for event detection. In IEEE Conference on Computer Vision and Pattern Recognition , pages 1798--1807, Boston, MA, USA, 2015. IEEE Computer Society

    2015

  56. [65]

    Specification-driven video search via foundation models and formal verification

    Yunhao Yang, Jean-Rapha \"e l Gaglione, Sandeep Chinchali, and Ufuk Topcu. Specification-driven video search via foundation models and formal verification. arXiv preprint arXiv:2309.10171, 2023

    2023 arXiv

  57. [66]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024

    2024 arXiv

  58. [67]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023

    2023

  59. [68]

    Neural-symbolic VQA: disentangling reasoning from vision and language understanding

    Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, and Josh Tenenbaum. Neural-symbolic VQA: disentangling reasoning from vision and language understanding. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicol \` o Cesa - Bianchi, and Roman ...

    2018

  60. [69]

    A probabilistic graphical model based on neural-symbolic reasoning for visual relationship detection

    Dongran Yu, Bo Yang, Qianhao Wei, Anchen Li, and Shirui Pan. A probabilistic graphical model based on neural-symbolic reasoning for visual relationship detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10599--10608, New Orleans, LA, USA, 2022. IEEE

    2022

  61. [70]

    Video-llama: An instruction-tuned audio-visual language model for video understanding

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding. arXiv preprint arXiv:2306.02858, 2023 a . URL https://arxiv.org/abs/2306.02858

    2023 arXiv

  62. [71]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint arXiv:2311.04145, 2023 b

    2023 arXiv

  63. [72]

    Abnormal event detection by a weakly supervised temporal attention network

    Xiangtao Zheng, Yichao Zhang, Yunpeng Zheng, Fulin Luo, and Xiaoqiang Lu. Abnormal event detection by a weakly supervised temporal attention network. CAAI Transactions on Intelligence Technology, 7 0 (3): 0 419--431, 2022

    2022

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.