Pith. sign in

REVIEW 5 major objections 6 minor 65 references

InterAct-Video: Reasoning-Rich Video QA for Urban Traffic

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces a traffic-specific VideoQA benchmark with 8 hours of intersection footage and over 25,000 QA pairs, and shows that fine-tuning on it improves state-of-the-art video-language models.

desk verdict A plausible new intersection-focused VideoQA benchmark whose ground-truth QA reliability is unquantified and whose internal counts disagree; worth refereeing, but only after the authors supply annotation statistics and fix the inconsistencies. read the letter →

arxiv 2507.14743 v3 pith:XYXGKQT7 submitted 2025-07-19 cs.CV

Joseph Raj Vishal , Divesh Basina , Rutuja Patil , Manas Srinivas Gowda , Katha Naik , Yezhou Yang , Bharatesh Chakravarthi This is my paper
classification cs.CV
keywords videoquestionansweringtrafficmonitoringintersectiondatasetspatiotemporalreasoningmulti-eventscenesfine-tuningvideo-languagemodelsbenchmarkintelligenttransportationsystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces InterAct VideoQA, a benchmark dataset for testing whether video question-answering models can make sense of real urban intersections. It compiles eight hours of roadside footage into 10-second clips and pairs them with over 25,000 questions about vehicle interactions, traffic incidents, counting, and counterfactual scenarios. The authors show that current state-of-the-art VideoQA models answer these questions poorly, and that fine-tuning them on the dataset produces measurable accuracy gains. If correct, the work establishes that traffic monitoring needs domain-specific VideoQA benchmarks rather than generic video datasets, and that such benchmarks can double as training data.

What carries the argument

The central object is the InterAct VideoQA dataset itself: a collection of 10-second real-world intersection clips with dense QA annotations generated by a human-plus-LLM pipeline. The pipeline records and segments traffic footage, uses a multimodal LLM to extract per-frame metadata and draft questions, then human annotators verify, correct, or discard pairs before release. The dataset is organized around five question types adapted from earlier traffic QA work (attribution, counting, event reasoning, reverse reasoning, counterfactual inference). The evaluation machinery is standard captioning metrics (BLEU, ROUGE, METEOR, CIDEr, SPICE) measured before and after LoRA fine-tuning of three video-language models.

What would settle it

Independently re-annotate a random sample of released clips with expert annotators who do not see the original labels and measure agreement; if agreement is low or many original answers are judged incorrect, the fine-tuning gains could be artifacts of noisy labels rather than real understanding.

Watch

Extended reading notes

Core claim

The paper's central claim is that real-world traffic intersections are too dense and multi-event for current VideoQA models, and that a purpose-built dataset can both reveal and reduce that weakness. InterAct VideoQA consists of eight hours of roadside footage from urban intersections, cut into 10-second clips and annotated with over 25,000 QA pairs (reported as 28,800) across five question types: attribution, counting, event reasoning, reverse reasoning, and counterfactual inference. Evaluated on this benchmark, three state-of-the-art video-language models show clear weaknesses in spatiotemporal and multi-agent reasoning; after fine-tuning on the dataset, their BLEU, ROUGE, METEOR, CIDEr, and SPICE scores improve, with the largest gains in reasoning-heavy categories. The author concludes that domain-specific, densely annotated traffic data is necessary to make VideoQA useful for intelligent transportation systems.

Load-bearing premise

The benchmark's ground truth is trustworthy: the 25,000+ question-answer labels are correct and unbiased, even though the paper reports no inter-annotator agreement or residual error rate.

Editorial extensions

If this is right

  • Fine-tuning VideoQA models on InterAct VideoQA improves their performance on traffic scenes, with the largest gains in event reasoning and reverse reasoning.
  • The benchmark provides a reusable testbed for measuring spatiotemporal and multi-agent reasoning in dense intersection footage.
  • General-purpose VideoQA datasets do not capture overlapping, concurrent traffic events; intersection-specific data is needed to evaluate and train deployable models.
  • Traffic monitoring applications such as incident detection and vulnerable-road-user safety can be evaluated in structured QA form rather than through detection-only metrics.
  • Public release of the dataset lets other groups reproduce the evaluations and extend the benchmark with new sites and seasons.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference beyond the paper: the reported fine-tuning gains were measured against models trained on general video data, so an ablation comparing equal-size generic versus traffic-specific fine-tuning would be needed to prove the dataset, not just extra data, drives the improvement.
  • Inference beyond the paper: because the QA pairs were drafted by an LLM and then corrected, template regularities from the generator could remain in the final labels; probing with rephrased questions would test whether fine-tuned models improved at understanding or at pattern-matching the question format.
  • Inference beyond the paper: if independent annotation confirms label quality, the counterfactual and reverse-reasoning categories could be repurposed as training signal for causal reasoning in autonomous driving and traffic-safety systems.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper introduces InterAct VideoQA, a new benchmark dataset for video question answering in urban traffic intersection monitoring. The dataset consists of 8 hours of real-world traffic footage segmented into 10-second clips with question-answer pairs generated by GPT-o3 and subsequently human-verified. The QA pairs are organized into five reasoning categories: attribution, counting, event reasoning, reverse reasoning, and counterfactual inference. The authors evaluate three existing VideoQA models (VideoLLaMA2, LLaVA-NeXT-Video, Qwen2-VL) using standard NLG metrics (BLEU, ROUGE, METEOR, CIDEr, SPICE) and report improvements after fine-tuning on the new dataset. The paper also includes a conceptual framework with equations for spatio-temporal reasoning, multi-agent interaction, and counterfactual inference, though these are not used in the experiments.

Significance. InterAct VideoQA addresses an underexplored area: traffic-specific VideoQA at intersections with dense multi-event annotations. If the dataset is reliable, it could be a valuable contribution to the ITS community, especially because it uses real-world footage rather than synthetic data. The paper's strengths include the public release of the dataset, the use of standard evaluation metrics, and the explicit effort to include human verification. However, the validity of the central claims is currently undermined by unquantified human verification, inconsistent dataset statistics, and unexplained fine-tuning results (including identical before/after rows). The significance of the contribution is conditional on addressing these validation gaps.

major comments (5)
  1. [Abstract / §3.4 / Table 1] The dataset size is reported inconsistently: the abstract states 'over 25,000 QA pairs', §3.4 states '28,800 question-answer pairs', and Table 1 lists '∼2.9k/34.6k' for InterAct VideoQA. Given 8 hours of 10-second clips, roughly 2,880 clips are expected, so 28,800 pairs implies exactly 10 per clip as in the prompt, but 34.6k pairs would require about 12 per clip. The authors should specify the exact number of clips and QA pairs, and reconcile these figures.
  2. [Section 4.1] Section 4.1 claims that human evaluators 'cross-referenced every question-answer pair against its video footage' and eliminated hallucinations, but no inter-annotator agreement, number of annotators, audited sample size, or residual error rate is reported. The benchmark's ground truth and the fine-tuning gains in Tables 2–3 are only as trustworthy as this verification step, so the authors must provide quantitative validation statistics, including agreement measures and the percentage of QA pairs discarded or corrected.
  3. [Section 8.1 / Figure 5] There is a direct contradiction between the description of the generation prompt and its actual text: Section 8.1 says the prompt 'prohibits revealing internal reasoning', but Figure 5 instructs the model to 'Think step-by-step, show internal reasoning; output only the final markdown table'. Additionally, the prompt's taxonomy uses 'Basic Understanding' and 'Attribution' (with counting under Attribution), while the paper elsewhere (Figure 1, §3.3, Table 2) treats 'Counting' as a separate category. This inconsistency affects dataset construction and evaluation rows, and the authors must clarify the actual category taxonomy.
  4. [Tables 2 and 3] The fine-tuning results are presented without error bars, confidence intervals, or significance tests, and several before/after rows in Table 2 are identical (e.g., CounterFactual for VideoLLaMA2 and LLaVA-NeXT-Video: BLEU -, ROUGE 31.25, METEOR 31.20, CIDEr -, SPICE 18.18). Identical scores across all five metrics before and after fine-tuning are implausible unless the model weights did not change, and the table caption does not explain this. Furthermore, Table 3 refers to 'Qwen2-VL-7B-Instruct' while Table 2 and §5.1 use different names ('Qwen2-VL-7B-hf' / 'Qwen2-VL-7B-Instruct'), so the model identity must be corrected. The central claim of 'notable performance improvements' requires statistical support and an explanation of the unchanged rows.
  5. [Section 5.3] Equations (1)–(3) in Section 5.3 are introduced as part of a 'Conceptual Evaluation Approach', but the paper never reports how these quantities were computed from the video data or what results were obtained. No experimental measurements are tied to these equations, so the framework is untestable as written. The authors should either remove this section or describe the concrete procedures and results for each equation.
minor comments (6)
  1. [Section 5.1] The sentence 'The models were selected based on a previous study conducted [?]' contains a missing citation placeholder; reference [42] should be properly cited here or the claim removed.
  2. [Section 5.2] There are typos in metric names: 'ROGUE' should be 'ROUGE', and 'BLUE score' should be 'BLEU score' in the discussion of reverse reasoning improvements.
  3. [Throughout] The dataset name is rendered inconsistently as 'InterAct VideoQA', 'InterAct-Video', and 'InterAct videoQA'; please standardize the spelling and capitalization.
  4. [Figures 9–14] Some example QA pairs appear mismatched or ungrammatical, e.g., Figure 11 asks 'What is the color of the sedan in the middle?' and answers 'No, there is no red sedan in the video.' Such samples undermine confidence in the claimed human verification and should be corrected or replaced with cleaner examples.
  5. [Section 8.3.2] The FCA score formula uses undefined variables Fdiv, Cdup, and Aexact; please define these terms and explain how the rejection threshold of 0.70 was determined.
  6. [Section 3.4] The phrase 'to hallucinate events' in the description of counterfactual questions is unclear; it should be rephrased, e.g., 'to test whether the model hallucinates events that do not occur.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark is grounded in external video footage and human verification; self-citations are ancillary, not load-bearing.

full rationale

The paper's central claims are the construction of the InterAct VideoQA benchmark and the evaluation/fine-tuning of VideoQA models on it. The QA labels are generated with GPT assistance but then externally grounded: Section 4.1 states that human evaluators 'cross-referenced every question-answer pair against its video footage' and manually corrected or discarded hallucinated or incorrect pairs. This is an external grounding step, not a derivation of the benchmark from its own evaluation targets. No equation in the paper defines a predicted quantity in terms of its own inputs; the formalisms in Section 5.3 (temporal dependency function, multi-agent interaction score, counterfactual probability inequality) are illustrative definitions rather than claimed first-principles results. The only self-citation is the model-selection rationale in Section 5.1, which references the authors' prior study [42]; this is peripheral to the dataset's construction and does not force the evaluation outcomes. The question taxonomy is adapted from the external SUTD-TrafficQA [48], not from a self-referential uniqueness claim. Fine-tuning and evaluating on the same benchmark is standard benchmark practice and is not circular by construction. The absence of inter-annotator agreement statistics and residual error rates is a validity and reproducibility concern, not a circularity. Therefore no circular step meets the required bar of quoting a specific reduction of a claimed result to its own inputs.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The benchmark depends on the reliability of GPT-o3-assisted annotations and human verification, neither of which is quantified. The design thresholds (FCA 0.70, 10-second clips) are procedural choices, not fitted parameters. No new physical or conceptual entities are introduced.

free parameters (1)
  • FCA rejection threshold = 0.70
    Hand-set quality gate in Section 8.3.2 that rejects GPT-generated QA batches scoring below 0.70; influences which questions enter the benchmark.
assumptions (4)
  • domain assumption Human verification of every QA pair is accurate and free of bias.
    Section 4.1 states human evaluators cross-referenced each QA pair against footage, but no inter-annotator agreement, sample size, or error rate is reported.
  • ad hoc to paper GPT-o3 generated QAs, after manual correction, constitute valid ground truth for traffic reasoning.
    Section 3.3 and Figure 2 rely on GPT-assisted QA generation; without independent ground-truth sources, annotation quality rests on the generative model's outputs.
  • domain assumption 10-second clips are sufficient to capture meaningful traffic events.
    Section 3.1 defends 10-second windows, but the sufficiency claim is asserted rather than measured.
  • domain assumption The five text metrics (BLEU, ROUGE, METEOR, CIDEr, SPICE) are meaningful proxies for answer correctness.
    Section 5.2 invokes these metrics as rigorous and reproducible without calibration against human judgments in this traffic domain.

how reviews work

0 comments
Cite this review

Pith. "Pith review of InterAct-Video: Reasoning-Rich Video QA for Urban Traffic." pith.science (2026). https://pith.science/paper/XYXGKQT7

@misc{pith2026250714743,
  author       = {Pith},
  title        = {Pith review of: InterAct-Video: Reasoning-Rich Video QA for Urban Traffic},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XYXGKQT7}},
  note         = {Machine review of arXiv:2507.14743}
}
read the original abstract

Traffic monitoring is crucial for urban mobility, road safety, and intelligent transportation systems (ITS). Deep learning has advanced video-based traffic monitoring through video question answering (VideoQA) models, enabling structured insight extraction from traffic videos. However, existing VideoQA models struggle with the complexity of real-world traffic scenes, where multiple concurrent events unfold across spatiotemporal dimensions. To address these challenges, this paper introduces \textbf{InterAct VideoQA}, a curated dataset designed to benchmark and enhance VideoQA models for traffic monitoring tasks. The InterAct VideoQA dataset comprises 8 hours of real-world traffic footage collected from diverse intersections, segmented into 10-second video clips, with over 25,000 question-answer (QA) pairs covering spatiotemporal dynamics, vehicle interactions, incident detection, and other critical traffic attributes. State-of-the-art VideoQA models are evaluated on InterAct VideoQA, exposing challenges in reasoning over fine-grained spatiotemporal dependencies within complex traffic scenarios. Additionally, fine-tuning these models on InterAct VideoQA yields notable performance improvements, demonstrating the necessity of domain-specific datasets for VideoQA. InterAct VideoQA is publicly available as a benchmark dataset to facilitate future research in real-world deployable VideoQA models for intelligent transportation systems. GitHub Repo: https://github.com/joe-rabbit/InterAct_VideoQA

Figures

Figures reproduced from arXiv: 2507.14743 by the authors.

Figure 1
Figure 1. InterAct Video analyzes a busy intersection and answers five kinds of questions: (1) Attribution, (2) Counting, (3) Event reasoning, (4) Reverse reasoning, and (5) Counter-factual inference. Abstract Traffic monitoring is crucial for urban mobility, road safety, and intelligent transportation systems (ITS). Deep learning has advanced video-based traffic monitoring through video question answering (VideoQA) models, e… view at source ↗
Figure 2
Figure 2. An overview of the InterAct VideoQA pipeline for creating a high-quality question-answer dataset from traffic videos. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Question distribution – (a) Vehicular vs. non-vehicular question counts by category, (b) Overall distribution of question types, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Analysis of (a) performance score distributions, (b) pre/post-fine-tuning comparisons, and (c) multi-metric improvements. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Prompt used to generate InterAct VideoQA data. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Lighting Conditions - Frames illustrate varying times of day: (top) sunrise (6:40 AM), (second) 7:30 AM, (third) 12:30 PM, [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Traffic Density Analysis - The first row (7:00 AM) shows minimal movement, the second (6:30 PM) captures light traffic, the [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Complex Interactions – The first row (4:30 PM) captures a skateboarder crossing against a red light, momentarily yielding to a [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Dawn - The figure illustrates question samples for pictures captured during dawn (6:30 AM) at the intersection. And all questions [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Morning - The figure illustrates question samples for pictures captured at 8:00 AM at the intersection. The questions based on [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Afternoon - The figure illustrates more complex, multi-faceted questions that combine object counting, traffic flow analysis, and [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Evening - The figure illustrates event-based reasoning questions. The questions focus on the presence of specific objects (e.g., [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Dusk - The figure illustrates a dusk scenario where reverse reasoning questions are explored. The QA set examines logical [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Night - The figure illustrates counterfactual inferencing, designed to test the model’s ability to avoid hallucinations in low-light [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

65 extracted references · 50 canonical work pages

  1. [1]

    Traffic violation detection system

    Sahara Ameer, Abdul Rahuman, Randhima Dinalankara, Udaya Wijenayake, et al. Traffic violation detection system. In Proceedings of Conference on Transdisciplinary Research in Engineering, volume 1, 2024. 2

    work page 2024

  2. [2]

    Spice: Semantic propositional image cap- tion evaluation

    Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propositional image cap- tion evaluation. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, Octo- ber 11-14, 2016, Proceedings, Part V 14 , pages 382–398. Springer, 2016. 6

    work page 2016

  3. [3]

    Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 2023. 6

    arXiv 2023

  4. [4]

    Meteor: An automatic metric for mt evaluation with improved correlation with hu- man judgments

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with hu- man judgments. In Proceedings of the acl workshop on in- trinsic and extrinsic evaluation measures for machine trans- lation and/or summarization, pages 65–72, 2005. 6

    work page 2005

  5. [5]

    Visual content based video retrieval on natural language queries

    Ravi Bansal and Sandip Chakraborty. Visual content based video retrieval on natural language queries. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Comput- ing, pages 212–219, 2019. 2

    work page 2019

  6. [6]

    Machine learning for autonomous vehicle’s trajectory prediction: A comprehen- sive survey, challenges, and future research directions

    Vibha Bharilya and Neetesh Kumar. Machine learning for autonomous vehicle’s trajectory prediction: A comprehen- sive survey, challenges, and future research directions. Ve- hicular Communications, 46:100733, 2024. 1

    work page 2024

  7. [7]

    Videollama 2: Advancing spatial-temporal modeling and audio understanding in video- llms, 2024

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and Lidong Bing. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video- llms, 2024. 6

    2024

  8. [8]

    Videollama 2: Advancing spatial- temporal modeling and audio understanding in video-llms

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial- temporal modeling and audio understanding in video-llms. arXiv preprint arXiv:2406.07476, 2024. 2

    arXiv 2024

Show all 65 references
  1. [9]

    Recent ad- vances in connected and automated vehicles

    David Elliott, Walter Keen, and Lei Miao. Recent ad- vances in connected and automated vehicles. Journal of Traffic and Transportation Engineering (English Edition) , 6(2):109–131, 2019. 1

    2019

  2. [10]

    A systematic re- view of traffic incident detection algorithms

    Osama ElSahly and Akmal Abdelfatah. A systematic re- view of traffic incident detection algorithms. Sustainability, 14(22):14859, 2022. 1

    2022

  3. [11]

    Agqa: A benchmark for compositional spatio-temporal reasoning

    Madeleine Grunde-McLaughlin, Ranjay Krishna, and Ma- neesh Agrawala. Agqa: A benchmark for compositional spatio-temporal reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11287–11297, 2021. 3

    2021

  4. [12]

    Venetianer, and Alan Lipton

    Niels Haering, P ´eter L. Venetianer, and Alan Lipton. The evolution of video surveillance: an overview. Machine Vi- sion and Applications , 19(5):279–290, 2008. Published on 2008/10/01. 1

    2008

  5. [13]

    Techniques of early incident detection and traffic monitoring centre in vanets: A review

    Mustafa Maad Hamdi, Lukman Audah, Sami Abduljabbar Rashid, and Mahmood Al Shareeda. Techniques of early incident detection and traffic monitoring centre in vanets: A review. J. Commun., 15(12):896–904, 2020. 1

    2020

  6. [14]

    Adoption of artificial intelligence in smart cities: A comprehensive review

    HMKKMB Herath and Mamta Mittal. Adoption of artificial intelligence in smart cities: A comprehensive review. Inter- national Journal of Information Management Data Insights, 2(1):100076, 2022. 1

    2022

  7. [15]

    Tgif-qa: Toward spatio-temporal reasoning in visual question answering

    Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim. Tgif-qa: Toward spatio-temporal reasoning in visual question answering. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 2758–2766, 2017. 2

    2017

  8. [16]

    High-resolution multi-source traffic data in new zealand.Sci- entific Data, 11(1):1216, 2024

    Bo Li, Ruotao Yu, Zijun Chen, Yingzhe Ding, Mingxia Yang, Jinghua Li, Jianxiao Wang, and Haiwang Zhong. High-resolution multi-source traffic data in new zealand.Sci- entific Data, 11(1):1216, 2024. Published on 2024/11/12. 1

    2024

  9. [17]

    Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024. 2

    2024 arXiv

  10. [18]

    Coda: A real-world road corner case dataset for object detection in autonomous driving

    Kaican Li, Kai Chen, Haoyu Wang, Lanqing Hong, Chao- qiang Ye, Jianhua Han, Yukuai Chen, Wei Zhang, Chunjing Xu, Dit-Yan Yeung, et al. Coda: A real-world road corner case dataset for object detection in autonomous driving. In European Conference on Computer Vision, pages 406–4...

    2022

  11. [19]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out , pages 74–81, 2004. 6

    2004

  12. [20]

    Improved baselines with visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023. 6

    2023

  13. [21]

    Llava-next: Im- proved reasoning, ocr, and world knowledge, January 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Im- proved reasoning, ocr, and world knowledge, January 2024. 6

    2024

  14. [22]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. 6

    2023

  15. [23]

    Largest: A benchmark dataset for large-scale traffic forecasting, 2023

    Xu Liu, Yutong Xia, Yuxuan Liang, Junfeng Hu, Yiwei Wang, Lei Bai, Chao Huang, Zhenguang Liu, Bryan Hooi, and Roger Zimmermann. Largest: A benchmark dataset for large-scale traffic forecasting, 2023. 2

    2023

  16. [24]

    Utd19: Understanding traffic capacity of urban networks

    Allister Loder, Lukas Amb ¨uhl, Monica Menendez, and Kay W Axhausen. Utd19: Understanding traffic capacity of urban networks. 2020. 2

    2020

  17. [25]

    Drama: Joint risk localization and captioning in driving

    Srikanth Malla, Chiho Choi, Isht Dwivedi, Joon Hee Choi, and Jiachen Li. Drama: Joint risk localization and captioning in driving. In Proceedings of the IEEE/CVF winter confer- ence on applications of computer vision , pages 1043–1052,

  18. [26]

    Lingoqa: Visual question answering for autonomous driv- ing

    Ana-Maria Marcu, Long Chen, Jan H ¨unermann, Alice Karn- sund, Benoit Hanotte, Prajwal Chidananda, Saurabh Nair, Vijay Badrinarayanan, Alex Kendall, Jamie Shotton, et al. Lingoqa: Visual question answering for autonomous driv- ing. In European Conference on Computer Vision , p...

    2024

  19. [27]

    Privacy-preserving pedestrian tracking with path image inpainting and 3d point cloud fea- tures

    Masakazu Ohno, Riki Ukyo, Tatsuya Amano, Hamada Rizk, and Hirozumi Yamaguchi. Privacy-preserving pedestrian tracking with path image inpainting and 3d point cloud fea- tures. Pervasive and Mobile Computing, 100:101914, 2024. 1

    2024

  20. [28]

    A review of artificial intelligence and machine learning for incident detectors in road transport systems

    Samuel Olugbade, Stephen Ojo, Agbotiname Lucky Imoize, Joseph Isabona, and Mathew O Alaba. A review of artificial intelligence and machine learning for incident detectors in road transport systems. Mathematical and Computational Applications, 27(5):77, 2022. 1

    2022

  21. [29]

    Chaotic world: A large and challenging benchmark for human behavior under- standing in chaotic events

    Kian Eng Ong, Xun Long Ng, Yanchao Li, Wenjie Ai, Kuangyi Zhao, Si Yong Yeo, and Jun Liu. Chaotic world: A large and challenging benchmark for human behavior under- standing in chaotic events. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20...

    2023

  22. [30]

    Goucher, Adam Perelman, and Aditya Ramesh et al

    OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, and Aditya Ramesh et al. Gpt-4o system card, 2024. 4

    2024

  23. [31]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages 311–318,

  24. [32]

    Real- time road hazard information system

    Carlos Pena-Caballero, Dongchul Kim, Adolfo Gonzalez, Osvaldo Castellanos, Angel Cantu, and Jungseok Ho. Real- time road hazard information system. Infrastructures, 5(9):75, 2020. 2

    2020

  25. [33]

    Traffic-domain video question answering with auto- matic captioning

    Ehsan Qasemi, Jonathan M Francis, and Alessandro Oltra- mari. Traffic-domain video question answering with auto- matic captioning. arXiv preprint arXiv:2307.09636 , 2023. 2

    2023 arXiv

  26. [34]

    Nuscenes-qa: a multi-modal visual question answering benchmark for autonomous driving scenario

    Tianwen Qian, Jingjing Chen, Linhai Zhuo, Yang Jiao, and Yu-Gang Jiang. Nuscenes-qa: a multi-modal visual question answering benchmark for autonomous driving scenario. In Proceedings of the Thirty-Eighth AAAI Conference on Artifi- cial Intelligence and Thirty-Sixth Conference ...

    2024

  27. [35]

    Shenoy, and Shanta Rangaswamy

    Roopa Ravish, Datthesh P. Shenoy, and Shanta Rangaswamy. Sensor-based traffic control system. In Jyotsna Kumar Man- dal and Somnath Mukhopadhyay, editors,Proceedings of the Global AI Congress 2019, pages 207–221, Singapore, 2020. Springer Singapore. 1

    2019

  28. [36]

    Find and focus: Retrieve and localize video events with natural language queries

    Dian Shao, Yu Xiong, Yue Zhao, Qingqiu Huang, Yu Qiao, and Dahua Lin. Find and focus: Retrieve and localize video events with natural language queries. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 200–216, 2018. 2

    2018

  29. [37]

    Vishnu, and C

    Dinesh Singh, C. Vishnu, and C. Krishna Mohan. Visual big data analytics for traffic monitoring in smart city. In 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 886–891, 2016. 1

    2016

  30. [38]

    Moviechat+: Question-aware sparse memory for long video question answering

    Enxin Song, Wenhao Chai, Tian Ye, Jenq-Neng Hwang, Xi Li, and Gaoang Wang. Moviechat+: Question-aware sparse memory for long video question answering. arXiv preprint arXiv:2404.17176, 2024. 2

    2024 arXiv

  31. [39]

    Movieqa: Understanding stories in movies through question- answering

    Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. Movieqa: Understanding stories in movies through question- answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4631–4640,

  32. [40]

    George Tom, Minesh Mathew, Sergi Garcia, Dimosthenis Karatzas, and C. V . Jawahar. Reading between the lanes: Text videoqa on the road, 2023. 3

    2023

  33. [41]

    Cider: Consensus-based image description evalua- tion

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evalua- tion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575, 2015. 6

    2015

  34. [42]

    Eyes on the road: State-of-the-art video question answering models assessment for traffic mon- itoring tasks, 2024

    Joseph Raj Vishal, Divesh Basina, Aarya Choudhary, and Bharatesh Chakravarthi. Eyes on the road: State-of-the-art video question answering models assessment for traffic mon- itoring tasks, 2024. 6

    2024

  35. [43]

    Omnivid: A generative framework for universal video understanding

    Junke Wang, Dongdong Chen, Chong Luo, Bo He, Lu Yuan, Zuxuan Wu, and Yu-Gang Jiang. Omnivid: A generative framework for universal video understanding. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18209–18220, 2024. 2

    2024

  36. [44]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Jun- yang Lin. Qwen2-vl: Enhancing vision-language model’s ...

    2024

  37. [45]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Jun- yang Lin. Qwen2-vl: Enhancing vision-language model’s ...

    2024 arXiv

  38. [46]

    Star: A benchmark for situated reason- ing in real-world videos

    Bo Wu, Shoubin Yu, Zhenfang Chen, Joshua B Tenenbaum, and Chuang Gan. Star: A benchmark for situated reason- ing in real-world videos. arXiv preprint arXiv:2405.09711,

    arXiv

  39. [47]

    Msr-vtt: A large video description dataset for bridging video and language

    Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5288–5296, 2016. 3

    2016

  40. [48]

    Sutd-trafficqa: A question answering benchmark and an efficient network for video rea- soning over traffic events

    Li Xu, He Huang, and Jun Liu. Sutd-trafficqa: A question answering benchmark and an efficient network for video rea- soning over traffic events. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 9878–9888, 2021. 2, 3, 4

    2021

  41. [49]

    Egocentric video task translation

    Zihui Xue, Yale Song, Kristen Grauman, and Lorenzo Tor- resani. Egocentric video task translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2310–2320, 2023. 3 10

    2023

  42. [50]

    A data-driven rule-based system for china’s traffic acci- dent prediction by considering the improvement of safety ef- ficiency

    Fei-Fei Ye, Long-Hao Yang, Ying-Ming Wang, and Haitian Lu. A data-driven rule-based system for china’s traffic acci- dent prediction by considering the improvement of safety ef- ficiency. Computers & Industrial Engineering, 176:108924,

  43. [51]

    Activitynet-qa: A dataset for understanding complex web videos via question answering

    Zhou Yu, Dejing Xu, Jun Yu, Ting Yu, Zhou Zhao, Yuet- ing Zhuang, and Dacheng Tao. Activitynet-qa: A dataset for understanding complex web videos via question answering. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 33, pages 9127–9134, 2019. 2

    2019

  44. [52]

    Llava- next: A strong zero-shot video understanding model, April

    Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. Llava- next: A strong zero-shot video understanding model, April

  45. [53]

    Tumtraffic-videoqa: A benchmark for unified spatio- temporal video understanding in traffic scenes

    Xingcheng Zhou, Konstantinos Larintzakis, Hao Guo, Wal- ter Zimmer, Mingyu Liu, Hu Cao, Jiajie Zhang, Venkat- narayanan Lakshminarasimhan, Leah Strand, and Alois C Knoll. Tumtraffic-videoqa: A benchmark for unified spatio- temporal video understanding in traffic scenes. arXiv ...

    2025 arXiv

  46. [54]

    5g- enabled v2x communications for vulnerable road users safety applications: a review

    Chaima Zoghlami, Rahim Kacimi, and Riadh Dhaou. 5g- enabled v2x communications for vulnerable road users safety applications: a review. Wireless Networks , 29(3):1237–1267, 2023. 2 11 System: You are a video questionnaire assistant for creating high-quality VideoQA data for th...

    2023

  47. [55]

    W ATCHthe clip carefully

  48. [56]

    Questions must be answerable using the clip alone

    GENERATE 10 diverse question-answer (QA) pairs that fit the five InterAct VideoQA categories – create 2 QAs per category (total = 10). Questions must be answerable using the clip alone. Keep language succinct and fact-based; avoid ambiguities and hallucinations

  49. [57]

    clip_videos_0 TAXONOMY & EXAMPLES:

    RETURN a markdown table with the columns: | Index | Video File Path | Question | Category | Answer | (Video File Path = the string inside <VIDEO>, e.g. clip_videos_0 TAXONOMY & EXAMPLES:

  50. [58]

    Example: Question: Is there a blue bus in the scene? ; Answer: Yes, a blue bus is stopped in the middle lane

    Basic Understanding: Presence/absence about salient objects, weather, lighting. Example: Question: Is there a blue bus in the scene? ; Answer: Yes, a blue bus is stopped in the middle lane

  51. [59]

    Example: Question: How many white cars are visible at 0–2 s? ; Answer: Four white cars

    Attribution: Static properties (colour, type, signage) or counts. Example: Question: How many white cars are visible at 0–2 s? ; Answer: Four white cars

  52. [60]

    Example: Question: Did any vehicle run the red light before the pedestrian crossed?; Answer: Yes, a black sedan did at 0.9 s

    Event Reasoning: Detect, localise or order dynamic events (starts, stops, turns, lane changes ). Example: Question: Did any vehicle run the red light before the pedestrian crossed?; Answer: Yes, a black sedan did at 0.9 s

  53. [61]

    Example: Question: Which occurred first: the cyclist entering the bike lane or the tram moving?; Answer: The cyclist entered first

    Reverse Reasoning: Ask about an event in reverse temporal order or which happened first. Example: Question: Which occurred first: the cyclist entering the bike lane or the tram moving?; Answer: The cyclist entered first

  54. [62]

    Example: Question: Is it true that no black car turned onto the bike lane? ; Answer: False; one black car did turn at 0.9 s

    CounterFactual Inference: Negate or invert a fact to test hallucination resistance. Example: Question: Is it true that no black car turned onto the bike lane? ; Answer: False; one black car did turn at 0.9 s. Question Generation Template Figure 5. Prompt used to generate Inter...

  55. [63]

    video questionnaire

    Supplementary Material 8.1. Prompt For Generating QA’s This prompt fully specifies a controlled generation task for our VideoQA assistant. First, it assigns the assistant the “video questionnaire” role and bounds it to the Inter- Act benchmark, ensuring consistency with our ev...

  56. [64]

    Each frame is tiled, super-resolved ( ×2 Real-ESRGAN) and fed to YOLOv8-small (person class) and a fine-tuned licence- plate detector

    Stage 1 – Automatic masking. Each frame is tiled, super-resolved ( ×2 Real-ESRGAN) and fed to YOLOv8-small (person class) and a fine-tuned licence- plate detector. • Bodies are pixelated ( 6 × 6 blocks). • Faces and plates are Gaussian-blurred ( σ = 21 and σ = 31, respectively...

  57. [65]

    irreversible

    Stage 2 – Human audit. Two annotators review the anonymised clip at 1× and 0.25× speed. Residual PII (i) triggers re-processing or (ii) causes permanent ex- clusion, subject to inter-rater agreement κ ≥ 0.9. Table 5. Key hyper-parameters for Stage 1 automatic masking. Componen...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.