Pith. sign in

REVIEW 4 major objections 6 minor 46 references

FriendsQA: A New Large-Scale Deep Video Understanding Dataset with Fine-grained Topic Categorization for Story Videos

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper introduces FriendsQA, a 44.6K-question benchmark for deep video understanding of story videos, and shows current video-QA models fall far short on it.

desk verdict Useful new DVU benchmark, but the quality claim only holds for season 1; treat the rest as LLM-filtered until an audit shows otherwise. read the letter →

arxiv 2412.17022 v1 pith:2RD3PLW3 submitted 2024-12-22 cs.CV

Zhengqian Wu , Ruizhe Li , Zijun Xu , Zhongyuan Wang , Chunxia Xiao , Chao Liang This is my paper · ORCID
classification cs.CV
keywords deepvideounderstandingquestionansweringstoryvideosfine-grainedtopicsmulti-agentLLMbenchmarkdatasetcross-episodereasoningdifficultymeasurement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the field's deep-video-understanding (DVU) benchmarks are too narrow to test story comprehension, and that a new dataset called FriendsQA fixes that. FriendsQA supplies 44.6K multiple-choice questions across 14 fine-grained story-topic categories, generated automatically by a multi-agent LLM pipeline called StoryMind and filtered by two independent LLM reviewers. The paper reports that ten current video-QA models score far lower on FriendsQA than on factoid benchmarks—for example, VideoChat2 drops from 61.70% on NExT-QA to 44.05%—suggesting the dataset measures something existing tests miss. If the dataset is as clean as claimed, it gives the community a balanced, per-topic diagnostic for story video understanding and a pipeline that can scale to other shows.

What carries the argument

The mechanism is StoryMind, a multi-agent pipeline that generates and filters the dataset. A generator LLM receives script, timeline, character detection bounding boxes (from a shot-based instance search) and writes questions with an assigned fine-grained topic; a feedback loop iterates until each of the 14 topics reaches the same quota, producing a balanced distribution. Two independent LLM reviewers then drop any question whose answer they do not both agree on and cannot verify against the script-video information. Each question also carries a difficulty score $\sigma^i = \sigma_l^i/\mu_l + \sigma_c^i/\mu_c$, where $\sigma_l^i$ is the ratio of whole-video length to relevant-video length and $\sigma_c^i$ the ratio of whole-video to relevant-video instance count; this score is intended to let users separate 'hard' from 'easy' questions. The 14 fine-grained topics (C, A, L, CA, CL, AL, CAL times P and I) are the unit of analysis that lets the paper pinpoint where models fail, e.g., character recognition.

What would settle it

Sample, say, 300 questions at random from seasons 2-10, have a human verify that (a) the question is answerable from the episode and (b) exactly one choice is correct. If the human retention rate falls well below the 90-94% observed on season 1, the reported model accuracies would need to be re-estimated on a corrected subset.

Watch

Extended reading notes

Core claim

The central claim is that FriendsQA is a large-scale, balanced benchmark for deep video understanding of story videos, and that current state-of-the-art models fail it. The dataset is derived from 234 episodes of Friends, averaging 1,358 seconds each, and contains 44.6K questions split evenly across 14 fine-grained topics—the seven story topics character (C), action (A), location (L), and their combinations, each posed in both perception (P) and inference (I) form. Roughly 9,470 questions span four consecutive episodes, requiring cross-episode reasoning. The paper's evaluation of ten VLM and MLLM models shows accuracy consistently near or below chance for many topic types, with character-related perception questions being the hardest; the best single-episode result is VideoLLaMA2 at 47.12%. The paper interprets these results as evidence that story-level understanding, not just factoid retrieval, is an open problem that FriendsQA exposes.

Load-bearing premise

The entire 44.6K dataset is validated only by computer reviewers; humans checked just the first season's 4,790 questions, so the benchmark's trustworthiness rests on the assumption that the automatic filters stay just as accurate on the other 39,800+ questions.

Editorial extensions

If this is right

  • If the benchmark is accepted, per-topic scores (e.g., P-C versus I-L) give model developers a diagnostic for exactly which story elements—characters, actions, locations, or their combinations—their system fails to model.
  • Cross-episode questions extend evaluation to long-range storyline reasoning beyond the typical single-scene clip, so progress on FriendsQA would signal genuine narrative understanding rather than clip-level pattern matching.
  • The difficulty measure separates easy, medium, and hard questions, allowing training and evaluation to target harder types of reasoning.
  • The automated StoryMind pipeline suggests that similar balanced, topic-tagged benchmarks can be produced for other story-rich media without the labor cost of handcrafted annotations.
  • The reported 44.05% accuracy of VideoChat2 (vs. 61.70% on NExT-QA) indicates that models tuned for factoid VideoQA do not transfer their performance to storyline understanding, setting a clear baseline for future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the generator and reviewers see the same script text, a model could in principle answer many questions from subtitles alone; an ablation that removes the video stream would reveal how much of the benchmark is genuinely visual, which the paper does not report.
  • The fact that inference questions score higher than perception questions for most models may reflect language priors in the LLM backbones rather than deep video reasoning; testing with shuffled answer orders could quantify that.
  • The paper's difficulty score assumes longer relevant spans make questions harder, but this is only validated indirectly; a direct human study comparing perceived difficulty with the score would strengthen the metric's use.
  • The same pipeline might be applied to non-English or lower-resource story videos, but that would require the character-detection and script-alignment stages to generalize beyond the sitcom's studio format.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces FriendsQA, a large-scale deep video understanding dataset built from 234 episodes of the sitcom Friends. The authors propose StoryMind, a multi-agent LLM framework in which a Gemini 1.5 Pro generator produces multiple-choice questions conditioned on scripts enriched with detected character bounding boxes and timestamps, and two LLM reviewers (Gemini 1.5 Pro and Claude 3.5 Sonnet) filter questions for relevance and correctness. The dataset contains 44,600 questions, including 9,470 cross-episode questions, balanced across 14 fine-grained topics formed by crossing seven story topics (C, A, L, CA, CL, AL, CAL) with two attributions (perception, inference). A difficulty score is defined from length and content ratios, and questions are bucketed into easy, medium, and hard levels. Ten state-of-the-art VideoQA models are evaluated zero-shot, with reported accuracy substantially lower on FriendsQA than on factoid benchmarks (e.g., VideoChat2 44.05% vs. 61.70% on NExT-QA). The paper also reports manual verification of 4,790 first-season questions and compares model accuracy on that verified subset with the automatically filtered version.

Significance. If the quality of the full dataset holds, FriendsQA would be a valuable community resource: it is the first DVU dataset with deliberate fine-grained topic balance (Gini 0.927, entropy 3.794), it combines long videos with a large question count (98,874.8 K question-seconds), and it includes cross-episode questions and a per-question difficulty annotation. The StoryMind pipeline, with explicit topic feedback and dual-LLM review, is a practical recipe for lowering annotation cost, and the paper ships code and dataset links. The S1 manual check (94.44% single / 90.95% cross retention after revision) is a useful sanity check. However, the significance is conditional: full-corpus quality and visual answerability are not established, and the absence of a human baseline limits the interpretability of the reported model gaps. The dataset is likely to be useful for benchmarking long-form video QA, but the paper's central 'high-quality benchmark' claim needs stronger evidence.

major comments (4)
  1. [Dataset Quality (Table 2)] The manual verification covers only FriendsQA-S1 (4,790 questions, about 10.7% of the 44.6K pool) and is used to extrapolate retention rates to all ten seasons. Since the first-season retention after revision is 94.44% for single-episode and 90.95% for cross-episode questions, a comparable error rate on the remaining roughly 39.8K questions implies thousands of questions with wrong, ambiguous, or non-unique ground truth. The paper should either manually audit a stratified sample across all seasons and both question types, or prominently restrict the quality claim to FriendsQA-S1 and provide error-rate bounds for the full set. Without this, the per-topic accuracies in Table 4 and the difficulty analysis in Figure 11 are not trustworthy as benchmark numbers.
  2. [Question Filtration (Figure 6, Appendix B.2)] Both reviewers receive exactly the same 'video information' as the generator: script text with inserted character bounding boxes and timestamps. They therefore cannot catch errors that require visual verification (e.g., whether a character is actually on screen, whether an action is visible), and their judgments are correlated with the generator since they read the same source. The manual revision in Table 2 also uses the script-based video information rather than the raw video. Consequently, the reported quality reflects answerability from transcripts, not necessarily visual answerability. The claim that VideoQA models 'fall short' on deep video understanding requires a human baseline on the same video-question pairs, or at minimum a visual-verification study on a sample. As written, the performance gap could partly reflect questions that are not visually grounded.
  3. [Difficulty Measure (Eq. 1, Figure 10)] The difficulty score sigma_i = sigma_l / mu_l + sigma_c / mu_c is a heuristic ratio of whole-video to relevant-video length and instance counts, with no justification for the additive form or for the 9:3:1 bucketing into easy, medium, and hard levels. Figure 10(b) shows a downward trend in average accuracy as difficulty increases, but this is aggregated and does not establish that the measure is calibrated for individual questions; SeViLA shows no difficulty effect. The content factor counts characters and locations from the video information, not semantic content, so it may conflate long-range reasoning with simple sparsity. The authors should validate the measure with item-level analysis (e.g., accuracy vs. difficulty within topic and attribution, comparison with an LLM-based difficulty rating, or human difficulty judgments) before using it to draw conclusions such as 'P questions are harder than I questions'.
  4. [Evaluation Setting] All 10 models are evaluated zero-shot with official defaults, and no human or fine-tuned upper bound is reported. Table 4 therefore shows relative ordering among zero-shot models, but the headline claim that state-of-the-art models 'fall far short' on FriendsQA lacks a reference point. A human-accuracy estimate on a sample, using the same multiple-choice interface, would let the reader separate model limitations from dataset artifacts. This is particularly important given the reviewer-groundedness issue above: without a human baseline, the gap between 61.70% on NExT-QA and 44.05% on FriendsQA cannot be cleanly attributed to the dataset's deep-understanding requirements.
minor comments (6)
  1. [Table 2 and main text] There are typographical errors: 'Manul' should be 'Manual' in the Table 2 header, and 'FirendsQA-M' should be 'FriendsQA-M' in the paragraph after Table 3.
  2. [Dataset Statistics (Figure 8)] The fine-grained topic distribution figure appears to contain corrupted glyphs in the provided manuscript; the figure should be replaced with a legible rendering so that the balanced distribution claim can be verified visually.
  3. [Figure 2 and Appendix A] The LLM-based categorization of existing datasets into 14 topics is reported without a human-agreement check; a small human annotation study on the categorization prompt would strengthen the comparison in Figure 2.
  4. [Question Generation (Figure 5)] The threshold used for balanced topic coverage is not stated; the paper should specify the target number of questions per fine-grained topic and how the 'same threshold' is set for single-episode and cross-episode questions.
  5. [Evaluation Result (Figure 10a)] The claim that P questions have higher overall difficulty than I questions is based on averaged difficulty factors without confidence intervals or significance testing; adding variability estimates would make the comparison more convincing.
  6. [Data Preparation] The choice of concatenating 4 consecutive episodes for cross-episode questions is attributed to the generator's context limit; this is an architectural constraint rather than a story-structure choice and should be acknowledged as a limitation in the cross-episode design.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; difficulty and quality are defined from explicit measured inputs, with only ordinary sampling limitations.

full rationale

FriendsQA is an empirical dataset paper rather than a formal derivation chain. The central pipeline, StoryMind, generates questions from script-plus-detection metadata, filters them with two LLM reviewers, and manually verifies a 4,790-question first-season subset; no benchmark number or question-answer pair is obtained by solving an equation whose input already contains the output. The difficulty score sigma_i = sigma_i^l / mu_l + sigma_i^c / mu_c is explicitly defined from measured relevant-video length and relevant-instance counts (Eq. 1), and it is used descriptively to sort questions into easy/medium/hard; the observed accuracy decrease across difficulty bins is an empirical trend, not a quantity fitted to that trend. The paper's self-citations (PAINS script metadata from Niu et al. 2023; shot-based instance search from Li et al. 2023; story-topic framing from Guo, Liang, and Wang 2023) supply tools and conceptual vocabulary, but the benchmark's validity does not reduce to those citations: the first-season manual revision and the 10-model zero-shot evaluation are independent of the cited methods' claims. Two limitations noted in the manuscript weigh on correctness risk rather than circularity: (1) only 4,790 of 44.6K questions receive human verification (Table 2), so the 91-94% retention rates are extrapolated to the full corpus; and (2) both reviewers receive the same script-based video information as the generator (Figure 6), so their agreement checks textual self-consistency rather than independent visual answerability. These are sampling and measurement-validity concerns, not self-definitional reductions, so the circularity score remains 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central dataset claim rests mainly on domain assumptions about source scripts, LLM review quality, and the proposed topic taxonomy. There are no novel physical or mathematical entities. The two main hand-chosen numerical choices are the difficulty bucketing ratio and the cross-episode window.

free parameters (2)
  • Difficulty bucket ratio = 9:3:1
    Questions are split into easy, medium, and hard by ascending difficulty score using a 9:3:1 ratio. This is a hand-chosen analysis choice, not fitted to data.
  • Cross-episode window size = 4 episodes
    Four consecutive episodes are concatenated, chosen because the scripts reach the generator's context limit.
assumptions (4)
  • domain assumption PAINS scripts provide accurate scene boundaries, locations, character presence, and dialogue timestamps for all 234 episodes.
    Used in Data Preparation to build the 'video information' prompts; if scripts are misaligned, generated questions inherit errors.
  • domain assumption LLM reviewers' relevance and correctness judgments are reliable proxies for human annotation quality.
    Question Filtration retains a question only when both Gemini 1.5 Pro and Claude 3.5 Sonnet agree; this assumes commercial LLM agreement indicates factual correctness.
  • ad hoc to paper The 14 fine-grained topic taxonomy (7 story topics crossed with perception/inference) captures the dimensions needed to evaluate deep video understanding.
    The taxonomy is proposed in this paper and used to generate and analyze all questions; it is a modeling choice, not an externally established standard.
  • ad hoc to paper The difficulty score defined as sigma = sigma_l / mu_l + sigma_c / mu_c is a valid proxy for question difficulty.
    Used to split into levels and analyze model performance; validated only indirectly by showing accuracy decreases with difficulty.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FriendsQA: A New Large-Scale Deep Video Understanding Dataset with Fine-grained Topic Categorization for Story Videos." pith.science (2026). https://pith.science/paper/2RD3PLW3

@misc{pith2026241217022,
  author       = {Pith},
  title        = {Pith review of: FriendsQA: A New Large-Scale Deep Video Understanding Dataset with Fine-grained Topic Categorization for Story Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2RD3PLW3}},
  note         = {Machine review of arXiv:2412.17022}
}
read the original abstract

Video question answering (VideoQA) aims to answer natural language questions according to the given videos. Although existing models perform well in the factoid VideoQA task, they still face challenges in deep video understanding (DVU) task, which focuses on story videos. Compared to factoid videos, the most significant feature of story videos is storylines, which are composed of complex interactions and long-range evolvement of core story topics including characters, actions and locations. Understanding these topics requires models to possess DVU capability. However, existing DVU datasets rarely organize questions according to these story topics, making them difficult to comprehensively assess VideoQA models' DVU capability of complex storylines. Additionally, the question quantity and video length of these dataset are limited by high labor costs of handcrafted dataset building method. In this paper, we devise a large language model based multi-agent collaboration framework, StoryMind, to automatically generate a new large-scale DVU dataset. The dataset, FriendsQA, derived from the renowned sitcom Friends with an average episode length of 1,358 seconds, contains 44.6K questions evenly distributed across 14 fine-grained topics. Finally, We conduct comprehensive experiments on 10 state-of-the-art VideoQA models using the FriendsQA dataset.

Figures

Figures reproduced from arXiv: 2412.17022 by the authors.

Figure 1
Figure 1. Comparisons of factoid VideoQA and DVU. (MLLM) technologies, significant progress has been made in recent years (Yu et al. 2023; Zhong et al. 2022). However, VideoQA models’ performance significantly declines on deep video understanding (DVU) task. For in￾stance, VideoChat2 achieves 61.70% accuracy on the factoid NExT-QA dataset but drops to 44.05% on our DVU Friend￾sQA dataset. As illustrated in [PITH_FULL_IMAGE:f… view at source ↗
Figure 2
Figure 2. The distribution of 5 datasets questions across the [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The workflow diagram of the multi-agent collaboration framework StoryMind. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: The flowchart of data preparation. Metadata The script and episode video represent the tex￾tual structure and visual presentation of the storyline, respec￾tively. Therefore, we use them as the metadata for question generation. The script from PAINS dataset (Niu et al. …
Figure 6
Figure 6. Figure 6: The flowchart of question filtration. (a) Length difficulty ( ) 22:49 00:00 Relevant: 00:49 05:07 Whole: (b) Content difficulty ( ) Relevant: Whole: … … Count (characters) + Count (locations) + + [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Illustration of two difficulty factors. level of obtaining storylines as the generator, allowing them to fairly assess the questions’ accuracy. Generated questions is the output of question generation phase. Reviewers evalu￾ate the generated questions based on the vide…
Figure 8
Figure 8. Figure 8: The distribution of fine-grained topics. [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 10
Figure 10. Figure 10: The influence of the two difficulties on accuracy [PITH_FULL_IMAGE:figures/full_fig_p006_10.png]
Figure 11
Figure 11. Figure 11: Accuracy (%) of 10 SOTA models across differ [PITH_FULL_IMAGE:figures/full_fig_p007_11.png]
Figure 12
Figure 12. Figure 12: Quantity distributions of single-episode and [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 13
Figure 13. Figure 13: Question input and the prompt for MLLM. solely utilize the default parameters provided by the official repository for performance evaluation. E.2 Text Input of Evaluation This paper evaluates two types of VideoQA models, VLM and MLLM. The similarity between them durin…
Figure 14
Figure 14. Figure 14: The prompt template for assessing fine-grained topics of different DVU datasets. [PITH_FULL_IMAGE:figures/full_fig_p013_14.png]
Figure 15
Figure 15. Figure 15: The prompt template for generator [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]
Figure 16
Figure 16. Figure 16: The prompt template for reviewer [PITH_FULL_IMAGE:figures/full_fig_p015_16.png]
Figure 17
Figure 17. Figure 17: Examples of the fine-grained topic questions (single-episode questions) [PITH_FULL_IMAGE:figures/full_fig_p016_17.png]
Figure 18
Figure 18. Figure 18: Examples of the fine-grained topic questions (cross-episode questions) [PITH_FULL_IMAGE:figures/full_fig_p017_18.png]
Figure 19
Figure 19. Figure 19: Samples of FriendsQA in different difficulty levels [PITH_FULL_IMAGE:figures/full_fig_p018_19.png]
Figure 20
Figure 20. Figure 20: Qualitative results across 10 SOTA models on single-episode questions. [PITH_FULL_IMAGE:figures/full_fig_p019_20.png]
Figure 21
Figure 21. Figure 21: Qualitative results across 10 SOTA models on cross-episode questions. [PITH_FULL_IMAGE:figures/full_fig_p020_21.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 35 canonical work pages

  1. [1]

    Ammanabrolu, P.; Cheung, W.; Broniec, W.; and Riedl, M. O. 2021. Automated Storytelling via Causal, Commonsense Plot Ordering. Proceedings of the AAAI Conference on Artificial Intelligence, 35(7): 5859--5867

    work page 2021

  2. [2]

    Chen, M.; Chu, Z.; Wiseman, S.; and Gimpel, K. 2021. Summscreen: A dataset for abstractive screenplay summarization. arXiv preprint arXiv:2104.07091

    arXiv 2021

  3. [3]

    Cheng, Z.; Leng, S.; Zhang, H.; Xin, Y.; Li, X.; Chen, G.; Zhu, Y.; Zhang, W.; Luo, Z.; Zhao, D.; and Bing, L. 2024. VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs. arXiv preprint arXiv:2406.07476

    arXiv 2024

  4. [4]

    Choi, J.; Lee, S.; Chu, J.; Choi, M.; and Kim, H. J. 2024. vid-TLDR: Training Free Token Merging for Light-weight Video Transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 18771--18781

    work page 2024

  5. [5]

    W.; Hou, L.; Longpre, S.; Zoph, B.; Tay, Y.; Fedus, W.; Li, E.; Wang, X.; Dehghani, M.; Brahma, S.; Webson, A.; Gu, S

    Chung, H. W.; Hou, L.; Longpre, S.; Zoph, B.; Tay, Y.; Fedus, W.; Li, E.; Wang, X.; Dehghani, M.; Brahma, S.; Webson, A.; Gu, S. S.; Dai, Z.; Suzgun, M.; Chen, X.; Chowdhery, A.; Narang, S.; Mishra, G.; Yu, A.; Zhao, V.; Huang, Y.; Dai, A.; Yu, H.; Petrov, S.; Chi, E. H.; Dean, J.; Devlin, J.; Roberts, A.; Zhou, D.; Le, Q. V.; and Wei, J. 2022. Scaling In...

    work page 2022

  6. [6]

    Curtis, K.; Awad, G.; Godil, A.; and Soboroff, I. 2023. The ACM Multimedia 2023 Deep Video Understanding Grand Challenge. In Proceedings of the 31st ACM International Conference on Multimedia, MM '23, 9606–9609. New York, NY, USA: Association for Computing Machinery. ISBN 9798400701085

    work page 2023

  7. [7]

    Curtis, K.; Awad, G.; Rajput, S.; and Soboroff, I. 2020. HLVU: A New Challenge to Test Deep Understanding of Movies the Way Humans do. In Proceedings of the 2020 International Conference on Multimedia Retrieval, ICMR '20, 355–361. New York, NY, USA: Association for Computing Machinery. ISBN 9781450370875

    work page 2020

  8. [8]

    Curtis, K.; Awad, G.; Rajput, S.; and Soboroff, I. 2022. The ACM Multimedia 2022 Deep Video Understanding Grand Challenge. In Proceedings of the 30th ACM International Conference on Multimedia, MM '22, 7075–7078. New York, NY, USA: Association for Computing Machinery. ISBN 9781450392037

    work page 2022

Show all 46 references
  1. [9]

    G., Victor Escorcia; and Niebles, J

    Fabian Caba Heilbron, B. G., Victor Escorcia; and Niebles, J. C. 2015. ActivityNet: A Large-Scale Video Benchmark for Human Activity Understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 961--970

    2015

  2. [10]

    Y.; Wang, L.; and Liu, Z

    Fu, T.-J.; Li, L.; Gan, Z.; Lin, K.; Wang, W. Y.; Wang, L.; and Liu, Z. 2023. An Empirical Study of End-to-End Video-Language Transformers With Masked Visual Modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 22898--22909

    2023

  3. [11]

    Fung, Y.; Wang, H.; Wang, T.; Kebarighotbi, A.; Bansal, M.; Ji, H.; and Natarajan, P. 2023. DeepMaven: Deep question answering on long-distance movie/TV show videos with multimedia knowledge extraction and synthesis. In Proceedings of the 17th Conference of the European Chapte...

    2023

  4. [12]

    Garcia, N.; Otani, M.; Chu, C.; and Nakashima, Y. 2020. KnowIT VQA: Answering Knowledge-Based Questions about Videos. Proceedings of the AAAI Conference on Artificial Intelligence, 34(07): 10826--10834

    2020

  5. [13]

    Guo, J.; Liang, C.; and Wang, Z. 2023. Who, What and Where: Composite-semantic Instance Search for Story Videos. In 2023 IEEE International Conference on Multimedia and Expo (ICME), 858--863. IEEE

    2023

  6. [14]

    K.; Jia, M.; Cao, X.; Shah, A.; Shrivastava, A.; and Lim, S.-N

    He, B.; Li, H.; Jang, Y. K.; Jia, M.; Cao, X.; Shah, A.; Shrivastava, A.; and Lim, S.-N. 2024. Ma-lmm: Memory-augmented large multimodal model for long-term video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13504--13514

    2024

  7. [15]

    Jin, P.; Takanobu, R.; Zhang, W.; Cao, X.; and Yuan, L. 2024. Chat-univi: Unified visual representation empowers large language models with image and video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13700--13710

    2024

  8. [16]

    Kim, K.-M.; Heo, M.-O.; Choi, S.-H.; and Zhang, B.-T. 2017. Deepstory: Video story qa by deep embedded memory networks. arXiv preprint arXiv:1707.00836

    2017 arXiv

  9. [17]

    Lei, J.; Yu, L.; Bansal, M.; and Berg, T. L. 2018. TVQA: Localized, Compositional Video Question Answering. In Empirical Methods in Natural Language Processing

    2018

  10. [18]

    Lei, J.; Yu, L.; Berg, T.; and Bansal, M. 2020. TVQA +: Spatio-Temporal Grounding for Video Question Answering. In Jurafsky, D.; Chai, J.; Schluter, N.; and Tetreault, J., eds., Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 8211--8225...

    2020

  11. [19]

    Li, K.; Wang, Y.; He, Y.; Li, Y.; Wang, Y.; Liu, Y.; Wang, Z.; Xu, J.; Chen, G.; Luo, P.; Wang, L.; and Qiao, Y. 2024. MVBench: A Comprehensive Multi-modal Video Understanding Benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR...

    2024

  12. [20]

    Li, L.; Chen, Y.-C.; Cheng, Y.; Gan, Z.; Yu, L.; and Liu, J. 2020. HERO : Hierarchical Encoder for V ideo+ L anguage Omni-representation Pre-training. In Webber, B.; Cohn, T.; He, Y.; and Liu, Y., eds., Proceedings of the 2020 Conference on Empirical Methods in Natural Languag...

    2020

  13. [21]

    Li, R.; Guo, J.; Li, M.; Wu, Z.; and Liang, C. 2023. A Hierarchical Deep Video Understanding Method with Shot-Based Instance Search and Large Language Model. In Proceedings of the 31st ACM International Conference on Multimedia, MM '23, 9425–9429. New York, NY, USA: Associatio...

    2023

  14. [22]

    Liang, C.; Zhang, Y.; Cheng, J.; Xu, C.; and Lu, H. 2009. A Novel Role-Based Movie Scene Segmentation Method. In Muneesawang, P.; Wu, F.; Kumazawa, I.; Roeksabutr, A.; Liao, M.; and Tang, X., eds., Advances in Multimedia Information Processing - PCM 2009, 917--922. Berlin, Hei...

    2009

  15. [23]

    Ma, K.; Jurczyk, T.; and Choi, J. D. 2018. Challenging reading comprehension on daily conversation: Passage completion on multiparty dialog. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tec...

    2018

  16. [24]

    Maaz, M.; Rasheed, H.; Khan, S.; and Khan, F. S. 2024. Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)

    2024

  17. [25]

    Mangalam, K.; Akshulakov, R.; and Malik, J. 2023. EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding. In Oh, A.; Naumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds., Advances in Neural Information Processing Systems, volume 36...

    2023

  18. [26]

    Nguyen, T.; Hu, Z.; Wu, X.; Nguyen, C.-D.; Ng, S.-K.; and Luu, A. T. 2024. Encoding and Controlling Global Semantics for Long-form Video Question Answering. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Empirical Methods in Natural...

    2024

  19. [27]

    Niu, Y.; Liang, C.; Lu, A.; Huang, B.; Wang, Z.; and Guo, J. 2023. Person-action Instance Search in Story Videos: An Experimental Study. ACM Trans. Inf. Syst., 42(2)

    2023

  20. [28]

    Ren, S.; Yao, L.; Li, S.; Sun, X.; and Hou, L. 2024. Timechat: A time-sensitive multimodal large language model for long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14313--14323

    2024

  21. [29]

    Sang, J.; Liang, C.; Xu, C.; and Cheng, J. 2011. Robust movie character identification and the sensitivity analysis. In 2011 IEEE International Conference on Multimedia and Expo, 1--6

    2011

  22. [30]

    Song, E.; Chai, W.; Wang, G.; Zhang, Y.; Zhou, H.; Wu, F.; Chi, H.; Guo, X.; Ye, T.; Zhang, Y.; et al. 2024. Moviechat: From dense token to sparse memory for long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18221--18232

    2024

  23. [31]

    Tapaswi, M.; Zhu, Y.; Stiefelhagen, R.; Torralba, A.; Urtasun, R.; and Fidler, S. 2016. Movieqa: Understanding stories in movies through question-answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, 4631--4640

    2016

  24. [32]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozi \`e re, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023 a . Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  25. [33]

    Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023 b . Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  26. [34]

    Wang, X.; Chen, J.; Wang, Z.; Liu, W.; Satoh, S.; Liang, C.; and Lin, C.-W. 2020. When Pedestrian Detection Meets Nighttime Surveillance: A New Benchmark. In Bessiere, C., ed., Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20 ...

    2020

  27. [35]

    Xiao, J.; Shang, X.; Yao, A.; and Chua, T.-S. 2021. NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9777--9786

    2021

  28. [36]

    Xu, D.; Zhao, Z.; Xiao, J.; Wu, F.; Zhang, H.; He, X.; and Zhuang, Y. 2017. Video Question Answering via Gradually Refined Attention over Appearance and Motion. In ACM Multimedia

    2017

  29. [37]

    Xu, Y.; Wei, Y.; and Wu, B. 2023. Query-aware Long Video Localization and Relation Discrimination for Deep Video Understanding. In Proceedings of the 31st ACM International Conference on Multimedia, MM '23, 9591–9595. New York, NY, USA: Association for Computing Machinery. ISB...

    2023

  30. [38]

    Yang, A.; Miech, A.; Sivic, J.; Laptev, I.; and Schmid, C. 2021. Just Ask: Learning to Answer Questions from Millions of Narrated Videos. In ICCV

    2021

  31. [39]

    Yang, Z.; and Choi, J. D. 2019. FriendsQA: Open-domain question answering on TV show transcripts. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, 188--197

    2019

  32. [40]

    Yu, S.; Cho, J.; Yadav, P.; and Bansal, M. 2023. Self-Chained Image-Language Model for Video Localization and Question Answering. In NeurIPS

    2023

  33. [41]

    Yu, Y.; Kim, J.; and Kim, G. 2018. A Joint Sequence Fusion Model for Video Question Answering and Retrieval. In Proceedings of the European Conference on Computer Vision (ECCV)

    2018

  34. [42]

    Yu, Z.; Xu, D.; Yu, J.; Yu, T.; Zhao, Z.; Zhuang, Y.; and Tao, D. 2019. ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01): 9127--9134

    2019

  35. [43]

    Zhang, B.; Fang, Y.; Ren, T.; and Wu, G. 2022. Multimodal analysis for deep video understanding with video language transformer. In Proceedings of the 30th ACM International Conference on Multimedia, 7165--7169

    2022

  36. [44]

    Zhong, Y.; Ji, W.; Xiao, J.; Li, Y.; Deng, W.; and Chua, T.-S. 2022. Video Question Answering: Datasets, Algorithms and Challenges. In Goldberg, Y.; Kozareva, Z.; and Zhang, Y., eds., Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 6439-...

    2022

  37. [45]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  38. [46]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.