Pith. sign in

REVIEW 3 major objections 4 minor 35 references

StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read StreamArena introduces an hour-scale, open-ended benchmark for streaming video understanding, and StreamMind, a two-tier system with persistent asynchronous memory, ranks first among streaming systems on all four capabilities while…

desk verdict A genuinely useful hour-scale streaming-video benchmark with a clean reference architecture, but the headline accuracy claims rest on an unequal baseline harness and an unverified tool metric. read the letter →

arxiv 2608.05703 v1 pith:BZAVZ4T3 submitted 2026-08-06 cs.CV

Xichen Zhang , Guankai Li , Yinghao Zhu , Shijian Wang , Sitong Wu , Shaozuo Yu , Meng Chu , Yuan Lu
show 1 more author
Jiaya Jia
This is my paper · ORCID
classification cs.CV
keywords streamingvideounderstandinglong-horizonmemorymultimodalagentsproactiveinteractionbenchmarkopen-endedevaluationtoolusepersistent
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that current streaming-video evaluation is misleading and that a decoupled system design can fix both measurement and performance. It introduces StreamArena, 243 hour-scale videos with 3,646 open-ended, timestamped question-answer pairs covering real-time perception, historical retrospection, proactive alerting, and tool use. Under this test, minimal baselines that only look at the latest few frames, methods that convert past video into text, and methods that repeatedly compress visual memory all fail on at least one capability. The paper's StreamMind separates fast frontend interaction from asynchronous backend memory construction and retrieval, and the reported results improve over the strongest streaming baseline in each capability by 53.7% to 228.1% while reducing pooled query-to-answer latency by 66.2% on a shared backbone. If these results hold, always-on agents can be both more responsive and more accurate by reusing persistent multimodal state built before the question arrives.

What carries the argument

The load-bearing mechanism is the two-tier StreamMind architecture. A frontend tier pairs a Front Worker, which answers, delegates, or spawns monitors, with independently scheduled Monitor Workers that track future conditions without blocking interaction. A backend tier runs a Memory Writer that continuously turns incoming frames and speech into a Memory Bank containing a hierarchical event layer (micro, macro, and super events), an entity-relation graph, and persistent key frames, plus Router, Recall, and Search workers that retrieve and synthesize evidence on demand. Because memory is built before a query arrives and is reused across questions, the latency-critical path shrinks to frontend dispatch, targeted retrieval, and final inference; the same persistent state also preserves multimodal evidence that text summaries or repeated compression lose.

What would settle it

Run the recent-window baselines in their native continuous-ingestion mode with state persisted across all questions on StreamArena, then compare historical-retrospection and tool accuracy; if those numbers rise to or above StreamMind's reported 34.9% HR and 11.6% Tool, the claimed 53.7%-228.1% improvements are artifacts of the replay harness.

Watch

Extended reading notes

Core claim

The paper's central claim is that long-horizon streaming video understanding is best evaluated as a causal, open-ended, multi-capability task rather than as short-clip multiple choice. Its benchmark forces each system to answer without answer choices, with the query timestamp and every supporting evidence segment annotated separately, so a model cannot cheat by recency or language priors. The central finding is a tension: recent-window models miss distant events, text-summary models lose visual evidence, and model-internal compression models lose fine detail over time. StreamMind addresses the tension with two tiers: frontend workers handle immediate interaction and condition monitoring, while backend workers asynchronously build a hierarchical event memory, an entity-relation graph, and key frames, then retrieve on demand. Under a shared 397B-parameter backbone, StreamMind ranks first among streaming systems on all four capabilities, improving on the strongest corresponding baseline by 58.4% on real-time perception, 53.7% on historical retrospection, 228.1% on tool use, and 54.7% on proactive interaction, while reducing pooled query-to-answer latency by 66.2% and retaining 89.7% of pooled accuracy.

Load-bearing premise

The load-bearing premise, stated in Appendix C.1, is that running each streaming baseline through its released query-triggered replay interface—which reconstructs a causal prefix or recent window per task and carries no hidden video state across questions—faithfully represents how that system would perform under continuous ingestion; if always-on models are substantially better in their native mode, the reported 53.7%-228.1% gains would shrink.

Editorial extensions

If this is right

  • Short-clip, multiple-choice streaming benchmarks can be gamed: a system using only the last four frames can match or beat complex streaming models, so hour-scale open-ended tasks are needed to expose real comprehension.
  • No single memory design covers all four capabilities; recent windows, text summaries, and internal compression each fail on at least one axis, so continuous interaction and long-horizon comprehension require separate execution paths.
  • Persistent multimodal memory with retrievable key frames improves historical retrospection at every evidence-to-query gap, not just at short gaps.
  • Reusing state built before a query reduces query-to-answer latency by 66.2% on a shared backbone while retaining 89.7% of pooled accuracy.
  • Proactive monitoring needs dedicated workers with a timed alert rule; a turn-based model that only answers when prompted cannot receive credit for autonomous alerting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the reported gains compare against baseline harnesses that reconstruct a causal prefix or recent window per question and carry no hidden video state; running recent-window systems in true continuous-ingestion mode with state preserved across tasks could narrow the 53.7%-228.1% improvements.
  • Editorial extension: if the 66.2% latency reduction transfers beyond this benchmark, the same frontend-backend decoupling could apply to embodied or wearable agents where continuous sensing and interactive response compete for the same model.
  • Editorial extension: ingestion at 2 fps with five-second audio windows makes very short transient events invisible to StreamMind; adaptive sampling or higher capture rates near detected salience might further lift real-time perception.
  • Editorial extension: the benchmark's 243 videos are heavily Mandarin-dominant (189 of 243), so the reported capability numbers may not fully reflect performance on other languages and cultural contexts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces StreamArena, a benchmark for hour-scale interactive streaming video understanding, containing 243 full-length videos (88.8 minutes on average) and 3,646 open-ended, human-validated question-answer pairs with separate query and evidence timestamps. Four capabilities are targeted: real-time multimodal perception, historical retrospection, proactive interaction, and multimodal tool utilization. The paper also proposes StreamMind, a two-tier architecture in which latency-critical frontend workers handle interaction and monitoring while backend workers asynchronously build persistent multimodal memory and perform recall and external search. Experiments compare offline turn-based models, recent-window methods, text-summary methods, model-internal compression methods, and StreamMind, reporting that StreamMind ranks first among streaming systems on all four capabilities, with relative improvements of 53.7% to 228.1% over the strongest corresponding streaming baselines and a 66.2% pooled query-to-answer latency reduction under a shared Qwen3.5-397B-A17B backbone.

Significance. If the main comparative claim holds, StreamArena is a valuable addition to the streaming-video evaluation landscape: it uses hour-scale videos, open-ended answers rather than multiple choice, dual temporal grounding for queries and evidence, and a four-capability coverage that existing benchmarks do not jointly provide. The paper also ships substantial artifacts, including the annotation data under CC BY 4.0, evaluation harness code, baseline adapters, and the StreamMind implementation, and it explicitly documents configuration details and reproducibility settings. The diagnostic analysis of modality availability, frame budgets, resolution, and reasoning mode is useful and appropriately caveated. My assessment is conditional, however, because the central comparative claim rests on an unequal evaluation harness for the streaming baselines, the Tool metric does not verify tool invocation despite being presented as a tool-utilization capability, and all quantitative results come from a single run with no uncertainty estimates.

major comments (3)
  1. [Section 3.1, Appendix C.1, Table 10] The headline claim that StreamMind outperforms 'existing streaming baselines' is evaluated under an unequal harness. StreamMind ingests the stream continuously with persistent state (Eq. (11)-(12)), while every baseline is evaluated through query-triggered replay that reconstructs a causal prefix or recent window per task and carries no hidden video state across questions, as stated in Appendix C.1 and Table 10. For always-on systems such as AURA and MiniCPM-o-4.5, this may understate their capabilities: for example, MiniCPM-o's reactive protocol in Appendix C.3 is a half-duplex recent-window pass, not the native full-duplex sliding-context interface used for proactive tasks. Because the abstract and contributions claim improvement over streaming baselines, the comparison should either run these baselines in a continuous, stateful mode where such a mode exists, or the claims should be explicitly restricted to query-triggered replay baselines. This is load-bearing for all four headline improvement percentages.
  2. [Section 2.1, Eq. (1), Appendix C.4, Section 3.2] The Tool metric is not a measure of tool utilization. Section 2.1 explicitly states that 'answer correctness alone does not establish whether or how a tool was invoked,' and Appendix C.4 notes for ThinkStream that its Tool score 'measures answer correctness without guaranteed tool invocation.' Yet Table 3 and Section 3.2 present Tool as one of the four capabilities and headline a 228.1% relative improvement. Because Tool questions ask for facts such as actor parent nationalities that may already be in the backbone's parametric memory, a correct answer can be produced with zero tool calls. One of the four headline capabilities and the largest reported gain are therefore not actually validated as tool use. The metric should require a traceable tool invocation whose returned evidence contributes to the answer, or the capability should be renamed and qualified as tool-enabled question answering.
  3. [Appendix C.6, C.7] All accuracy and latency numbers in Table 3, Table 4, Table 5, and Figure 3 come from a single evaluation run, and the paper states that standard deviations and significance tests are not reported. Given that the central claims are quantitative relative improvements (53.7%-228.1% accuracy gains and a 66.2% latency reduction), single-run point estimates provide no indication of variability or robustness. At minimum, repeated runs or bootstrap confidence intervals on a subset such as the 616-question diagnostic set should be provided, or the claims should be softened to single-run observations.
minor comments (4)
  1. [Table 1 caption] The column abbreviations in Table 1 (e.g., 'Omni', 'MT', 'Pro.') are not defined in the caption; please spell them out or add a legend.
  2. [Section 2.1, Eq. (1)] The judge function Judge(·) is used in Eq. (1) before the judge model (Gemini 3.1 Pro) is introduced in Section 3.1; moving the judge specification to Section 2.1 would improve readability.
  3. [Figure 2] Figure 2 is very dense; labeling the worker roles directly in the figure or adding a short legend for the solid and dashed flows would make the architecture easier to follow.
  4. [Appendix C.5] The large number of hyperparameters in Appendix C.5 would be easier to digest as a summary table, with the prose paragraphs reserved for details.

Circularity Check

0 steps flagged · score 2.0 of 10

No formal circularity: the results are empirical benchmark measurements with one non-load-bearing self-citation and an explicitly disclosed Tool-metric validity caveat.

full rationale

StreamArena's central claims are empirical measurements over a newly constructed benchmark, not derivations from fitted parameters, and no equation reduces a reported result to its own input. Accuracy (Eq. 1) is an external LLM-judge factual-core match applied uniformly to all systems; latency is wall-clock; Proactive-Acc (Eq. 2) uses an independently specified timing rule. The judge (Gemini 3.1 Pro) is distinct from the evaluated Gemini 3.5 Flash baseline, so the main comparison has an external assessor. Authors designing both the benchmark and the method is standard practice and does not by itself constitute circularity. The one self-citation, HyperEyes [35], is used only for the crop-based image-search protocol inside one worker (Appendix C.5, Search Worker); it is a protocol detail, not a load-bearing premise of the accuracy or latency claims. The paper is also transparent about the strongest caveat: Tool accuracy is defined as tool-enabled end-to-end answer accuracy, with Section 2.1 stating 'answer correctness alone does not establish whether or how a tool was invoked,' and Appendix C.4 noting that ThinkStream's Tool score 'measures answer correctness without guaranteed tool invocation.' This is a construct-validity limitation: a correct answer could in principle come from parametric memory rather than an actual tool call, so the 228.1% Tool improvement is best read as end-to-end answer accuracy on tool-use questions, not as a verified tool-invocation rate. That caveat affects one headline interpretation, but it is an explicitly disclosed measurement scope rather than a circular reduction: the metric is not fitted to the answer, the judge is external, and no equation makes Tool accuracy equal to the method's own parameters. Similarly, the query-triggered-replay evaluation of always-on baselines (Appendix C.1) is a fairness concern openly discussed in the paper, not a circularity. Overall, the derivation chain is self-contained and the circularity score is therefore low.

Assumptions & free parameters 9 free parameters · 5 assumptions · 3 invented entities

The benchmark and system rest on hand-set hyperparameters that are not fitted by automated search, but also not validated by ablations or significance tests. The evaluation depends on an external LLM judge and on annotator accuracy, both taken as background assumptions. The main invented constructs are software components whose individual contributions are not isolated.

free parameters (9)
  • Memory Writer interval delta_W = 4 s
    Controls how often visual observations are consolidated into events; hand-set, no automated search reported.
  • Front Worker recent frames N_F = 12 frames
    Recent context used for direct answers; affects the latency-accuracy trade-off.
  • Monitor check interval delta_M = 2 s
    Frequency at which Monitor Workers check the latest frames; hand-set and affects proactive timing sensitivity.
  • Monitor trigger confidence theta_M = 0.6
    Alert fires only when the JSON verdict confidence reaches this value; hand-set and directly affects proactive accuracy.
  • Router ReAct rounds R_R = 4
    Maximum reasoning rounds for retrieval composition; hand-set.
  • Recall rounds R_C = 4
    Maximum Observe-Reason-Act rounds for memory retrieval; hand-set.
  • Search rounds R_S = 3
    Maximum rounds for external search; hand-set.
  • Entity canonicalization threshold = 0.85
    Similarity threshold for merging entity names; hand-set and affects entity graph quality.
  • FrameStore image cap = 4000 images
    Maximum retained key frames; hand-set and bounds memory while preserving visual evidence.
assumptions (5)
  • domain assumption Gemini 3.1 Pro used as judge makes reliable factual-equivalence decisions for open-ended video QA.
    All accuracy numbers depend on this judge; no human-judge agreement or calibration is reported (Section 3.1, Eq. 1).
  • domain assumption Annotated ground truth and timestamps are correct after the three-stage validation pipeline.
    Benchmark validity rests on annotator accuracy; no inter-annotator agreement statistics are reported (Section 2.1).
  • domain assumption Open-ended answers can be reduced to a single factual core that a strict binary judge can detect.
    The metrics treat each question as having a unique factual core, which may not hold for multi-part or open-ended questions.
  • ad hoc to paper Evaluating streaming baselines via their released replay interfaces without persistent state is a faithful deployment proxy.
    This assumption is load-bearing for the comparative claim; the paper discloses it in Appendix C.1 but does not validate it against a continuous-ingestion variant.
  • domain assumption YouTube video IDs remain accessible or have archived substitutes.
    Reproducibility depends on video availability; the paper tracks and substitutes expired videos.
invented entities (3)
  • Hierarchical event memory (micro, macro, super events)
    purpose: Stores observations at multiple time scales for later recall.
    No ablation isolates the contribution of the hierarchy; Figure 3 diagnostics only vary frame count, resolution, and reasoning mode of an offline model, not the memory hierarchy.
  • Monitor Worker
    purpose: Continuously checks a user-registered condition and fires an alert without a new prompt.
    Proactive gains are measured, but the component is not ablated; no separate evidence that the worker mechanism, rather than the backbone, causes the improvement.
  • Router-mediated Recall/Search composition
    purpose: Interleaves historical memory retrieval with external search.
    Tool metric does not verify tool invocation, so the coordination benefit is not independently evidenced.

how reviews work

0 comments
Cite this review

Pith. "Pith review of StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding." pith.science (2026). https://pith.science/paper/BZAVZ4T3

@misc{pith2026260805703,
  author       = {Pith},
  title        = {Pith review of: StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BZAVZ4T3}},
  note         = {Machine review of arXiv:2608.05703}
}
read the original abstract

Deploying autonomous multimodal agents in continuous, real-world environments requires them to ingest unbounded audio-visual streams and maintain hour-scale memory. However, current evaluations predominantly rely on brief clips and multiple-choice formats. This design allows minimal baselines that process only the last four frames to match or surpass complex streaming models, while answer options also expose language shortcuts. We introduce StreamArena, a benchmark for hour-scale, interactive streaming video understanding. StreamArena contains 243 full-length videos averaging 88.8 minutes and 3,646 rigorously annotated, open-ended question-answer pairs that evaluate real-time perception, historical retrospection, proactive interaction, and multimodal tool utilization. Evaluation across diverse systems exposes a tension between continuous interaction and long-horizon multimodal comprehension. Methods that retain only recent frames cannot recover distant events, methods that convert past observations into text lose visual evidence, and methods that repeatedly compress visual memory struggle to preserve fine-grained details over time. We address this tension with StreamMind, a two-tier architecture that assigns latency-critical interaction and proactive monitoring to independently scheduled frontend workers, while backend workers asynchronously construct persistent multimodal memory and perform historical recall and external search. StreamMind outperforms existing streaming baselines across all four capabilities and reduces query-to-answer latency by reusing persistent state.

Figures

Figures reproduced from arXiv: 2608.05703 by the authors.

Figure 1
Figure 1. Overview of StreamArena. The top row summarizes (a) source domains, (b) task types, (c) video durations, and (d) temporal gaps between queries and supporting evidence. The remaining panels show one example of each evaluated capability. Red boxes mark the audio-visual evidence required to answer a question, while dark-blue boxes mark the query time and, for proactive tasks, the expected response time. Capability Laye… view at source ↗
Figure 2
Figure 2. Overview of StreamMind. The frontend handles interaction and monitoring; the backend maintains memory and performs retrieval. The Memory Bank combines hierarchical events, entity relations, and key frames. Temporal latency and triggering accuracy. For reactive tasks, we measure query-to-answer response latency as L = tresp − tquery. This interval includes every operation triggered by the query, including routing, re… view at source ↗
Figure 3
Figure 3. Accuracy (%) on the diagnostic subset under changes to (a) frame count, (b) resolution, and (c) reasoning mode. Method ACC ↑ Mean latency (s) ↓ A (RTP) B (HR) D (TOOL) Gemini 3.5 Flash† 63.1 86.9 123.9 139.5 Kimi-K2.6 54.5 162.8 187.9 128.2 Qwen3.5-397B-A17B 54.2 83.8 117.4 62.8 Qwen3.5-Omni† 44.6 50.8 56.4 103.7 MiMo-V2.5 43.3 224.5 266.5 187.4 StreamMind (Ours) 48.6 12.9 30.7 28.1 [PITH_FULL_IMAGE:figures/full_fi… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 14 canonical work pages

  1. [1]

    A simple baseline for streaming video under- standing.arXiv preprint arXiv:2604.02317, 2026

    Yujiao Shen, Shulin Tian, Jingkang Yang, and Ziwei Liu. A simple baseline for streaming video under- standing.arXiv preprint arXiv:2604.02317, 2026

    arXiv 2026

  2. [2]

    Improving patient safety through video monitoring.Rehabilitation Nursing, 2016

    Michele Cournan, Benjamin Fusco-Gessick, and Laura Wright. Improving patient safety through video monitoring.Rehabilitation Nursing, 2016

    work page 2016

  3. [3]

    Safefac: Video- based smart safety monitoring for preventing industrial work accidents.Expert Systems with Applications, 215:119397, 2023

    Jungmo Ahn, JaeYeon Park, Sung Sik Lee, Kyu-Hyuk Lee, Heesung Do, and JeongGil Ko. Safefac: Video- based smart safety monitoring for preventing industrial work accidents.Expert Systems with Applications, 215:119397, 2023

    work page 2023

  4. [4]

    When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysis.Accident Analysis & Prevention, 219:108077, 2025

    Ruixuan Zhang, Beichen Wang, Juexiao Zhang, Zilin Bian, Chen Feng, and Kaan Ozbay. When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysis.Accident Analysis & Prevention, 219:108077, 2025

    work page 2025

  5. [5]

    Interactive language: Talking to robots in real time.IEEE Robotics and Automation Letters, 2023

    Corey Lynch, Ayzaan Wahid, Jonathan Tompson, Tianli Ding, James Betker, Robert Baruch, Travis Armstrong, and Pete Florence. Interactive language: Talking to robots in real time.IEEE Robotics and Automation Letters, 2023

    2023

  6. [6]

    Robix: A unified model for robot interaction, reasoning and planning.arXiv preprint arXiv:2509.01106, 2025

    Huang Fang, Mengxi Zhang, Heng Dong, Wei Li, Zixuan Wang, Qifeng Zhang, Xueyun Tian, Yucheng Hu, and Hang Li. Robix: A unified model for robot interaction, reasoning and planning.arXiv preprint arXiv:2509.01106, 2025

    arXiv 2025

  7. [7]

    Video-llama: An instruction-tuned audio-visual language model for video understanding

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding. InProceedings of the 2023 conference on empirical methods in natural language processing: system demonstrations, pages 543–553, 2023

    2023

  8. [8]

    Video-llava: Learning united visual representation by alignment before projection

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. InProceedings of the 2024 conference on empirical methods in natural language processing, pages 5971–5984, 2024

    2024

Show all 35 references
  1. [9]

    Video-chatgpt: Towards detailed video understanding via large vision and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12585...

    2024

  2. [10]

    Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    2024 arXiv

  3. [11]

    Bytedance Seed. Seed2. 0 model card: Towards intelligence frontier for real-world complexity.arXiv preprint arXiv:2607.00248, 2026

    2026 arXiv

  4. [12]

    GPT-Realtime-2: A multimodal speech-to-speech large language model

    OpenAI. GPT-Realtime-2: A multimodal speech-to-speech large language model. https://openai. com, 2026. Accessed: 2026-07-15

    2026

  5. [13]

    Aura: Always-on understanding and real-time assistance via video streams

    Xudong Lu, Yang Bo, Jinpeng Chen, Shuhan Li, Xintong Guo, Huankang Guan, Fang Liu, Dunyuan Xu, Peiwen Sun, Heyang Sun, et al. Aura: Always-on understanding and real-time assistance via video streams. arXiv preprint arXiv:2604.04184, 2026

    2026 arXiv

  6. [14]

    Interaction models: A scalable approach to human-ai collaboration.Thinking Machines Lab: Connectionism, May 2026

    Thinking Machines Lab. Interaction models: A scalable approach to human-ai collaboration.Thinking Machines Lab: Connectionism, May 2026. https://thinkingmachines.ai/blog/interaction-models/

    2026

  7. [15]

    Videollm-online: Online video large language model for streaming video

    Joya Chen, Zhaoyang Lv, Shiwei Wu, Kevin Qinghong Lin, Chenan Song, Difei Gao, Jia-Wei Liu, Ziteng Gao, Dongxing Mao, and Mike Zheng Shou. Videollm-online: Online video large language model for streaming video. InProceedings of the IEEE/CVF Conference on Computer Vision and Pa...

    2024

  8. [16]

    Dispider: Enabling video llms with active real-time interaction via disentangled perception, decision, and reaction

    Rui Qian, Shuangrui Ding, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Dahua Lin, and Jiaqi Wang. Dispider: Enabling video llms with active real-time interaction via disentangled perception, decision, and reaction. InProceedings of the Computer Vision and Pattern Recogniti...

    2025

  9. [17]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. InProceedings of the IEEE/CVF conference on ...

    2025

  10. [18]

    Streamingbench: Assessing the gap for mllms to achieve streaming video understanding

    Junming Lin, Zheng Fang, Chi Chen, Haoxuan Cheng, Zihao Wan, Fuwen Luo, Ziyue Wang, Peng Li, Yang Liu, and Maosong Sun. Streamingbench: Assessing the gap for mllms to achieve streaming video understanding. InICASSP 2026-2026 IEEE International Conference on Acoustics, Speech a...

    2026

  11. [19]

    Omnimmi: A comprehensive multi-modal interaction benchmark in streaming video contexts

    Yuxuan Wang, Yueqian Wang, Bo Chen, Tong Wu, Dongyan Zhao, and Zilong Zheng. Omnimmi: A comprehensive multi-modal interaction benchmark in streaming video contexts. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18925–18935, 2025

    2025

  12. [20]

    Minicpm-o 4.5: Towards real-time full-duplex omni-modal interaction

    Junbo Cui, Bokai Xu, Chongyi Wang, Tianyu Yu, Weiyue Sun, Yingjing Xu, Tianran Wang, Zhihui He, Wenshuo Ma, Tianchi Cai, et al. Minicpm-o 4.5: Towards real-time full-duplex omni-modal interaction. arXiv preprint arXiv:2604.27393, 2026

    2026 arXiv

  13. [21]

    Video streaming thinking: Videollms can watch and think simultaneously.arXiv preprint arXiv:2603.12262, 2026

    Yiran Guan, Liang Yin, Dingkang Liang, Jianzhong Ju, Zhenbo Luo, Jian Luan, Yuliang Liu, and Xiang Bai. Video streaming thinking: Videollms can watch and think simultaneously.arXiv preprint arXiv:2603.12262, 2026

    2026 arXiv

  14. [22]

    Streamforest: Efficient online video understanding with persistent event memory

    Xiangyu Zeng, Kefan Qiu, Qingyu Zhang, Xinhao Li, Jing Wang, Jiaxin Li, Ziang Yan, Kun Tian, Meng Tian, Xinhai Zhao, et al. Streamforest: Efficient online video understanding with persistent event memory. Advances in Neural Information Processing Systems, 38:75804–75835, 2026

    2026

  15. [23]

    Thinking in streaming video.arXiv preprint arXiv:2603.12938, 2026

    Zikang Liu, Longteng Guo, Handong Li, Ru Zhen, Xingjian He, Ruyi Ji, Xiaoming Ren, Yanhao Zhang, Haonan Lu, and Jing Liu. Thinking in streaming video.arXiv preprint arXiv:2603.12938, 2026

    2026

  16. [24]

    Memdreamer: Decoupling perception and reasoning for long video understanding via hierarchical graph memory and agentic retrieval mechanism.arXiv preprint arXiv:2606.07512, 2026

    Cong Chen, Guo Gan, Kaixiang Ji, ChaoYang Zhang, Zhen Yang, Guangming Yao, Hao Chen, Jingdong Chen, Yi Yuan, and Chunhua Shen. Memdreamer: Decoupling perception and reasoning for long video understanding via hierarchical graph memory and agentic retrieval mechanism.arXiv prepr...

    2026 arXiv

  17. [25]

    Seeing, listening, remembering, and reasoning: A multimodal agent with long-term memory.arXiv preprint arXiv:2508.09736, 2025

    Lin Long, Yichen He, Wentao Ye, Yiyuan Pan, Yuan Lin, Hang Li, Junbo Zhao, and Wei Li. Seeing, listening, remembering, and reasoning: A multimodal agent with long-term memory.arXiv preprint arXiv:2508.09736, 2025

    2025

  18. [26]

    Egolife: Towards egocentric life assistant

    Jingkang Yang, Shuai Liu, Hongming Guo, Yuhao Dong, Xiamengwei Zhang, Sicheng Zhang, Pengyun Wang, Zitang Zhou, Binzhu Xie, Ziyue Wang, et al. Egolife: Towards egocentric life assistant. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 28885–28900, 2025

    2025

  19. [27]

    Agentic very long video understanding.arXiv preprint arXiv:2601.18157, 2026

    Aniket Rege, Arka Sadhu, Yuliang Li, Kejie Li, Ramya Korlakai Vinayak, Yuning Chai, Yong Jae Lee, and Hyo Jin Kim. Agentic very long video understanding.arXiv preprint arXiv:2601.18157, 2026

    2026 arXiv

  20. [28]

    Longvideobench: A benchmark for long-context interleaved video-language understanding.Advances in Neural Information Processing Systems, 37:28828– 28857, 2024

    Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-context interleaved video-language understanding.Advances in Neural Information Processing Systems, 37:28828– 28857, 2024

    2024

  21. [29]

    Mlvu: Benchmarking multi-task long video understanding

    Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zhang, et al. Mlvu: Benchmarking multi-task long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1369...

    2025

  22. [30]

    Ovo-bench: How far is your video-llms from real-world online video understanding? InProceedings of the Computer Vision and Pattern Recognition Conference, pages 18902–18913, 2025

    Junbo Niu, Yifei Li, Ziyang Miao, Chunjiang Ge, Yuanhang Zhou, Qihao He, Xiaoyi Dong, Haodong Duan, Shuangrui Ding, Rui Qian, et al. Ovo-bench: How far is your video-llms from real-world online video understanding? InProceedings of the Computer Vision and Pattern Recognition C...

    2025

  23. [31]

    Online video understanding: Ovbench and videochat-online

    Zhenpeng Huang, Xinhao Li, Jiaqi Li, Jing Wang, Xiangyu Zeng, Cheng Liang, Tao Wu, Xi Chen, Liang Li, and Limin Wang. Online video understanding: Ovbench and videochat-online. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 3328–3338, 2025

    2025

  24. [32]

    Rtv-bench: Benchmarking mllm continuous perception, understanding and reasoning through real-time video.Advances in Neural Information Processing Systems, 38, 2026

    Shuhang Xun, Sicheng Tao, Jungang Li, Yibo Shi, Zhixin Lin, Zhanhui Zhu, Yibo Yan, Hanqian Li, Linghao Zhang, Shikang Wang, et al. Rtv-bench: Benchmarking mllm continuous perception, understanding and reasoning through real-time video.Advances in Neural Information Processing ...

    2026

  25. [33]

    Ost- bench: Evaluating the capabilities of mllms in online spatio-temporal scene understanding.Advances in Neural Information Processing Systems, 38, 2026

    Jingli Lin, Chenming Zhu, Runsen Xu, Xiaohan Mao, Xihui Liu, Tai Wang, and Jiangmiao Pang. Ost- bench: Evaluating the capabilities of mllms in online spatio-temporal scene understanding.Advances in Neural Information Processing Systems, 38, 2026

    2026

  26. [34]

    Can vision-language models answer face to face questions in the real-world?arXiv preprint arXiv:2503.19356, 2025

    Reza Pourreza, Rishit Dagli, Apratim Bhattacharyya, Sunny Panchal, Guillaume Berger, and Roland Memisevic. Can vision-language models answer face to face questions in the real-world?arXiv preprint arXiv:2503.19356, 2025

    2025

  27. [35]

    what room number did the actress mention?

    Guankai Li, Jiabin Chen, Yi Xu, Xichen Zhang, and Yuan Lu. Hypereyes: Dual-grained efficiency-aware reinforcement learning for parallel multimodal search agents.arXiv preprint arXiv:2605.07177, 2026. 12 A Related work Interactive streaming video understanding.Traditional video...

    2026 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.