REVIEW 9 cited by
StreamChat: Chatting with Streaming Video
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
This paper presents StreamChat, a novel approach that enhances the interaction capabilities of Large Multimodal Models (LMMs) with streaming video content. In streaming interaction scenarios, existing methods rely solely on visual information available at the moment a question is posed, resulting in significant delays as the model remains unaware of subsequent changes in the streaming video. StreamChat addresses this limitation by innovatively updating the visual context at each decoding step, ensuring that the model utilizes up-to-date video content throughout the decoding process. Additionally, we introduce a flexible and efficient crossattention-based architecture to process dynamic streaming inputs while maintaining inference efficiency for streaming interactions. Furthermore, we construct a new dense instruction dataset to facilitate the training of streaming interaction models, complemented by a parallel 3D-RoPE mechanism that encodes the relative temporal information of visual and text tokens. Experimental results demonstrate that StreamChat achieves competitive performance on established image and video benchmarks and exhibits superior capabilities in streaming interaction scenarios compared to state-of-the-art video LMM.
Forward citations
Cited by 9 Pith papers
-
Think in Sets for Streaming Video Token Compression
NovaCov uses a bounded, recency-weighted historical reference bank and a dual-branch submodular coverage objective to select streaming video tokens, outperforming training-free baselines.
-
Vinci2: Providing Proactive Assistance in Continuous Egocentric Videos
EgoMemo uses multi-scale temporal summaries, a knowledge graph, and visual archives to decide whether and when to intervene proactively on continuous egocentric video, setting baselines on the new EgoServe benchmark o...
-
ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding
Training-free latent-object memory anchors let frozen Video-LLMs retain object histories under a tight token budget and improve streaming and long-video QA.
-
FOLIO: Focused Semantic Memory for Streaming Video Understanding
Entity-centered focus-guided streaming memory lifts Qwen3-VL-8B to 82.0/69.1 Perception/Backward on OVO-Bench and 74.5 on StreamingBench while cutting writer tokens by ~32%.
-
Learning Streaming Video Representation via Multitask Training
A causal-temporal-attention video backbone, trained jointly on global, temporal, and spatial visual-language tasks, transfers to multiple streaming video benchmarks with strong efficiency.
-
LiveCC: Learning Video LLM with Streaming Speech Transcription at Scale
A video LLM trained on millions of YouTube videos with timestamp-aligned automatic speech recognition transcripts achieves state-of-the-art 7B-scale video QA and wins against 72B models on a custom real-time commentar...
-
Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark
Video-MMLU: a 1,065-video lecture benchmark where most AI video models score 10-50%, but text-only models answer 40% of quiz questions without video.
-
Existence of small semi-vortex solutions for the cubic nonlinear Schr\"{o}dinger system with Rashba type Spin-Orbit coupling on $\mathbb{R}^2$
Small semi-vortex and ground-state solutions of the cubic NLS system with Rashba SOC on R² exist as energy minimizers under small mass, via concentration-compactness.
-
cuRAMSES: Scalable AMR Optimizations for Large-Scale Cosmological Simulations
Recursive k-section domain decomposition, Morton-key hashing, and GPU dispatch cut communication and memory bottlenecks in RAMSES while preserving conservation to ~0.5%.
Discussion (0). Continue with ORCID to comment.