Pith. sign in

REVIEW 4 cited by

VISA: Reasoning Video Object Segmentation via Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.11325 v1 pith:4EHNOXAF submitted 2024-07-16 cs.CV

Cilin Yan , Haochen Wang , Shilin Yan , Xiaolong Jiang , Yao Hu , Guoliang Kang , Weidi Xie , Efstratios Gavves This is my paper · ORCID
classification cs.CV
keywords segmentationreasoningvideocomplexknowledgevisaworldobject
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Existing Video Object Segmentation (VOS) relies on explicit user instructions, such as categories, masks, or short phrases, restricting their ability to perform complex video segmentation requiring reasoning with world knowledge. In this paper, we introduce a new task, Reasoning Video Object Segmentation (ReasonVOS). This task aims to generate a sequence of segmentation masks in response to implicit text queries that require complex reasoning abilities based on world knowledge and video contexts, which is crucial for structured environment understanding and object-centric interactions, pivotal in the development of embodied AI. To tackle ReasonVOS, we introduce VISA (Video-based large language Instructed Segmentation Assistant), to leverage the world knowledge reasoning capabilities of multi-modal LLMs while possessing the ability to segment and track objects in videos with a mask decoder. Moreover, we establish a comprehensive benchmark consisting of 35,074 instruction-mask sequence pairs from 1,042 diverse videos, which incorporates complex world knowledge reasoning into segmentation tasks for instruction-tuning and evaluation purposes of ReasonVOS models. Experiments conducted on 8 datasets demonstrate the effectiveness of VISA in tackling complex reasoning segmentation and vanilla referring segmentation in both video and image domains. The code and dataset are available at https://github.com/cilinyan/VISA.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Beyond Frame Selection: Generative Latent Evidence Aggregation for Long-Video Understanding

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A query-conditioned latent evidence aggregator after frozen frame selection improves long-video QA by up to +5.2 average / +10.1 LVBench with 0.11–0.40% token overhead.

  2. Unleashing Hierarchical Reasoning: An LLM-Driven Framework for Training-Free Referring Video Object Segmentation

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A training-free pipeline parses a video query with an LLM, tracks candidates with SAM2, then uses coarse motion plus conditional pose reasoning to segment the referred object.

  3. Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaboration

    cs.CV 2025-05 conditional novelty 6.0 of 10

    Omni-R1 uses GRPO reinforcement learning to train a global reasoning model that selects keyframes and rewrites queries for a detail model, improving video and audio-visual segmentation and out-of-domain QA.

  4. Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Codec-guided sparse patch selection plus a lightweight speak/silent gate yields a 4B streaming VLM that is competitive on static tasks, stronger on video/spatial benchmarks, and much cheaper at inference.

Pith tools