Pith. sign in

REVIEW 9 cited by

GroundingGPT:Language Enhanced Multi-modal Grounding Model

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.06071 v5 pith:J54DOTMC submitted 2024-01-11 cs.CV cs.CL

Zhaowei Li , Qi Xu , Dong Zhang , Hang Song , Yiqing Cai , Qi Qi , Ran Zhou , Junting Pan
show 4 more authors
Zefeng Li Van Tu Vu Zhida Huang Tao Wang
This is my paper
classification cs.CVcs.CL
keywords multi-modalmodelmodelsinformationtasksacrossdatasetgroundinggpt
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Multi-modal large language models have demonstrated impressive performance across various tasks in different modalities. However, existing multi-modal models primarily emphasize capturing global information within each modality while neglecting the importance of perceiving local information across modalities. Consequently, these models lack the ability to effectively understand the fine-grained details of input data, limiting their performance in tasks that require a more nuanced understanding. To address this limitation, there is a compelling need to develop models that enable fine-grained understanding across multiple modalities, thereby enhancing their applicability to a wide range of tasks. In this paper, we propose GroundingGPT, a language enhanced multi-modal grounding model. Beyond capturing global information like other multi-modal models, our proposed model excels at tasks demanding a detailed understanding of local information within the input. It demonstrates precise identification and localization of specific regions in images or moments in videos. To achieve this objective, we design a diversified dataset construction pipeline, resulting in a multi-modal, multi-granularity dataset for model training. The code, dataset, and demo of our model can be found at https: //github.com/lzw-lzw/GroundingGPT.

Discussion (0). Sign in to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DisTime: Distribution-based Time Representation for Video Large Language Models

    cs.CV 2025-05 conditional novelty 7.0 of 10

    A single learnable time token, decoded into a probability distribution over time bins, improves temporal grounding in Video-LLMs and is trained partly on a new 1.25M-event pseudo-labeled dataset.

  2. Teaching MLLMs to Say No: Generalized Referring Expression Comprehension via Refusal Calibrated GRPO

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A recalibrated GRPO reinforcement learning method lets multimodal LLMs say 'None' for nonexistent referring expressions without sacrificing localization accuracy on objects that do exist.

  3. ReMeREC: Relation-aware and Multi-entity Referring Expression Comprehension

    cs.CV 2025-07 conditional novelty 6.0 of 10

    ReMeREC introduces a relation-aware multi-entity referring expression comprehension framework and the ReMeX dataset, reporting state-of-the-art grounding and relation prediction, with some evaluation caveats.

  4. AV-Reasoner: Improving and Benchmarking Clue-Grounded Audio-Visual Counting for MLLMs

    cs.CV 2025-06 reject novelty 6.0 of 10

    A clue-grounded audio-visual counting benchmark over 497 long videos and an RL-trained counting model, whose headline result is undermined by training on the DVD-Counting evaluation benchmark.

  5. Encode Once, Decode Never: Reusing Audio LM Internals for Efficient Temporal Localization

    cs.LG 2026-02 conditional novelty 5.0 of 10

    Attaching a frame-level prediction head (trained with a Poisson-process-style loss) to an audio LM's decoder outputs localizes events in audio directly, avoiding autoregressive timestamp generation.

  6. KnowDR-REC: A Benchmark for Referring Expression Comprehension with Real-World Knowledge

    cs.LG 2025-08 conditional novelty 5.0 of 10

    KnowDR-REC is a benchmark that tests image-and-text AI models on object finding that needs real-world knowledge, and on 16 current models most of them fail.

  7. IntentVCNet: Bridging Spatio-Temporal Gaps for Intention-Oriented Controllable Video Captioning

    cs.CV 2025-07 conditional novelty 5.0 of 10

    IntentVCNet uses per-frame object coordinates, red-box visual prompts, and a lightweight box adapter to make video captioning focus on a user-selected object, reporting 225.19 CIDEr on the IntentVC public test set.

  8. Grounded Gesture Generation: Language, Motion, and Space

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A dataset and framework that unifies two gesture corpora into HumanML3D format and demonstrates improved spatially controlled pointing gestures via fine-tuned OmniControl.

  9. Video Understanding by Design: How Datasets Shape Video Models

    cs.CV 2025-09 reject novelty 4.0 of 10

    A dataset-centric framework that explains video architectures as responses to structural properties of benchmark datasets.

Pith tools