DC-Scene: Data-Centric Learning for 3D Scene Understanding

Ruicheng Zhang; Ting Huang; Yang Zhao; Zeyu Zhang

REVIEW 5 major objections 5 minor 3 cited by

DC-Scene: Data-Centric Learning for 3D Scene Understanding

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DC-Scene shows that a CLIP-scored, curriculum-ordered 75% subset of training pairs outperforms the full dataset for 3D scene captioning while cutting training epochs by two-thirds.

desk verdict Plausible data-centric idea for 3D captioning, but the paper contradicts itself on the central training-cost claim; worth reviewing but needs major revision. read the letter →

arxiv 2505.15232 v1 pith:5F46QIOL submitted 2025-05-21 cs.CV

Ting Huang , Zeyu Zhang , Ruicheng Zhang , Yang Zhao This is my paper

classification cs.CV

keywords 3Dscenecaptioningdata-centriclearningcurriculumCLIP-basedqualityscoringdual-indicatorfiltertrainingefficiencyScanReferNr3D

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether 3D scene captioning needs all of its training data, and answers no. It claims that scoring each scene–caption pair with a CLIP-based dual-indicator measure and then training on the top 75% of pairs, fed through a three-stage curriculum, produces better captions than the full dataset while using only a third of the training epochs. On the ScanRefer and Nr3D benchmarks with the 3D CoCa backbone, the filtered subset reaches 86.10 and 53.60 CIDEr, beating the full-data baseline's 85.42 and 52.84 at 1,080 epochs. The practical claim is that a moderate, carefully selected subset can outperform exhaustive training for 3D scene understanding, which matters because 3D data is expensive to annotate and costly to train on.

What carries the argument

The load-bearing object is the DIQ region in a two-dimensional quality space. Each sample gets a CLIP score $s_j = S_{\mathrm{clip}}(x_j^s) \cdot T_{\mathrm{clip}}(x_j^t)$ from the dot product of scene-encoder and text-encoder features, and a caption loss $l_j = -\sum_t \log P_\theta(Y_{j,t}|Y_{j,<t}, X_j)$ that doubles as perplexity. The DIQ filter keeps only samples inside $L_{\min} \le l_j \le L_{\max}$ and $S_{\min} \le s_j \le S_{\max}$, with the bounds set at the 5th and 95th percentiles, and the curriculum scheduler partitions the region into stages and raises the thresholds $L_p = L_{\min}+k\Delta L$ and $S_p = S_{\min}+k\Delta S$ as training proceeds. This mechanism creates a difficulty continuum from easy, well-aligned pairs to noisier ones, and it is what lets the model converge in one-third of the epochs.

What would settle it

Train the same backbone twice on the same 75% sample budget, once with DIQ selection and once with random sampling; if the random subset matches or beats the DIQ-filtered CIDEr, the quality filter is not responsible for the reported gain.

Watch

Extended reading notes

Core claim

Working within the 3D CoCa architecture, the paper introduces a data-centric training recipe: a dual-indicator quality (DIQ) filter that keeps only samples whose CLIP scene–caption alignment score and caption-loss perplexity both fall between the 5th and 95th percentiles, and a curriculum scheduler that trains first on the top 25% of DIQ-qualified pairs, then the top 50%, then the top 75%. On ScanRefer and Nr3D, this recipe reaches state-of-the-art CIDEr with one-third of the training epochs of the full-data baseline, and the gain is consistent across a second backbone (Vote2Cap-DETR++). The central claim is that data selection and ordering, not a larger model or more data, drive the improvement: semantically aligned, moderately complex samples are enough.

Load-bearing premise

The filter assumes that a high CLIP alignment score and a moderate caption-loss value really mark a scene–caption pair as high quality; neither indicator is checked against human judgments or an independent quality measure before samples are discarded.

Editorial extensions

If this is right

Using the top-75% DIQ subset cuts training epochs from 1,080 to 360 while raising CIDEr on both ScanRefer (from 85.42 to 86.10) and Nr3D (from 52.84 to 53.60) with the 3D CoCa backbone.
The same pattern holds for Vote2Cap-DETR++, showing the data-centric recipe transfers across architectures without changing the model.
Because only the training schedule changes, DC-Scene can be added to existing 3D captioning backbones as a drop-in efficiency upgrade.
The three curriculum stages (top 25%, 50%, 75%) give a natural trade-off between data volume and training cost, with even the smallest subset training fastest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If DIQ-style filtering generalizes, the same two-indicator recipe could be applied to 3D visual question answering or open-vocabulary 3D grounding, where annotations are scarcer than in captioning.
A direct comparison against random subset selection would isolate whether the quality filter or the smaller training budget is responsible for the speedup; the paper does not run that control.
Because the filter relies only on pretrained encoders and the model's own loss, it may transfer across datasets without retraining the quality scorer, letting a model trained on ScanRefer prune noisy pairs in a newly collected 3D dataset.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear Colleague,

Short version: this paper has a data-centric idea worth testing, but the manuscript is self-contradictory about what was actually trained, and the headline claim of a two-thirds training-cost reduction doesn't hold up as written. I would not cite it in its current form, but I'd let it go to review so the authors are forced to clarify the setup.

The new bit is the combination of two cheap quality signals for 3D scene–caption pairs: a CLIP scene–text alignment score and the caption model's own loss/perplexity. Using both to filter the training data, with a curriculum that supposedly expands from a top-25% to a top-75% pool, is a reasonable idea, and testing on two backbones (3D CoCa, Vote2Cap-DETR++) and two benchmarks (ScanRefer, Nr3D) is decent breadth. The qualitative examples do show the filtered model producing richer captions than the full-data baseline. The "intermediate similarity" notion (dropping both very low and very high CLIP scores) is interesting and not something I've seen in 3D captioning before.

The soft spots are load-bearing. Section 4.2 describes a three-phase curriculum with epoch switches at 360 and 720 and training ending at 1,080. Table 1 lists the top-75% row as a 360-epoch run, and the abstract and conclusion use that row to claim a two-thirds saving. Either the 75% result is the endpoint of a 1,080-epoch curriculum (no saving) or it's a standalone 360-epoch run (the curriculum was never evaluated). The paper can't have it both ways. Worse, Eqs. 6–7 have the pool shrink as k increases because the thresholds only move upward, which contradicts the expansion described in the text. So there's no coherent experiment specification that yields the central result.

Beyond that, no error bars or seeds; the gains (86.10 vs 85.42 CIDEr) are small enough to be noise. The 75% ratio is chosen post hoc from the same ablation table. The perplexity is computed with the model being trained, which makes the quality metric circular, and the paper doesn't say when or how often the filter scores are recomputed or what that costs. The SOTA claim rests only on comparison to the paper's own full-data baseline, not to published methods.

The idea deserves a proper test, but this submission doesn't provide one. With code, a corrected schedule, random-subset baselines, and variance reporting, it could be a solid paper. As is, it needs major revision before it should appear.

I'd accept it for peer review because the potential significance is real and the flaws are fixable, but a referee should insist on the above. It's also a useful teaching example of how experimental inconsistencies can sink an otherwise interesting claim.

Best

Referee Report

5 major / 5 minor

Summary. The paper proposes DC-Scene, a data-centric training framework for 3D scene captioning that combines a CLIP-based dual-indicator quality (DIQ) filter with a three-stage curriculum scheduler. The DIQ filter scores each scene–caption pair by a CLIP alignment score and by the captioning model's own loss/perplexity, and retains samples in a bounded region of this two-dimensional quality space. The curriculum is described as progressively expanding the training pool from the top 25% to the top 75% of DIQ-qualified samples. Experiments on ScanRefer and Nr3D with two backbones (3D CoCa and Vote2Cap-DETR++) are reported, and the paper claims that training on the top-75% subset for 360 epochs achieves higher CIDEr than training on the full dataset for 1,080 epochs, reducing training cost by approximately two-thirds. The central efficiency claim, the formal scheduler definition, and the reported experimental protocol are mutually inconsistent in ways that prevent the main result from being reproducible from the manuscript.

Significance. If the central claim were sound, the paper would address a practically important problem: reducing the high training cost of 3D scene captioning while maintaining or improving quality through data selection and curriculum learning. The authors identify a real pain point, evaluate on two standard benchmarks, and include two backbone architectures and qualitative examples. The proposed DIQ filter and curriculum are intuitively plausible. However, the significance is severely undermined by internal contradictions: the curriculum described in Section 4.2 is not the protocol that produced Table 1, the formal scheduler in Section 3.4 shrinks the training set as k increases rather than expanding it, and the headline improvements (86.10 vs. 85.42 CIDEr on ScanRefer; 53.60 vs. 52.84 on Nr3D) are small, single-run values with no error bars. As written, the paper does not provide a coherent, reproducible experiment supporting the two-thirds cost-reduction claim.

major comments (5)

[§4.2 and Table 1] The training protocol described in Section 4.2 is incompatible with the experiments reported in Table 1. Section 4.2 states that the curriculum runs for 1,080 epochs with pool switches at epochs 360 and 720: epochs 1–360 use the top 25%, epochs 361–720 use the top 50%, and epochs 721–1,080 use the full DIQ-qualified set. Table 1 instead lists a '75%' DIQ ratio with exactly 360 epochs, and Sections 4.3 and 5 use this 360-epoch, top-75% row to claim a two-thirds training-time reduction. If the top-75% row is a standalone 360-epoch run, then the Section 4.2 curriculum was never evaluated and the claimed savings are not attributable to the curriculum. If the top-75% row is the endpoint of the Section 4.2 curriculum, its actual cost is 1,080 epochs, not 360, and the two-thirds reduction claim collapses. The manuscript does not provide logs, checkpoints, or code to resolve which specification generated Table 1.
[§3.4, Eqs. (6)–(7)] The formal curriculum scheduler contradicts the paper's own description of progressive expansion. Eq. (6) defines C_k = {d_j | Lp ≤ l_j, Sp ≤ s_j}, and Eq. (7) sets Lp = Lmin + k∆L and Sp = Smin + k∆S. As k increases, both thresholds increase, so the set C_k of admissible samples becomes strictly smaller (the constraints become harder to satisfy). The text, however, states that the curriculum 'progressively expands the training pool from the top 25% to 75%' and that the Advanced Phase (k=2) exposes the model to 'the most challenging data.' These are opposite behaviors: a growing pool with increasing k would require thresholds that decrease with k or reverse inequality signs. The formal mechanism as written cannot produce the 25%→50%→75% expansion described in Section 4.2 and emphasized throughout the paper.
[Table 1 and §4.3] The central empirical claim rests on single-run results with no error bars, no variance reporting, and no statistical significance testing. The reported gains of the top-75% DIQ subset over the full-data baseline are small (86.10 vs. 85.42 CIDEr on ScanRefer; 53.60 vs. 52.84 on Nr3D for 3D CoCa; similar for Vote2Cap-DETR++). In captioning benchmarks, differences of this magnitude are commonly within run-to-run variance, especially without multiple seeds. Without error bars or significance tests, the claim that the top-75% subset 'in some cases even improv[es]' over full-data training is not supported.
[§3.3, Eq. (3)] The DIQ filter uses the captioning model's own loss/perplexity to judge data quality, but this proxy is never validated against human judgments or an independent data-quality measure. Eq. (3) defines l_j as the negative log-likelihood assigned by the model being trained to the ground-truth caption; this quantity reflects the model's current fit, not intrinsic caption quality, and can change as training progresses. The manuscript gives no evidence that samples with moderate loss (the DIL region in Eq. (4)) are genuinely higher quality, nor that filtering by this self-referential criterion avoids discarding hard but informative examples. Without such validation, the DIQ filter's role in the reported gains is not established.
[§4.3 and Table 1] The choice of the 75% DIQ ratio is made after inspecting the results, not as a pre-specified configuration. Section 4.3 states that 'the top-75% DIQ split achieves the best trade-off' after presenting Table 1, which reports results for 25%, 50%, 75%, and 100% ratios. Selecting the best-performing row from a table of multiple configurations introduces a selection bias; the reported gains of the chosen split are not an unbiased estimate of the method's expected performance. The paper should either pre-register the target ratio or report multiple splits with proper multiple-comparison awareness, and should clarify whether the other ratios were also run with the intended curriculum protocol (which, as noted above, is itself ambiguous).

minor comments (5)

[Eq. (3)] The quantity l_j in Eq. (3) is a negative log-likelihood, which is commonly called the caption loss, not the perplexity. Perplexity is typically exp(l_j). The terminology should be corrected to avoid ambiguity, especially since 'perplexity' is used throughout Section 3.3.
[Figure 1 caption] The caption states that filtering out the lowest-quality 80% of samples 'cuts training time by 75%,' but no experiment, dataset, or protocol behind this figure is described in the text. The claimed reduction also differs from the two-thirds reduction emphasized elsewhere; the figure should be either integrated with the experimental section or removed.
[Table 1 and §4.2] Table 1 labels the 100% row as 'baseline' with 1,080 epochs, but Section 4.2 says the final phase (epochs 721–1,080) uses the 'full DIQ-qualified dataset.' It is unclear whether this 100% baseline is the full raw dataset or the dataset after DIQ filtering. The paper should define '100%' precisely.
[§2 and references] The related-work section cites several 3D captioning and curriculum-learning papers, but it omits the recent literature on data quality scoring for 2D image-text data (e.g., CLIP-score-based filtering), which is directly relevant to the proposed DIQ filter. Adding such references would help position the contribution.
[Throughout] The paper contains several formatting and clarity issues: 'V ote2Cap' appears with an unwanted space, the notation Sclip(·) and Tclip(·) is introduced in Eq. (1) but the figure uses different symbols, and the meaning of 'top-75% DIQ subset' (top by what order?) is never formally defined. These should be polished.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the efficiency claim is an external empirical comparison, not a result entailed by the DIQ filter equations.

full rationale

The paper's central claim is empirical: training on the top-75% DIQ subset for 360 epochs achieves 86.10 CIDEr on ScanRefer and 53.60 on Nr3D versus 85.42 and 52.84 for the full-data 1,080-epoch baseline. This inequality is not derivable from the method's equations; Eqs. (1)-(7) define a filtering heuristic, and success is measured by held-out CIDEr, which is external to the filter construction. The self-referential quality indicator in Eq. (3) (caption loss computed with the captioning model's own parameters) combined with Eq. (5) (using that loss to select training data) is an explicit self-paced feedback loop rather than a logical derivation; because the evaluation metric is not l_j, the outcome could have failed, so the result is not forced by construction. The only overlapping-author citation is [17] for the 3D CoCa backbone and its CLIP 3D scene encoder; that prior work is used as a fixed backbone for both the baseline and the proposed method, so it is not load-bearing for the data-centric efficiency claim. The mismatch between the 1,080-epoch three-phase curriculum in Section 4.2 and the 360-epoch '75%' rows in Table 1 is a reproducibility or correctness concern, not a circular dependence. No step in the derivation chain reduces to its own input.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on two unvalidated proxies for data quality (CLIP alignment and model perplexity), on arbitrary percentile thresholds and step sizes, and on split ratios chosen after inspecting the experimental table. No new physical entities are introduced.

free parameters (4)

CLIP score bounds Smin/Smax = not reported (5th and 95th percentiles)
Chosen empirically from the CLIP score distribution; determines the DIS subset in Eq. (2).
Caption loss bounds Lmin/Lmax = not reported (5th and 95th percentiles)
Chosen empirically from the caption loss distribution; determines the DIL subset in Eq. (4).
Curriculum step sizes and stage ratios = 25%, 50%, 75% at phases k=0,1,2
Stage boundaries and subset ratios are chosen by hand and validated via Table 1; no principled selection criterion is given.
Final subset ratio and epoch count = 75% and 360 epochs
The 75% ratio is the best-performing split in Table 1, so the headline comparison is selected from the ablation rather than specified in advance.

assumptions (3)

domain assumption CLIP alignment scores measure 3D scene-caption semantic quality.
Section 3.2 uses the CLIP dot product as a quality score without demonstrating that it correlates with captioning quality for point cloud scenes.
ad hoc to paper Caption perplexity of the decoder being trained is a valid data-quality indicator.
Eq. (3) defines perplexity using the model's own probability distribution, then Eq. (5) uses it to filter training data, coupling the filter to the model under training.
domain assumption CLIP text-image alignment transfers to 3D point-cloud scenes.
The CLIP 3D scene encoder is used without evidence that its score space is well-calibrated for 3D caption quality.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DC-Scene: Data-Centric Learning for 3D Scene Understanding." pith.science (2026). https://pith.science/paper/5F46QIOL

@misc{pith2026250515232,
  author       = {Pith},
  title        = {Pith review of: DC-Scene: Data-Centric Learning for 3D Scene Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5F46QIOL}},
  note         = {Machine review of arXiv:2505.15232}
}

read the original abstract

3D scene understanding plays a fundamental role in vision applications such as robotics, autonomous driving, and augmented reality. However, advancing learning-based 3D scene understanding remains challenging due to two key limitations: (1) the large scale and complexity of 3D scenes lead to higher computational costs and slower training compared to 2D counterparts; and (2) high-quality annotated 3D datasets are significantly scarcer than those available for 2D vision. These challenges underscore the need for more efficient learning paradigms. In this work, we propose DC-Scene, a data-centric framework tailored for 3D scene understanding, which emphasizes enhancing data quality and training efficiency. Specifically, we introduce a CLIP-driven dual-indicator quality (DIQ) filter, combining vision-language alignment scores with caption-loss perplexity, along with a curriculum scheduler that progressively expands the training pool from the top 25% to 75% of scene-caption pairs. This strategy filters out noisy samples and significantly reduces dependence on large-scale labeled 3D data. Extensive experiments on ScanRefer and Nr3D demonstrate that DC-Scene achieves state-of-the-art performance (86.1 CIDEr with the top-75% subset vs. 85.4 with the full dataset) while reducing training cost by approximately two-thirds, confirming that a compact set of high-quality samples can outperform exhaustive training. Code will be available at https://github.com/AIGeeksGroup/DC-Scene.

Figures

Figures reproduced from arXiv: 2505.15232 by the authors.

**Figure 1.** Effect of CLIP-score–based data filtering on training cost. Filtering out the lowest-quality 80% of samples cuts training time by 75%, motivating our data-centric learning strategy. 3D scene understanding plays a vital role in a wide range of real-world applications, including robotics, augmented reality(AR), virtual reality(VR), and autonomous driving [13, 14, 22, 31, 32]. For instance, robots must interpret comp… view at source ↗

**Figure 2.** Framework of DC-Scene. Raw point cloud scene and candidate captions are first encoded by a Scene Encoder and a Text Encoder. Quality Scoring module computes the CLIP score for each scene–text pair. Dual-Indicator Quality (DIQ) Filter then selects samples that lie within a predefined quality region, retaining only the top-k candidates per scene. These filtered representations are passed to the Curriculum Scheduler, w… view at source ↗

**Figure 3.** Ablation study comparing different Dual-Indicator Quality (DIQ) regions on the [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Data quality buckets visualized in 3-D embedding space. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Qualitative comparison of scene captions. For three validation scenes from the ScanRefer [8] dataset, we present the rendered point cloud mesh (top row), followed by captions generated by three sources: the full-data baseline model (in pink), our DC-Scene model trained…

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

ConsiSpace: Learning Geometric Consistency Matters for Video Spatial Reasoning
cs.CV 2026-07 conditional novelty 6.0 of 10

A geometry-consistent memory plus a self-supervised consistency-reinforcement stage improves video spatial reasoning by roughly 7–15 points on three benchmarks, with an advertised average gain of 12.6 points.
3D-R1: Enhancing Reasoning in 3D VLMs for Unified Scene Understanding
cs.CV 2025-07 conditional novelty 6.0 of 10

3D-R1 uses a synthetic chain-of-thought cold start plus GRPO reinforcement learning with perception, semantic, and format rewards, and reports best published results across 3D dense captioning, QA, grounding, dialogue...
Nav-R1: Reasoning and Navigation in Embodied Scenes
cs.RO 2025-09 reject novelty 4.0 of 10

Nav-R1 uses a 110K synthetic CoT dataset, GRPO with three rewards, and a fast-in-slow system to set new SOTA on R2R-CE, RxR-CE, and HM3D-OVON.

Reference graph

Works this paper leans on

36 extracted references · 21 canonical work pages · cited by 3 Pith papers

[1]

Referit3d: Neural listeners for fine-grained 3d object identification in real- world scenes

Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, and Leonidas Guibas. Referit3d: Neural listeners for fine-grained 3d object identification in real- world scenes. 16th European Conference on Computer Vision (ECCV), 2020

work page 2020
[2]

Leveraging self-paced semi-supervised learning with prior knowledge for 3d object detection on a lidar-camera system

Pei An, Junxiong Liang, Xing Hong, Siwen Quan, Tao Ma, Yanfei Chen, Liheng Wang, and Jie Ma. Leveraging self-paced semi-supervised learning with prior knowledge for 3d object detection on a lidar-camera system. Remote Sensing, 15(3):627, 2023. doi: 10.3390/rs15030627

work page doi:10.3390/rs15030627 2023
[3]

Scanqa: 3d question answering for spatial scene understanding

Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: 3d question answering for spatial scene understanding. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19129–19139, 2022

2022
[4]

METEOR: An automatic metric for MT evaluation with improved correlation with human judgments

Satanjeev Banerjee and Alon Lavie. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Jade Goldstein, Alon Lavie, Chin- Yew Lin, and Clare V oss, editors,Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan...

work page 2005
[5]

Curriculum learning

Yoshua Bengio, Jérémie Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learn- ing, pages 41–48. ACM, 2009. doi: 10.1145/1553374.1553380

arXiv 2009
[6]

Medical ai for early detection of lung cancer: A survey

Guohui Cai, Ying Cai, Zeyu Zhang, Yuanzhouhan Cao, Lin Wu, Daji Ergu, Zhinbin Liao, and Yang Zhao. Medical ai for early detection of lung cancer: A survey. arXiv preprint arXiv:2410.14769, 2024. TING HUANG AND ZEYU ZHANG ET AL.: DC-SCENE 11

arXiv 2024
[7]

Msdet: Recep- tive field enhanced multiscale detection for tiny pulmonary nodule

Guohui Cai, Ruicheng Zhang, Hongyang He, Zeyu Zhang, Daji Ergu, Yuanzhouhan Cao, Jinman Zhao, Binbin Hu, Zhinbin Liao, Yang Zhao, et al. Msdet: Recep- tive field enhanced multiscale detection for tiny pulmonary nodule. arXiv preprint arXiv:2409.14028, 2024

arXiv 2024
[8]

Scanrefer: 3d object local- ization in rgb-d scans using natural language

Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. Scanrefer: 3d object local- ization in rgb-d scans using natural language. 16th European Conference on Computer Vision (ECCV), 2020

work page 2020

Show all 36 references

[9]

D3net: A speaker-listener architecture for semi-supervised dense captioning and visual ground- ing in rgb-d scans

Dave Zhenyu Chen, Qirui Wu, Matthias Nießner, and Angel X Chang. D3net: A speaker-listener architecture for semi-supervised dense captioning and visual ground- ing in rgb-d scans. arXiv preprint arXiv:2112.01551, 2021

2021 arXiv
[10]

DaveZhenyu Chen, Ali Gholami, Matthias Niesner, and AngelX. Chang. Scan2cap: Context-aware dense captioning in rgb-d scans. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2021. doi: 10.1109/cvpr46437. 2021.00321

2021
[11]

End- to-end 3d dense captioning with vote2cap-detr

Sijin Chen, Hongyuan Zhu, Xin Chen, Yinjie Lei, Gang Yu, and Tao Chen. End- to-end 3d dense captioning with vote2cap-detr. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 11124–11133, Jun 2023. doi: 10.1109/cvpr52729.2023.01070

2023
[12]

V ote2cap-detr++: Decoupling localization and describing for end-to-end 3d dense captioning

Sijin Chen, Hongyuan Zhu, Mingsheng Li, Xin Chen, Peng Guo, Yinjie Lei, Gang Yu, Taihao Li, and Tao Chen. V ote2cap-detr++: Decoupling localization and describing for end-to-end 3d dense captioning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(11):7331–73...

2024
[13]

Tightcap: 3d human shape capture with clothing tightness field

Xin Chen, Anqi Pang, Yang Wei, Wang Peihao, Lan Xu, and Jingyi Yu. Tightcap: 3d human shape capture with clothing tightness field. ACM Transactions on Graphics (Presented at ACM SIGGRAPH), 2021

2021
[14]

Sportscap: Monocular 3d human motion capture and fine-grained understanding in challeng- ing sports videos

Xin Chen, Anqi Pang, Wei Yang, Yuexin Ma, Lan Xu, and Jingyi Yu. Sportscap: Monocular 3d human motion capture and fine-grained understanding in challeng- ing sports videos. International Journal of Computer Vision , page 2846–2864, Oct
[15]

Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner

Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2017

2017
[16]

Scene-llm: Extend- ing language model for 3d visual reasoning

Rao Fu, Jingyu Liu, Xilun Chen, Yixin Nie, and Wenhan Xiong. Scene-llm: Extend- ing language model for 3d visual reasoning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2025

2025
[17]

3d coca: Contrastive learners are 3d captioners

Ting Huang, Zeyu Zhang, Yemin Wang, and Hao Tang. 3d coca: Contrastive learners are 3d captioners. arXiv preprint arXiv:2504.09518, 2025

2025 arXiv
[18]

Sceneverse: Scaling 3d vision-language learning for grounded scene understanding

Baoxiong Jia, Yixin Chen, Huangyue Yu, Yan Wang, Xuesong Niu, Tengyu Liu, Qing Li, and Siyuan Huang. Sceneverse: Scaling 3d vision-language learning for grounded scene understanding. In European Conference on Computer Vision (ECCV), 2024. 12 TING HUANG AND ZEYU ZHANG ET AL.: DC-SCENE

2024
[19]

More: Multi-order relation mining for dense captioning in 3d scenes

Yang Jiao, Shaoxiang Chen, Zequn Jie, Jingjing Chen, Lin Ma, and Yu-Gang Jiang. More: Multi-order relation mining for dense captioning in 3d scenes. InIn Proceedings of the European conference on computer vision, page 528–545, Jan 2022. doi: 10.1007/ 978-3-031-19833-5_31

2022
[20]

Tod3cap: Towards 3d dense captioning in outdoor scenes

Bu Jin, Yupeng Zheng, Pengfei Li, Weize Li, Yuhang Zheng, Sujie Hu, Xinyu Liu, Jin- wei Zhu, Zhijie Yan, Haiyang Sun, Kun Zhan, Peng Jia, Xiaoxiao Long, Yilun Chen, and Hao Zhao. Tod3cap: Towards 3d dense captioning in outdoor scenes. In In Pro- ceedings of the European confer...

2025 doi
[21]

Duoduo clip: Efficient 3d under- standing with multi-view images

Han-Hung Lee, Yiming Zhang, and Angel X Chang. Duoduo clip: Efficient 3d under- standing with multi-view images. In International Conference on Learning Represen- tations (ICLR), 2025

2025
[22]

Point cloud instance segmentation with semi-supervised bounding-box min- ing

Yongbin Liao, Hongyuan Zhu, Yanggang Zhang, Chuangguan Ye, Tao Chen, and Jian- chao Fan. Point cloud instance segmentation with semi-supervised bounding-box min- ing. Cornell University - arXiv,Cornell University - arXiv, Nov 2021

2021
[23]

ROUGE: A package for automatic evaluation of summaries

Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out , pages 74–81, Barcelona, Spain, July 2004. As- sociation for Computational Linguistics. URL https://aclanthology.org/ W04-1013/

2004
[24]

View selection for 3d captioning via diffusion ranking

Tiange Luo, Justin Johnson, and Honglak Lee. View selection for 3d captioning via diffusion ranking. In Computer Vision – ECCV 2024, pages 180–197. Springer, 2024. doi: 10.1007/978-3-031-72751-1_11

2024 doi
[25]

Sqa3d: Situated question answering in 3d scenes

Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang. Sqa3d: Situated question answering in 3d scenes. arXiv preprint arXiv:2210.07474, 2022

2022 arXiv
[26]

Bleu: a method for au- tomatic evaluation of machine translation

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics , ACL ’02, page 311–318, USA, 2002. Association for Computation...

2002
[27]

Dense multimodal alignment for open-vocabulary 3d scene understanding

Li Ruihuang, Zhang Zhengqiang, He Chenhang, Ma Zhiyuan, Patel Vishal M., and Zhang Lei. Dense multimodal alignment for open-vocabulary 3d scene understanding. In ECCV, 2024

2024
[28]

Curriculum learning: A survey

Paula Soviany, Radu Tudor Ionescu, Nicolae Catalin Ristea, and Marius Leordeanu. Curriculum learning: A survey. International Journal of Computer Vision , 130(4): 1096–1133, 2022. doi: 10.1007/s11263-021-01555-0

2022 doi
[29]

Lawrence Zitnick, and Devi Parikh

Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. Cider: Consensus- based image description evaluation. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4566–4575, 2015. doi: 10.1109/CVPR.2015. 7299087. TING HUANG AND ZEYU ZHANG ET AL....

2015 doi
[30]

Spatiality-guided trans- former for 3d dense captioning on point clouds

Heng Wang, Chaoyi Zhang, Jianhui Yu, and Weidong Cai. Spatiality-guided trans- former for 3d dense captioning on point clouds. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence , page 1393–1400, Jul 2022. doi: 10.24963/ijcai.2022/194

2022 doi
[31]

Open vocabulary 3d scene understanding via geometry guided self-distillation

Pengfei Wang, Yuxi Wang, Shuai Li, Zhaoxiang Zhang, Zhen Lei, and Lei Zhang. Open vocabulary 3d scene understanding via geometry guided self-distillation. In Computer Vision – ECCV 2024 , pages 442–460. Springer, 2024. doi: 10.1007/ 978-3-031-72633-0\_25

2024
[32]

Dcnet: Large-scale point cloud semantic segmentation with discriminative and efficient feature aggregation

Fukun Yin, Zilong Huang, Tao Chen, Guozhong Luo, Gang Yu, and Bin Fu. Dcnet: Large-scale point cloud semantic segmentation with discriminative and efficient feature aggregation. IEEE Transactions on Circuits and Systems for Video Technology , page 1–1, Jan 2023. doi: 10.1109/t...

2023
[33]

Meddet: Generative adversarial distillation for efficient cervical disc herniation detection

Zeyu Zhang, Nengmin Yi, Shengbo Tan, Ying Cai, Yi Yang, Lei Xu, Qingtai Li, Zhang Yi, Daji Ergu, and Yang Zhao. Meddet: Generative adversarial distillation for efficient cervical disc herniation detection. In 2024 IEEE International Conference on Bioinfor- matics and Biomedici...

2024
[34]

Peddet: Adaptive spectral optimization for multimodal pedestrian detection

Rui Zhao, Zeyu Zhang, Yi Xu, Yi Yao, Yan Huang, Wenxin Zhang, Zirui Song, Xi- uying Chen, and Yang Zhao. Peddet: Adaptive spectral optimization for multimodal pedestrian detection. arXiv preprint arXiv:2502.14063, 2025

2025 arXiv
[35]

Cur- ricular object manipulation in lidar-based object detection

Ziyue Zhu, Qiang Meng, Xiao Wang, Ke Wang, Liujiang Yan, and Jian Yang. Cur- ricular object manipulation in lidar-based object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1132–1141. IEEE, 2023

2023
[2021]

URL http://dx.doi.org/10

doi: 10.1007/s11263-021-01486-4. URL http://dx.doi.org/10. 1007/s11263-021-01486-4

doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Referit3d: Neural listeners for fine-grained 3d object identification in real- world scenes

Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, and Leonidas Guibas. Referit3d: Neural listeners for fine-grained 3d object identification in real- world scenes. 16th European Conference on Computer Vision (ECCV), 2020

work page 2020

[2] [2]

Leveraging self-paced semi-supervised learning with prior knowledge for 3d object detection on a lidar-camera system

Pei An, Junxiong Liang, Xing Hong, Siwen Quan, Tao Ma, Yanfei Chen, Liheng Wang, and Jie Ma. Leveraging self-paced semi-supervised learning with prior knowledge for 3d object detection on a lidar-camera system. Remote Sensing, 15(3):627, 2023. doi: 10.3390/rs15030627

work page doi:10.3390/rs15030627 2023

[3] [3]

Scanqa: 3d question answering for spatial scene understanding

Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: 3d question answering for spatial scene understanding. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19129–19139, 2022

2022

[4] [4]

METEOR: An automatic metric for MT evaluation with improved correlation with human judgments

Satanjeev Banerjee and Alon Lavie. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Jade Goldstein, Alon Lavie, Chin- Yew Lin, and Clare V oss, editors,Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan...

work page 2005

[5] [5]

Curriculum learning

Yoshua Bengio, Jérémie Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learn- ing, pages 41–48. ACM, 2009. doi: 10.1145/1553374.1553380

arXiv 2009

[6] [6]

Medical ai for early detection of lung cancer: A survey

Guohui Cai, Ying Cai, Zeyu Zhang, Yuanzhouhan Cao, Lin Wu, Daji Ergu, Zhinbin Liao, and Yang Zhao. Medical ai for early detection of lung cancer: A survey. arXiv preprint arXiv:2410.14769, 2024. TING HUANG AND ZEYU ZHANG ET AL.: DC-SCENE 11

arXiv 2024

[7] [7]

Msdet: Recep- tive field enhanced multiscale detection for tiny pulmonary nodule

Guohui Cai, Ruicheng Zhang, Hongyang He, Zeyu Zhang, Daji Ergu, Yuanzhouhan Cao, Jinman Zhao, Binbin Hu, Zhinbin Liao, Yang Zhao, et al. Msdet: Recep- tive field enhanced multiscale detection for tiny pulmonary nodule. arXiv preprint arXiv:2409.14028, 2024

arXiv 2024

[8] [8]

Scanrefer: 3d object local- ization in rgb-d scans using natural language

Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. Scanrefer: 3d object local- ization in rgb-d scans using natural language. 16th European Conference on Computer Vision (ECCV), 2020

work page 2020