{"id":"16db4321-2d94-4ce2-ac41-e5ef02345e34","arxiv_id":"2501.03879","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CL3DOR pairs 8,192-point inputs, GPT-4o-generated hard-negative response triplets, and an odds-ratio contrastive loss to achieve state-of-the-art results on 3D scene understanding benchmarks.","lead":"This paper introduces CL3DOR, a 3D vision-language model that combines denser point clouds, AI-generated wrong answers, and an odds-ratio contrastive loss to improve how models understand and describe 3D scenes. The authors report state-of-the-art scores on several 3D question-answering, captioning, and hallucination benchmarks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2's SOTA claim is confounded by the LLM backbone: CL3DOR uses LLaMA3-8B-Instruct while LEO's LLM is unspecified, so the reported gains may not be attributable to the proposed components.","rationale":"The paper is a solid empirical contribution with carefully designed within-model ablations: high vs low resolution (Table 4), hard vs easy negatives (Table 4), OR vs PR vs SFT (Figure 5 and Tables 6-9) all demonstrate the value of the components. However, the headline claim of state-of-the-art performance depends on a fair comparison with the strongest general-purpose baseline. The paper does not control for the LLM backbone when comparing with LEO, and LEO's architecture is not specified in the manuscript. This is the same concern the reader identified as the weakest assumption, and it is the most load-bearing because it affects the primary SOTA assertion on three major benchmarks. The concrete test I propose would directly settle whether the gap is due to the proposal or to the choice of language model. The reader's verdict of CONDITIONAL remains appropriate: the concern is significant but addressable, not a fundamental flaw requiring rejection. The secondary issues (no error bars, data-release promise, negative-quality validation) add to the conditionality but do not change the verdict.","tokens_in":22009,"tokens_out":5165,"duration_ms":49531,"concrete_test":"Obtain or reimplement LEO with the same LLaMA3-8B-Instruct backbone, or conversely retrain CL3DOR with the LLM used by LEO (e.g., Vicuna-7B if that is LEO's backbone), and re-run ScanQA, SQA3D, and Scan2Cap under identical training data and evaluation protocol. If the reported gains shrink to ≤1–2 pp refined EM@1 and ≤3 CIDEr, the SOTA claim is confounded by the LLM; if the gains persist, the proposed components are supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, 'CL3DOR achieves state-of-the-art performance in 3D scene understanding and reasoning benchmarks,' is supported primarily by Table 2, where the only general-purpose baseline is LEO. The comparison is not backbone-matched: Section 4 states CL3DOR uses LLaMA3-8B-Instruct, while LEO's language backbone is not disclosed in the paper and is not controlled for. The within-model ablations (Tables 4, 6, 7, 8, 9 and Figure 5) all hold LLaMA3 fixed, but LEO was likely built with a different, possibly weaker, LLM. Consequently, the 5.3 pp refined EM@1 gain on ScanQA and the 21-point CIDEr gain on Scan2Cap could largely reflect the newer, stronger instruction-following and language-generation capabilities of LLaMA3 rather than the 8,192-point inputs, hard negatives, or odds-ratio loss. If much of the gap vanishes under a backbone-matched comparison, the state-of-the-art attribution in the abstract does not follow. The paper's own acknowledgment of single inference runs (Appendix A.2) and the underspecified object-existence sampling are secondary; the backbone confound is the most load-bearing because it threatens the headline claim across multiple benchmarks.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CL3DOR, a 3D large multimodal model that combines three ingredients: (i) higher-resolution point clouds (8,192 points per object instead of the common 1,024), (ii) training triplets with GPT-4o-generated hard negative responses, and (iii) an odds-ratio (ORPO) auxiliary loss added to the standard negative log-likelihood loss during instruction tuning. The model is trained in three stages and evaluated on ScanQA, SQA3D, Scan2Cap, and 3D-POPE. The authors report state-of-the-art results on these benchmarks compared with single-task, task-adaptive, and general-purpose baselines, particularly LEO, and support their components with within-model ablations.","tokens_in":22248,"tokens_out":4630,"duration_ms":39590,"significance":"If the gains prove robust, the paper contributes a useful and simple recipe: high-resolution point cloud sampling and contrastive instruction tuning with hard negatives are modifications that could be adopted by other 3D LMMs. The within-model ablations (Table 4, Figure 5, Appendix Tables 6-9) are carefully designed and show consistent improvements from each component, and the authors commit to releasing the constructed datasets, which would benefit the community. However, the headline state-of-the-art claim is currently not established because the main comparison is confounded by the LLM backbone, and the evaluations lack error bars.","major_comments":[{"comment":"The state-of-the-art claim is supported by a comparison against LEO in which the LLM backbone is not controlled. CL3DOR uses LLaMA3-8B-Instruct, as stated in Section 4 (Implementation details), while LEO's language model is not specified in the paper. The within-model ablations (Tables 4, 6-9) all fix LLaMA3, so they do not resolve this confound. To attribute the reported gains (e.g., 5.3 pp refined EM@1 on ScanQA, 21 CIDEr on Scan2Cap) to the proposed components, a backbone-matched comparison is needed.","section":"§4, Table 2"},{"comment":"The statement that all evaluations are conducted using a single inference run means the numbers in Tables 2-4 and 6-9 lack error bars. Given that many differences are small (e.g., 3D-POPE accuracy differences around 2-5 points between settings, or λ ablation differences of ~1-2 points), the paper should report variance across at least a few seeds or explicitly refrain from making fine-grained claims. At minimum, the main text should draw attention to this limitation, not only the appendix.","section":"Appendix A.2"},{"comment":"The quality and difficulty of the GPT-4o-generated negatives are not validated. The prompt includes filters (\"REMOVE THIS SAMPLE...\"), but no statistics on the filter rate or a human evaluation of the remaining negatives are provided. If many negatives are mislabeled or not actually hard, the contrastive term could be learning from noise. Since these datasets are claimed as a contribution, the paper should include a validation study (e.g., human agreement or at least a sample of checked examples).","section":"§3.3, Appendix A.3"}],"minor_comments":[{"comment":"The text says hard negatives are shown with a green line and easy negatives with a blue line, but the Figure 6 caption states the opposite. The color assignment should be corrected to be consistent.","section":"Section 5.2 and Figure 6"},{"comment":"The text refers to a \"Discussion section\" that does not exist in the manuscript; these references should point to the relevant sections (e.g., \"Experimental Results\" or \"Appendix\").","section":"Section 5.2, Appendix A.4"},{"comment":"There are minor formatting issues in baseline names: \"V ote2Cap-DETR\" has a stray space, and \"V oteNet+MCAN\" appears with inconsistent spacing in the text.","section":"Table 2"},{"comment":"The limitation section is only one sentence and does not mention the LLM backbone confound or the single-run evaluation; consider expanding it to include these caveats.","section":"Limitation section"}],"recommendation":"major_revision","confidential_remarks":"The paper has a strong within-model ablation story, but the SOTA claim is overstated given the LLM backbone confound. The editor should consider requiring a backbone-matched comparison or a revised claim. Also, the odds-ratio loss is directly borrowed from ORPO, so the novelty is primarily in its application to 3D LMMs; the contribution is still reasonable but should be framed accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is a competent empirical contribution, and the stress-test concern is real: the headline SOTA claim in Table 2 is not backbone-matched. CL3DOR uses LLaMA3-8B-Instruct; LEO's LLM is not disclosed, so the 5.3 pp EM@1 gain and 21-point CIDEr gain on Scan2Cap could partly reflect a newer language model rather than the three proposed components. That said, the within-model ablations hold the LLM fixed and give the paper its real value: high-res 8,192 points beats 1,024, hard negatives beat easy negatives, and the odds-ratio loss beats SFT and probability-ratio across λ values (Tables 4, 6-9, Figure 5). Those are clean, controlled comparisons, and the improvements are consistent. The hard-negative dataset construction is the most genuinely new artifact—GPT-4o prompts that filter illogical QA pairs and create plausible wrong answers for captioning, with intentional hallucinations. That is reusable beyond this specific model.\n\nSoft spots, in order. First, the SOTA framing overclaims because of the backbone confound; the authors should provide a backbone-matched comparison or soften the claim. Second, all evaluations are single-run with no error bars (Appendix A.2), so the small metric differences across λ values should be read as trends, not guarantees. Third, the object-existence negative sampling is underspecified—\"frequency and co-occurrence\" thresholds are not given, so this piece is not reproducible as written. Fourth, the data release is promissory; since the dataset is a claimed contribution, the code/data should be available for hard-negative quality to be checked. The paper's own limitation section (few epochs, no million-scale data) is honest about what it does not show.\n\nThe ORPO loss is borrowed verbatim from Hong et al. and the authors cite it correctly; the novelty is the combination and the data, not the math. For a subfield where 1,024-point inputs were standard, the high-resolution result alone is a useful practical finding. I'd send it to review: the recipe is clear, the ablations are honest, and the problems are addressable rather than fatal. The main revision asks should be a matched backbone or a qualified SOTA claim, plus error bars or multiple seeds, and the object-existence sampling details.\n\nWorth your time if you work on 3D LMMs or hallucination in multimodal models; less so if you are outside the area.","headline":"A solid empirical recipe for 3D LMMs whose component ablations hold up, but the headline SOTA claim is confounded by an unmatched LLM backbone and needs either a matched comparison or a softer claim.","tokens_in":22830,"tokens_out":2165,"would_cite":true,"duration_ms":19440,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CL3DOR claims that enriching point-cloud density to 8,192 points per object and adding GPT-4o-generated hard negatives trained with an odds-ratio contrastive loss makes a general-purpose 3D large multimodal model state of the art on…","keywords":["3D large multimodal models","point cloud understanding","contrastive learning","odds ratio loss","hard negative responses","instruction tuning","object hallucination","3D scene understanding"],"falsifier":"Retrain CL3DOR while holding the LLM backbone identical to the previous general-purpose baseline (and retrain that baseline with LLaMA3-8B-Instruct), keeping point-cloud resolution, hard negatives, and the OR loss exactly the same; if the 5.3-point ScanQA EM gain and 21-point CIDEr gain collapse, the proposed components are not the cause. Separately, have human annotators judge a random sample of GPT-4o-generated negatives for correctness and hardness; if many are wrong or easy, the negative-quality premise fails.","tokens_in":21787,"feed_emoji":"☁️","tokens_out":9154,"duration_ms":71753,"temperature":0.7,"pith_summary":"CL3DOR is a 3D large multimodal model that answers questions and writes captions about 3D scenes from point clouds. The paper argues that existing training data for such models is information-poor in two ways: point clouds are sampled too sparsely, and instruction responses come without informative wrong answers. To fix this, the authors sample 8,192 points per object, use GPT-4o to generate plausible-but-wrong hard negative responses, and add an odds-ratio contrastive term to the standard language-modeling loss. They report state-of-the-art results on four benchmarks, including a 5.3-point gain in refined exact-match on ScanQA and a 21-point CIDEr gain on Scan2Cap over the previous general-purpose model. The central bet is that richer visual input plus explicit penalization of near-miss answers yields finer spatial understanding and fewer object hallucinations.","feed_headline":"High-res point clouds and odds-ratio loss lift 3D LMMs to SOTA","feed_subtitle":"Denser inputs plus GPT-4o hard negatives beat prior generalists on QA, captioning, and hallucination benchmarks.","key_machinery":"The load-bearing mechanism is the odds-ratio contrastive loss, adapted from preference optimization: $\\mathrm{odds}_{\\theta}(y\\mid x)=P_{\\theta}(y\\mid x)/(1-P_{\\theta}(y\\mid x))$, with $\\mathcal{L}_{\\mathrm{OR}}=-\\log\\sigma\\big(\\log(\\mathrm{odds}_{\\theta}(y^+\\mid x)/\\mathrm{odds}_{\\theta}(y^-\\mid x))\\big)$ added as an auxiliary term to negative log-likelihood. The odds ratio matters because it suppresses hard negatives without crushing their logits, which the authors argue is more stable than a raw probability-ratio loss. Around this loss sits a three-stage training pipeline: object alignment on single-object captions, scene alignment on referring-expression and scene-caption data, then spatial contrastive instruction tuning on triplet data. The visual side uses a frozen PointBERT encoder consuming 8,192 points per object, a spatial transformer for inter-object relations, and special point tokens injected into a LLaMA3-8B-Instruct backbone.","core_discovery":"The discovery claimed is that a single general-purpose 3D LMM can beat task-specific and generalist models on four benchmarks by combining three ingredients: (1) sampling 8,192 points per object instead of the usual 1,024; (2) augmenting instruction data with hard negatives that are plausible but swap location, color, count, or existence; and (3) training with $L = \\mathcal{L}_{\\mathrm{NLL}} + \\lambda \\mathcal{L}_{\\mathrm{OR}}$, where $\\mathcal{L}_{\\mathrm{OR}} = -\\log \\sigma\\big(\\log \\frac{\\mathrm{odds}_{\\theta}(y^+\\mid x)}{\\mathrm{odds}_{\\theta}(y^-\\mid x)}\\big)$ and the odds of a response are its generation probability divided by one minus that probability. Ablations within the paper show each ingredient matters: dropping to low resolution lowers Scan2Cap CIDEr from 93.4 to 71.7, easy negatives underperform hard negatives, and removing the odds-ratio term by setting $\\lambda=0$ reduces ScanQA CIDEr by roughly 10 points. The same model keeps its yes-rate close to the true object-existence rate on 3D-POPE, indicating less bias toward hallucinating objects than baselines.","pith_inferences":["The paper's headline comparison is not backbone-matched, and the authors do not ablate the language model; a rerun that swaps LLaMA3-8B-Instruct for the exact backbone used by the previous general-purpose baseline would isolate how much of the gain comes from the three proposed components rather than from a newer LLM.","The GPT-4o-generated hard negatives are not human-validated; if a share of them are wrong or only trivially different, the measured contrastive benefit could partly reflect dataset filtering or noise, so a small human audit of negative quality would tighten the claim.","Because the odds-ratio loss is not specific to point clouds, the same triplet-construction and OR training recipe could transfer to 2D vision-language models or any generative instruction-following setting where near-miss hallucinations are the dominant failure mode.","The paper's own limitation note says resources allowed only a few epochs and prevented million-scale training; whether the reported SOTA gaps persist under longer training and larger data remains untested by the paper."],"forward_implications":["High-resolution point clouds are nearly free: the paper reports the point encoder is only 0.27% of the model's parameters, so moving from 1,024 to 8,192 points raises fidelity without a meaningful compute penalty.","Hard negatives are doing real work beyond mere data augmentation: the hard-negative setting beats an easy-negative triplet setting on almost every metric, and the log-odds and reward-margin curves show hard negatives keep the model challenged throughout training.","The odds-ratio loss is stable to its weight: performance stays high for lambda from 5e-2 to 5e-1 and beats both plain SFT and the probability-ratio variant on ScanQA, SQA3D, and Scan2Cap.","Object hallucination is reduced as a by-product: on 3D-POPE, CL3DOR's yes-rate is far closer to the balanced baseline than prior 3D LMMs, which were heavily biased toward saying objects exist.","A single general-purpose model can beat task-specific and task-adaptive models on 3D captioning and QA without fine-tuning per benchmark."],"supporting_citations":[{"why":"Supplies the odds-ratio objective that CL3DOR reinterprets as an auxiliary contrastive loss.","marker":"(Hong et al., 2024)"},{"why":"Provides the main general-purpose baseline and the benchmark numbers, refined EM metric, and instruction templates CL3DOR compares against.","marker":"(Huang et al., 2024)"},{"why":"Defines the ScanQA dataset and metric used for the main 3D question-answering evaluation.","marker":"(Azuma et al., 2022)"},{"why":"Defines the SQA3D situated-question-answering benchmark evaluated in the paper.","marker":"(Ma et al., 2022)"},{"why":"Defines the Scan2Cap dataset and captioning metric used for 3D object-in-scene captioning.","marker":"(Chen et al., 2021)"},{"why":"Defines the 3D-POPE hallucination benchmark used to evaluate object-existence bias.","marker":"(Yang et al., 2024)"},{"why":"Provides PointBERT, the frozen point-cloud encoder that consumes the 8,192-point per-object inputs.","marker":"(Yu et al., 2022)"},{"why":"Supplies the spatial transformer architecture used to model inter-object relations in scenes.","marker":"(Chen et al., 2022)"},{"why":"Provides LLaMA3-8B-Instruct, the language-model backbone whose weights are trained in the final stage.","marker":"(Dubey et al., 2024)"}],"fun_headline_variants":["Odds-ratio contrastive loss sharpens 3D LMMs on dense points","High-res point clouds plus odds-ratio loss beat 3D generalists","CL3DOR: harder negatives and odds-ratio lift 3D multimodal models","8K points per object and odds-ratio loss push 3D LMMs to SOTA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The main gain over the previous general-purpose model is measured without keeping the language-model backbone fixed, so a stronger newer LLM rather than the 8,192-point inputs, hard negatives, or odds-ratio loss could explain part of the improvement.","fun_headline_variants_meta":{"raw":{"variants":["Odds-ratio contrastive loss sharpens 3D LMMs on dense points","High-res point clouds plus odds-ratio loss beat 3D generalists","CL3DOR: harder negatives and odds-ratio lift 3D multimodal models","8K points per object and odds-ratio loss push 3D LMMs to SOTA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000167,"raw_usage":{"total_tokens":1312,"prompt_tokens":1053,"completion_tokens":259,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":669,"completion_tokens_details":{"reasoning_tokens":166}},"tokens_in":669,"tokens_out":259,"duration_ms":2907,"temperature":1.0,"reasoning_tokens":166,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:44:55.026670+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain CL3DOR while holding the LLM backbone identical to the previous general-purpose baseline (and retrain that baseline with LLaMA3-8B-Instruct), keeping point-cloud resolution, hard negatives, and the OR loss exactly the same; if the 5.3-point ScanQA EM gain and 21-point CIDEr gain collapse, the proposed components are not the cause. Separately, have human annotators judge a random sample of GPT-4o-generated negatives for correctness and hardness; if many are wrong or easy, the negative-quality premise fails.","supporting_citations":[{"cited_title":"An embodied generalist agent in 3d world","cited_arxiv_id":null,"evidence_quote":"Provides the main general-purpose baseline and the benchmark numbers, refined EM metric, and instruction templates CL3DOR compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Scan2Cap dataset and captioning metric used for 3D object-in-scene captioning."},{"cited_title":"Language conditioned spatial relation reasoning for 3d object grounding","cited_arxiv_id":null,"evidence_quote":"Supplies the spatial transformer architecture used to model inter-object relations in scenes."}],"review_version":1}