{"id":"58d1d72e-cc5c-4a59-abca-182c368a3fa9","arxiv_id":"2505.20639","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Open-Det trains an open-ended detector on Visual Genome and reports higher zero-shot LVIS accuracy than GenerateU using 1.5% of the data and 31 epochs instead of 149.","lead":"Open-Det is a new training framework for open-ended object detection, a task where a model must draw boxes around objects and write a name for each one without being given a fixed vocabulary at test time. It reports matching or better accuracy than the prior GenerateU system while using about 1.5% of the training data, about a fifth of the training epochs, and less GPU hardware.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline efficiency claim is not yet controlled: the '+1.0% APr with 1.5% of data' compares Open-Det to GenerateU trained on VG+GRIT5M, while '20.8% of epochs (31 vs 149)' compares to a different GenerateU run (VG only), and neither reference was re-run in the train-LLM setting.","rationale":"The paper's strongest claim is that Open-Det is strictly more efficient and more accurate than GenerateU on zero-shot LVIS MiniVal. Three conditions must hold for that claim: (i) the reference GenerateU numbers must be comparable or re-produced under the same protocol; (ii) the abstract must describe a single paired comparison; and (iii) the method description must be faithful enough to reproduce the trained system. Condition (ii) fails as written: the abstract's '1.5% of the data' and '+1.0% APr' reference the VG+GRIT5M GenerateU row while '20.8% of the epochs (31 vs 149)' references the VG-only row; these are different models with different training budgets, so the composite headline cannot be verified from any single table row. Condition (i) is the load-bearing concern: the only in-paper GenerateU re-run is the frozen-LLM variant (Table 7), not the train-LLM setting that the headline compares against. Because the authors already run GenerateU for Table 7 and Figures 8–9, the missing controlled train-LLM re-run is a genuine gap rather than a practical necessity. Condition (iii) is threatened by Eq. 7 (distillation loss written with Qd rather than Pvl) and by Eq. 3's log applied to a cosine sum that can be negative; a reader cannot implement the exact losses, so the ablated gains in Tables 4–5 cannot be independently checked. Credit where due: the epoch arithmetic in Appendix B.1 checks out (180,000×64/77,398≈149; 300,000×8/77,398≈31), the paper includes a controlled GenerateU† re-run in the frozen-LLM setting (Table 7) where Open-Det wins by +5.6 APr, and the released code makes the proposed controlled re-run feasible. The ablated gains in Tables 3–5 are directionally consistent, and the architecture combines known components in a plausible way. None of the identified issues is a mathematical error that would invalidate the construction outright; rather, the evidence supporting the headline claim is not yet as controlled as the strength of the claim requires. On this basis I agree with the reader's verdict of CONDITIONAL: accept with conditions that include a controlled train-LLM GenerateU re-run under Open-Det's hardware and evaluator, corrected and implementation-faithful specifications of Eqs. 3 and 7, and disclosure of the VLM/LoRA configuration and total GPU-hours. Since my read does not change the reader's verdict, verdict_should_be is UNCHANGED.","tokens_in":22026,"tokens_out":17587,"duration_ms":160377,"concrete_test":"Run the official GenerateU training code in the full train-LLM configuration (unfrozen FlanT5, Swin-Tiny, the setup that produced the 17.4 APr reference) on the same 4×V100/batch-8 hardware as Open-Det, for the full 149-epoch budget (~1.44M iterations at batch 8), and evaluate the resulting checkpoint with Open-Det's exact LVIS MiniVal evaluator (same pre-trained text encoder, same name-similarity matching and AP code). If the reproduced row matches the published 17.4/22.4/29.6/25.4 within noise, the Table 1 and Table 6 comparisons and the abstract's efficiency claims are controlled. If it deviates materially, the +1.0% and +3.6% APr advantages are not established against a controlled baseline.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—strictly higher APr with 1.5% of the data, 20.8% of the epochs, and fewer GPUs—has no single reference baseline. In Table 1, the data-size comparison (0.077M vs 5.077M) and the +1.0% APr gain refer to GenerateU trained on VG+GRIT5M (APr 20.0), whereas the 31-vs-149 epoch comparison refers to GenerateU trained on VG only (APr 17.4). The abstract therefore asserts a composite that no single row of Table 1 supports: against the VG-only model the gain is +3.6 APr, not +1.0%; against the GRIT5M model the epoch count of the reference is not reported. Either way, the comparison depends on re-cited GenerateU numbers produced with batch size 64 on 16 A100s over 180k iterations. Open-Det uses batch size 8 on 4 V100s over 300k iterations; the Appendix B.1 epoch conversion (180k×64/77,398 ≈ 149; 300k×8/77,398 ≈ 31) is arithmetically correct, but it does not control for optimizer schedule, gradient noise at different batch sizes, or evaluator identity. The authors demonstrably can run GenerateU: Table 7 reports a GenerateU† re-run on 4 V100s, and Figures 8–9 report GenerateU score distributions, so the absence of a controlled train-LLM re-run of the reference row is a concrete, fixable omission rather than an infeasibility. If the re-cited GenerateU numbers shift under Open-Det's evaluator or hardware, the headline advantage reflects protocol rather than the proposed architecture. Secondary hazards—Eq. 7 writing LVLD with Qd instead of the VL-prompts Pvl that Section 3.4 distills, and Eq. 3 taking log(Salign) of a cosine sum in [−2,2]—indicate the method description is not yet implementation-faithful enough to reproduce the comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Open-Det, a framework for open-ended object detection that combines an object detector with a generative language model. The main contributions are a decoupled one-to-many/one-to-one decoder and threshold-based query selection, a bidirectional vision-language alignment module, a prompts distiller that transfers VLM knowledge into VL-prompts, a LoRA-based object name generator with text denoising, and two loss functions (Masked Alignment Loss and Joint Loss). The authors claim that Open-Det matches or exceeds GenerateU while using only 1.5% of the training data, 20.8% of the training epochs, and fewer GPU resources, and they support the claim with experiments on LVIS MiniVal, COCO, and Objects365, together with extensive ablations.","tokens_in":22468,"tokens_out":5317,"duration_ms":52265,"significance":"If the efficiency claims are validated, Open-Det would be a meaningful step toward practical open-ended detection: it addresses a real training-cost bottleneck in GenerateU and introduces several mechanistically motivated components (distilled VL-prompts, masked alignment, and a joint classification loss). The paper is commendable for releasing source code, providing component-wise ablations, and reporting score-distribution analyses. The main risk is the headline comparison: the claimed simultaneous advantages in data, epochs, and accuracy are assembled from two different GenerateU baselines, and no controlled re-run of the reference model in the train-LLM setting is provided. The contribution is therefore plausible but not yet established at the level claimed.","major_comments":[{"comment":"The headline efficiency claim combines two different GenerateU baselines. The '1.5% of training data (0.077M vs 5.077M)' and '+1.0% APr' numbers compare Open-Det on VG with GenerateU trained on VG+GRIT5M (APr 20.0), while the '20.8% of epochs (31 vs 149)' number compares with GenerateU trained on VG only (APr 17.4). No single row of Table 1 supports all three ratios simultaneously, and the +1.0% APr gain does not correspond to the 31-vs-149 epoch comparison. The paper should either report a single matched baseline for all three quantities or clearly separate the two comparisons in the abstract and conclusions.","section":"Abstract and Table 1"},{"comment":"The conversion from iterations to epochs (180,000 x 64 / 77,398 = 149; 300,000 x 8 / 77,398 = 31) is arithmetically correct, but it does not equate the training protocols: GenerateU uses batch size 64 on 16 A100 GPUs, while Open-Det uses batch size 8 on 4 V100 GPUs, and the optimizer schedules and evaluation harness are not matched. Since the paper demonstrates in Table 7 that GenerateU can be re-run on 4 V100 GPUs with the public official code (in the frozen-LLM setting), a controlled train-LLM re-run of the reference row is feasible and should be supplied. Without such a re-run, the claimed data/epoch/GPU advantages cannot be attributed to the proposed architecture rather than to protocol differences.","section":"Appendix B.1 and Table 7"},{"comment":"The total-loss decomposition defines the distillation loss as LVLD = 1 - cosine(Qd, Te), using the decoder queries Qd, whereas Section 3.4 and Figure 3 state that the cosine-similarity supervision is applied to the distilled VL-prompts Pvl. This is a material inconsistency for reproducibility: if the implementation follows Section 3.4, Eq. (7) is wrong; if Eq. (7) is correct, the description of the Prompts Distiller is wrong. Please reconcile the notation and specify exactly which tensors enter the cosine loss.","section":"Section 3.4 and Appendix A.5, Eq. (7)"}],"minor_comments":[{"comment":"The symbol M is used both for the number of text embeddings and for the binary mask matrix, which makes the dimension of the product VLalign x M easy to misread; please use distinct symbols (e.g., M for the mask and M_text or K for the number of embeddings).","section":"Section 3.6, Eq. (3)"},{"comment":"The set notation for Qid and the statement that decoder queries Qd are selected from E according to Qid is not fully precise; please specify how the selected indices are converted into query embeddings and how variable-length query sets are handled by the transformer decoder.","section":"Section 3.2, Eq. (1)"},{"comment":"For the GenerateU trained on VG+GRIT5M row, the training epoch count is omitted, so the reader cannot verify the '20.8% of epochs' comparison against that baseline; please report the epoch count or explicitly state that it is not available.","section":"Section 4.1 and Table 1"},{"comment":"There is a typo in the table: 'GenereteU' should be 'GenerateU'; also, 'LoRa' and 'LoRA' are used inconsistently throughout the paper.","section":"Appendix B.1, Table 6"},{"comment":"The symbol N is reused for the number of queries in Eq. (4) and for the number of tokens in the output sequence in Eq. (9); please use a different symbol (e.g., T) for the token count to avoid ambiguity.","section":"Appendix A.5, Eq. (9)"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a computer-vision or machine-learning venue, and the proposed method appears novel and potentially useful. My recommendation of major revision is driven by the uncontrolled baseline comparison and the inconsistency in the distillation loss definition, both of which are fixable. I do not have concerns about citation patterns: the self-citations are peripheral to the central claims. The authors should be asked to provide a controlled GenerateU re-run in the train-LLM setting and to align the abstract's efficiency statement with a single baseline or explicitly separate the data/epoch comparisons."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Open-Det is a genuinely useful systems paper: it takes the GenerateU-style open-ended detection pipeline and makes it trainable on a fraction of the data and compute, with ablations and code to back up the component choices. The one-to-many/one-to-one decoupled decoder, LoRA head, text denoising, and the masked alignment/Joint losses are all adapted from known ideas, but the integration is new and the gains on LVIS are large. If the headline efficiency numbers hold under a controlled comparison, this is a practical advance worth taking seriously.\n\nThe soft spot is exactly the headline. The abstract says '+1.0% APr with 1.5% of the data, 20.8% of the epochs.' As the stress-test note details, those two ratios refer to two different GenerateU runs: the data ratio and +1.0% come from the VG+GRIT5M row (APr 20.0), while the epoch ratio comes from the VG-only row (APr 17.4). No single row of Table 1 supports the abstract's composite claim. Against the VG-only model the gain is +3.6 APr; against the GRIT5M model the epoch count isn't reported. The appendix's iteration-to-epoch conversion is arithmetically fine, but it doesn't control for batch size, optimizer schedule, or the evaluator. The authors can run GenerateU—Table 7 reports a re-run on 4 V100s—so the missing controlled train-LLM re-run is a fixable omission, not an impossibility.\n\nThere are also two genuine method-description bugs: Eq. 7 writes the VLD loss using Qd instead of the VL-prompts Pvl that Sec. 3.4 says are distilled, and Eq. 3 takes log(Salign) where Salign is the sum of two cosine similarities, which can be negative. Neither kills the architecture, but both suggest the text isn't yet implementation-faithful enough to reproduce the comparison. The free parameters (thresholds, α, LoRA rank) are reported but sensitivity is not explored, which is a minor gap for a systems paper.\n\nWho's this for? Anyone working on vocabulary-free detection or efficient VLM-distilled detectors. It deserves a serious referee: the idea is timely, the ablations are thorough, and the concerns are about experimental control rather than fatal flaws. I'd send it to review but with a clear request: run GenerateU train-LLM on the same hardware/evaluator, report the GRIT5M-baseline epoch count, and fix the equation inconsistencies before publication.","headline":"Solid systems paper with a plausible efficiency story, but the headline comparison mixes two GenerateU baselines and needs a controlled re-run before I'd trust the +1.0% APr claim.","tokens_in":23042,"tokens_out":2271,"would_cite":true,"duration_ms":22511,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Open-Det achieves open-ended object detection with 1.5% of GenerateU's training data, 20.8% of its epochs, and fewer GPUs, while raising APr on zero-shot LVIS by 1.0 point.","keywords":["open-ended object detection","zero-shot object detection","vision-language alignment","knowledge distillation","efficient training","generative language model","object name generation","LVIS"],"falsifier":"Run a controlled experiment that trains the original GenerateU code on the same 4 V100 GPUs, batch size 8, optimizer configuration, and 31 epochs used for Open-Det, then compare on LVIS MiniVal; if GenerateU reaches an equal or higher APr under that budget, the claimed efficiency gain is not caused by the proposed architecture.","tokens_in":21794,"feed_emoji":"🎯","tokens_out":7354,"duration_ms":66654,"temperature":0.7,"pith_summary":"Open-Det is a framework for open-ended object detection—locating objects and generating their category names in free-form text without any fixed vocabulary at inference. The paper's claim is that this task does not need the web-scale grounding datasets and heavy GPU budgets used by the existing GenerateU approach. Trained on the 77,398 Visual Genome images (1.5% of GenerateU's 5.077M images) for 31 epochs (20.8% of GenerateU's 149) on 4 V100 GPUs, Open-Det reports an APr of 21.0 on zero-shot LVIS MiniVal, 1.0 point higher than GenerateU trained with GRIT5M. The framework reaches this efficiency by rebuilding the detector and name generator, distilling vision-language knowledge into 'VL-prompts', and replacing contradictory alignment supervision with masked and joint losses. If the efficiency comparisons survive matched training budgets, cheap and practical open-ended detection becomes a realistic target for modest compute settings.","feed_headline":"Open-Det matches, then beats, GenerateU with 1.5% of its data","feed_subtitle":"Uses 31 epochs and 4 V100 GPUs, yet raises zero-shot LVIS APr by 1.0 point over GenerateU.","key_machinery":"The central mechanism is the VL-prompt: a decoder query transformed into an image-like representation aligned to text-embedding space, produced by VLD-M through deformable cross-attention over backbone and encoder features, MLP projection, and linear fusion, then supervised against the VLM's text embeddings by cosine similarity. The VL-prompt, rather than the raw vision query, is what the LLM sees, which is what lets Open-Det bridge the cross-modal gap and train the name generator without large paired data. Around it sit the bidirectional alignment score $S_{\\text{align}} = \\cos(Q_d M_{VL}, T_e) + \\cos(Q_d, T_e M_{LV})$, the decoupled one-to-many/one-to-one decoder, and the two loss corrections: Masked Alignment Loss (binarized text-text mask set by threshold 0.99) and Joint Loss combining binary, IoU, and alignment scores.","core_discovery":"On its own terms, Open-Det claims that accelerating open-ended detection is a matter of architecture and supervision design rather than data scale. Its Object Detector uses a decoupled decoder: the first four layers train with one-to-many matching to speed box convergence, the last two use one-to-one matching to suppress duplicates, and a threshold on encoder-token scores selects queries so the model detects a variable number of objects. Instead of feeding raw vision queries to the T5-based name generator, a Vision-to-Language Distillation Module (VLD-M) enriches each query with backbone and encoder context through deformable cross-attention, projects it into the text-embedding space, and distills a frozen CLIP's knowledge into these 'VL-prompts' with a cosine-similarity loss. A Bidirectional Vision-Language Alignment module (BVLA-M) computes alignment scores in both V-to-L and L-to-V directions, a Masked Alignment Loss zeroes out contradictory same-category negative pairs, and a Joint Loss couples binary classification with IoU and alignment scores. The reported outcome is a model that, from 0.077M images and 31 epochs, outperforms GenerateU trained on 5.077M images for 149 epochs, and scales further with larger backbones (APr 31.2 with Swin-Large versus 22.3).","pith_inferences":["Editorial inference: the design suggests the real bottleneck in open-ended detection is the modality gap rather than data volume; if so, applying VL-prompt distillation to other generative detectors could yield similar efficiency gains.","Editorial inference: the reported efficiency advantage depends on the epoch conversion in Appendix B.1; a controlled re-run of GenerateU under Open-Det's batch size and optimizer would cleanly separate architecture effects from schedule and hardware effects.","Editorial inference: freezing the language model head and training a LoRa head during early epochs, plus text denoising, may generalize as a recipe for fine-tuning any heavy generative head on noisy multimodal inputs.","Editorial inference: threshold-based query selection makes the detector's object count dynamic, a property that could be extended to other DETR-like detectors beyond the open-ended setting."],"forward_implications":["Open-ended detection can be trained from a single small labeled dataset (Visual Genome, 77k images) rather than multi-million-image grounding corpora, reducing data acquisition cost by roughly 65 times.","Rare-class performance, historically the weak spot in zero-shot detection, improves most: +3.6% APr over GenerateU at equal data, +1.0% APr over GenerateU with GRIT5M, and +8.9% APr with a larger backbone.","The same framework transfers zero-shot to COCO2017 and Objects365 with +2.2 and +3.3 AP over GenerateU, so the data-efficiency gain is not confined to the LVIS benchmark.","Convergence speed is compressed to about one fifth of the epochs, making iterative experimentation on open-ended detection practical on 4 V100 GPUs instead of 16 A100s."],"supporting_citations":[{"why":"Defines the OED task and provides the GenerateU baseline, including its training data and epoch counts that Open-Det compares against.","marker":"Lin et al., 2024"},{"why":"Supplies the Visual Genome training set of 77,398 images that Open-Det uses.","marker":"Krishna et al., 2017"},{"why":"Provides the zero-shot LVIS benchmark and the APr, APc, APf, and AP metrics used for evaluation.","marker":"Gupta et al., 2019"},{"why":"Supplies the frozen VLM whose text embeddings are used for distillation, alignment scoring, and name-similarity evaluation.","marker":"Radford et al., 2021"},{"why":"Supplies the T5 generative language model whose heavy head is frozen and replaced by a LoRa Head in the Object Name Generator.","marker":"Raffel et al., 2020"},{"why":"Provides the deformable cross-attention operation used in VLD-M and in the detector design.","marker":"Zhu et al., 2020"},{"why":"Supplies the DINO-style detector base and anchor-box denoising ideas that the Object Detector builds on.","marker":"Zhang et al., 2022a"},{"why":"Provides the GRIT5M grounding data used in GenerateU's larger-scale pretraining, the comparison point for Open-Det's 1.5% data claim.","marker":"Peng et al., 2023"}],"fun_headline_variants":["Open-Det beats GenerateU with 1.5% of the data","Open-Det: 98.5% less data, 20% epochs, 1 point higher APr","Open-Det: tiny data, tiny compute, better open-ended detection","Open-Det cuts data by 98.5% and still tops GenerateU","Open-Det: 31 epochs on 4 V100s outclass 149 on 16 A100s"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim depends on the epoch conversion in Appendix B.1 faithfully equating GenerateU's and Open-Det's training budgets despite different batch sizes, GPU types, and optimizer schedules; if those budgets are not equivalent, the reported efficiency advantage could be an artifact of schedule and hardware rather than the proposed architecture.","fun_headline_variants_meta":{"raw":{"variants":["Open-Det beats GenerateU with 1.5% of the data","Open-Det: 98.5% less data, 20% epochs, 1 point higher APr","Open-Det: tiny data, tiny compute, better open-ended detection","Open-Det cuts data by 98.5% and still tops GenerateU","Open-Det: 31 epochs on 4 V100s outclass 149 on 16 A100s"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000427,"raw_usage":{"total_tokens":2268,"prompt_tokens":1113,"completion_tokens":1155,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":729,"completion_tokens_details":{"reasoning_tokens":1038}},"tokens_in":729,"tokens_out":1155,"duration_ms":10454,"temperature":1.0,"reasoning_tokens":1038,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:50:31.813016+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled experiment that trains the original GenerateU code on the same 4 V100 GPUs, batch size 8, optimizer configuration, and 31 epochs used for Open-Det, then compare on LVIS MiniVal; if GenerateU reaches an equal or higher APr under that budget, the claimed efficiency gain is not caused by the proposed architecture.","supporting_citations":[],"review_version":1}