{"id":"a2c6ba8a-85ef-41d8-8dd4-014e2177de29","arxiv_id":"2412.07161","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"ULAO, a CLIP-based framework with object-first sequential prediction and prediction-based hard negative contrastive training, reports state-of-the-art compositional zero-shot learning results on MIT-States, UT-Zappos, and C-GQA.","lead":"A new CLIP-based system, ULAO, improves compositional zero-shot learning by first recognizing the object in an image, then using that object as context to identify its attribute, and by training a separate branch to avoid linking the right attribute to the wrong object. The authors report state-of-the-art accuracy on three standard benchmarks in both closed-world and open-world settings, though the gains are modest on two datasets and no code or error bars are provided.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Both novel modules inherit UAO's object predictions; the paper's own failure cases show wrong cues can mislead, so the SOTA claim is conditional on object recognition reliability. An oracle-cue test would settle it.","rationale":"I read the paper as an empirical CZSL contribution whose headline claim is state-of-the-art HM/AUC on three benchmarks. The reader's weakest-assumption analysis points to the reliability of object-first predictions, and the manuscript itself supplies independent evidence for that concern: the Limitations paragraph admits errors when object cues are wrong, and Fig. 5 shows failure cases where adding UAO to C+AO changes a correct or plausible prediction into an incorrect one (Ancient-Computer to Old-Computer, Browned-Cake to Sliced-Cake). Both of the paper's novel mechanisms depend on UAO probabilities, so the concern is not peripheral. The ablation study in Table 3 does show that C+UAO improves over C+AO on HM and AUC, but that comparison does not tell us how often the predicted object cues are correct on unseen compositions, nor whether the gains would survive a systematic object-recognition failure. In addition, the lack of released code, missing error bars, and single-seed parameter studies already noted by the reader make the small SOTA margins hard to interpret. I do not think this warrants rejection: the method is clearly motivated, the ablations are consistent with the design story, and the reported numbers are plausible. However, the central claim should be conditional on an explicit test of the acknowledged object-cue failure mode. Since the reader's verdict is already CONDITIONAL, my read does not move the verdict; it strengthens the reason for the condition.","tokens_in":13812,"tokens_out":7077,"duration_ms":85378,"concrete_test":"Run the published ULAO pipeline on MIT-States and UT-Zappos with the object-cue mechanism in Eq. (3) replaced by the ground-truth object (set k=1, rk=1, e'_3=w_co) and with hard negatives in Eqs. (7)-(8) selected from the ground-truth object rather than the UAO argmax, keeping all other hyperparameters and training settings identical. If the oracle-cue version improves CW or OW HM/AUC by a large margin (e.g., more than 1 point HM on MIT-States), the model depends on object recognition being correct and the acknowledged failure mode is central; if the oracle-cue version is comparable or worse, the method is robust to object-cue errors and the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that ULAO consistently outperforms prior methods on HM and AUC rests on two modules that both route through the UAO object probabilities. In Eq. (3), the attribute prompt is built from a top-k weighted sum of predicted object embeddings, so an incorrect object prediction directly corrupts the contextual attribute cue. In Eqs. (7)-(8), hard negatives are selected by argmax over the same UAO attribute and object probabilities, so the contrastive loss can reinforce exactly the confusions that UAO got wrong. The paper itself acknowledges this in the Limitations section: 'Transitioning to UAO can lead to errors if the object cues are incorrect.' The qualitative results in Fig. 5 also show concrete failures attributable to bad object cues, e.g., C+UAO turning Ancient-Computer into Old-Computer and Browned-Cake into Sliced-Cake. The ablations (C+AO to C+UAO to ULAO) show aggregate gains, but they do not isolate whether those gains require object predictions to be correct, and no object-only accuracy on unseen compositions is reported. Because several headline margins are small -- MIT-States CW HM is 39.3 vs 39.2 for CDSCZSL -- a systematic weakness in object cues could plausibly flip the comparison. This is not a fatal flaw, but it is a load-bearing assumption that is currently untested.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ULAO, a CLIP-based framework for compositional zero-shot learning (CZSL) with two modules. The UAO module performs object-first sequential prediction and uses top-k predicted object embeddings as contextual cues in attribute prompts (Eq. 3). The LAO module adds a direct composition-prediction branch and a contrastive loss whose hard negatives are generated from the UAO module's most probable attribute/object mispredictions (Eqs. 7-8), with an adaptively updated threshold (Eqs. 9-11). The authors report state-of-the-art HM and AUC results on MIT-States, UT-Zappos, and C-GQA in both closed-world and open-world settings, supported by ablations and qualitative examples.","tokens_in":14090,"tokens_out":4731,"duration_ms":51537,"significance":"If the reported gains reproduce, ULAO is a useful contribution to CLIP-based CZSL: it directly targets two known failure modes of CLIP, namely primitive misunderstanding and attribute-object mislinking. The idea of constructing textual hard negatives from the model's own most-confusing predictions is interesting and is tested against random and composition-branch alternatives. The paper also explicitly discusses limitations, which is commendable. The main uncertainty is empirical: the central SOTA claim rests on small margins for some datasets, and the results are reported without variance estimates, while the key design assumption about the reliability of UAO object cues is not directly tested.","major_comments":[{"comment":"The claim that ULAO 'consistently outperforms other methods on HM and AUC across all datasets' is not yet supported with statistical evidence. No error bars, confidence intervals, or multiple-seed results are reported. On MIT-States CW, the HM margin over CDSCZSL is 0.1 point (39.3 vs. 39.2), and on MIT-States OW the unseen accuracy is actually lower than CDSCZSL (21.6 vs. 21.8). Please run at least three seeds and report mean ± standard deviation (or comparable dispersion measures) for the main comparisons, and state whether the differences are consistent across seeds.","section":"Comparisons with SOTAs (Table 2)"},{"comment":"Both novel modules inherit the UAO object predictions: Eq. (3) constructs the attribute-prompt object token as a probability-weighted combination of top-k predicted object embeddings, and Eqs. (7)-(8) select hard negatives by argmax over the same UAO attribute and object probabilities. The paper itself notes that 'Transitioning to UAO can lead to errors if the object cues are incorrect,' and Fig. 5 shows concrete failures (e.g., Ancient-Computer → Old-Computer, Browned-Cake → Sliced-Cake). Because several headline margins are small, the SOTA claim is conditional on the reliability of these object cues. I request an oracle-cue ablation that feeds ground-truth object labels into Eq. (3) and/or Eqs. (7)-(8), together with object-only accuracy on unseen compositions, to determine whether the gains are robust when object cues are correct.","section":"Method (Eqs. 3, 7-8) and Limitations"},{"comment":"The generated negatives must belong to the seen composition set Y^S, but the paper does not specify what happens when no seen composition (a', co) or (ca, o') exists for a given ground-truth pair. In sparse datasets such as C-GQA, such missing negatives could exclude many training examples from the contrastive loss. Please clarify whether those examples are skipped or receive a fallback, and report the coverage rate of negative generation per dataset.","section":"Eqs. (7)-(8), Hard negative generation"}],"minor_comments":[{"comment":"There are several typos and formatting artifacts: 'Prediciton' in Fig. 1, 'an then' in the Introduction, 'modu les' in the Final prediction and training paragraph, and 'GPICOL' vs. 'GIPCOL' in Table 2 versus the reference title.","section":"Throughout"},{"comment":"The row labels in Table 4 are not readable; the columns 'C', 'Random', 'UAO', 'Mean', 'Max', 'Fixed', 'up' need explicit row headers and checkmarks so the reader can tell which variant each row corresponds to.","section":"Table 4"},{"comment":"The text states that 'The provided Supplementary materials include codes, environment requirements, and detailed config files,' but no supplementary material or link is accessible in the arXiv version. Please provide an anonymized code/data link or include the config files as actual supplementary material.","section":"Implementation Details"},{"comment":"The threshold update th_t = max(th_{t-1}, delta_{t-1}) makes the threshold monotonically nondecreasing up to the bound; the term 'adaptive' would be more accurate if the threshold could also decrease when the model improves. Please justify the one-sided ratchet or test a two-sided update.","section":"Eq. (10)"}],"recommendation":"major_revision","confidential_remarks":"The paper has a clear and plausible method, but the empirical evidence needs to be strengthened before acceptance: no variance estimates, a small margin on a key dataset, and an untested load-bearing assumption about object-cue reliability. I would require the authors to supply an oracle-cue experiment and multi-seed results. The claim about provided supplementary code should also be verified by the editors, as no materials are currently available."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a competently executed CZSL paper with a genuinely novel combination of ideas. The UAO module (object-first sequential prediction, with top-k predicted objects as contextual hints for attribute classification) and the LAO module (hard negatives generated from UAO's own top confusions, with an adaptive margin) are both reasonable and are not just re-labeled prior work. The ablations in Table 3 show each component contributes, and the gains on UT-Zappos (HM 52.7 to 57.2 CW) and C-GQA OW are real and substantial. The authors are also honest: the limitations section explicitly says that transitioning to UAO can cause errors if object cues are wrong, and Fig. 5 shows concrete failures of exactly that kind.\n\nThe main soft spots are exactly what the reader flagged. No error bars or multiple seeds for any table, so we cannot tell whether the MIT-States CW HM lead of 39.3 vs 39.2 over CDSCZSL is noise. The supplementary code and configs are promised but not actually accessible on arXiv. And a lot of hyperparameters (rk, k, rao, rc, alpha, up, rm) are tuned on the same three benchmarks, so the usual overfitting-to-benchmark caveat applies.\n\nThere is one structural concern worth taking seriously. Both novel components route through the UAO object probabilities: the attribute prompt in Eq. (3) is a top-k weighted sum of predicted object embeddings, and the hard negatives in Eqs. (7)-(8) are argmax over the same UAO probabilities. So if object prediction is wrong, both the cue and the contrastive loss can be corrupted by the same error. The paper's own failure cases show this happens. This does not sink the paper—the ablations show aggregate gains—but it is a load-bearing assumption that is currently untested. An oracle-cue experiment (feed ground-truth object cues during attribute prediction, and/or generate negatives from oracle object labels) would cleanly separate the benefit of the idea from the reliability of UAO's object predictions. That should be requested in review.\n\nThe citation pattern looks fair. The related work covers the relevant CLIP-based CZSL methods, and self-citations are to prior work in the same group, which is normal here.\n\nBottom line: this paper deserves a serious referee. It is incremental in the niche sense, but the two modules are reusable ideas and the reported empirical results are mostly consistent. I would ask for code/error bars and the oracle-cue analysis before trusting the SOTA claim, but the work itself is honest and technically coherent.","headline":"Solid, honest CZSL paper with a genuinely novel two-module design; the SOTA claim is conditional on the promised code/error bars and on how much object-cue errors propagate.","tokens_in":14620,"tokens_out":1766,"would_cite":true,"duration_ms":17591,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A CLIP-based framework called ULAO claims state-of-the-art compositional zero-shot learning by predicting objects before attributes and adaptively contrasting hard negatives.","keywords":["compositional zero-shot learning","vision-language models","CLIP","sequential primitive prediction","hard negative generation","adaptive contrastive loss","attribute-object composition","open-world recognition"],"falsifier":"Take a set of images in which object recognition is artificially corrupted—crop out or mask the object regions, or replace the top-1 object with a wrong but plausible object prompt—and measure whether UAO's attribute accuracy drops more than that of a joint feature baseline; if attribute accuracy is robust to wrong object cues, the contextual-hint mechanism is not doing the claimed work. Alternatively, train UAO with k=0 (no object hints) and check whether the reported HM/AUC gap on MIT-States persists.","tokens_in":13588,"feed_emoji":"🧩","tokens_out":4490,"duration_ms":38684,"temperature":0.7,"pith_summary":"This paper aims to fix a specific failure mode of CLIP-based models in compositional zero-shot learning: they confuse attributes for objects (calling a draped silk dress 'bright orange') and link attributes to the wrong object ('ruffled pants' instead of 'ruffled jacket'). The proposed ULAO framework solves this by predicting the object first, using the top-k predicted objects as weighted contextual hints to refine attribute prediction, and then training a contrastive module that generates hard negative compositions from the model's own most-confused predictions and pushes them away with an adaptively growing margin. The paper reports state-of-the-art harmonic-mean and AUC numbers across MIT-States, UT-Zappos, and C-GQA in both closed-world and open-world settings. If correct, the gain comes from modeling, not from changing the CLIP backbone.","feed_headline":"Predict the object first, then the attribute: a new CZSL recipe","feed_subtitle":"CLIP-based ULAO beats prior models on MIT-States, UT-Zappos, and C-GQA by adding object cues and adaptive hard negatives.","key_machinery":"The load-bearing mechanism is the two-module pipeline. UAO first computes object probabilities from an object-specific visual attention stream using the prompt 'a photo of [object]', then replaces the generic 'object' token in the attribute prompt with a probability-weighted mixture of the top-k predicted objects' embeddings, detached from gradient flow; the composition score is the product of attribute and object probabilities. LAO adds a composition branch whose visual features are the image feature plus a learned mixer output, and a fully learnable prompt '[a] [photo] [of] [attribute] [object]'; it generates exactly two hard negatives per training image—the most probable wrong attribute paired with the true object, and the true attribute paired with the most probable wrong object, both restricted to seen compositions—and applies a contrastive loss with a per-negative threshold that starts at zero and grows with the running mean similarity gap, capped by an upper bound.","core_discovery":"The central claim is that splitting CZSL into sequential primitive prediction—object first, attribute second with object context—and then linking primitives through contrastive learning with model-generated hard negatives delivers consistent state-of-the-art results. On the three benchmarks the paper reports the best HM and AUC in both closed-world and open-world evaluation, with the largest relative gain (19.6%) on the open-world setting of C-GQA, where the search space is largest. The paper attributes the gain to two mechanisms: UAO reduces attribute-object confusion by letting object recognition proceed without attribute interference, and LAO shrinks the effective search space by penalizing exactly the mislinkages the model is most prone to.","pith_inferences":["The object-first design implicitly assumes objects are more reliably recognized than attributes; in scenes with multiple objects or occlusion, the top-k object mix could average over unrelated objects and degrade attribute cues—a testable prediction: performance should drop more on multi-object images than single-object ones.","Because the hard negatives are restricted to seen compositions, the model never sees a negative involving an unseen object or attribute; extending generation to the open-world label space with feasibility filtering could further shrink the search space.","The failures the paper reports (general labels penalized toward specific ones, e.g., Sliced-Fruit -> Sliced-Apple) suggest the adaptive threshold systematically punishes high-frequency generic labels; a frequency-aware margin could counteract this.","The composition visual mixer is a simple additive residual on top of CLIP features; plugging the same UAO+LAO losses into stronger visual representations should compound the gains if the mechanism is the claimed modeling advantage."],"forward_implications":["CZSL models built on frozen CLIP can be improved without fine-tuning the backbone, by changing the order and context of primitive prediction.","Hard negatives derived from the model's own confusions are more useful than random or composition-branch confusions for learning attribute-object linkages.","The adaptive margin lets the model start with easy distinctions and tighten as training proceeds, which the ablations show beats fixed or max-difference thresholds.","On datasets with highly similar primitives (UT-Zappos), the contrastive module contributes most; on datasets with abstract overlapping categories (MIT-States), sequential object-first prediction with object hints contributes most."],"supporting_citations":[{"why":"Supplies the frozen CLIP image and text encoders that both UAO and LAO build on.","marker":"(Radford et al. 2021)"},{"why":"CSP is the structured-prompt baseline that first brought CLIP prompting to CZSL and is a direct comparison point.","marker":"(Nayak, Yu, and Bach 2023)"},{"why":"HPL is a multi-branch prompt-learning baseline whose attribute/object/composition design ULAO extends and outperforms.","marker":"(Wang et al. 2023a)"},{"why":"CDSCZSL is the strongest prior baseline and the source of implementation practices such as frozen-CLIP training.","marker":"(Li et al. 2024)"},{"why":"Troika is a competing multi-path method and the source of the finding that fine-tuning CLIP degrades CZSL performance.","marker":"(Huang et al. 2024)"},{"why":"DFSP contributes the fully learnable composition prompt format that the LAO branch adopts.","marker":"(Lu et al. 2023)"},{"why":"SCEN represents the random-negative contrastive learning strategy that LAO's hard-negative generation is designed to replace.","marker":"(Li et al. 2022)"},{"why":"Provides evidence that CLIP behaves like a bag of words, motivating the primitive-understanding and linkage goals of ULAO.","marker":"(Yuksekgonul et al. 2022)"}],"fun_headline_variants":["Object first, then attribute: a winning CZSL order","ULAO splits CZSL: object cues plus adaptive contrast","Adaptive hard negatives lift zero-shot composition scores","Sequential object-to-attribute beats prior CZSL on three benchmarks","Contextual object hints sharpen attribute-object linkage in CZSL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline stands on the assumption that the model's object-first predictions are accurate enough to serve as trustworthy contextual cues; the paper itself notes that 'Transitioning to UAO can lead to errors if the object cues are incorrect'.","fun_headline_variants_meta":{"raw":{"variants":["Object first, then attribute: a winning CZSL order","ULAO splits CZSL: object cues plus adaptive contrast","Adaptive hard negatives lift zero-shot composition scores","Sequential object-to-attribute beats prior CZSL on three benchmarks","Contextual object hints sharpen attribute-object linkage in CZSL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000191,"raw_usage":{"total_tokens":1302,"prompt_tokens":860,"completion_tokens":442,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":476,"completion_tokens_details":{"reasoning_tokens":357}},"tokens_in":476,"tokens_out":442,"duration_ms":5458,"temperature":1.0,"reasoning_tokens":357,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:04:14.659429+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of images in which object recognition is artificially corrupted—crop out or mask the object regions, or replace the top-1 object with a wrong but plausible object prompt—and measure whether UAO's attribute accuracy drops more than that of a joint feature baseline; if attribute accuracy is robust to wrong object cues, the contextual-hint mechanism is not doing the claimed work. Alternatively, train UAO with k=0 (no object hints) and check whether the reported HM/AUC gap on MIT-States persists.","supporting_citations":[{"cited_title":"V.; Yu, P.; and Bach, S","cited_arxiv_id":null,"evidence_quote":"CSP is the structured-prompt baseline that first brought CLIP prompting to CZSL and is a direct comparison point."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Troika is a competing multi-path method and the source of the finding that fine-tuning CLIP degrades CZSL performance."}],"review_version":1}