{"id":"259028e1-2194-48ff-b6c4-515c55214de2","arxiv_id":"2412.11484","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Contrastively learned visual prompts, combined through guided attention, improve zero-shot visual domain adaptation of embodied RL policies.","lead":"The paper introduces CONPE, a method that adds trainable visual prompts to a pretrained CLIP model to help reinforcement learning agents adapt to new visual conditions without retraining. It reports higher zero-shot success in navigation, manipulation, and driving simulators than several existing baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Data confound: CONPE's prompt learning uses extra expert demonstrations, but baselines are not shown to receive the same data, so the SOTA comparison is not yet established.","rationale":"I read the paper as claiming that the prompt ensemble, not the data or the CLIP backbone, is what enables zero-shot policy adaptation. The strongest evidence for this is Table 1, but the comparison is confounded: CONPE's prompt learning consumes expert demonstrations that the baselines are not described as receiving. This is not an internal inconsistency, and the paper's Section 6 limitation about unseen factors is honest; rather, it is an unaddressed gap in the evaluation protocol. If the authors can show a control where a non-prompted CLIP encoder is given the same expert data and still underperforms, the concern is settled. The reader's weakest-assumption about predefined domain factors is real but secondary, and the paper explicitly acknowledges that scope limitation. Because the verifiability of the SOTA comparison is central to acceptance, the conditional verdict stands: accept only with code/data and the control condition. Unless the control fails, I would not move to reject.","tokens_in":24774,"tokens_out":10035,"duration_ms":91389,"concrete_test":"Re-run the AI2THOR ObjectNav evaluation in Table 1(a) with an additional control condition: take the same frozen CLIP ViT-B/32 encoder used by EmbCLIP, fine-tune it with the same contrastive losses on the same 28,464 expert demonstration samples used for CONPE's prompt learning, then train the same PPO policy on the same 4 source domains and evaluate on the same 30 seen and 10 unseen target domains. If this 'CLIP+data, no prompts' control matches or beats CONPE's 79.7% unseen-target success rate, the 20.7% gain over EmbCLIP is not attributable to the prompt ensemble.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing uncertainty is whether the reported superiority over EmbCLIP and the other baselines is caused by the prompt ensemble or by the extra expert demonstration data used for prompt-based contrastive learning. Section 4 says CONPE learns visual prompts on expert demonstrations (10 episodes per domain factor; the AI2THOR dataset has 28,464 samples, Table 17), while EmbCLIP is a frozen-CLIP baseline and Appendix E.2 does not state that LUSR, CURL, ATC, ACO, or EmbCLIP are given these demonstrations during representation learning. Table 1(a) attributes the 20.7% unseen-domain gain over EmbCLIP to CONPE, but the protocol does not control for the additional data. The internal ablations (Tables 3-4) show the ensemble mechanism helps relative to simple averaging, but they do not establish superiority over a CLIP encoder fine-tuned on the same demonstrations without prompts. If a simple 'CLIP plus same contrastive data' baseline reaches CONPE's numbers, the central claim is unsupported; if it does not, the concern is resolved.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CONPE, a framework for zero-shot policy adaptation in embodied RL that combines a frozen CLIP visual encoder with a set of contrastively learned visual prompts, each associated with a factor that affects egocentric observation (e.g., camera position, stride length, illumination). A guided-attention ensemble weights the prompted embeddings via cosine-similarity guidance, and the resulting state representation is jointly optimized with the policy. Experiments in AI2THOR navigation, egocentric-Metaworld manipulation, and CARLA driving report state-of-the-art zero-shot performance on unseen target domains, improved sample efficiency, and an extension that adapts a pretrained policy by updating only the attention module. The appendix provides implementation details, hyperparameters, dataset descriptions, and additional ablations.","tokens_in":25028,"tokens_out":4999,"duration_ms":43974,"significance":"If the central claim is established, the contribution is valuable: a parameter-efficient way to adapt a large vision-language model to embodied RL without fine-tuning the backbone, with a clean ensemble mechanism and evidence of interpretability. The paper is thorough in scope (three environments, multiple tasks, many ablations) and provides detailed appendices on hyperparameters and datasets. The internal ablations (Tables 3-4) convincingly show that the guided-attention ensemble outperforms uniform averaging and composition schemes. The main caveat is that the headline comparison to state-of-the-art baselines is currently confounded by unequal data access, so the claimed superiority is not yet established.","major_comments":[{"comment":"The comparison with baselines in Table 1 is confounded by unequal access to expert demonstrations. CONPE's prompt-based contrastive learning uses a dedicated expert dataset (10 episodes per domain factor; 28,464 samples for AI2THOR per Table 17; 3,840 for Metaworld; 7,394 for CARLA), while the descriptions of LUSR, CURL, ATC, ACO, and EmbCLIP in Appendix E.2-E.4 do not indicate that any of these baselines receive the same demonstrations. In particular, EmbCLIP is a frozen-CLIP baseline, so it cannot benefit from this data. The 20.7% unseen-domain gain over EmbCLIP in Table 1(a) could therefore be caused by the extra expert data rather than by the prompt ensemble mechanism. The authors should either provide the same expert demonstrations to all baselines (e.g., by fine-tuning a CLIP encoder with the same contrastive objectives on the same data and using it in EmbCLIP or as a representation for the other baselines), or include an ablation that trains the CONPE encoder on the same contrastive data without the prompt ensemble. Without such a control, the central claim of state-of-the-art zero-shot performance is not supported.","section":"Section 4, Appendix E.2"},{"comment":"The CARLA results are statistically fragile. With only three seeds and no significance tests, the reported differences often fall within the baselines' large standard deviations; for example, Map 1 seen-target performance is 1732.2±588.6 for EmbCLIP vs. 1738.0±163.5 for CONPE, and Map 2 seen-target is 2139.1±655.9 for EmbCLIP vs. 2422.5±79.6 for CONPE. The unseen-target improvements are more sizable, but the variance overlap across methods makes it unclear whether the ranking is stable. The authors should report confidence intervals, run a significance test (e.g., paired bootstrap or a nonparametric test over seeds and domains), or provide additional seeds for the CARLA experiments. This matters because the paper claims superior performance in all three environments, not only in AI2THOR.","section":"Section 4.1, Table 1(c)"}],"minor_comments":[{"comment":"The citation key '[clr]' in the sentence 'As in [clr], for latent vectors x, y...' is undefined; this should be replaced with the intended reference (likely SimCLR, [51]).","section":"Equation (3)"},{"comment":"The headers contain typos: 'Souce' and 'Scoure' should be 'Source'.","section":"Table 2 and Table 15"},{"comment":"The table caption has a typo: 'Tabel' should be 'Table'.","section":"Table 4"},{"comment":"The term 'zero-shot' is applied to seen target domains, but the prompt-based contrastive learning step has already consumed data from those domains; the paper should consistently distinguish zero-shot at the policy level from domain generalization to entirely unseen domains, and the abstract should be qualified accordingly.","section":"Abstract and Section 4"},{"comment":"The abstract reports sample-efficiency numbers for ATC ('less than 50.0% and 16.7%'), but Section 4.1 only quantifies the comparison to EmbCLIP; please clarify how the ATC numbers are derived from Figure 4 or the appendix.","section":"Section 4.1 and Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper appears to be a NeurIPS 2023 submission posted to arXiv in 2024; the header still contains the NeurIPS 2023 footnote. The data-confound issue is the most serious concern and should be the primary focus of the revision. If the authors can add a controlled comparison where baselines receive the same expert demonstrations, the paper would be much stronger; without it, the SOTA claims are not convincing. The appendix is extensive and generally helpful, but the inability to verify the exact experimental protocol (e.g., no code link, no dataset URL in the text) adds to the uncertainty."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Good to know: this is a real attempt to use per-domain-factor contrastive visual prompts plus a guided-attention ensemble for zero-shot RL adaptation in embodied agents. That specific combination is new as far as I can tell from the citations. The internal ablations (Table 4) show the guided-attention ensemble beats uniform/weighted averaging of the prompted embeddings by 3.5 and 14.2 points on seen/unseen targets, which is evidence the mechanism does something, not just the prompt count. The dataset release is a plus, and the limitation section honestly notes that adaptation is constrained to the enumerated domain factors.\n\nThe soft spots are real though. The most serious is the data confound: CONPE learns its prompts on expert demonstrations (10 episodes per domain factor; roughly 28k samples in AI2THOR), but the baselines, especially EmbCLIP with its frozen CLIP encoder, are not given that data. So the 20.7% unseen-domain win over EmbCLIP might just be the extra demonstrations. The paper needs the obvious control: a CLIP encoder fine-tuned on the same contrastive data without prompts, or baselines trained with the same expert demos. The stress-test note is correct on this. If that control matches CONPE, the central claim collapses; if it does not, the concern is resolved.\n\nOther issues, in proportion: three seeds with no significance tests, and CARLA variances are large enough that some numbers are not separable (e.g., ATC 1684±368 vs CONPE 1738±163 on Map1 seen). The sample-efficiency claims are inconsistent between the intro (50%/16.7% vs ATC, 60%/50% vs EmbCLIP) and Section 4.1 (60% seen, 50% unseen vs EmbCLIP). There is a dangling citation [clr] in Section 3.2 that never resolves. No code release. The loss equation (3) looks like a valid contrastive form, but the denominator excluding the positive is unusual enough that I would want it checked.\n\nThe paper is for people working on prompt tuning or domain generalization for embodied RL. The idea deserves a serious referee; the evaluation needs the missing control and more seeds before the SOTA claim holds. I would accept it for review but expect heavy revision.","headline":"The prompt-ensemble idea is worth a look, but the headline zero-shot results are confounded by the extra expert data used for prompt learning, and the paper omits the control that would settle it.","tokens_in":25510,"tokens_out":2798,"would_cite":false,"duration_ms":26281,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CONPE learns one contrastive visual prompt per domain factor and mixes the resulting CLIP embeddings with cosine-guidance attention, yielding zero-shot policy adaptation that outperforms prior state-of-the-art across navigation…","keywords":["zero-shot adaptation","embodied reinforcement learning","visual prompt ensemble","contrastive learning","CLIP vision-language model","domain generalization","policy adaptation","attention mechanism"],"falsifier":"Build a target domain that changes only a visual property outside the enumerated prompt pool, for example lens distortion or snow accumulation, while holding all enumerated factors at their source values; if a CONPE policy's zero-shot success drops to roughly the level of a plain CLIP baseline on that domain, then the claimed generalization is limited to the factor list and the central claim fails.","tokens_in":24613,"feed_emoji":"🤖","tokens_out":6319,"duration_ms":52382,"temperature":0.7,"pith_summary":"The paper claims that an embodied reinforcement learning agent can adapt to unseen visual domains without any retraining if the visual encoder is augmented with a pool of learned prompts, one per domain factor such as camera position, stride length, or illumination. Each prompt is trained with a contrastive loss on a few expert demonstrations so that it pulls together observations that share behavior while separating observations across domains. During policy learning, the prompted CLIP embeddings are combined by an attention module whose weights are steered by cosine similarity between the image and each prompted embedding, giving a state representation that is both domain-invariant and task-specific. The authors show this yields higher zero-shot success than prior methods in AI2THOR navigation, egocentric-Metaworld manipulation, and CARLA driving, and it needs fewer environment samples to reach comparable performance.","feed_headline":"Zero-shot robot adaptation via contrastive prompt ensemble","feed_subtitle":"Per-domain-factor prompts plus guided attention keep CLIP-based policies effective in unseen environments.","key_machinery":"A visual prompt is a short sequence of learnable vectors prepended to the patch embeddings of the CLIP vision encoder; the paper learns one prompt per domain factor with a contrastive loss (Equation 3) on expert demonstrations, forming a prompt pool. The ensemble module G computes, for each observation, prompted embeddings $z_i = T_\\phi(o, p_i^v)$, a guidance score $g_i$ equal to the cosine similarity between the base embedding $z_0$ and $z_i$, and softmax attention weights $\\omega_i$; the final state is $Z = z_0 + \\sum_i \\omega_i z_i$. This guided attention is the load-bearing mechanism: it lets the policy downweight prompts irrelevant to the current observation and upweight the one whose domain factor is active, which is what makes the representation adapt zero-shot.","core_discovery":"CONPE establishes that zero-shot policy adaptation to visual domain shifts can be reduced to learning a small set of visual prompts, each encoding invariance to one named domain factor, and then letting an attention module weight those prompted embeddings for the task at hand. On the paper's own terms, the central discovery is that prompt-based contrastive learning plus cosine-guidance attention makes the CLIP embedding space reusable across unseen combinations of known visual factors, so policies trained in a few source domains keep working in target domains without adaptation. The strongest reported evidence is a 20.7 percentage point gain over EmbCLIP on unseen AI2THOR object-navigation domains, with similar gains in manipulation and driving.","pith_inferences":["If the learned prompts are approximately orthogonal in CLIP space, the attention weights can be read as a soft estimate of which domain factors are active in the current observation, which would make CONPE usable for domain-factor identification and calibration rather than only policy execution; the paper does not claim this explicitly.","Because each prompt is trained from only 10 expert episodes per factor, a practical extension the paper does not test is building prompt pools from short user-collected videos instead of rule-based expert demonstrations, which could make the method applicable to real robots where hand-designed experts are unavailable.","The enumerated-factor assumption implies a sharp testable boundary: if deployment introduces a shift outside the prompt pool, such as lens distortion or sensor noise, zero-shot performance should degrade; the paper's own limitation statement says adaptability is constrained when domain changes extend beyond the considered factors.","The semantic-regularization results hint that language descriptions could eventually replace the manually enumerated domain factors, letting the prompt pool be defined by text rather than by hand; the paper lists this as future work."],"forward_implications":["Zero-shot transfer: policies trained with CONPE in four source domains maintain high success in 30 seen and 10 unseen target domains, with the largest reported gap over the strongest baseline being 20.7 percentage points in AI2THOR object navigation.","Sample efficiency: to reach comparable zero-shot success, CONPE uses less than 60% of the timesteps EmbCLIP needs on seen target domains and 50% on unseen target domains in the same task.","Pretrained-policy adaptation: when only the attention module and a task prompt are updated (about 10% of the training samples), zero-shot target performance improves by 3.5 to 7.0 points in AI2THOR and by 9.0 to 57.6 points in egocentric-Metaworld.","Scalability: increasing the prompt pool from 2 to 10 factors raises zero-shot success by 42.8 points on seen domains and 36.7 points on unseen domains, and performance stays stable for 10 to 18 prompts.","Semantic regularization: adding object-level text descriptions to regularize augmented prompted embeddings improves robustness to larger augmentation noise, indicating that language information can further strengthen the ensemble."],"supporting_citations":[{"why":"The EmbCLIP baseline that uses pretrained CLIP embeddings for embodied AI; CONPE is compared directly against it and beats it by up to 20.7 percentage points.","marker":"[7]"},{"why":"Supplies the decoupled representation-learning paradigm and the timestep-driven contrastive task that CONPE reuses to learn prompts.","marker":"[1]"},{"why":"Supplies the CURL contrastive unsupervised representation method for RL, used as a baseline and as the source of augmentation-driven contrastive learning.","marker":"[2]"},{"why":"Supplies the behavior-driven contrastive task using expert actions, used as one of the positive-pair schemes for prompt learning.","marker":"[12]"},{"why":"Provides the prompt-learning approach for vision-language models that CONPE adapts from text prompts to visual prompts.","marker":"[17]"},{"why":"Provides the visual prompt tuning technique of appending learnable vectors to a vision transformer, the basis of the visual prompt pool.","marker":"[16]"},{"why":"The AI2THOR simulator used for object and point navigation experiments with camera, stride, and illumination domain factors.","marker":"[9]"},{"why":"The CARLA simulator used for autonomous driving experiments with weather, time-of-day, and camera domain factors.","marker":"[10]"},{"why":"The Meta-World benchmark adapted to egocentric observations for the manipulation tasks.","marker":"[14]"},{"why":"Provides the contrastive predictive coding loss formulation that Equation (3) is based on for prompt learning.","marker":"[13]"}],"fun_headline_variants":["Contrastive prompts enable zero-shot robot adaptation","Prompt ensemble gives embodied agents instant domain shifts","Zero-shot policy adaptation via contrastive prompt ensemble","Robots adapt to unseen visuals with contrastive prompts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a short, hand-enumerated list of domain factors (camera pose, stride, illumination, weather, and a few others) together with about 10 expert episodes per factor captures every visual shift an agent will face at deployment, so that unseen domains are only unseen combinations of known factors.","fun_headline_variants_meta":{"raw":{"variants":["Contrastive prompts enable zero-shot robot adaptation","Prompt ensemble gives embodied agents instant domain shifts","Zero-shot policy adaptation via contrastive prompt ensemble","Robots adapt to unseen visuals with contrastive prompts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00016,"raw_usage":{"total_tokens":1202,"prompt_tokens":882,"completion_tokens":320,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":261}},"tokens_in":498,"tokens_out":320,"duration_ms":3427,"temperature":1.0,"reasoning_tokens":261,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:52:05.653812+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a target domain that changes only a visual property outside the enumerated prompt pool, for example lens distortion or snow accumulation, while holding all enumerated factors at their source values; if a CONPE policy's zero-shot success drops to roughly the level of a plain CLIP baseline on that domain, then the claimed generalization is limited to the factor list and the central claim fails.","supporting_citations":[{"cited_title":"Simple but Effective: CLIP embeddings for embodied AI","cited_arxiv_id":null,"evidence_quote":"The EmbCLIP baseline that uses pretrained CLIP embeddings for embodied AI; CONPE is compared directly against it and beats it by up to 20.7 percentage points."},{"cited_title":"Decoupling representation learning from reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Supplies the decoupled representation-learning paradigm and the timestep-driven contrastive task that CONPE reuses to learn prompts."},{"cited_title":"CURL: Contrastive unsupervised repre- sentations for reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Supplies the CURL contrastive unsupervised representation method for RL, used as a baseline and as the source of augmentation-driven contrastive learning."},{"cited_title":"Learning to drive by watching youtube videos: Action-conditioned contrastive policy pretraining","cited_arxiv_id":null,"evidence_quote":"Supplies the behavior-driven contrastive task using expert actions, used as one of the positive-pair schemes for prompt learning."},{"cited_title":"Learning to prompt for vision-language models","cited_arxiv_id":null,"evidence_quote":"Provides the prompt-learning approach for vision-language models that CONPE adapts from text prompts to visual prompts."},{"cited_title":"Visual prompt tuning","cited_arxiv_id":null,"evidence_quote":"Provides the visual prompt tuning technique of appending learnable vectors to a vision transformer, the basis of the visual prompt pool."},{"cited_title":"CARLA: An open urban driving simulator","cited_arxiv_id":null,"evidence_quote":"The CARLA simulator used for autonomous driving experiments with weather, time-of-day, and camera domain factors."},{"cited_title":"Meta-World: A benchmark and evaluation for multi-task and meta reinforce- ment learning","cited_arxiv_id":null,"evidence_quote":"The Meta-World benchmark adapted to egocentric observations for the manipulation tasks."}],"review_version":1}