{"id":"f40b08a2-8c73-4fed-9773-3adc8fe44946","arxiv_id":"2411.10063","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"PLAN adapts CLIP in federated learning by synchronizing local prompts with a global reference and then selectively aggregating them, reaching the highest average accuracy on PACS, OfficeHome, VLCS, and DomainNet among 15 baselines.","lead":"This paper presents PLAN, a federated learning method that lets separate data owners adapt a shared vision-language model to new tasks by exchanging small trainable prompt vectors instead of raw images or model weights. In tests on four benchmark datasets, PLAN beats 15 existing methods while sending far less data between server and clients.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim is fragile: all margins over the strongest baseline FedMaPLe are at most 1.52 points, no error bars are reported, and the privacy-preservation claim is asserted without testing despite broadcasting all local prompts to every client.","rationale":"The paper is a competent, clearly written extension of prompt learning to federated domain generalization, and the released code is a useful artifact. The central claim is empirical: PLAN is the best method on four FedDG benchmarks. For that claim to hold, the differences against the strongest baseline must be real rather than noise. The margins over FedMaPLe are small, and the paper provides only three-run averages without spread. Under typical run-to-run variation in prompt learning, a 0.45-point edge on PACS and a 1.42-point edge on VLCS are not reassuring. The ablation study is internally consistent and shows the proposed components help, but it does not establish statistical significance. I do not see an internal contradiction in the method itself; the issue is that the evidence does not yet support the strength of the stated claim. The reader's weakest assumption about the KL reference in Eq. (9) is real but secondary: the ablation quantifies a positive effect on the tested benchmarks, so the risk is about untested distributional regimes rather than the current results. The privacy claim is also asserted rather than demonstrated, and the protocol's choice to send all local prompts to all clients makes it a concrete vulnerability worth checking. The accuracy claim is the one most directly load-bearing for 'SOTA', so the recommended verdict remains CONDITIONAL: require per-seed statistics and, ideally, a prompt-inversion or membership-inference check before the SOTA and 'secure bridge' claims are accepted.","tokens_in":19344,"tokens_out":8525,"duration_ms":103367,"concrete_test":"Ask the authors to release per-seed results for PLAN and FedMaPLe under identical hyperparameter search budgets on all four datasets, then compute paired differences with 95% confidence intervals or a permutation test. If any confidence interval for the paired difference includes zero, especially on PACS or VLCS, the claim that PLAN is the highest-accuracy method is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline result is the averaged accuracy advantage over FedMaPLe: +0.45 on PACS, +1.52 on OfficeHome, +1.42 on VLCS, +1.38 on DomainNet (Tables II-V). The paper reports only three-run averages, with no standard deviations, per-run values, or significance tests. With three seeds, differences near one point are within typical run-to-run variation for prompt tuning, so the SOTA ranking is not established. This is load-bearing because the paper's contribution is precisely that reference-based prompt learning and attention aggregation beat simple averaging; the ablation differences are also small, e.g., removing both aggregators costs 1.11 points on OfficeHome. In addition, the 'more secure bridge' motivation is a central part of the claimed contribution, yet Section IV-C asserts that exchanging local prompts 'does not introduce the risk of data leakage' solely because prompts are learned rather than sampled. The protocol actually broadcasts every client's optimized prompt tokens to all other clients during aggregation, so a malicious participant receives K-1 other clients' prompts. Prompt tokens are optimized on private images and can encode class- and domain-specific information; prompt inversion is a known risk. Neither claim is tested. The accuracy issue directly affects 'new state-of-the-art performance'; the privacy issue affects whether the method satisfies the privacy-preserving FedDG setting it is designed for.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PLAN, a federated domain generalization (FedDG) method that adapts a frozen CLIP model by learning and exchanging only prompt tokens together with lightweight attention-based aggregators. In each federated round, clients first learn local text and visual prompts using a cross-entropy loss plus a KL-divergence regularizer that aligns local predictions with the global prompts from the previous round; then all local prompts are broadcast to all clients, where attention-based aggregators are trained to form global prompts. The method is evaluated on PACS, OfficeHome, VLCS, and DomainNet against CL-, FL-, FedDG-, and PEFT-based baselines, with ablations, few-shot experiments, cost analyses, and visualizations. The central claims are that PLAN achieves state-of-the-art FedDG accuracy while being communication- and computation-efficient, and that sharing learned prompts is a more secure bridge than sharing spectrum information, prototypes, or style statistics.","tokens_in":19684,"tokens_out":3355,"duration_ms":39710,"significance":"If the empirical claims are substantiated, PLAN would be a useful contribution: it is a conceptually simple, parameter-efficient way to perform FedDG with pre-trained VLMs, and the combination of reference-based synchronization with attention-based aggregation is a reasonable design. The paper explicitly releases code, reports extensive comparisons across four benchmarks, and provides ablations and cost analyses, which are all strengths. However, the headline 'new state-of-the-art' claim currently rests on small average-accuracy margins with no reported variance, and the privacy-preservation motivation is asserted without any security analysis. Both issues are load-bearing for the paper's stated contributions, so they need to be addressed before the claims can be accepted at face value.","major_comments":[{"comment":"The central claim that PLAN achieves state-of-the-art performance is not statistically supported. The tables report only the average of three runs, with no standard deviations, per-run values, or significance tests. The margins over the strongest PEFT baseline FedMaPLe are small: +0.45 on PACS, +1.52 on OfficeHome, +1.42 on VLCS, and +1.38 on DomainNet. With three seeds and prompt-tuning hyperparameters, these differences are within typical run-to-run variation. The ablation margins in Table VI are of the same order (e.g., removing both aggregators costs 1.11 points on OfficeHome), so the component-wise conclusions are also affected. Please report standard deviations or confidence intervals and, if possible, add paired significance tests against FedMaPLe and the best ablated variant.","section":"V-D, Tables II–V"},{"comment":"The privacy claim is asserted without support and, as written, appears overstated. Section IV-C states that exchanging local prompts between clients 'does not introduce the risk of data leakage,' but Algorithm 1 sends the full set of local prompts {T^k} and {V^k} to every client, and each client's prompts are optimized on private images. Learned prompts can encode class- and domain-specific information, and prompt-inversion or membership-inference attacks are a recognized risk. To justify the 'more secure bridge' motivation that appears in the abstract and introduction, the paper needs either a threat model and empirical privacy evaluation (e.g., prompt-inversion or membership attacks) or a substantially more cautious phrasing that compares the method's exposure to raw data sharing without claiming the absence of leakage risk.","section":"IV-C and Algorithm 1"},{"comment":"The method's core regularizer assumes that the previous round's global prompts form a trustworthy common reference for all clients, but the paper does not analyze when this assumption holds. If early-round global prompts are dominated by one or a few domains, or if client data is heavily imbalanced, the KL term could pull local prompts away from useful domain-specific features. The existing ablation (Table VII) varies only the weight alpha; it does not test sensitivity to reference quality, client imbalance, or the number of early rounds. A concrete experiment on an imbalanced or skewed client partition, or a comparison against using a fixed reference, would clarify the conditions under which the reference-based mechanism helps rather than hurts.","section":"IV-B, Eq. (9)"}],"minor_comments":[{"comment":"There is a typo in the introduction: 'mutiple' should be 'multiple'.","section":"I"},{"comment":"The first round is special-cased in the text (using CLIP's zero-shot prediction distribution as the KL reference), but Algorithm 1 does not show this special case; please make the pseudocode consistent so that the initialization T^g and V^g for round 1 is explicit.","section":"IV-B and Algorithm 1"},{"comment":"Reference [38] (the KL-divergence citation) lists only the article title and journal, without author names or the year; please provide the full citation.","section":"References"},{"comment":"The cost analysis is useful, but the figures do not clearly state the unit of 'GPU time' or the exact protocol for measuring communication size; please add the units and a sentence describing whether the reported megabytes include both downlink and uplink transfers.","section":"V-G, Fig. 4"},{"comment":"On DomainNet, only PEFT-based methods are compared because reproducing all models is 'prohibitively expensive'; the text should state this limitation in the main results discussion, since the claim 'only method to exceed 70%' is relative to that subset.","section":"V-D, Table V"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the mechanics are genuinely new in combination: each round has local prompt learning pulled toward the previous global prompt via a KL term, followed by attention-weighted aggregation of all clients' prompts. That two-stage loop is not in the PromptFL, FedAPT, FedPR, or FedMaPLe baselines, and the ablations show both pieces matter (removing both aggregators costs ~1.1 points on OfficeHome, ~1.4 on VLCS; dropping the KL term costs ~0.9 points on OfficeHome). Second, the empirical case for \"state of the art\" is thinner than the tables suggest: the margins over FedMaPLe are 0.45 to 1.52 points across the four benchmarks, and the paper gives only three-run averages with no standard deviations, per-run values, or significance tests. With prompt tuning, those gaps are within normal run-to-run noise.\n\nWhat the paper does well: it is clearly written, the problem framing is sensible, the cost analysis is useful (especially the convergence-speed argument against FedMaPLe), and the code is released. The few-shot experiments and hyperparameter sweeps are consistent with the main story. The KL-alignment trick is a simple, plausible way to reduce client drift without sharing data, and the attention aggregator is lightweight and easy to reproduce.\n\nSoft spots, in proportion: the privacy claim is the weakest part. The paper says exchanging local prompts \"does not introduce the risk of data leakage\" and calls prompts \"a more secure bridge,\" but the protocol broadcasts every client's optimized prompt tokens to all other clients. Prompts are trained on private images and can encode class- and domain-specific information; the paper offers no attack, inversion test, or differential-privacy analysis. That is an assertion, not a result, and the motivation section leans on it heavily. A second, lesser concern: the KL regularizer assumes the global prompt from the previous round is a trustworthy reference for every client. If early-round global prompts are dominated by a few domains, the regularizer could pull local prompts away from useful domain-specific features. The ablations do not probe regimes where that happens. The baseline tuning is described only vaguely (\"following settings in original papers or validation results\"), which matters when the claimed wins are small.\n\nWho is this for: researchers working on prompt-based federated learning or FedDG who want a strong, reproducible baseline and a reasonable design to compare against. The contribution is incremental, not a breakthrough, but it is a legitimate step.\n\nMy recommendation: send it to peer review with the expectation of heavy revision. The reviewer should demand error bars or significance tests, a rewritten privacy discussion with at least a basic leakage evaluation, and more transparency on baseline hyperparameters. With those fixes the paper would be a solid acceptance. As it stands, the central ranking claim is not established, but the method and the writing are worth engaging with.","headline":"A competent prompt-learning FedDG paper whose SOTA claim needs error bars and whose privacy motivation needs proof.","tokens_in":20190,"tokens_out":1392,"would_cite":true,"duration_ms":16653,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PLAN shows that learned prompts, synchronized to a global reference and attention-aggregated, achieve top accuracy in federated domain generalization on four benchmarks.","keywords":["federated domain generalization","prompt learning","vision-language models","CLIP","attention aggregation","federated learning","domain generalization","parameter-efficient fine-tuning"],"falsifier":"Run PLAN on a federation where the previous-round global prompts are deliberately corrupted or generated from a single dominant domain; if accuracy on the held-out target domain does not fall below a variant without the KL term, then the reference-synchronization mechanism is not responsible for the reported gains. A complementary test is a membership-inference attack against the shared prompts: if client samples can be reconstructed or re-identified from the prompt tokens, the paper's 'more secure bridge' claim is not supported by evidence.","tokens_in":19168,"feed_emoji":"🧩","tokens_out":9065,"duration_ms":87007,"temperature":0.7,"pith_summary":"The paper sets out to show that in federated domain generalization, the safest and most effective way to transfer knowledge between isolated clients is through learned prompt tokens rather than through sample-derived statistics such as spectra, prototypes, or style moments. It proposes PLAN, which freezes a pre-trained CLIP model and trains only small text and visual prompts on each client, adding a KL-divergence term that pulls local prompts toward the global prompts from the previous round so that clients are indirectly synchronized. A lightweight attention aggregator then weighs each client's prompts and combines them into new global prompts. On PACS, OfficeHome, VLCS, and DomainNet, the reported leave-one-domain-out accuracy is the highest among the 15 methods compared, while the communication cost per round is far below full-model federated learning. If these results hold, prompt-only adaptation with a common-reference regularizer is a better privacy-conscious transfer mechanism for FedDG than sharing data-derived knowledge.","feed_headline":"Prompts alone outdo 15 baselines in federated domain generalization","feed_subtitle":"Aligning client prompts to one global reference and attention-aggregating them tops PACS, OfficeHome, VLCS, DomainNet.","key_machinery":"The load-bearing objects are the learnable prompt tokens, namely text prompts $T^g_l$ and visual prompts $V^g_l$ inserted into each Transformer block of CLIP's frozen encoders, together with two lightweight attention-based aggregators $A_t$ and $A_v$ that map a set of local prompts to a global prompt via a learnable query and bottleneck MLPs (Eqs. (12)--(13)). The regularizer that carries the method is the KL divergence in Eq. (9), which makes each client's predictions mimic those of the previous round's global prompts; in the first round, CLIP's zero-shot prediction distribution plays that role. These pieces jointly turn distributed private data into a synchronized set of small vectors whose selective average defines the model used on unseen target domains.","core_discovery":"On its own terms, the paper's discovery is that a pre-trained vision-language model can be adapted for federated domain generalization by exchanging only prompts, provided the local prompts are pulled toward a shared reference and then selectively merged. PLAN inserts learnable text and visual prompt tokens into every block of CLIP's encoders; each client trains its prompts on local data with a cross-entropy loss plus a KL loss (Eq. (9)) that aligns its prediction distribution to that of the previous round's global prompts. The server then broadcasts all local prompts to all clients, each client optimizes a pair of lightweight attention aggregators that recombine the prompts, and the averaged aggregators produce the new global prompts. The reported results are 97.40% average accuracy on PACS, 86.73% on OfficeHome, 85.29% on VLCS, and 71.05% on DomainNet, exceeding the compared FL, DG, FedDG, and prompt-learning baselines while training only prompt tokens and small aggregators. The paper further claims that this design lowers communication cost by up to 15.29 times relative to FedAvg with ViT-Base/16 and converges faster than the FedMaPLe baseline, and that learned prompts are a more secure bridge than sharing information extracted directly from local samples.","pith_inferences":["Beyond the paper, the KL-alignment step is effectively a distillation from the global prompt to local prompts, so its benefit should depend on the quality of that teacher; an extension would adapt the alignment weight $\\alpha$ per round or per client based on how well the global prompts predict each client's validation set.","The privacy argument is comparative rather than formal; a natural follow-up is to run membership-inference or gradient-inversion attacks on shared prompts to see whether any residual sample information survives, since even learned prompts may encode trends of local data.","Since the aggregation stage broadcasts every client's prompts to every client, the scheme assumes that such exposure is acceptable; under stricter privacy or bandwidth limits, one could test partial visibility or secure aggregation of prompts within the same attention framework.","One could transfer the same reference-based synchronization idea to other parameter-efficient modules, such as adapters or LoRA parameters, replacing prompts and testing whether a common reference plus attention aggregation is the general mechanism driving the gains."],"forward_implications":["Federated domain generalization can be achieved by exchanging only prompt tokens and aggregator parameters, shrinking per-round communication to about 5.62 MB on OfficeHome versus 85.94 MB for FedAvg with ViT-Base/16.","Aligning local prompts to a common global reference is necessary for the gains: removing the KL loss drops average accuracy by 0.85, 0.92, and 0.24 percentage points on PACS, OfficeHome, and VLCS respectively.","Selective aggregation matters: replacing either the text or visual attention aggregator with simple averaging reduces performance, and removing both causes declines of 1.11 and 1.36 points on OfficeHome and VLCS.","Because PLAN surpasses FedMaPLe's best accuracy after a single round, its total training compute and communication cost can be lower than that of a per-round-cheaper prompt baseline.","In few-shot settings with 2 to 16 samples per class, PLAN outperforms both PromptFL and FedMaPLe on PACS and OfficeHome, with the exception of the one-shot case."],"supporting_citations":[{"why":"Supplies the frozen pre-trained vision-language model whose text and image encoders PLAN adapts through prompts.","marker":"[13]"},{"why":"Provides the multi-modal prompt design of text and visual prompts at every block that PLAN builds on and the FedMaPLe baseline it compares against.","marker":"[16]"},{"why":"The federated prompt-learning baseline that averages local prompts with fixed weights, which PLAN contrasts with its reference-based synchronization and attention aggregation.","marker":"[17]"},{"why":"A federated visual-prompt baseline that constrains updates to the null space of the global prompt, used as a comparison method.","marker":"[18]"},{"why":"The frequency-spectrum sharing FedDG method, representing the data-derived knowledge transfer that PLAN argues is less secure.","marker":"[9]"},{"why":"The style-transfer FedDG baseline that shares channel-wise statistics, another data-derived mechanism PLAN replaces with prompts.","marker":"[11]"},{"why":"Defines the PACS benchmark dataset used in the main comparison.","marker":"[41]"},{"why":"Defines the OfficeHome benchmark dataset used in the main comparison and hyperparameter analysis.","marker":"[42]"},{"why":"Defines the VLCS benchmark dataset used in the main comparison.","marker":"[43]"},{"why":"Defines the DomainNet benchmark dataset, the largest-scale evaluation of PLAN's generalization claim.","marker":"[44]"}],"fun_headline_variants":["Prompts alone outdo 15 baselines in federated domain generalization","Exchanging only prompts, not data, wins federated DG","PLAN: selective prompt aggregation tops federated DG baselines","Attention-aggregated prompts outperform 15 FedDG baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the previous round's global prompts form a trustworthy common reference for every client; if those prompts are biased toward a few dominant domains or are poor in early rounds, the KL regularizer could pull local prompts away from their useful domain-specific features, and the method also assumes source and unseen domains share the same category space.","fun_headline_variants_meta":{"raw":{"variants":["Prompts alone outdo 15 baselines in federated domain generalization","Exchanging only prompts, not data, wins federated DG","PLAN: selective prompt aggregation tops federated DG baselines","Attention-aggregated prompts outperform 15 FedDG baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000317,"raw_usage":{"total_tokens":1859,"prompt_tokens":1076,"completion_tokens":783,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":692,"completion_tokens_details":{"reasoning_tokens":709}},"tokens_in":692,"tokens_out":783,"duration_ms":8126,"temperature":1.0,"reasoning_tokens":709,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:00:31.473678+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run PLAN on a federation where the previous-round global prompts are deliberately corrupted or generated from a single dominant domain; if accuracy on the held-out target domain does not fall below a variant without the KL term, then the reference-synchronization mechanism is not responsible for the reported gains. A complementary test is a membership-inference attack against the shared prompts: if client samples can be reconstructed or re-identified from the prompt tokens, the paper's 'more secure bridge' claim is not supported by evidence.","supporting_citations":[{"cited_title":"Promptfl: Let federated participants cooperatively learn prompts instead of models-federated learning in age of foundation model,","cited_arxiv_id":null,"evidence_quote":"The federated prompt-learning baseline that averages local prompts with fixed weights, which PLAN contrasts with its reference-based synchronization and attention aggregation."},{"cited_title":"Learning federated visual prompt in null space for mri reconstruction,","cited_arxiv_id":null,"evidence_quote":"A federated visual-prompt baseline that constrains updates to the null space of the global prompt, used as a comparison method."},{"cited_title":"Feddg: Federated do- main generalization on medical image segmentation via episodic learn- ing in continuous frequency space,","cited_arxiv_id":null,"evidence_quote":"The frequency-spectrum sharing FedDG method, representing the data-derived knowledge transfer that PLAN argues is less secure."},{"cited_title":"Federated domain generaliza- tion for image recognition via cross-client style transfer,","cited_arxiv_id":null,"evidence_quote":"The style-transfer FedDG baseline that shares channel-wise statistics, another data-derived mechanism PLAN replaces with prompts."}],"review_version":1}