{"id":"e27caace-e386-4f72-b126-ec3d9208ed5e","arxiv_id":"2608.04791","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"In federated remote sensing, LoRA tuning of a frozen CLIP model achieves the best accuracy-to-communication trade-off, while full fine-tuning causes severe catastrophic forgetting of pretrained knowledge.","lead":"This paper compares four ways to adapt a vision-language model inside federated learning for remote sensing image classification: full fine-tuning, image or text encoder fine-tuning, prompt learning, and LoRA. It finds that LoRA gives the best balance of accuracy and communication cost, while full fine-tuning causes severe loss of general knowledge.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never reports communication rounds or total communication; the central LoRA-efficiency claim rests on per-round parameter counts, so if LoRA needs more rounds, the 'more than two orders of magnitude' reduction may not hold.","rationale":"The reader's weakest assumption identifies the shared learning rate and missing number of communication rounds as the main weakness. I agree with that diagnosis, but I would sharpen it: the missing round count is not merely a reporting gap; it directly affects the quantitative claim of 'more than two orders of magnitude' communication reduction, because total communication depends on rounds. The per-round parameter counts in Table 2 are informative but not sufficient for a practical guideline. The paper otherwise provides a clear experimental design, publicly available code, and a plausible interpretation of the observed catastrophic forgetting, so the concern is addressable rather than fatal. The appropriate verdict remains CONDITIONAL: the central claim is plausible but should be accepted only after the authors report communication rounds, total communication, and some robustness check on learning rate and convergence.","tokens_in":11240,"tokens_out":4434,"duration_ms":55544,"concrete_test":"Re-run FFT, LoRA, and text-encoder fine-tuning on the BigEarthNet-S2 federated setup, recording the cumulative number of communication rounds needed to reach each strategy's best in-distribution micro-mAP (or a fixed convergence criterion). Compute total communicated parameters = trainable parameters per round x number of rounds. If LoRA's total communicated parameters are not at least about 100x smaller than FFT at matched performance, the 'more than two orders of magnitude' claim and recommendation #3 fail. Additionally, run a small learning-rate sweep (e.g., 1e-6, 1e-5, 1e-4) for LoRA and CoOp to verify that the Table 1 ranking is stable across reasonable hyperparameters.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central conclusion is that LoRA offers the best performance-communication trade-off because it achieves competitive in-distribution performance (79.1 micro-mAP vs. 78.3 for FFT) while communicating only 1.08M trainable parameters per round versus 427.62M for FFT. Table 2 defines communication cost as the number of trainable parameters exchanged per communication round. In federated learning, total communication cost is the product of per-round parameters and the number of communication rounds. The paper states that 'the same FL training procedure was adopted across all adaptation strategies' (Section 4.2) but never reports the number of communication rounds used for Table 1. Figure 1 mentions communication rounds C, but no values are given. If LoRA converges slower than FFT in terms of rounds, the per-round ratio of roughly 396x could shrink substantially, potentially undermining the 'more than two orders of magnitude' claim and the recommendation to prefer LoRA. A second, related issue is that all strategies are compared at a single learning rate (1e-5), three local epochs, and an unreported round count with no error bars. The in-distribution differences between LoRA and FFT are small (79.1 vs. 78.3 micro-mAP; 58.7 vs. 61.8 macro-mAP), so without evidence that both strategies are at comparable convergence and without variance estimates, the ranking may reflect configuration mismatch rather than intrinsic strategy properties. This is an addressable experimental-reporting issue, not a demonstrated internal inconsistency, but it is load-bearing because the paper's headline guideline rests on this comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a comparative study of CLIP adaptation strategies (full fine-tuning, image/text encoder fine-tuning, CoOp prompt learning, and LoRA) in federated learning for remote sensing image classification. Experiments use BigEarthNet-S2 as the federated training set, with EuroSAT, RESISC45, and ImageNet for out-of-distribution evaluation. The paper reports in-distribution and OOD accuracy, local training FLOPs/MACs, and per-round communication cost, and derives a guideline recommending LoRA as the best performance/communication trade-off, text encoder fine-tuning for cross-dataset generalization, prompt learning only under severe bandwidth constraints, and FFT with PAINT interpolation for controlled adaptation. The code is publicly available.","tokens_in":11540,"tokens_out":5383,"duration_ms":59751,"significance":"This is a timely and practically useful comparison for federated remote sensing; it appears to be the first systematic study of these VLM adaptation strategies in this setting. Strengths include the realistic non-IID partition by country, evaluation of catastrophic forgetting on a large natural image benchmark, detailed FLOPs/MACs and parameter counts, and public code. However, the headline ranking is not fully supported by the current experimental reporting: the number of communication rounds and hence total communication volume are missing, all strategies share a single learning rate without sensitivity analysis, and no variance estimates are provided. If these issues are addressed, the paper would provide a solid empirical guideline.","major_comments":[{"comment":"The paper states in Section 4.2 that 'the same FL training procedure was adopted across all adaptation strategies' but never reports the number of communication rounds used for the results in Table 1 or for the curves in Figure 1. Table 2 defines communication cost as the number of trainable parameters exchanged per round, and the conclusion in Section 6 claims that LoRA reduces communicated parameters by more than two orders of magnitude compared to FFT. In federated learning, total communication volume is per-round parameters multiplied by the number of rounds. If LoRA requires more rounds than FFT to reach the reported performance, the per-round ratio of about 396x could shrink substantially, possibly undermining the central recommendation. Please report the round count, convergence behavior, and total communication volume for each strategy, or justify that all strategies are compared at a common, converged round count.","section":"§4.2, §5.5, Table 2"},{"comment":"All strategies are trained with AdamW at a single learning rate of 1e-5, with no per-strategy learning-rate search or sensitivity analysis. CoOp, LoRA, and full fine-tuning typically operate in different optimization regimes, so the large gap between CoOp (66.5 micro-mAP) and LoRA/FFT (79.1/78.3) may reflect hyperparameter mismatch rather than intrinsic capacity differences. To support the guideline that prompt learning is only suitable under bandwidth constraints and that LoRA is the best trade-off, please provide a per-strategy hyperparameter sweep or at least a small sensitivity analysis over learning rates and show that the ranking is stable across reasonable choices.","section":"§4.2"},{"comment":"No error bars, confidence intervals, or multiple seeds are reported. The in-distribution difference between LoRA and FFT is small (micro-mAP 79.1 vs 78.3; macro-mAP 58.7 vs 61.8), and the OOD differences are also computed from single runs. Without variance estimates, the claim that LoRA achieves competitive in-distribution performance is not statistically substantiated. Please run at least three independent federated trials with different random seeds and report mean and standard deviation (or confidence intervals) for the main results.","section":"§5.1, Table 1"}],"minor_comments":[{"comment":"The number of clients is implied by the eight listed countries, but K is never explicitly stated; please report K and per-client sample counts to allow reproduction.","section":"§4.1"},{"comment":"Figure 1's caption mentions fine-tuning duration defined by E and C, but no concrete values are given; please report the E and C used for each curve and label the interpolation coefficient axis.","section":"§5.3, Figure 1"},{"comment":"The phrase 'around 14% below zero-shot' for RESISC45 should be 'approximately 14.3 percentage points' to avoid confusion between relative and absolute differences.","section":"§5.2"},{"comment":"The FLOPs/MACs table reports per-forward-pass costs; please clarify whether these counts include the backward pass and whether they are measured or computed analytically, since the no-caching rows for FFT and encoder-specific fine-tuning are identical by construction.","section":"§5.4"},{"comment":"The PAINT analysis is presented qualitatively and Figure 1 does not provide numerical results; since PAINT is listed as a recommended strategy in the conclusion, please report the quantitative trade-off (e.g., the best interpolation coefficient and the resulting accuracies on BigEarthNet-S2 and ImageNet) that supports that recommendation.","section":"§5.3, §6"},{"comment":"Reference [12] is listed as 'Hou et al.' without full author names; please ensure all references are complete and consistently formatted.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The main blocker is the missing communication-round count and total communication volume, which directly affects the central LoRA-efficiency claim. The hyperparameter and variance issues are also addressable within the manuscript's scope. I would encourage the editor to request a revision rather than a rejection; the underlying experimental design is reasonable and the code availability makes verification feasible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Briefly: this is the first systematic comparison of FFT, encoder fine-tuning, CoOp, and LoRA under federated remote sensing classification, with OOD evaluation on EuroSAT, RESISC45, and ImageNet. That alone is worth something; the RS-FL literature has been scattered, and practitioners need guidance. The headline results are plausible: LoRA gives near-FFT in-distribution accuracy at a tiny per-round parameter cost, FFT catastrophically forgets ImageNet, and text-encoder fine-tuning transfers best to EuroSAT. The code is public, and the datasets are standard.\n\nThe main issue is the load-bearing communication-efficiency claim. Table 2 reports trainable parameters per round, but the paper never states the number of communication rounds used for Table 1. If the same fixed round budget was used for all strategies, then the per-round ratio is also the total-communication ratio; if different budgets or early stopping were used, the conclusion could change. Either way, the number should be in the paper, and total communication (params × rounds) should be reported. The stress-test note highlights exactly this; I think it's the right concern, though it is addressable with one missing number.\n\nTwo smaller issues: there are no error bars or multiple seeds, so the rankings between LoRA (79.1) and FFT (78.3) micro-mAP are not statistically robust. And all strategies share a single learning rate of 1e-5, which likely disadvantages CoOp and LoRA (both usually need higher LR). That CoOp still underperforms is probably true, but the magnitude is uncertain. Also, the CoOp context length is described as 61 tokens, which is odd and inconsistent with the '5 learnable context vectors' mentioned in Section 5.4; a typo, but confusing.\n\nThe paper is honest about limitations (e.g., only RGB, single VLM backbone) and does not overclaim. The PAINT interpolation section is a nice secondary result.\n\nWho is this for? Practitioners in federated RS who need a starting point for choosing an adapter. It deserves a serious referee; the experimental gaps are fixable with re-runs or even just better reporting. I would accept it for review.","headline":"A useful first systematic comparison of VLM adaptation strategies in federated remote sensing, with a plausible LoRA recommendation, but the communication-efficiency claim needs round counts and variance to be fully convincing.","tokens_in":12110,"tokens_out":2688,"would_cite":true,"duration_ms":28973,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Low-rank adaptation (LoRA) offers the most favorable accuracy–communication trade-off when fine-tuning a vision-language model federated on remote sensing data.","keywords":["federated learning","vision-language models","parameter-efficient fine-tuning","image classification","remote sensing","LoRA","prompt learning","catastrophic forgetting"],"falsifier":"Repeat the BigEarthNet-S2 federated experiment with per-strategy hyperparameter tuning (especially a learning-rate grid for CoOp and LoRA) and a fixed, reported number of communication rounds; if prompt learning then matches LoRA's in-distribution accuracy at comparable communication cost, or if full fine-tuning's catastrophic forgetting disappears under a longer or differently scheduled training run, the paper's guideline would be overturned.","tokens_in":11059,"feed_emoji":"🛰️","tokens_out":10120,"duration_ms":96128,"temperature":0.7,"pith_summary":"This paper asks which way of adapting a pretrained vision–language model (CLIP) works best when training is federated across remote-sensing image archives that cannot be pooled. It compares four families of adaptation strategies—full fine-tuning, encoder-specific fine-tuning, prompt learning (CoOp), and low-rank adaptation (LoRA)—on a country-based non-IID partition of BigEarthNet-S2, and evaluates each on in-distribution accuracy, cross-dataset transfer, communication cost, and local compute. The central result is a practical guideline: LoRA delivers competitive in-distribution accuracy while sending more than two orders of magnitude fewer parameters than full fine-tuning, making it the recommended strategy when accuracy and bandwidth both matter. The paper also finds that full fine-tuning severely forgets pretrained knowledge, that text-encoder fine-tuning transfers best to related remote-sensing datasets, and that prompt learning is mainly useful when communication is the dominant constraint.","feed_headline":"LoRA wins the accuracy-bandwidth trade-off in federated remote sensing","feed_subtitle":"LoRA matches full fine-tuning on accuracy while sending about 400x fewer parameters.","key_machinery":"The central object is the choice of trainable parameter subset $\\theta_{\\text{train}} \\subseteq \\{\\theta_v, \\theta_t\\}$ in a CLIP model, since this subset determines both what is transmitted to the server each federated round and how flexibly the model can specialize. The argument is carried by a systematic comparison under a fixed non-IID country-based partition of BigEarthNet-S2: for each strategy the paper measures in-distribution micro/macro-mAP, out-of-distribution accuracy on EuroSAT, RESISC45, and ImageNet, plus FLOPs and trainable-parameter counts. LoRA (low-rank adaptation injecting small trainable matrices into the attention query and value projections) is the mechanism that occupies the favorable middle: it updates both visual and textual representations while keeping the backbone frozen, which the paper finds preserves pretrained features and keeps communication near 1 million parameters.","core_discovery":"On the paper's own terms, the discovery is that the choice of trainable parameter subset $\\theta_{\\text{train}} \\subseteq \\{\\theta_v, \\theta_t\\}$ for federated CLIP adaptation largely determines a three-way trade-off, and that this trade-off has a clear winner for typical remote-sensing deployments. LoRA, which injects low-rank matrices into the query and value projections of the transformer attention blocks, reaches 79.1 micro-mAP on BigEarthNet-S2 versus 78.3 for full fine-tuning, while exchanging only 1.08M trainable parameters per round compared with 427.62M for FFT. At the same time, LoRA preserves 75.1% top-1 accuracy on ImageNet, within 0.5 points of zero-shot CLIP, whereas FFT collapses to 7.8%. The paper therefore concludes that adapting a VLM in a federated setting beats both a from-scratch ResNet-50 baseline and zero-shot CLIP, and that LoRA provides the most favorable performance–communication trade-off; it also finds that text-encoder fine-tuning is the best fine-tuned choice for cross-dataset transfer, and that PAINT-style weight interpolation can control forgetting only at the price of full fine-tuning's high communication cost.","pith_inferences":["The headline ranking may partly reflect the paper's decision to use the same learning rate and training procedure for all strategies; separate tuning could move CoOp or LoRA up or down.","The communication comparison counts raw trainable-parameter counts; with quantization or compression of updates, the practical bandwidth gap between full fine-tuning and LoRA could narrow.","Catastrophic forgetting is probed only with ImageNet; other held-out distributions, or multispectral inputs, might show a different forgetting pattern.","The conclusions rest on a general-purpose CLIP backbone; if remote-sensing-specific foundation models were used, the trade-off could shift, as the paper itself notes as future work."],"forward_implications":["A practitioner who needs both strong task accuracy and low bandwidth should prefer LoRA over full fine-tuning, since it matches in-distribution performance while reducing communicated parameters by roughly a factor of 400.","Text-encoder fine-tuning is the best fine-tuned strategy when the goal is transfer to related remote-sensing datasets, as it achieves the highest EuroSAT and RESISC45 accuracy among that group.","Prompt learning should be chosen only when communication is the dominant constraint, because it sends just 46.85K parameters but gives the weakest accuracy and worst generalization.","Full fine-tuning should be avoided when the model must retain general visual knowledge, because it drops ImageNet top-1 accuracy from 75.6% to 7.8%.","Weight interpolation between zero-shot and fine-tuned CLIP weights (PAINT) can control forgetting, but only with full fine-tuning's high communication overhead."],"supporting_citations":[{"why":"supplies the CLIP vision-language model whose encoders and zero-shot behavior are the substrate for all adaptation strategies.","marker":"[6]"},{"why":"prior federated CLIP prompt-learning framework for remote sensing that this study extends into a systematic comparison.","marker":"[10]"},{"why":"supplies CoOp, the prompt-learning strategy whose learnable context vectors define one of the four compared adaptation routes.","marker":"[11]"},{"why":"supplies LoRA, the low-rank adaptation method whose trainable matrices are the basis of the paper's main recommendation.","marker":"[18]"},{"why":"supplies BigEarthNet-S2, the multi-label dataset used for the federated in-distribution evaluation.","marker":"[19]"},{"why":"supplies the ILSVRC benchmark whose ImageNet evaluation serves as the proxy for catastrophic forgetting.","marker":"[23]"},{"why":"supplies the federated averaging procedure and the from-scratch ResNet-50 baseline used as the lower bound for comparison.","marker":"[25]"},{"why":"supplies PAINT, the weight-interpolation technique used to mitigate catastrophic forgetting in the full fine-tuning case.","marker":"[26]"}],"fun_headline_variants":["LoRA tops federated remote sensing accuracy-bandwidth trade-off","LoRA leads federated VLM tuning for remote sensing efficiency","Federated remote sensing: LoRA beats full fine-tuning in accuracy and bandwidth","LoRA's 400x parameter savings match full fine-tuning in federated RS","For federated remote sensing, LoRA is the efficient VLM adaptation choice"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that one shared learning rate and an unreported common number of communication rounds give every adaptation strategy a fair comparison, so the observed ranking could reflect hyperparameter mismatch rather than intrinsic properties of the strategies.","fun_headline_variants_meta":{"raw":{"variants":["LoRA tops federated remote sensing accuracy-bandwidth trade-off","LoRA leads federated VLM tuning for remote sensing efficiency","Federated remote sensing: LoRA beats full fine-tuning in accuracy and bandwidth","LoRA's 400x parameter savings match full fine-tuning in federated RS","For federated remote sensing, LoRA is the efficient VLM adaptation choice"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000175,"raw_usage":{"total_tokens":1373,"prompt_tokens":1123,"completion_tokens":250,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":739,"completion_tokens_details":{"reasoning_tokens":167}},"tokens_in":739,"tokens_out":250,"duration_ms":3054,"temperature":1.0,"reasoning_tokens":167,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:29:38.908274+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the BigEarthNet-S2 federated experiment with per-strategy hyperparameter tuning (especially a learning-rate grid for CoOp and LoRA) and a fixed, reported number of communication rounds; if prompt learning then matches LoRA's in-distribution accuracy at comparable communication cost, or if full fine-tuning's catastrophic forgetting disappears under a longer or differently scheduled training run, the paper's guideline would be overturned.","supporting_citations":[{"cited_title":"Learning transferable visual models from natural language supervision,","cited_arxiv_id":null,"evidence_quote":"supplies the CLIP vision-language model whose encoders and zero-shot behavior are the substrate for all adaptation strategies."},{"cited_title":"FedRSCLIP: Federated learning for remote sensing scene classification using vision-language models,","cited_arxiv_id":null,"evidence_quote":"prior federated CLIP prompt-learning framework for remote sensing that this study extends into a systematic comparison."},{"cited_title":"Learning to prompt for vision-language models,","cited_arxiv_id":null,"evidence_quote":"supplies CoOp, the prompt-learning strategy whose learnable context vectors define one of the four compared adaptation routes."},{"cited_title":"LoRA: Low-rank adaptation of large language models,","cited_arxiv_id":null,"evidence_quote":"supplies LoRA, the low-rank adaptation method whose trainable matrices are the basis of the paper's main recommendation."},{"cited_title":"reBEN: Refined BigEarthNet dataset for remote sensing image analysis,","cited_arxiv_id":null,"evidence_quote":"supplies BigEarthNet-S2, the multi-label dataset used for the federated in-distribution evaluation."},{"cited_title":"ImageNet large scale visual recognition challenge,","cited_arxiv_id":null,"evidence_quote":"supplies the ILSVRC benchmark whose ImageNet evaluation serves as the proxy for catastrophic forgetting."},{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"supplies the federated averaging procedure and the from-scratch ResNet-50 baseline used as the lower bound for comparison."},{"cited_title":"Patching open-vocabulary models by interpolating weights,","cited_arxiv_id":null,"evidence_quote":"supplies PAINT, the weight-interpolation technique used to mitigate catastrophic forgetting in the full fine-tuning case."}],"review_version":1}