{"id":"00f91293-47d4-4cf2-9d38-2fb1c4e0cfd3","arxiv_id":"2412.12640","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"GDBR reconstructs batch label counts from a single shared non-output layer's gradient by propagating gradients to the logits using auxiliary data estimates.","lead":"This paper introduces GDBR, an attack that recovers a federated learning client's private label distribution from just one unencrypted layer of shared gradients. It shows that encrypting only the final classification layer, a popular lightweight defense, does not protect label privacy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GDBR only evaluates models that have been customized to satisfy its key positivity assumption; standard architectures with biases, BatchNorm, and default initialization are never tested, so the claimed generality is unsubstantiated.","rationale":"The reader's weakest assumption correctly identifies Assumptions 1 and 2 and the need for auxiliary estimates, but the more load-bearing issue is that the theoretical derivation itself requires all ReLU activations to be strictly positive, and the experimental protocol enforces this by modifying the model: removing biases, replacing average pooling, and using positive weight initialization. This is not a minor implementation detail; it is what makes Eqs. (17)-(19) valid in the tested architectures. Under a standard training setup, dead ReLUs and negative pre-activations invalidate the per-sample identity used in Theorem 1, so the gradient bridge reconstructs the wrong logit gradient. The paper's robustness check for standard initialization is limited to one MLP and does not cover the convolutional architectures central to the headline results. The instance-level label assignment from the count vector λ in Eq. (21) is also left unspecified, which is a secondary concern about the reported InsAcc metric. Both issues reinforce the reader's conditional verdict: the attack direction is plausible, but the evidence as presented does not substantiate the general claim that any shared non-output layer leaks labels in standard models. A single controlled experiment on an unmodified ResNet-18 with default initialization would settle whether the concern lands.","tokens_in":15231,"tokens_out":9674,"duration_ms":99328,"concrete_test":"Run GDBR unchanged on an unmodified CIFAR-100 ResNet-18 with PyTorch default Kaiming initialization, bias terms, BatchNorm, and the original average pooling layer (no conv replacement). Share only the penultimate conv layer's gradients, use batch size 64 and the same auxiliary-data protocol, and compare InsAcc and the relative reconstruction error ||∇z̄_est − ∇z̄_true||_2 / ||∇z̄_true||_2 against the paper's positive-init/no-bias setup across early training rounds. If InsAcc falls below 80% or the reconstruction error is large, GDBR's success depends on the paper's custom model modifications rather than on a general property of partial gradient sharing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The bridge derivation divides by activation vectors: Eq. (12) and Theorem 1/2 explicitly require all elements of a to be non-zero, and Eq. (17) divides by estimated features ea. With standard Kaiming initialization, ReLU units die for a nontrivial fraction of inputs, so Lemma 4's identity z = a used in Theorem 1 fails and the recursive bridge in Eqs. (15)-(19) produces an incorrect batch-averaged logit gradient. The paper's implementation avoids this by altering the model under attack: Section V.A replaces the average pooling layer with a convolutional layer, ignores bias terms in the bottom layers, and initializes those weights uniformly in [0.01, 0.2], ensuring that every pre-activation is positive and no dead ReLU occurs. This same custom initialization also makes Assumption 1 easier to satisfy, since all samples yield similar positive features. Yet the abstract and introduction claim the attack works across ordinary convolutional and transformer architectures, and the only evidence against standard initialization is a single 6-layer MLP experiment (Fig. 5b), not the ResNet/VGG/AlexNet configurations in Figs. 3 and 6. Thus the central claim that sharing even one non-output layer leaks labels in real FL deployments is not established by the evaluated settings.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GDBR, a white-box label-recovery attack against FedSGD when a client shares gradients of only one non-output layer. The attack builds a 'gradient bridge' through stacks of FC-ReLU and Conv-ReLU layers to estimate the batch-averaged gradient with respect to the output logits, and then recovers per-class label counts using Eq. (21) with auxiliary estimates of the final-layer features and softmax probabilities. Experiments on MNIST, SVHN, CIFAR-10/100, and ImageNet report instance-level accuracy above 80% and claim robustness to gradient pruning and noise perturbation. The algebraic chain in Eqs. (6)-(21) is coherent under explicit positivity and rank conditions, but the implementation and validation impose conditions that are not satisfied by standard unmodified architectures.","tokens_in":15496,"tokens_out":9438,"duration_ms":86901,"significance":"If the claimed generality held, GDBR would be an important result because it challenges the common defense of encrypting only classification-head gradients in federated learning. The paper is among the first analytical label-recovery attacks to operate from a single unencrypted non-output layer, and the comparison against ZLG and LLG is informative, especially because the baselines are given part of the ground-truth final-layer gradient while GDBR is not. The derivation is self-contained and the experimental scope is broad at first sight. However, the results are obtained on models that are modified to guarantee the attack's core positivity assumption, and the batch-averaging step is a heuristic with no error analysis; the significance is therefore conditional on a much narrower setting than the abstract and conclusion claim.","major_comments":[{"comment":"The passage from per-sample gradients to batch averages is an unproven approximation. For each sample n, the shared-layer gradient has the form diag(∇W^(1)(n) W^(1)⊤) ⊘ a^(1)(n) or the analogous Conv expression, and Eq. (17) sums the numerators while replacing the per-sample denominators by a single auxiliary estimate ea^(1). This is valid only if a^(1)(n) is nearly constant across the batch, whereas Assumption 1 asserts only that the features have 'similar Gaussian distributions'; Fig. 2 shows visible spread for five selected dimensions of one model at one training point. The approximation error propagates through Eqs. (18)-(19) into ∇z^[L] and hence into the recovered labels. The same issue affects Assumption 2 and Eq. (21), where per-sample deviations of p^(n) from the auxiliary estimate ep are not accounted for. The paper should either provide an error bound in terms of the variance of a^(1)(n) and p^(n), or evaluate the attack in settings where activation variation is not suppressed by custom initialization.","section":"Section IV.C, Eq. (17)"},{"comment":"The attacked models are modified to guarantee the positivity condition of Theorems 1-2: the average pooling layer in ResNet is replaced by a convolutional layer, bias terms in the bottom layers are ignored, and bottom-layer weights are initialized uniformly in [0.01, 0.2]. This initialization makes every pre-activation positive, so dead ReLUs essentially do not occur and the identity a = z used in the proof of Theorem 1 holds. The only experiment with standard PyTorch initialization is the 6-layer MLP in Fig. 5b, where zero features are additionally replaced by the mean of non-zero elements; no standard ResNet, VGG, or AlexNet is evaluated without these modifications. The abstract's claim that the attack applies to 'convolutional and transformer-based networks' in ordinary FL deployments is therefore not established by the reported experiments.","section":"Section V.A, Implementation Details"},{"comment":"The bridge equations rely on unstated algebraic conditions. Eq. (9) and its recursive use in Eqs. (15)-(19) require inverting WW^⊤, which demands that each FC weight matrix have full row rank and that its output dimension not exceed its input dimension; the paper neither states nor verifies these conditions. Theorem 2 is derived only for convolution outputs of spatial shape 1×1, yet the theorem statement and its use in Eq. (14) do not carry this restriction forward. These conditions are necessary for the derivation to be well-defined and should appear as explicit assumptions in Section IV.C, with rank and shape checks in the experiments.","section":"Section IV.B, Lemma 2 and Theorem 2"},{"comment":"The derivation covers only stacks of FC-ReLU and Conv-ReLU with bias terms ignored. Standard ResNet, VGG, and AlexNet architectures contain BatchNorm layers, residual connections, or bias parameters, each of which introduces additional gradient paths not represented in Eqs. (12)-(19). The ResNet experiments replace the average pooling and ignore biases, but they do not remove BatchNorm or residual connections, and the paper does not explain how these components are incorporated into the gradient bridge. Without an analysis of these extra terms, the validity of the bridge for standard ResNet-style architectures is unsupported.","section":"Section IV.B and Section V.A"},{"comment":"The label recovery formula assumes that the batch size B is known and that the reconstructed ∇z^[L] is the exact batch-averaged gradient. In practice the gradient bridge produces an estimate, so the vector B·(ep - ∇z^[L]) is not guaranteed to be a nonnegative integer count summing to B; the paper does not describe how λ is projected, rounded, or constrained. This is not merely cosmetic, because any error in ∇z^[L] is multiplied by B and can dominate at the large batch sizes shown in Fig. 3.","section":"Section IV.D, Eq. (21)"}],"minor_comments":[{"comment":"The text says the 6-layer MLP experiment uses the SVHN and CIFAR-10 datasets, while the Fig. 4 caption says MNIST and CIFAR-10; please correct the inconsistency.","section":"Section V.D.1 and Fig. 4"},{"comment":"The abstract and introduction mention transformer-based networks, but no transformer architecture appears in the experiments; either add such an experiment or remove that claim.","section":"Abstract and Section V"},{"comment":"The notation for batch-averaged gradients (e.g., ∇a[1] versus ∇a^(1)(n)) is easy to confuse because the overline is not consistently rendered throughout the text; a single notation for sample-level and batch-averaged quantities would improve readability.","section":"Section IV.C"},{"comment":"The model name 'Resnet18' appears with inconsistent capitalization; please unify it with 'ResNet18' used elsewhere.","section":"Section V.E"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible analytical attack under a narrow set of conditions, but the headline generality is not yet supported by the evidence. The Section V.A implementation modifies the victim's model in ways that directly enforce the core positivity assumption, and the batch-averaging step in Section IV.C remains a heuristic without an error analysis. I would ask for experiments on unmodified standard architectures and a formal treatment of the batch-averaging approximation before acceptance. I did not identify citation or novelty concerns beyond those stated in the report."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the gradient bridge: an analytical way to recover label counts from a single non-output layer's gradient, when the final-layer gradient is encrypted. Prior label recovery attacks (iDLG, GI, RLG, ZLG, LLG, iLRG) all consume the output-layer gradient, so GDBR targets a real gap in lightweight defenses like PE-MPC. The core algebra for FC-ReLU and Conv-ReLU stacks is elementary backpropagation chaining, and under the stated conditions it is correct. Credit where due: the paper runs broad experiments across MNIST, SVHN, CIFAR-10/100, and ImageNet, with ablations on batch size, class distribution, auxiliary vs. dummy data, and defenses. That is more than most attack papers do.\n\nThe soft spots are serious, though. The paper's own implementation details say that for ResNet/VGG/AlexNet they replace the average pooling layer with a convolutional layer, ignore bias terms in the bottom layers, and initialize those bottom weights uniformly in [0.01, 0.2]. This is exactly what makes the bridge work, because the bridge divides by activation vectors and fails when ReLU units are dead. Standard Kaiming initialization produces dead ReLUs for a nontrivial fraction of inputs, so the claimed generality across \"convolutional and transformer architectures\" is not established by the evaluated settings. The only default-initialization experiment is a 6-layer MLP, and there they patch the problem by replacing zero features with the mean of non-zero ones. That patch is not a principled fix; it is a workaround.\n\nThe theory also omits bias, pooling, BatchNorm, and residual connections, and the theorems never state the full-rank and non-zero-activation conditions that the derivations need. Assumptions 1 and 2, which let the batch-averaged equations be treated like a single-sample formula, are only visually verified on one model at one early training point. The attacker also needs to know the batch size B, which is not always realistic. None of this makes the attack wrong; it makes the paper's central claim—that sharing even one unencrypted layer leaks labels in real FL deployments—stronger than the evidence supports.\n\nOn circularity: Eq. (21) is just a rearrangement of the cross-entropy gradient identity ∇z = p − y, so the novelty is entirely in the bridge, not the label-count formula. That is fine, but the paper should say so more clearly.\n\nWho is this for? Researchers working on gradient leakage and lightweight defenses in FL. They will find the bridge idea useful and worth engaging with, but they should not take the headline claim at face value. This paper deserves a serious referee: the direction is novel, the gap is real, and the weaknesses are fixable with more honest experiments on standard architectures and stated conditions. I would send it to review with an expectation of major revision, and I would want the revised version to include code and quantitative verification of the assumptions.","headline":"Novel attack direction against partial gradient encryption, but the generality claim is unsubstantiated because every evaluated architecture was customized to keep activations positive.","tokens_in":16032,"tokens_out":2049,"would_cite":true,"duration_ms":20562,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Encrypting only the classification head's gradients does not protect label privacy in federated learning: a server that sees any one lower-layer gradient can recover a client's batch label counts with over 80% instance-level accuracy.","keywords":["federated learning","gradient leakage","label recovery attack","partial gradient encryption","gradient bridge","privacy attack","label distribution recovery","FedSGD"],"falsifier":"On a late- or fully-trained model, take a batch containing images from visually distant classes so that penultimate-layer features diverge, run GDBR with the same auxiliary data, and measure instance-level label accuracy; if it remains above 80%, the similarity assumptions are not the limiting factor, and if it collapses, the attack is confined to the early-training regime the paper focuses on.","tokens_in":14999,"feed_emoji":"🕵️","tokens_out":9211,"duration_ms":76698,"temperature":0.7,"pith_summary":"This paper tries to establish that partial gradient encryption—specifically, hiding only the final classification layer's gradients—does not protect label privacy in federated learning. It introduces GDBR, a white-box attack in which a semi-honest server takes the gradient of a single non-output layer and reconstructs the batch-averaged gradient of the output logits through a chain of layer-wise gradient identities. From that reconstructed gradient, the attack solves a linear equation to recover the per-class counts of the victim's training batch. Experiments across five datasets and five model families report more than 80% instance-level label recovery, and the recovered labels are a prerequisite for stronger attacks such as data reconstruction and membership inference. If the paper is right, any shared lower layer becomes a sufficient channel for label leakage in FedSGD, not just the final layer.","feed_headline":"Sharing one non-output gradient layer leaks 80% of labels","feed_subtitle":"White-box server recovers client label counts even when the final layer's gradients are hidden from it.","key_machinery":"The central object is the 'gradient bridge', a chain of gradient-propagation identities connecting a shared lower-layer gradient to the gradient of the output logits. Lemma 1–4 establish the per-layer identities, Theorem 1 and Theorem 2 extend them through FC-ReLU and Conv-ReLU stacks, and Equations (17)–(19) convert the chain into a batch-averaged recursive reconstruction. The load-bearing formula is $\\lambda = B(\\hat p - \\overline{\\nabla z}^{[L]})$, which turns the reconstructed logit gradient into a vector of integer label counts because $\\lambda$ sums to $B$ and each entry counts how many samples in the batch belong to that class.","core_discovery":"GDBR's central claim is that the gradient of one unencrypted layer—for example the first fully connected or convolutional layer in the bottom stack, or the penultimate layer before the classifier—is enough to recover the label multiset of a client's batch. The proof chain uses four gradient identities: in an FC layer $\\nabla x x^\\top = W^\\top \\nabla W$, $\\nabla z z^\\top = \\nabla W W^\\top$, and $\\nabla z \\odot z = \\operatorname{diag}(\\nabla W W^\\top)$; in a Conv layer $\\langle \\nabla W_k, W_k\\rangle_F = \\langle \\nabla Z_k, Z_k\\rangle_F$; and in a ReLU layer $\\nabla z \\odot z = \\nabla a \\odot a$. Recursively propagating from the shared layer to the logits yields the reconstructed batch-averaged logit gradient $\\overline{\\nabla z}^{[L]}$ (Equations 17–19). With two approximations—that per-sample features in the shared layer and per-sample output probabilities are each close to auxiliary-data estimates—the attack obtains $\\lambda = B(\\hat p - \\overline{\\nabla z}^{[L]})$, where $\\lambda$ is the vector of per-class label counts and $B$ is the batch size. The paper argues this defeats final-layer-only encryption defenses.","pith_inferences":["The same bridge construction could in principle be adapted to architectures with batch normalization, attention, or residual connections, but the paper only derives identities for FC, Conv, and ReLU layers.","Because the recovered counts $\\lambda$ must sum to the batch size $B$, an attacker can in principle self-calibrate an unknown batch size from the same equation, although the paper does not discuss this.","The similarity assumptions behind the bridge are most plausible early in training, which the paper acknowledges; a defense that shifts to later-stage training or amplifies per-sample feature diversity could blunt the attack.","A concrete extension would be to test GDBR on a converged model; the paper's early-training focus leaves open whether the bridge survives once features and probabilities become sample-specific."],"forward_implications":["A defense that encrypts or hides only the final fully connected layer's gradients leaves label information reachable through any other layer whose gradient is shared.","GDBR achieves over 80% instance-level label recovery across MNIST, SVHN, CIFAR-10/100, and ImageNet using MLP, LeNet, AlexNet, VGG, and ResNet models.","The attack remains effective under moderate gradient pruning and noise; only aggressive defenses (pruning threshold at or above 0.9 or noise scale at or above 0.2) substantially degrade it.","Recovered label counts are a stepping stone for data reconstruction and membership inference, so partial-encryption systems must be re-evaluated for this expanded attack surface.","GDBR needs only auxiliary data—or, on simple datasets, dummy Gaussian data—rather than access to the victim's private batch."],"supporting_citations":[{"why":"Defines FedSGD, the federated-learning protocol whose gradient sharing GDBR attacks.","marker":"[1]"},{"why":"ZLG baseline that estimates posterior probabilities from auxiliary data; GDBR compares against it.","marker":"[7]"},{"why":"LLG baseline that needs final-layer gradients; GDBR shows the final layer is unnecessary.","marker":"[8]"},{"why":"DLG gradient-inversion attack that motivates label leakage from gradients and provides comparison context.","marker":"[10]"},{"why":"PE-MPC defense that encrypts the FC layer closest to the data source; GDBR challenges this class of defense.","marker":"[18]"},{"why":"iDLG attack that ties the last FC layer's gradient to the one-hot label; GDBR moves this relation to reconstructed logits.","marker":"[27]"}],"fun_headline_variants":["One unencrypted gradient layer leaks label counts","Partial gradient encryption defeated by GDBR attack","GDBR: Recover labels even when final layer is hidden","Encrypting output layer isn't enough: GDBR attack","Label leakage from a single unencrypted layer"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack's load-bearing premise is that every sample in the victim's batch has roughly the same penultimate-layer features and roughly the same output probabilities as the attacker's auxiliary-data estimates, so that batch-averaged equations behave like single-sample formulas.","fun_headline_variants_meta":{"raw":{"variants":["One unencrypted gradient layer leaks label counts","Partial gradient encryption defeated by GDBR attack","GDBR: Recover labels even when final layer is hidden","Encrypting output layer isn't enough: GDBR attack","Label leakage from a single unencrypted layer"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000384,"raw_usage":{"total_tokens":2120,"prompt_tokens":1118,"completion_tokens":1002,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":734,"completion_tokens_details":{"reasoning_tokens":923}},"tokens_in":734,"tokens_out":1002,"duration_ms":7886,"temperature":1.0,"reasoning_tokens":923,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:53:21.589444+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a late- or fully-trained model, take a batch containing images from visually distant classes so that penultimate-layer features diverge, run GDBR with the same auxiliary data, and measure instance-level label accuracy; if it remains above 80%, the similarity assumptions are not the limiting factor, and if it collapses, the attack is confined to the early-training regime the paper focuses on.","supporting_citations":[{"cited_title":"User-level label leakage from gradients in federated learning,","cited_arxiv_id":null,"evidence_quote":"LLG baseline that needs final-layer gradients; GDBR shows the final layer is unnecessary."},{"cited_title":"Deep leakage from gradients,","cited_arxiv_id":null,"evidence_quote":"DLG gradient-inversion attack that motivates label leakage from gradients and provides comparison context."},{"cited_title":"Partially encrypted multi- party computation for federated learning,","cited_arxiv_id":null,"evidence_quote":"PE-MPC defense that encrypts the FC layer closest to the data source; GDBR challenges this class of defense."}],"review_version":1}