{"id":"ae7637d6-c9da-4188-a8d3-2df070541874","arxiv_id":"2506.16460","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A purely black-box attacker can infer whether a task was included in multitask training by measuring the variance or pairwise inner products of embedding vectors, with near-perfect success when tasks correspond to labels and marginal success for user personalization.","lead":"This paper shows that an attacker with black-box access to a shared representation in multitask learning can often tell whether a specific task was used in training, using only fresh samples from that task. The result matters because shared representations are the minimal piece of information that organizations exchange in collaborative learning, so any leakage there is a core privacy risk.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported weak-adversary attacks are calibrated with a leave-one-out pool of 2T tasks and cross-task percentile thresholds, so they do not instantiate the single-task black-box adversary of the Section 3 threat model.","rationale":"Good-faith reading: the paper asks a meaningful privacy question, the codependency hypothesis is plausible, and the generalization-gap experiments in Section 6.6 provide some independent support. However, the most load-bearing step is not the hypothesis itself but the match between threat model and evaluation. The Section 3 security game gives the adversary one batch from one challenge task, and the paper explicitly disclaims sampling access to the underlying data distribution. Yet every reported statistic in Section 6 is computed using a leave-one-out whitening covariance estimated from the other 2T-1 tasks, and thresholds are selected as percentiles of the pooled IN/OUT statistic distribution. This makes the evaluated adversary a batch adversary with a calibration pool, not the single-task black-box adversary described in the abstract. If the reported numbers survive a re-run without that pool, the central claim is supported; if not, the current evidence cannot establish it. The reader's weakest_assumption concerned the codependency hypothesis; my concern is upstream and distinct, though the reader's rationale already noted that percentile-threshold evaluation uses information unavailable to a single-task adversary. I keep the reader's CONDITIONAL verdict, sharpening the condition to require a single-task re-evaluation before the central claim can be accepted.","tokens_in":29488,"tokens_out":8421,"duration_ms":87547,"concrete_test":"Re-run the Table 2 Stack Overflow weak-adversary row (multiple learning problems) under the Section 3 security game: estimate Algorithm 2's whitening transform and centering using only the 16 challenge-task samples (or omit whitening entirely), and set the decision threshold a priori from public pretrained-model statistics rather than from percentiles of the pooled IN/OUT distribution. If weak-adversary AUC and TPR-at-1%-FPR collapse to chance, the reported success is an artifact of the multi-task calibration pool and the central claim fails as stated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that a purely black-box adversary holding only fresh samples from a single challenge task and query access can infer task inclusion. The evaluation does not instantiate that adversary. In Section 5.1.3, Algorithm 2's whitening transform is estimated by 'pooling all of the embeddings available to the adversary, regardless of task,' and the implementation 'compute[s] whitening transformations for each of the 2T tasks that we input to our attack, leaving out the data from one task each time.' Every reported operating point in Section 6, including AUC, TPR at fixed FPR, and the 50th/75th/90th percentile thresholds in Tables 1-2, is computed across the full set of IN and OUT tasks, so the adversary is given a calibration pool containing the OUT distribution. The Section 3 security game (Step 4) gives the adversary only one batch from one task, and the paper explicitly disclaims 'sampling access to the underlying data distribution.' Thus the evaluated adversary is strictly stronger than the threat model, and the headline numbers do not support the stated 'given only fresh samples from a particular task' claim. This is not a critique of the codependency hypothesis itself; it is a mismatch between the claimed access and the access used to produce the evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces a task-inference privacy threat model for multitask learning, in which an adversary with a batch of samples from a challenge task and black-box query access to the shared representation must decide whether that task participated in training. Two attacks are proposed: a coordinate-wise variance statistic over the embeddings of the challenge batch, and a whitened pairwise inner-product/cosine statistic. A theoretical analog is developed for tracing a task in a Gaussian mean-estimation problem, with Theorems 4.1–4.3 separating strong and weak adversaries. The attacks are evaluated on CelebA, FEMNIST, and Stack Overflow under two task definitions (personalization and multiple learning problems), along with synthetic ablations over embedding dimension, samples per task, and number of tasks.","tokens_in":29676,"tokens_out":6648,"duration_ms":69164,"significance":"The paper addresses a relevant and timely question: whether shared representations in MTL leak task-level membership even to an adversary who never saw training samples. The proposed attacks are simple, efficient, and require no shadow models, which is a useful advance over prior membership and property inference pipelines. The Gaussian analysis in Appendix B is clean and correctly derived, and the evaluation is extensive across vision and language domains. The main caveat is that both the theoretical and empirical adversaries are given auxiliary information that the Section 3 security game does not provide, so the headline claim is stronger than the evidence presented.","major_comments":[{"comment":"The attacks as evaluated do not instantiate the single-task black-box adversary of Section 3. In Section 5.1.3, the whitening transform for Algorithm 2 is estimated by pooling all of the embeddings available to the adversary, regardless of task, and the implementation computes a separate transform for each of the 2T tasks in the evaluation set. Section 6.4.1 then runs the attack on all 512 tasks (256 IN and 256 OUT), and Tables 1–2 report AUC as well as TPR/FPR at percentile thresholds computed over this full pool. The security game in Section 3 gives the adversary only one batch from one challenge task and explicitly disclaims sampling access to the underlying task distribution. Consequently, the reported operating points rely on a calibration pool containing OUT tasks and their inclusion labels, which is strictly stronger access than the threat model allows. Please either evaluate a genuinely single-task adversary, for example with thresholds fixed before seeing the challenge or calibrated from data the threat model permits, or clearly reframe the claims as applying to a calibrated or transductive adversary and soften the abstract's statement about inferring inclusion from only fresh task samples.","section":"§5.1.3 and §6.4"},{"comment":"The theoretical tracing attack assumes knowledge that the threat model does not provide. Step 3 of the attack in Section 4.1 computes z = , requiring the population mean mu_bar of the task distribution Q, but Section 3 states that the adversary has no sampling access to the underlying data distribution and receives only the challenge batch X*. Theorems 4.1–4.3 are correct for an adversary who knows mu_bar, but this is an additional axiom not present in the security game. Please state this assumption explicitly, and either provide a version of the analysis where mu_bar is estimated from data available to the adversary or clarify that the theory addresses a stronger known-distribution adversary and therefore only partially explains the empirical separation.","section":"§4.1"},{"comment":"In the multiple-learning-problems experiment on Stack Overflow, the weak-adversary setting is confounded by overlap between tasks. The paper states that because the posts in the Stack Overflow dataset can contain multiple topics, the training data is not disjoint between tasks. Thus a post sampled from an OUT topic may nevertheless have appeared in the training set as part of an IN topic's task batch. For such posts, the adversary is no longer mounting a purely task-level attack with fresh samples; the shared representation may recognize the exact input, which is a sample-membership signal unrelated to whether the target topic was included as a task. This could explain the unusually small strong/weak gap in Figure 5b, where the variance attack achieves AUC 0.918 for the strong adversary and 0.909 for the weak adversary. Please either construct the OUT-task challenge sets so that no challenge input appears in D under any task, or report analyses that separate task-level from sample-level leakage.","section":"§6.5.2"}],"minor_comments":[{"comment":"The sentence 'In two of the the seminal works' contains a duplicated 'the'.","section":"§2.2.3"},{"comment":"Section 6.4.3 refers to 'Figure 6.4.3' instead of Figure 3c, and the FEMNIST weak 75th-percentile row in Table 1 has a formatting error ('52.2%,(11.2%,8.8%)').","section":"§6.4.3 and Table 1"},{"comment":"The whitening regularization parameter lambda is described only as a small constant; please report the values used for each experiment or point to the appendix.","section":"§5.1.3"},{"comment":"Please clarify in the threat model whether the weak adversary in the personalization setting is allowed to possess held-out samples from the same user, since this is a meaningful privacy assumption that differs from the multiple-learning-problems setting.","section":"§3"}],"recommendation":"major_revision","confidential_remarks":"The calibration-pool mismatch is the main decision driver: the reported percentile-threshold operating points and AUCs are computed over a pool that includes OUT tasks, which is stronger access than the Section 3 security game grants. This is fixable by reframing the claims or adding a genuinely single-task evaluation. The overlap issue in the Stack Overflow multiple-learning-problems experiment also needs attention. If the authors can provide a clean single-task evaluation or explicitly scope the claims, the paper would be a useful contribution to the privacy literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the task-inference threat model is a genuine addition, and the multi-problem empirical result is real, but the headline weak-adversary numbers come from an adversary that has a pool of OUT tasks to calibrate the whitening transform—something Section 3 explicitly denies. The reader's conditional verdict is fair.\n\nWhat is new: the strong/weak task-inference distinction is a clean way to frame leakage at the task level, and the paper is right that prior attacks leaned on shadow models or reference corpora. The two attacks are simple and the codependency hypothesis is plausible. The empirical separation between personalization (weak attack near random, AUC 0.53–0.62) and multiple learning problems (weak AUC 0.91–0.94 on Stack Overflow topics) is a substantive observation, and the generalization-gap experiments are a good addition.\n\nWhere the paper slips: the evaluation does not instantiate the threat model. Section 5.1.3 fits the whitening covariance by pooling embeddings from all 2T tasks, including the OUT tasks, and every reported ROC and percentile threshold is computed across that joint pool. So the 'weak' adversary effectively has an auxiliary reference set of non-member tasks. The abstract's 'given only fresh samples from a particular task' is not what was tested. The variance attack and raw inner products may survive without the pool, but the paper does not show it. The theory is clean but models a released mean and needs mu_bar known to the adversary, so it is motivation, not evidence for the deep-learning results. No code or data released, which hurts reproducibility for an empirical attack paper.\n\nWho this is for: privacy and FL researchers, especially those working on inference attacks against shared encoders. They will want to read it and will likely want to fix the calibration gap before relying on the numbers.\n\nRecommendation: send it to review. The core observation deserves referee time, but the revision should either re-run the attacks within the stated threat model (no cross-task whitening, thresholds set per-task) or explicitly add an auxiliary-pool adversary and justify it as realistic. With that, the paper could be solid.","headline":"The task-inference threat model is a real contribution and the multi-problem empirical result is striking, but the headline weak-adversary numbers are produced by an adversary with access to an OUT-task calibration pool that Section 3 explicitly denies.","tokens_in":30245,"tokens_out":3886,"would_cite":true,"duration_ms":40837,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A black-box adversary can tell whether a task was in MTL training using only fresh samples and query access to the shared representation.","keywords":["task inference","multitask learning","shared representations","black-box privacy attack","embedding codependency","membership inference","property inference","user inference"],"falsifier":"Train an MTL encoder with a regularizer that explicitly whitens embeddings per task so that within-task covariance equals across-task covariance, then run both attacks with the weak adversary; if AUC does not fall to chance, the codependency assumption is not the sole mechanism. A simpler empirical test is to measure whether the two statistics separate IN and OUT tasks on a dataset engineered to have independent task distributions.","tokens_in":29246,"feed_emoji":"🕵️","tokens_out":6971,"duration_ms":69244,"temperature":0.7,"pith_summary":"The paper claims that the shared representation learned in multitask learning leaks which tasks were used to train it, even to an adversary who has never seen the training data. It formalizes task-inference: given a small batch of fresh samples from a target task and black-box query access to the shared encoder, the adversary decides whether that task was IN or OUT. Two simple statistics, the coordinate-wise variance of a batch of embeddings and the mean absolute pairwise inner product of those embeddings, suffice to make this decision on vision and language benchmarks. A theoretical analysis on Gaussian mean estimation supports the attacks and shows a strict separation between a strong adversary with training samples and a weak adversary with fresh samples. If right, the 'minimal shared information' of MTL is not minimal from a privacy perspective.","feed_headline":"Shared representations leak which tasks trained them","feed_subtitle":"Even with only fresh samples and no shadow models, two simple embedding statistics break task privacy.","key_machinery":"The carrying object is the 'task-inference security game,' where a challenge task is drawn from the same distribution as the training tasks, and the adversary sees only $\\{h(x)\\}$ for a batch of its samples. The operational mechanism is the paper's codependency hypothesis: in a well-trained shared representation, embeddings of samples from the same task exhibit statistically detectable positive and negative correlations, analogous to augmentation invariance in contrastive encoders. That signal is harvested by Algorithm 1, which thresholds the average coordinate-wise variance $\\operatorname{tr}(\\operatorname{Cov}(E))$, and Algorithm 2, which thresholds the mean absolute inner product $\\frac{1}{\\binom{k}{2}}\\sum_{i