{"id":"a33e64a4-d4ec-4aef-9271-487951484c12","arxiv_id":"2608.04477","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"DeepInvert uses unlabeled obfuscated embeddings to train an inversion model that recovers up to 73.5% of original tokens against ObfusLM, versus 26.2% for the previous best attack.","lead":"DeepInvert, a semi-supervised attack, recovers original text tokens from obfuscated language model embeddings far more accurately than prior attacks, including 73.5% token recovery against the ObfusLM defense. A smart generalist should read it because it suggests that a broad class of lightweight privacy defenses for cloud AI services is much weaker than claimed.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unsupervised branch of Algorithm 1 appears to consume victim plaintext prompts via the real model, contradicting the §3.1 threat model; without code, whether the headline attack runs under the stated adversary model is unresolved.","rationale":"The reader identified the transductive inclusion of the evaluation target as the weakest assumption. I agree that is a real caveat: the 0.735 headline is for training on the evaluation embeddings. But the paper itself provides Figure 7 showing 0.545 when the target is excluded, still roughly 2.1x the 0.262 baseline, so the central qualitative claim survives in the inductive setting. The more serious issue is the Algorithm 1 discrepancy, because if the unsupervised branch literally uses victim plaintext, the attack is not an inversion attack; it is memorization. The paper's own descriptions and ablations suggest the implementation is correct and the pseudocode is imprecise, but without code the reader cannot verify. I therefore do not move the verdict, but I would make the condition explicit: code release or a corrected algorithm, plus inductive numbers for the headline defenses. Credit where due: the paper includes a compute-matched single-shadow comparison, out-of-domain transfer experiments, and a non-transductive ablation, which are independent checks that support the core claim.","tokens_in":30856,"tokens_out":11879,"duration_ms":104988,"concrete_test":"Inspect the released implementation (or, failing that, have the authors provide a data-flow trace) for the unsupervised branch of Algorithm 1: verify that the tensor fed to L_u is read from a precomputed store of observed obfuscated embeddings produced by the victim's deployment, not generated by calling the real model on raw victim prompts. As an independent numerical check, re-run the ObfusLM (RoBERTa, SST-2) headline experiment using only the 1,821 non-transductive unlabeled embeddings of Figure 7 and confirm top-1 ≈ 0.545; if reproducing that number requires raw victim prompts, the headline result is invalid under the threat model. If the code is not released, the ambiguity cannot be settled and the conditional should stand.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is an unresolved data-access ambiguity in the attack's central training loop, not the transductive setting. In Section 4.2, Algorithm 1 lists 'Victim's prompt dataset D_real' as input, and the unsupervised branch (lines 14-15) reads 'Sample a prompt from D_real; x←extract(d_theta_real,prompt,l)'. Taken literally, the adversary runs the real target model on the victim's plaintext prompts to produce the unlabeled embeddings used for training. That contradicts Section 3.1, where the adversary observes only transmitted obfuscated embeddings and never sees plaintext prompts; if this were the actual procedure, the attack would be circular and the advertised 73.5% recovery would not measure inversion under the stated threat model. The surrounding text and the non-transductive ablation in Figure 7 indicate the implementation probably consumes a precomputed store of observed obfuscated embeddings, which would make the pseudocode simply wrong. However, the paper releases no code, and the discrepancy is never acknowledged or corrected in the manuscript. Since every headline result depends on which data the unsupervised branch consumes, this ambiguity is load-bearing; it should be resolved before the strong advertised numbers are taken at face value. The transductive issue is secondary: Figure 7 already shows 0.545 inductive top-1, so the qualitative claim survives, but with a smaller margin.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DeepInvert, a semi-supervised embedding inversion attack that combines supervised training on labeled shadow obfuscated embeddings with an unsupervised consistency objective (soft top-k KL divergence between an EMA teacher and a student on masked augmented views) over unlabeled target embeddings. The training alternates between supervised and unsupervised steps via a linear schedule. Defense-aware additions include multiple shadow models, renormalization, and PCA denoising. The attack is evaluated against nine obfuscation defenses (ObfusLM, SentinelLMs, TextObfuscator, DPNR, CAPE, DP-Forward, Santext, Custext, Datamix), five tasks, and four models (BERT, RoBERTa, LLaMA3-8B, Qwen3.5-27B). The headline result is 73.5% top-1 token recovery against ObfusLM (epsilon=0.1) on RoBERTa/SST-2 versus 26.2% for InvBert. The paper concludes that obfuscation defenses provide far less protection than previously believed on token-level and generation tasks, with a task-dependent privacy-utility tension.","tokens_in":31186,"tokens_out":5980,"duration_ms":49276,"significance":"If the empirical claims hold, DeepInvert represents a substantial advance in embedding inversion: it is, to my knowledge, the first attack to show that unlabeled obfuscated embeddings from the target deployment can be exploited through a semi-supervised objective, and the improvement over InvBert on ObfusLM is large. The evaluation is broad (nine defenses, five tasks, four architectures) and the paper includes a thoughtful analysis of why some DP-based defenses survive (CAPE, DP-Forward) and why DPNR's behavior is model-dependent (Appendix F). The PCA denoising lemma is correctly proved. The central privacy-utility tension finding is useful, though partly a restatement of the empirical pattern. The weakest point is the unresolved data-access ambiguity in Algorithm 1 and the reliance on a transductive evaluation for the headline number; both need to be fixed before the strong advertised claims are taken at face value.","major_comments":[{"comment":"The unsupervised branch (lines 14-15) samples a prompt from D_real and computes x <- extract(d_theta_real, prompt, l), meaning the adversary runs the real target model on the victim's plaintext prompts. This directly contradicts the threat model in Section 3.1, which states that the adversary observes only transmitted obfuscated embeddings and never has access to the victim's plaintext prompts. If the attack actually proceeds this way, it is circular and the advertised 73.5% recovery does not measure inversion under the stated adversary model. Appendix D describes the 'real split' as the full source of unlabeled embeddings available to the attacker, which suggests the implementation uses a precomputed store of observed obfuscated embeddings; in that case Algorithm 1 is simply wrong and should be corrected. Because no code is released, this ambiguity is unresolved and load-bearing; please clarify and fix, and release the code.","section":"§4.2, Algorithm 1"},{"comment":"The headline number 73.5% (vs 26.2%) is obtained in the transductive setting, where the evaluation target is included in the unlabeled training pool. The paper's own ablation shows that excluding the target from the unlabeled set drops top-1 to 0.545, which is still a meaningful improvement over InvBert but a much smaller margin. The abstract and conclusion advertise 73.5% without mentioning the transductive protocol. Since the transductive setting is not a realistic description of an adversary attacking future prompts, the main claims should be restated to report both numbers and the abstract should qualify the headline.","section":"§6.4.6, Figure 7, Abstract/Conclusion"}],"minor_comments":[{"comment":"The Open Science appendix states only that datasets and models are open-source; no code or reproduction instructions are provided. Given the Algorithm 1 ambiguity, releasing the code (or a precise data-flow diagram) is necessary.","section":"Appendix: Open Science"},{"comment":"No error bars or multiple-seed variance are reported for any table or figure; please report at least 3 seeds for the headline numbers and key ablations.","section":"§6.2/§6.4"},{"comment":"The text states the transductive setting is 'consistent with prior prompt-inversion works [18], [30]' but cites no specifics; please justify that the transductive setting is a meaningful adversary model.","section":"§6.1.6"},{"comment":"The first row of Table 5 is described as 'an approximate upper bound for InvBert with 7 shadow models,' but this configuration uses the unlabeled FixMatch objective, which InvBert does not have; it is a semi-supervised baseline, not an upper bound for InvBert. Please rephrase.","section":"§6.4.1, Table 5"},{"comment":"The input list uses 'Victim's prompt dataset D_real'; rename this to 'observed obfuscated embeddings' to match the threat model and Appendix D.","section":"§4.2, Algorithm 1 input list"},{"comment":"The sentence 'both attacks suffer a larger (relative) performance drop' is ambiguous; the table shows DeepInvert drops from 0.735 to 0.471 and InvBert from 0.262 to 0.184, so please specify which comparison is meant.","section":"§6.4.5, Table 7"}],"recommendation":"major_revision","confidential_remarks":"The Algorithm 1 data-access ambiguity is the key technical issue; if the authors clarify that D_real is a store of observed obfuscated embeddings and correct the pseudocode, the paper could be acceptable after major revision. The transductive-vs-inductive gap should also be addressed in the abstract. The experimental breadth is a strength, but the lack of released code makes verification difficult."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a real, well-engineered attack result that should make people working on obfuscation-based defenses nervous. The novelty is genuine. Nobody has used unlabeled target embeddings for embedding inversion before, and the FixMatch-style consistency loss with an EMA teacher is a sensible adaptation. The evaluation is broad: nine defenses, five tasks, four architectures, and the ablations are informative. The main claim, that DeepInvert roughly triples top-1 recovery against ObfusLM relative to the prior best, is plausible and supported by the experiments, with an important caveat.\n\nThe caveat is the transductive setting: the headline 73.5% number includes the evaluation target in the unlabeled training pool. The paper's own Figure 7 shows that without that, recovery drops to 54.5%. That is still much better than the 26.2% baseline, so the qualitative conclusion survives, but the advertised number is the most favorable one and should be presented that way from the start.\n\nThe bigger problem is a load-bearing ambiguity in Algorithm 1. The pseudocode says the unsupervised branch samples from the victim's prompt dataset and runs the real target model to extract embeddings. Taken literally, that means the adversary has plaintext prompts, which contradicts the paper's own threat model in Section 3.1, where the server only sees transmitted obfuscated embeddings. If the attack actually needs plaintext prompts, it is not an inversion attack at all. The surrounding text and the dataset appendix suggest the implementation really uses precomputed obfuscated embeddings, so the pseudocode is probably just wrong. But without code, it is unresolved, and this is the one issue that could sink the central claim if the authors cannot clarify it. It should be fixed before the paper is taken at face value.\n\nOther soft spots are minor: no error bars or seed variance, some comparisons give DeepInvert more compute than baselines (though the single-shadow ablation is compute-matched and still wins), and the generation experiments cover only four defenses. The PCA denoising lemma is simple but correct. The citation pattern looks fine, and the paper engages honestly with prior attacks.\n\nBottom line: this is a strong candidate for peer review. The method is new, the evaluation is serious, and the privacy-utility tension finding is worth taking seriously. The authors should be asked to fix the pseudocode, report inductive numbers prominently, and ideally release code. Whoever referees it should push on the data-access question first.","headline":"Genuinely new semi-supervised inversion attack that likely breaks several obfuscation defenses, but the pseudocode contradicts its own threat model and the headline number is transductive; still deserves a serious referee.","tokens_in":31715,"tokens_out":2772,"would_cite":true,"duration_ms":24213,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Obfuscation-based defenses for cloud language models protect far less than claimed: a semi-supervised embedding-inversion attack recovers 73.5% of tokens from the leading defense, and defenses that resist inversion collapse utility.","keywords":["embedding inversion","obfuscation defenses","semi-supervised learning","differential privacy","language model privacy","prompt recovery","consistency regularization","token recovery"],"falsifier":"Reproduce the ObfusLM experiments in the strictly non-transductive setting, excluding the evaluation split from the unlabeled pool as in Figure 7; if the top-1 recovery on SST-2 does not exceed the supervised-only InvBert baseline of 0.262 under matched hyperparameters, then the advertised unsupervised advantage is an artifact of transductive leakage rather than a general property of obfuscated embeddings.","tokens_in":30645,"feed_emoji":"🔓","tokens_out":6498,"duration_ms":57337,"temperature":0.7,"pith_summary":"This paper claims that the main class of lightweight defenses for cloud language models, which obfuscate prompts before transmission, protects far less than their papers suggest. It introduces DeepInvert, a semi-supervised attack that trains an inversion model on both labeled shadow embeddings and unlabeled obfuscated embeddings captured from the target service. The attack recovers 73.5% of tokens with its single best guess against the strongest tested defense, versus 26.2% for the previous best attack. If the claim holds, any obfuscation that preserves enough signal for token-level or generative tasks also preserves enough structure to be inverted, while defenses that successfully resist inversion destroy task utility.","feed_headline":"Semi-supervised attack recovers 73.5% of obfuscated prompt tokens","feed_subtitle":"It exploits semantic structure that survives obfuscation, exposing a privacy–utility tradeoff in cloud defense designs.","key_machinery":"The argument turns on two mechanisms. First, a semi-supervised consistency objective: unlabeled obfuscated embeddings are viewed under weak and strong masking; an exponential-moving-average teacher produces soft top-$k$ targets for the weak view, and the student minimizes a top-$k$ KL divergence to those targets on the strong view, while a supervised loss on shadow embeddings anchors the mapping. Second, for noise-adding defenses, a linear denoising step uses PCA projection, whose effect is captured by the identity $\\mathbb{E}\\|PP^{T}g\\|^{2}/\\mathbb{E}\\|g\\|^{2}=K/H$ for isotropic zero-mean noise $g$, so projecting to $K$ of $H$ dimensions amplifies the signal-to-noise ratio roughly by $H/K$.","core_discovery":"The central discovery is that obfuscation perturbs individual token embeddings but preserves enough relative semantic geometry for a semi-supervised training loop to reconstruct the original tokens. DeepInvert uses labeled shadow embeddings for a supervised cross-entropy loss and unlabeled target embeddings for a consistency loss between a weakly augmented view, scored by an exponential-moving-average teacher, and a strongly augmented view, scored by the student, with soft top-k targets to avoid committing to unreliable pseudo-labels. Alternating the two objectives by epoch prevents gradient conflict. Under previously recommended privacy parameters, the strongest tested defense yields 0.735 top-1 token recovery on SST-2, 0.818 on AG News, and 0.852 on a 27B-parameter autoregressive model in medical question answering at epsilon=1; on token-level NER and generation, defenses that resist recovery collapse utility to near random.","pith_inferences":["Beyond the paper: if the transductive-versus-held-out gap seen in Figure 7 generalizes, then a server-side adversary with continuous access to a target's prompts should be modeled as operating near the transductive curve, making the practical threat larger than the non-transductive numbers suggest.","Beyond the paper: the revealed task-dependent tension suggests a design rule that could be tested directly: any obfuscation that preserves token-level utility on NER or generation is likely invertible, so defenses should be evaluated on whether they can mask token identities while preserving only sentence-level statistics.","Beyond the paper: the same consistency-based inversion objective could be adapted to pooled representations in retrieval-augmented pipelines, where the transmitted vector is one summary embedding rather than per-token states; the consistency target would need to shift from token IDs to span-level pseudo-labels.","Beyond the paper: the PCA denoising lemma implies that, for a fixed privacy budget, increasing embedding dimension while keeping sensitivity bounded may make DP-based defenses more invertible rather than less, an empirically testable inversion of the usual high-dimensional-privacy intuition."],"forward_implications":["Under recommended privacy settings, the strongest tested defense loses its claimed protection: DeepInvert achieves 73.5% top-1 token recovery on SST-2 versus 26.2% for the previous best attack, with similar gains at epsilon=1 on generation tasks.","For token-level and generation workloads, no evaluated scheme simultaneously preserves utility and limits recovery; defenses that resist inversion, CAPE and DP-Forward, drive task utility to near random, for example an F1 of 0 on CoNLL-2003.","On sentence-level classification, some DP-based defenses can maintain both accuracy and low token recovery, so the privacy-utility tension is task-dependent rather than universal.","Attack performance degrades gracefully without transductive access: excluding the evaluation target from the unlabeled pool still yields 54.5% top-1 recovery on the strongest tested defense, and out-of-domain shadow data leaves most defenses above 70% recovery.","The attack transfers to large autoregressive generation models at 8B and 27B parameter scales, indicating the finding is not limited to small encoder models."],"supporting_citations":[{"why":"Supplies the leading obfuscation defense, its recommended privacy parameters, and the threat model that DeepInvert attacks.","marker":"[9]"},{"why":"Defines the prior supervised inversion attack InvBert, the main baseline and the origin of the shadow-model training setting.","marker":"[19]"},{"why":"Provides the nearest-neighbor KNN attack baseline that demonstrates simple embedding matching is foiled by obfuscation.","marker":"[20]"},{"why":"Supplies the direct prompt-embedding optimization baseline TBS that DeepInvert must outperform.","marker":"[18]"},{"why":"Contributes the consistency and pseudo-labeling idea that DeepInvert adapts from image classification to large-vocabulary token recovery.","marker":"[32]"},{"why":"Contributes the exponential-moving-average teacher mechanism used to stabilize the unsupervised consistency target.","marker":"[33]"},{"why":"Defines the DPNR embedding-level DP defense with min-max normalization, one of the defenses tested and analyzed for noise-to-signal behavior.","marker":"[12]"},{"why":"Defines the DP-Forward defense with Frobenius normalization, the strongest DP baseline that resists recovery at the cost of collapsed utility.","marker":"[13]"}],"fun_headline_variants":["DeepInvert attack recovers 73.5% of obfuscated prompt tokens","Semi-supervised attack defeats prompt obfuscation defenses","New attack exposes privacy flaw in prompt obfuscation","DeepInvert recovers 73.5% of tokens despite obfuscation","Semi-supervised inversion cracks obfuscated LLM embeddings"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline numbers assume the adversary can collect the very prompts it will later be judged on into its unlabeled training pool; when those prompts are held out, recovery on the leading defense drops from 73.5% to 54.5%.","fun_headline_variants_meta":{"raw":{"variants":["DeepInvert attack recovers 73.5% of obfuscated prompt tokens","Semi-supervised attack defeats prompt obfuscation defenses","New attack exposes privacy flaw in prompt obfuscation","DeepInvert recovers 73.5% of tokens despite obfuscation","Semi-supervised inversion cracks obfuscated LLM embeddings"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000762,"raw_usage":{"total_tokens":3409,"prompt_tokens":996,"completion_tokens":2413,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":2316}},"tokens_in":612,"tokens_out":2413,"duration_ms":16913,"temperature":1.0,"reasoning_tokens":2316,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:38:13.525862+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce the ObfusLM experiments in the strictly non-transductive setting, excluding the evaluation split from the unlabeled pool as in Figure 7; if the top-1 recovery on SST-2 does not exceed the supervised-only InvBert baseline of 0.262 under matched hyperparameters, then the advertised unsupervised advantage is an artifact of transductive leakage rather than a general property of obfuscated embeddings.","supporting_citations":[{"cited_title":"Obfuslm: Privacy-preserving language model service against embedding inversion attacks,","cited_arxiv_id":null,"evidence_quote":"Supplies the leading obfuscation defense, its recommended privacy parameters, and the threat model that DeepInvert attacks."},{"cited_title":"InvBERT: Reconstructing Text from Contextualized Word Embeddings by inverting the BERT pipeline","cited_arxiv_id":"2109.10104","evidence_quote":"Defines the prior supervised inversion attack InvBert, the main baseline and the origin of the shadow-model training setting."},{"cited_title":"Natural language understanding with privacy-preserving BERT,","cited_arxiv_id":null,"evidence_quote":"Provides the nearest-neighbor KNN attack baseline that demonstrates simple embedding matching is foiled by obfuscation."},{"cited_title":"Depth gives a false sense of privacy: LLM internal states inversion,","cited_arxiv_id":null,"evidence_quote":"Supplies the direct prompt-embedding optimization baseline TBS that DeepInvert must outperform."},{"cited_title":"Fixmatch: Simplifying semi- supervised learning with consistency and confidence,","cited_arxiv_id":null,"evidence_quote":"Contributes the consistency and pseudo-labeling idea that DeepInvert adapts from image classification to large-vocabulary token recovery."},{"cited_title":"Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results,","cited_arxiv_id":null,"evidence_quote":"Contributes the exponential-moving-average teacher mechanism used to stabilize the unsupervised consistency target."},{"cited_title":"Differentially private representation for NLP: formal guarantee and an empirical study on privacy and fair- ness,","cited_arxiv_id":null,"evidence_quote":"Defines the DPNR embedding-level DP defense with min-max normalization, one of the defenses tested and analyzed for noise-to-signal behavior."},{"cited_title":"Dp-forward: Fine-tuning and inference on language models with differential privacy in forward pass,","cited_arxiv_id":null,"evidence_quote":"Defines the DP-Forward defense with Frobenius normalization, the strongest DP baseline that resists recovery at the cost of collapsed utility."}],"review_version":2}