Pith. sign in

REVIEW 2 major objections 3 minor 12 references

RULER: Representation-Level Verification of Machine Unlearning

T0 review · 2 major / 3 minor · reviewed 2026-06-29 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Representation-level checks show machine unlearning methods leave residual encoding of forgotten data even when output tests pass.

desk verdict RULER shows output checks miss representation-level traces but the oracle comparison in M2 needs controls for training stochasticity. read the letter →

arxiv 2605.27569 v2 pith:FO52FBCU submitted 2026-05-26 cs.AI

Georgina Cosma , Axel Finke This is my paper · ORCID
classification cs.AI
keywords machineunlearningrepresentationverificationresidualencodingoraclecomparisonmembershipinferenceforget-setaccuracylinearmixed-effectsmodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes that standard verification of machine unlearning, which relies on membership inference attacks and accuracy on retain and forget sets, can be satisfied while forgotten records remain encoded in a model's intermediate layers. It introduces RULER metrics that compare representational positions of forget-set items against an oracle model retrained without them. Across four approximate unlearning approaches, the oracle-comparative metric M2 identifies significant residuals in ten of twelve conditions under a linear mixed-effects analysis, with larger effects as the forget fraction grows. An oracle-free metric M4 identifies the same issue from internal similarity structure alone and flags memorization in face-recognition models where no method fully erases the signal.

What carries the argument

RULER's M2 metric, which tests whether forget-set records occupy the same representational position in the unlearned model as they do in an oracle model retrained without those records.

What would settle it

An experiment in which models flagged by M2 show no measurable difference from the oracle model on any task or attack that operates on internal representations.

Watch

Extended reading notes

Core claim

Four approximate unlearning methods all pass output-level evaluation, yet under a linear mixed-effects model M2 detects significant residuals in 10 of 12 conditions (p<0.05), with effect sizes growing as the forget fraction increases. A fifth method, Bad Teacher, shows the same residuals despite a different forgetting mechanism. M4 acts as a pre-unlearning diagnostic across tabular, image, clinical text, and face-identity settings.

Load-bearing premise

That the oracle retrained model supplies the correct ground-truth representational positions and that detected differences indicate residual encoding of the forgotten records rather than other modeling artifacts.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper introduces RULER, a framework of representation-level metrics for verifying machine unlearning. It claims that four approximate unlearning methods satisfy standard output-level criteria (membership inference, retain/forget accuracy) yet fail representation-level checks: the oracle-comparative metric M2 detects significant residuals (via linear mixed-effects model, p<0.05 in 10/12 conditions) whose effect sizes increase with forget-set size, while the oracle-free metric M4 identifies memorization pre-unlearning across tabular, image, text, and face-identity domains. A fifth method (Bad Teacher) exhibits the same pattern.

Significance. If the central empirical claim holds after addressing variance controls, the work is significant because it demonstrates a concrete gap between output-level and representation-level verification and supplies falsifiable, oracle-free diagnostics (M4) that could be adopted as standard practice. The multi-domain evaluation and use of mixed-effects modeling provide a reproducible template for future unlearning audits.

major comments (2)
  1. [Abstract / M2 description] Abstract and Methods (M2 definition): the claim that M2 residuals indicate residual encoding of forgotten records rests on the oracle retrained model serving as unbiased ground truth. Because both the unlearned and oracle models are trained from scratch on different data subsets, representational differences can arise from optimization stochasticity, random initialization, or minor distribution shifts; the linear mixed-effects model specification does not appear to include terms that isolate these sources of variance, so the reported p<0.05 results may reflect training artifacts rather than forgotten-record encoding.
  2. [Results paragraph on M2] Results (effect-size trend): the observation that effect sizes grow with forget fraction is consistent with the alternative that any two independently trained models diverge more as their training sets become increasingly dissimilar; without quantified oracle stability (e.g., variance across multiple independent retrainings of the oracle) or explicit controls for training stochasticity, the trend cannot be attributed specifically to residual encoding.
minor comments (3)
  1. Provide the exact mathematical definitions of M2 and M4 (including any distance or similarity measures) as numbered equations in the main text rather than relying on prose descriptions.
  2. Report the full specification of the linear mixed-effects model (fixed effects, random effects, covariance structure) and any data-exclusion or preprocessing rules applied before fitting.
  3. Clarify whether the reported p-values are corrected for multiple comparisons across the 12 conditions.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for highlighting potential confounds from training stochasticity in the interpretation of M2. These observations are constructive and point to ways the analysis can be strengthened. We address each major comment below, proposing concrete revisions that directly target the concerns about oracle stability and variance controls while preserving the core empirical findings.

read point-by-point responses

  1. Referee: [Abstract / M2 description] Abstract and Methods (M2 definition): the claim that M2 residuals indicate residual encoding of forgotten records rests on the oracle retrained model serving as unbiased ground truth. Because both the unlearned and oracle models are trained from scratch on different data subsets, representational differences can arise from optimization stochasticity, random initialization, or minor distribution shifts; the linear mixed-effects model specification does not appear to include terms that isolate these sources of variance, so the reported p<0.05 results may reflect training artifacts rather than forgotten-record encoding.

    Authors: We agree that stochasticity, initialization, and minor distribution shifts between independently trained models constitute a plausible alternative source of representational differences, and that the current LME specification (fixed effects for method and forget fraction, random intercepts for modality and architecture) does not explicitly isolate seed-level variance. To address this, we will revise the Methods section to describe additional oracle retrainings performed with multiple random seeds. The resulting per-seed variance will be incorporated either as an expanded random-effects structure or as a separate baseline comparison. Updated LME results, including whether residuals remain significant after accounting for this variance, will be reported in the revised manuscript. revision: yes

  2. Referee: [Results paragraph on M2] Results (effect-size trend): the observation that effect sizes grow with forget fraction is consistent with the alternative that any two independently trained models diverge more as their training sets become increasingly dissimilar; without quantified oracle stability (e.g., variance across multiple independent retrainings of the oracle) or explicit controls for training stochasticity, the trend cannot be attributed specifically to residual encoding.

    Authors: The referee correctly notes that the increasing effect-size trend with forget fraction could arise from greater divergence between training sets of different sizes rather than from residual encoding. We will therefore add the multiple-oracle-retraining analysis described above. This will quantify baseline divergence at each forget fraction and test whether the observed M2 effect sizes for unlearned models exceed that baseline. The revised Results section will present both the original trend and the controlled comparison, allowing readers to evaluate whether the trend is specifically attributable to forgotten-record residuals. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity in metric definitions or statistical claims

full rationale

M2 is defined as an explicit comparison of representational positions between the unlearned model and an independently retrained oracle (retrained on the retain set only); M4 is defined from the unlearned model's internal similarity structure without reference to external fits. The linear mixed-effects model is then applied as a standard hypothesis test on these pre-defined quantities. No equations reduce a result to its own inputs by construction, no self-citations are load-bearing for the central claims, and no ansatzes or uniqueness theorems are imported. The empirical findings (significant residuals in 10/12 conditions) are therefore independent of the metric definitions themselves.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Based solely on the abstract, no free parameters, axioms, or invented entities are explicitly stated or required for the central claim; the metrics are introduced as new verification tools without reference to fitted constants or postulated entities.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RULER: Representation-Level Verification of Machine Unlearning." pith.science (2026). https://pith.science/paper/FO52FBCU

@misc{pith2026260527569,
  author       = {Pith},
  title        = {Pith review of: RULER: Representation-Level Verification of Machine Unlearning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FO52FBCU}},
  note         = {Machine review of arXiv:2605.27569}
}
read the original abstract

Machine unlearning aims to remove the influence of specific training records from a deployed model without retraining from scratch. Current protocols verify this at the output level through membership inference, retain accuracy, and forget-set accuracy, but a model can satisfy all three whilst still encoding forgotten records in its intermediate representations. We introduce RULER, a set of representation-level verification metrics. The oracle-comparative metric M2 measures whether forget-set records occupy the same representational position as in a model retrained without them. The oracle-free metric M4 detects residuals from the unlearned model's internal similarity structure alone, without retraining. Four approximate unlearning methods all pass output-level evaluation, yet under a linear mixed-effects model M2 detects significant residuals in 10 of 12 conditions (p<0.05), with effect sizes growing as the forget fraction increases. A fifth method, Bad Teacher, shows the same residuals despite a different forgetting mechanism. M4 acts as a pre-unlearning diagnostic across tabular, image, clinical text, and face-identity settings: it detects identity-level memorisation in face recognition models where no tested method fully erases the signal.

Figures

Figures reproduced from arXiv: 2605.27569 by the authors.

Figure 1
Figure 1. Discordance at ff = 5 % (N = 100 per method; 10 datasets × 10 seeds). (a) MIA accuracy falls within the ±0.05 pass window for all four methods and inde￾pendently retrained oracles. (b) The signed calibration gap M2 (Section 3) is negative for every method. Dashed lines: null values (0.50 for MIA; 0 for M2). forget-set information in intermediate layers in ways measurably distinct from a model that was never trained … view at source ↗
Figure 2
Figure 2. The tabular MLP (TabularMLP) used in all primary experiments. RULER ex￾tracts the penultimate-layer activation hθ(x) (the output of the second ReLU, high￾lighted in red), L2-normalises it, and feeds it to M2 and M4 (Eqs. 3, 7). The architecture is held constant across all ten datasets. Ten training seeds (i ∈ {0, . . . , 9}) are used per dataset; a single unlearning seed (j = 100) per training seed yields 10 runs pe… view at source ↗
Figure 3
Figure 3. [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: RULER on Bio_ClinicalBERT (clinical text). (a) M2 remains small at 1 % but shows increased variability at larger forget fractions. (b) M4 is near 0.50 across forget fractions, indicating weak or limited representation-level memorisation. Dashed lines mark the null valu…
Figure 5
Figure 5. Figure 5: Empirical calibration of M2 at ff = 5 %. Dashed line: null of zero. Left: M2 across all 10 2  = 45 oracle pairs per dataset, pooled across 10 datasets (450 values); approximately centred on zero, confirming the metric is well-calibrated under correct re￾training. Righ…
Figure 6
Figure 6. Figure 6: Per-dataset representation shift (M3) across all three forget fractions (rows: ff = 1 %, 5 %, 10 %; columns: 10 datasets; 10 training seeds per condition). Shift = mean change in cosine similarity to the oracle on forget-set records, after minus before unlearning. Dash…
Figure 7
Figure 7. Figure 7: shows the per-dataset distribution of M4 at ff = 5 %. The four unlearning methods produce similar distributions to the oracle baseline within each dataset, with deviations from 0.50 driven by the dataset’s feature-space geometry rather than the unlearning method. Grad.…
Figure 8
Figure 8. Figure 8: RULER on LFW face-identity unlearning (gender classification task, ResNet￾18, N = 5 seeds). Pre-unlearning M4 (leftmost bar/row) is far above the 0.50 null at every forget fraction, confirming the model encodes identity-level structure despite the gender-classification…
Figure 9
Figure 9. Figure 9: Image architectures used in the M4 diagnostic. RULER extracts the activation just before the classifier: the second fully-connected layer for the three-layer CNN (256- d), and the post-GAP activation for ResNet-18 (512-d). The ResNet-18 backbone is also used for the fa…
Figure 10
Figure 10. Figure 10: Bio_ClinicalBERT. The training objective is masked language modelling: 15% of words in each input sentence are replaced with a [MASK] token and the model is trained to recover the original word from the surrounding context, choosing from a vocabulary of around 30,000 …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 2 canonical work pages

  1. [1]

    In: 2021 IEEE Symposium on Security and Privacy (SP)

    Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C.A., Jia, H., Travers, A., Zhang, B., Lie, D., Papernot, N.: Machine unlearning. In: 2021 IEEE Symposium on Security and Privacy (SP). pp. 141–159. IEEE (2021)

    2021

  2. [2]

    In: 2015 IEEE Symposium on Security and Privacy

    Cao, Y., Yang, J.: Towards making systems forget with machine unlearning. In: 2015 IEEE Symposium on Security and Privacy. pp. 463–480. IEEE (2015)

    2015

  3. [3]

    In: AAAI

    Chundawat, V.S., Tarun, A.K., Mandal, M., Kankanhalli, M.S.: Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher. In: AAAI. pp. 7210–7217. AAAI Press (2023)

    2023

  4. [4]

    Towards adversarial evaluations for inexact machine unlearning,

    Goel, S., Prabhu, A., Sanyal, A., Lim, S.N., Torr, P., Kumaraguru, P.: Towards adversarial evaluations for inexact machine unlearning (2023),https://arxiv. org/abs/2201.06640

    work page arXiv 2023

  5. [5]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Golatkar, A., Achille, A., Soatto, S.: Eternal sunshine of the spotless net: Selective forgetting in deep networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9304–9312 (2020)

    2020

  6. [6]

    In: European Conference on Computer Vision

    Golatkar, A., Achille, A., Soatto, S.: Forgetting Outside the Box: Scrubbing deep networks of information accessible from input-output observations. In: European Conference on Computer Vision. pp. 383–398 (2020)

    2020

  7. [7]

    Hayes, J., Shumailov, I., Triantafillou, E., Khalifa, A., Papernot, N.: Inexact un- learning needs more careful evaluations to avoid a false sense of privacy (2024)

    2024

  8. [8]

    In: Chaudhuri, K., Salakhutdinov, R

    Kornblith, S., Norouzi, M., Lee, H., Hinton, G.: Similarity of neural network rep- resentations revisited. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 97, pp. 3519–3529. PMLR (09–15 Jun 2019)

    2019

Show all 12 references
  1. [9]

    In: Thirty-seventh Conference on Neural Information Process- ing Systems (2023)

    Kurmanji, M., Triantafillou, P., Hayes, J., Triantafillou, E.: Towards unbounded machine unlearning. In: Thirty-seventh Conference on Neural Information Process- ing Systems (2023)

    2023

  2. [10]

    In: 2017 IEEE Symposium on Security and Pri- vacy (SP)

    Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: 2017 IEEE Symposium on Security and Pri- vacy (SP). pp. 3–18. IEEE (2017)

    2017

  3. [11]

    In: 31st USENIX Security Symposium (USENIX Security 22) (2022)

    Thudi, A., Jia, H., Shumailov, I., Papernot, N.: On the necessity of auditable algo- rithmic definitions for machine unlearning. In: 31st USENIX Security Symposium (USENIX Security 22) (2022)

    2022

  4. [12]

    Wang, W., Zhang, C., Tian, Z., Yu, S., Su, Z.: Evaluation of machine unlearn- ing through model difference. IEEE Transactions on Information Forensics and Security20, 5211–5223 (2025) RULER: Representation-Level Verification 15 A Additional Multi-Seed Experiments and Robustnes...

    2025

Pith tools

Reviewed June 29, 2026 · model on record in the stance chip above.