REVIEW 4 cited by
Low-Complexity Probing via Finding Subnetworks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The dominant approach in probing neural networks for linguistic properties is to train a new shallow multi-layer perceptron (MLP) on top of the model's internal representations. This approach can detect properties encoded in the model, but at the cost of adding new parameters that may learn the task directly. We instead propose a subtractive pruning-based probe, where we find an existing subnetwork that performs the linguistic task of interest. Compared to an MLP, the subnetwork probe achieves both higher accuracy on pre-trained models and lower accuracy on random models, so it is both better at finding properties of interest and worse at learning on its own. Next, by varying the complexity of each probe, we show that subnetwork probing Pareto-dominates MLP probing in that it achieves higher accuracy given any budget of probe complexity. Finally, we analyze the resulting subnetworks across various tasks to locate where each task is encoded, and we find that lower-level tasks are captured in lower layers, reproducing similar findings in past work.
Forward citations
Cited by 4 Pith papers
-
Stochastic Parameter Decomposition
SPD uses stochastic masking and a learned causal importance function to decompose neural network parameters into sparsely active rank-one subcomponents, recovering ground-truth mechanisms in toy models where APD struggled.
-
Towards Unified Attribution in Explainable AI, Data-Centric AI, and Mechanistic Interpretability
A position paper unifying feature, data, and component attribution under three shared techniques, perturbation, gradient, and linear approximation, and proposing cross-attribution research directions.
-
From Indirect Object Identification to Syllogisms: Exploring Binary Mechanisms in Transformer Circuits
GPT-2 small performs syllogisms through truth-copying attention heads and a suppression-plus-MLP pathway that can output a negated truth value.
-
A Numerical PDEs Approach to Evolution Equations in Shape Analysis Based on Regularized Morphoelasticity
Regularized morphoelasticity yields a high-order elliptic system for continuous shape evolution that is solved by mixed finite elements in FEniCSx within an LDDMM-style optimal-control growth model.
Discussion (0). Continue with ORCID to comment.