Converting MLPs into Polynomials in Closed Form

Alice Rigg; Nora Belrose

REVIEW 3 major objections 6 minor 19 references

Converting MLPs into Polynomials in Closed Form

T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper derives closed-form least-squares polynomial approximations of MLPs and GLUs under Gaussian-mixture inputs, making their linear and quadratic structure directly computable from weights and moments.

desk verdict Useful idea, honest paper, but the closed-form derivation has two fixable algebraic errors and the headline quadratic result isn't actually computed in closed form. read the letter →

arxiv 2502.01032 v1 pith:KZSW3VPV submitted 2025-02-03 cs.LG stat.ML

Nora Belrose , Alice Rigg This is my paper · ORCID

classification cs.LGstat.ML

keywords closed-formpolynomialapproximationGaussianmixtureinputsmechanisticinterpretabilitysimplicitybiasgatedlinearunitsmultilayerperceptronsSVDadversarialexamplesfractionofvarianceunexplained

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that feedforward networks need not be treated as black boxes: under a Gaussian-mixture input assumption, the best least-squares polynomial approximation of any degree to an MLP or a gated linear unit can be computed exactly, with coefficients read off from the network weights and input moments. This gives access to the network's linear and quadratic structure without training an approximator or sampling from the model. The authors show that a quadratic approximant captures over 95% of an MNIST-trained MLP's output variance, and that directions found in the approximant's SVD transfer as effective adversarial attacks on the original network. They also use the approximants to watch how the network's effective complexity grows during training, finding a phase where the linear fit worsens while the quadratic fit holds steady—consistent with the idea that networks learn low-order statistics first.

What carries the argument

The load-bearing object is the paper's Master Theorem (Theorem 3.1), which reduces any expectation of the form $\mathbb{E}[g(X)\prod_{i=1}^n Y_i]$ over jointly Gaussian variables to a linear combination of univariate expectations $\mathbb{E}[g(X)X^k]$. The reduction works by regressing each $Y_i$ on $X$ to get $Y_i = \alpha_i+\beta_i X+\epsilon_i$, expanding the product combinatorially, and using Isserlis' theorem to evaluate products of the residuals, which are Gaussian and independent of $X$. This theorem turns the feature-map/OLS strategy for polynomial approximation into a closed-form computation: for Gaussian inputs the preactivations of an MLP or GLU are Gaussian, Stein's lemma gives the linear cross-covariances, and the theorem supplies the higher moments, while the law of total covariance extends the result to Gaussian mixtures.

What would settle it

Train a network on a non-Gaussian dataset (e.g., natural images with heavy-tailed pixel statistics or raw audio), fit the closed-form linear and quadratic approximants using a Gaussian mixture model of the inputs, and compare their $R^2$ against polynomials of the same degree fitted by SGD on the true samples; a large drop in $R^2$ for the Gaussian-derived approximant would show the closed-form optimality premise fails on that distribution.

Watch

Extended reading notes

Core claim

The paper's central claim is that the coefficients of the least-squares optimal polynomial approximant of an MLP or GLU, of any chosen degree, can be written analytically from the network's weights and biases together with the mean and covariance of a Gaussian input distribution (or the component means and covariances of a Gaussian mixture). The derivation reduces every required expectation to univariate Gaussian integrals via a master theorem, so the approximant is produced without stochastic optimization. The authors report that on MNIST, the quadratic approximant explains over 95% of the variance of the trained MLP's outputs, and the same approximant's top singular vectors, when ablated from the input, drive the original network's accuracy to near-chance—evidence that the polynomial captures causally relevant structure.

Load-bearing premise

The load-bearing premise is that the input genuinely follows (or is well-approximated by) a Gaussian mixture with known component means and covariances; the closed-form coefficients are least-squares optimal only under that distribution, and if the true data distribution is far from Gaussian, the interpretability and attack conclusions will not follow.

Editorial extensions

If this is right

For any single-hidden-layer MLP or GLU with Gaussian mixture inputs, a linear approximant is computed by inverting the input covariance and applying Stein's lemma; no data fitting is needed, only the network weights and the input moments.
Higher-degree approximants follow from the same feature-map trick, with the Master Theorem supplying all cross-moments; quadratic approximants are feasible for input dimensions up to thousands because the feature covariance has closed form.
On MNIST, the quadratic approximant explains more than 95% of output variance, and ablating the top four SVD directions of the linear approximant drops the original MLP's accuracy below 50%, so the approximants expose input directions that are causally load-bearing for the network.
Training-time $R^2$ trajectories show the linear fit degrading sharply after a phase transition while the quadratic fit stays tight, evidence that the network first learns linear statistics and then quadratic ones, consistent with a simplicity bias.
The same analytic machinery applies to GLUs because their preactivations are jointly Gaussian, so gated linear units inherit the closed-form polynomial approximation and its interpretations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial: the Master Theorem's structure suggests the method extends to any activation whose Gaussian moments are analytically or numerically tractable, including Swish, softplus, and possibly learned activations, without new theory.
Editorial: the sharp rise in linear FVU combined with flat quadratic FVU could be used as a diagnostic for when a network stops learning mean/covariance structure and starts exploiting higher moments; testing it on transformers or with different optimizers would reveal whether the 'quadratic phase' is a general inductive bias.
Editorial: because the linear approximant's SVD directions are computed analytically from weights alone, they could support zero-shot concept steering or erasure in feedforward modules, complementing approaches that require training a probe on model activations.
Editorial: the overcomplete eigenbasis of the quadratic approximant ($d^2$ vectors for residual-stream dimension $d$) may function as a fixed, data-independent analogue of sparse autoencoder features, potentially helping separate features the model computes from features that merely exist in the data.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this one because the intended theory is likely correct but the paper as written doesn't establish it. The core contribution is genuinely new: analytic formulas for least-squares polynomial approximants of MLPs and GLUs under Gaussian mixture inputs, extending Pearce et al.'s bilinear-layer work. Releasing code and tracing FVU across training is a nice application of the simplicity-bias idea. The paper is also transparent about its limitations and about the SGD finetuning used for the quadratic approximants on MNIST.

The algebra, however, has two load-bearing errors. Equation 6 omits the E[x_j]E[phi(y_i)] term in Stein's lemma for nonzero-mean Gaussians. On MNIST every class mean is nonzero, so the displayed linear coefficient is not the least-squares solution. Equation 16's coefficient on E[g(X)] should be alpha1*alpha2 + Cov(eps1, eps2), not alpha1*alpha2*Cov(eps1, eps2). Even for zero-mean inputs the quadratic coefficient is wrong when residuals are correlated. These aren't appendix typos; they feed the formulas used for the linear and quadratic approximants. The main theoretical claim as stated is therefore incorrect.

The empirical quadratic result also doesn't exercise the closed form. The footnote in Section 4.1 says the analytic solution for Gaussian mixtures is intractable on MNIST, so they finetune under N(0,1) with SGD. The "over 95% variance explained" plot is a property of the SGD-finetuned approximant, not the closed-form derivation. The adversarial attack lacks a baseline — no comparison to random projections or other ablations — and the curves are single-run, so the lockstep claim is weaker than presented.

These are serious but fixable issues. The intended approach — using Stein's lemma and Gaussian-moment identities to derive polynomial approximants — is sound in principle, and the errors are the kind a careful revision can correct. The paper is not incoherent or circular; it's a promising method with a flawed write-up.

This is for mechanistic interpretability readers who want a closed-form alternative to SGD-fitted proxies. It deserves a serious referee, but only with the promise of a heavy revision. I'd send it to review rather than desk-reject, then require corrections to Eq. 6 and Eq. 16, and ask for a closed-form quadratic demonstration on a lower-dimensional problem or a clear separation of the SGD-based empirical claims.

Referee Report

3 major / 6 minor

Summary. The paper proposes a method for converting one-hidden-layer MLPs and gated linear units (GLUs) into least-squares-optimal polynomial approximants in closed form, assuming the input distribution is Gaussian or a Gaussian mixture. The derivation combines Stein's lemma with a 'master theorem' that reduces multivariate Gaussian integrals of the form E[g(X)Y1...Yn] to univariate expectations. The authors apply the linear approximants to an MNIST-trained MLP, track the fraction of variance unexplained (FVU) over training time as evidence for the distributional simplicity bias, visualize top eigenvectors of the quadratic approximants, and show that SVD-based ablations of the linear approximant transfer as adversarial interventions to the original network. They also release a code library. The manuscript is transparent about its main limitations, notably the Gaussian-mixture assumption and the fact that the quadratic MNIST approximants are obtained by SGD finetuning rather than the closed-form formulas.

Significance. If the derivation is corrected, the paper offers a useful and reusable interpretability tool: analytic polynomial surrogates for common feedforward modules, with explicit integral formulas for ReLU and GELU, a clean way to quantify the order of statistics a network uses, and falsifiable predictions via adversarial transfer. The master-theorem reduction and the appendices with Gaussian integrals are valuable contributions, and the code release is a strength. The paper does not appear to assume its conclusions: the FVU measurements are not used to fit the polynomial coefficients beyond the disclosed finetuning. However, the headline quadratic results are currently produced by SGD finetuning, not the closed-form derivation, so the theoretical contribution and the strongest empirical validation are less tightly coupled than the abstract suggests.

major comments (3)

[Section 3.1, Eq. (6)] The displayed identity E[phi(y_i)x_j] = Cov(y_i,x_j)E[phi'(y_i)] is not correct for nonzero-mean jointly Gaussian variables; the general identity is E[phi(y_i)x_j] = E[x_j]E[phi(y_i)] + Cov(y_i,x_j)E[phi'(y_i)]. If Eq. (6) is used literally in the computation of E[f(x)x^T], the omitted E[x_j]E[phi(y_i)] term does not cancel in the subsequent subtraction defining Cov[f(x),x], because that subtraction is not shown; the final beta in Eq. (4) is therefore not justified as written. The derivation should either state and use the covariance identity Cov(phi(y_i),x_j) = Cov(y_i,x_j)E[phi'(y_i)] and show the cancellation, or carry the extra term through the computation.
[Section 3.3, Eq. (16)] The n=2 specialization of the master theorem is mis-derived. Expanding (alpha1+beta1X+epsilon1)(alpha2+beta2X+epsilon2) and using the independence of epsilon1,epsilon2 from X gives E[g(X)Y1Y2] = beta1beta2 E[g(X)X^2] + (alpha1beta2+alpha2beta1)E[g(X)X] + (alpha1alpha2 + Cov(epsilon1,epsilon2))E[g(X)]. Equation (16) writes the constant coefficient as alpha1alpha2 Cov(epsilon1,epsilon2), omitting the alpha1alpha2 E[g(X)] term. Since this identity feeds the quadratic coefficient derivation in Sections 3.2 and 3.5, the closed-form quadratic formulas are not established as written and must be corrected and re-derived downstream.
[Appendix B / Section 4.1, footnote 3] The quadratic FVU curves that support the abstract's claim that 'quadratic approximants explain over 95% of the variance' are not computed with the closed-form Gaussian-mixture solution. As the footnote and Appendix B state, that solution is intractable on MNIST, so the authors initialize with the standard-Gaussian closed form and finetune with SGD on Gaussian-mixture samples. This is disclosed, but it means the headline quadratic result and the quadratic branch of the simplicity-bias experiment validate a numerical fit to a convex objective, not the analytic derivation. Please qualify the abstract/conclusion accordingly, or supply convergence or error evidence that the SGD solution is close to the true least-squares coefficients.

minor comments (6)

[Theorem 3.1, Eq. (9)] The upper limit of the sum in Eq. (9) should be n, not n-1; the product of n linear factors has degree n, and the subsequent n=2 formula is consistent with k=0,1,2.
[Figure 2 caption] The caption reads 'Fraction of variance explained (FVU)', but FVU is the fraction of variance unexplained; the label should be corrected.
[Abstract and Section 4.2] The abstract and main text state the >95% quadratic FVU result without noting that the quadratic MNIST coefficients were SGD-finetuned rather than computed by the closed-form formulas; this should be flagged where the number is advertised.
[Section 2] The phrase 'the the simple' contains a duplicated article and should read 'the simple'.
[Eq. (7)] The feature map phi_2(x) is defined with n as the input dimension, but the rest of the paper uses d; the notation should be unified.
[Figure 1 caption] The caption says 'at step (y)' without defining y; it should reference the log-spaced step index used in the text, e.g., step 2^12 or a named axis variable.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the polynomial coefficients are computed from network weights and input moments, and the DSB evidence is measured with a new instrument.

full rationale

The formal derivation is not circular. The linear approximant is defined as the OLS solution beta = Cov[x]^-1 Cov[f(x), x]; the paper evaluates the needed Gaussian integrals analytically via Stein's lemma, so the coefficients are functions of the network weights and input moments, not of the target R^2. The master theorem (Theorem 3.1) reduces multivariate Gaussian expectations to univariate moments, and the Gaussian-mixture extension (Eq. 17) invokes the law of total covariance. None of these steps presupposes the FVU values or the conclusion that networks learn polynomials of increasing degree. In the experiments, the quadratic approximants are explicitly admitted to be finetuned with SGD (Section 4.1 footnote 3 and Appendix B), so the reported >95% R^2 is a measured fit quality, not a fitted parameter renamed as a prediction; this transparency removes any fitted-input-as-prediction circularity. The simplicity-bias narrative cites the authors' prior DSB work (Belrose et al., 2024), but the present FVU and feature-visualization evidence is generated independently with a new analytic instrument, making the self-citation motivational rather than load-bearing. A separate correctness concern, not a circularity concern, is that Eq. 6 appears to omit the centering term in Stein's lemma and Eq. 16's coefficient for E[g(X)] appears to be alpha1*alpha2 + Cov(epsilon1, epsilon2) rather than alpha1*alpha2*Cov(epsilon1, epsilon2); these are algebraic risks in the derivation, not a reduction of the claimed outputs to the inputs.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The theoretical derivation rests on standard Gaussian integral results and the Gaussian mixture input assumption. The empirical quadratic results on MNIST rely on an approximating SGD finetune rather than the closed-form solution, and on the assumption that MNIST is well-modeled as a Gaussian mixture. No new entities or mediators are introduced.

free parameters (1)

MNIST class-conditional Gaussian mixture parameters (per-class mean and covariance) = Empirical class means and covariance matrices estimated from the MNIST training set
Define the input measure with respect to which the approximants are least-squares optimal; the reported FVU curves and attack results are specific to this fitted input distribution.

assumptions (5)

domain assumption Inputs follow a Gaussian mixture with known component means and covariances
Used throughout Section 3 to make preactivations jointly Gaussian and to define the optimality measure; acknowledged as restrictive in the Limitations paragraph.
standard math Stein's lemma, Isserlis theorem, and Gaussian moment formulas are valid for the activation functions used
Invoked in Section 3.1 via Stein's lemma, Section 3.2 via Isserlis, and Appendix A via Owen integral tables for ReLU and GELU.
domain assumption MNIST is well-modeled as a Gaussian mixture
Justifies computing FVU under the fitted mixture rather than the true empirical distribution; cited to Belrose et al. 2023, Figure 2.
ad hoc to paper The SGD-finetuned quadratic approximant converges near the true least-squares solution on the Gaussian mixture
Appendix B states the exact solution requires a covariance matrix with 9.47e10 entries, so the fit is initialized under N(0,1) and tuned with SGD; the paper asserts convexity makes this accurate but does not verify it.
standard math Residuals from regressing Gaussian variables on X are independent of X and jointly Gaussian
Used in the proof of Theorem 3.1 to separate E[prod(s)] from E[g(X)X^k] when expanding the conditional expectation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Converting MLPs into Polynomials in Closed Form." pith.science (2026). https://pith.science/paper/KZSW3VPV

@misc{pith2026250201032,
  author       = {Pith},
  title        = {Pith review of: Converting MLPs into Polynomials in Closed Form},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KZSW3VPV}},
  note         = {Machine review of arXiv:2502.01032}
}

abstract

Recent work has shown that purely quadratic functions can replace MLPs in transformers with no significant loss in performance, while enabling new methods of interpretability based on linear algebra. In this work, we theoretically derive closed-form least-squares optimal approximations of feedforward networks (multilayer perceptrons and gated linear units) using polynomial functions of arbitrary degree. When the $R^2$ is high, this allows us to interpret MLPs and GLUs by visualizing the eigendecomposition of the coefficients of their linear and quadratic approximants. We also show that these approximants can be used to create SVD-based adversarial examples. By tracing the $R^2$ of linear and quadratic approximants across training time, we find new evidence that networks start out simple, and get progressively more complex. Even at the end of training, however, our quadratic approximants explain over 95% of the variance in network outputs.

Figures

Figures reproduced from arXiv: 2502.01032 by the authors.

**Figure 1.** Quadratic and linear features for class ‘3’, over the course of training. Discernible ‘3’ qualities arise from noise for both quadratic and linear features, aligning with the region where learning is happening. The linear 3 structure is most intuitively discernible at step (y), when FVU is minimal, before beginning to overfit. This can be interpreted as the MLP learning and relying on statistics of higher complexity… view at source ↗

**Figure 2.** Fraction of variance unexplained for linear and quadratic approximants on a Gaussian mixture distribution imitation the MNIST training set. There is a sharp increase in the linear FVU between 500 and 1K training steps, while the quadratic FVU is roughly constant over the same time period. proximants capture the out-of-distribution behavior of the network. We formulate interventions as input space transformations, pr… view at source ↗

**Figure 3.** KL divergence of linear and quadratic approximants from the network on which they were fit, evaluated on a Gaussian mixture distribution imitating the MNIST training set. The trend mirrors the FVU plot ( [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Top row: Adversarial ‘3’ for different intervention strengths, ranging from one to ten SVD components ablated (left to right). Bottom row: Random examples of each digit with all ten SVD components ablated. Examples bear high resemblance to the original images, despite …

**Figure 5.** Figure 5: Steering with an adversarial mask. Strikingly, the original network’s accuracy drops in perfect lockstep with that of its linear and quadratic approximants. Just four SVD components are needed to bring accuracy below 50%, and ablating the entire rowspace (only 10 out …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 8 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

Leace: Perfect linear concept erasure in closed form

Belrose, N., Schneider-Joseph, D., Ravfogel, S., Cotterell, R., Raff, E., and Biderman, S. Leace: Perfect linear concept erasure in closed form. arXiv preprint arXiv:2306.03819, 2023

arXiv 2023
[3]

T., and Fern, X

Belrose, N., Pope, Q., Quirke, L., Mallen, A. T., and Fern, X. Neural networks learn statistics of increasing complexity. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=IGdpKP0N6w

work page 2024
[4]

N., Fan, A., Auli, M., and Grangier, D

Dauphin, Y. N., Fan, A., Auli, M., and Grangier, D. Language modeling with gated convolutional networks. In International conference on machine learning, pp.\ 933--941. PMLR, 2017

2017
[5]

A., Mehta, H., Mishchenko, K., Khaled, A., and Cutkosky, A

Defazio, A., Yang, X. A., Mehta, H., Mishchenko, K., Khaled, A., and Cutkosky, A. The road less scheduled. arXiv preprint arXiv:2405.15682, 2024

arXiv 2024
[6]

Toy models of superposition

Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., et al. Toy models of superposition. arXiv preprint arXiv:2209.10652, 2022

arXiv 2022
[7]

and Gimpel, K

Hendrycks, D. and Gimpel, K. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016

arXiv 2016
[8]

R., Ewart, A., and Sharkey, L

Huben, R., Cunningham, H., Smith, L. R., Ewart, A., and Sharkey, L. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, 2023

2023

Show all 19 references

[9]

On a formula for the product-moment coefficient of any order of a normal frequency distribution in any number of variables

Isserlis, L. On a formula for the product-moment coefficient of any order of a normal frequency distribution in any number of variables. Biometrika, 12 0 (1/2): 0 134--139, 1918

1918
[10]

Gradient-based learning applied to document recognition

LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 0 (11): 0 2278--2324, 1998

1998
[11]

and Black, S

Millidge, B. and Black, S. The singular value decompositions of transformer weight matrices are highly interpretable. In AI Alignment Forum, pp.\ 17, 2022

2022
[12]

and Hinton, G

Mnih, A. and Hinton, G. Three new graphical models for statistical language modelling. In Proceedings of the 24th international conference on Machine learning, pp.\ 641--648, 2007

2007
[13]

L., Zhang, F., and Barak, B

Nakkiran, P., Kaplun, G., Kalimeris, D., Yang, T., Edelman, B. L., Zhang, F., and Barak, B. Sgd on neural networks learns functions of increasing complexity. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp.\ 3496--3506, 2019

2019
[14]

Owen, D. B. A table of normal integrals: A table. Communications in Statistics-Simulation and Computation, 9 0 (4): 0 389--419, 1980

1980
[15]

Automatically interpreting millions of features in large language models

Paulo, G., Mallen, A., Juang, C., and Belrose, N. Automatically interpreting millions of features in large language models. arXiv preprint arXiv:2410.13928, 2024

2024 arXiv
[16]

T., Dooms, T., Rigg, A., Oramas, J

Pearce, M. T., Dooms, T., Rigg, A., Oramas, J. M., and Sharkey, L. Bilinear mlps enable weight-based mechanistic interpretability. arXiv preprint arXiv:2410.08417, 2024

2024 arXiv
[17]

Neural networks trained with sgd learn distributions of increasing complexity

Refinetti, M., Ingrosso, A., and Goldt, S. Neural networks trained with sgd learn distributions of increasing complexity. In International Conference on Machine Learning, pp.\ 28843--28863. PMLR, 2023

2023
[18]

Glu variants improve transformer

Shazeer, N. Glu variants improve transformer. arXiv preprint arXiv:2002.05202, 2020

2002 arXiv
[19]

M., Hartmann, V., and Abbasnejad, E

Teney, D., Nicolicioiu, A. M., Hartmann, V., and Abbasnejad, E. Neural redshift: Random networks are not random functions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 4786--4796, 2024

2024

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

Leace: Perfect linear concept erasure in closed form

Belrose, N., Schneider-Joseph, D., Ravfogel, S., Cotterell, R., Raff, E., and Biderman, S. Leace: Perfect linear concept erasure in closed form. arXiv preprint arXiv:2306.03819, 2023

arXiv 2023

[3] [3]

T., and Fern, X

Belrose, N., Pope, Q., Quirke, L., Mallen, A. T., and Fern, X. Neural networks learn statistics of increasing complexity. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=IGdpKP0N6w

work page 2024

[4] [4]

N., Fan, A., Auli, M., and Grangier, D

Dauphin, Y. N., Fan, A., Auli, M., and Grangier, D. Language modeling with gated convolutional networks. In International conference on machine learning, pp.\ 933--941. PMLR, 2017

2017

[5] [5]

A., Mehta, H., Mishchenko, K., Khaled, A., and Cutkosky, A

Defazio, A., Yang, X. A., Mehta, H., Mishchenko, K., Khaled, A., and Cutkosky, A. The road less scheduled. arXiv preprint arXiv:2405.15682, 2024

arXiv 2024

[6] [6]

Toy models of superposition

Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., et al. Toy models of superposition. arXiv preprint arXiv:2209.10652, 2022

arXiv 2022

[7] [7]

and Gimpel, K

Hendrycks, D. and Gimpel, K. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016

arXiv 2016

[8] [8]

R., Ewart, A., and Sharkey, L

Huben, R., Cunningham, H., Smith, L. R., Ewart, A., and Sharkey, L. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, 2023

2023