Pith. sign in

REVIEW 3 major objections 4 minor 56 references

Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Tensorized neural networks are an underused route to smaller, more interpretable models, this position paper argues.

desk verdict A clear, honest position paper that makes a plausible case for tensorized networks; the main weakness is that its load-bearing premise about weight structure is asserted rather than tested. read the letter →

arxiv 2505.20132 v1 pith:BJA5CWKW submitted 2025-05-26 cs.LG cs.AIquant-ph

Safa Hamreras , Sukhbinder Singh , Román Orús This is my paper
classification cs.LGcs.AIquant-ph MSC 68T0715A69
keywords tensorizedneuralnetworkstensormodelcompressionmechanisticinterpretabilitymatrixproductoperatorbondindicesinductivebiasscalinglaws
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that tensorized neural networks (TNNs), where dense weight matrices are reshaped into higher-order tensors and approximated by low-rank tensor-network decompositions, are a powerful yet underexplored framework for deep learning. The authors claim that TNNs deserve more attention from both engineering and theoretical communities, not just as compression tools but as a flexible class of architectures with distinct scaling properties and interpretability advantages. A central assertion is that the bond indices of a tensor network create new latent spaces absent in conventional networks, potentially offering finer-grained views of how features evolve across layers. If this perspective holds, TNNs could become a practical path to smaller, faster, and more trustworthy AI systems.

What carries the argument

The central object is the tensor-network decomposition of weight matrices: a dense matrix is reshaped into a higher-order tensor and then decomposed into contracted tensors using methods such as the Matrix Product Operator (tensor train), Tucker, or CP decompositions, typically via repeated singular value decompositions. The load-bearing degrees of freedom are the bond indices—the summed or internal indices of the tensor network—whose dimensions control the strength of correlations captured by the decomposition. The paper's key interpretive move is the stack view, which represents an MPO layer as a stack of sparse fully-connected layers, so that bond dimensions become the input and output dimensions of standard linear layers and the inductive bias becomes visible as structured sparsity.

What would settle it

On a large suite of modern pretrained models and standard benchmarks, compute the maximum compression ratio achievable by post-hoc tensorization at a fixed accuracy tolerance; if the achievable compression is consistently near 1, or accuracy collapses at any real compression, then the paper's central practical motivation is not borne out.

Watch

Extended reading notes

Core claim

The paper's central claim is that tensorization—replacing dense weight matrices with low-rank tensor-network decompositions—offers a versatile framework for deep learning that goes well beyond model compression. The authors argue that bond indices, the internal contracted indices of a tensor network, introduce new latent spaces that are not present in conventional dense networks, and that these bond features can be interpreted as a temporal resolution of a layer's output features, aiding mechanistic interpretability. They also propose a 'stack view' in which an MPO layer is equivalent to a sequence of sparse fully-connected layers, making the inductive bias of TNNs concrete as structured sparsity and opening new scaling directions such as bond inflation and inserting tensors into existing layers. The paper concludes that fully tensorized networks, where activations and nonlinearities also live in the tensor-network domain, could enable training and inference without dense intermediates.

Load-bearing premise

The load-bearing premise is that real-world data is structured and correlated, and that neural network weights trained on such data inherit enough of that structure that low-rank tensor-network decompositions can approximate them without significant performance loss.

Editorial extensions

If this is right

  • Tensorization can be combined with pruning, quantization, and knowledge distillation, and combining these strategies often yields higher overall compression than any single method alone.
  • TNNs offer flexible scaling directions—enlarging tensor dimensions, inserting new tensors, or inflating bond dimensions—that may enable incremental training and help mitigate catastrophic forgetting.
  • Bond feature spaces provide a temporal resolution of layer outputs, and studying them across equivalent stack representations of the same layer could yield new mechanistic interpretability tools.
  • Tensorized sparse autoencoders could offer finer views of how polysemantic features are disentangled, and tensor-network information measures such as von Neumann entropy could be used to rank or categorize data and weights.
  • Fully tensorized neural networks could keep activations in the tensor-network domain throughout training, avoiding dense intermediates and potentially reducing memory and compute costs.
  • A pretrained TNN can be deepened or widened without retraining the whole model, and bond inflation can be performed dynamically during training based on accuracy thresholds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If bond spaces indeed carry semantically meaningful features, existing interpretability probes designed for dense networks could be applied directly to bond indices, and one could test whether bond features align with human-labeled concepts across different tensor network gauge choices.
  • A testable corollary not explored in the paper is that post-hoc tensorization quality should correlate with a measurable property of the weights, such as their effective correlation length; measuring this could give a cheap screening tool for deciding which layers to tensorize.
  • The scaling discussion suggests a concrete research program: compare scaling laws of tensorized architectures against dense ones while varying bond dimension versus hidden width, to see whether bond dimension acts as a separate scaling axis.
  • The paper's 'fully tensorized' vision implies that local activation functions and normalization layers designed for tensor networks are a rate-limiting step; one could test whether trainable local nonlinearities, as in Kolmogorov-Arnold-style networks, stabilize training in deep tensorized models.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This position paper argues that tensorized neural networks (TNNs), in which dense weight matrices are reshaped into higher-order tensors and approximated by tensor-network decompositions such as MPO/Tensor Train, Tucker, and CP, are a powerful but underexplored framework for deep learning. The paper provides a tutorial-level review of the relevant tensor-network machinery, introduces a "stack view" of MPO layers that exposes bond indices as internal feature-carrying dimensions, and enumerates claimed benefits: a useful inductive bias, flexible scaling directions, post-hoc compression of pretrained models, potential forward/backward acceleration, and novel interpretability tools. It also lists practical obstacles (hardware/software support, unclear inductive bias, a large hyperparameter space, and difficult integration with quantization) and sketches a research agenda toward fully tensorized networks in which activations, weights, and nonlinearities all remain in tensor-network form.

Significance. If the paper's central thesis is accepted, it could redirect research effort toward tensor-network architectures as a route to more parameter-efficient and interpretable models. The manuscript's main strengths are its accurate and compact technical exposition of MPO/Tucker/CP decompositions, the explicit identification of open problems in Section 4, and the honest admission in Section 4, item 2, that post-hoc tensorization may fail if trained weights do not exhibit the presumed low-rank tensor structure. The "stack view" of MPO layers and the discussion of MPO gauge freedom are genuinely clarifying and could be useful to both theorists and practitioners. However, the paper is a position piece rather than a source of new evidence, and several of its most attention-grabbing claims, especially about interpretability and speedups, are speculative. The overall case is plausible and coherent, but it would be strengthened if the authors clearly separated established results from hypotheses.

major comments (3)
  1. [Section 1 and Section 3 (first subsection)] The load-bearing premise that trained neural-network weights inherit low-rank tensor structure from structured real-world data is asserted rather than demonstrated. The citations given for this premise, [7] and [8], concern correlations in data streams and clinical survival data, not weight matrices, while [39] concerns gradient-feature alignment rather than MPO/Tucker low-rankness. Since the paper itself concedes in Section 4, item 2, that post-hoc tensorization may fail if such structure does not emerge during training, I recommend that the sentence "It is therefore reasonable to expect that the weights and activations ... would inherit similar structure" be explicitly framed as an open hypothesis, and that the authors cite direct measurements of tensor-rank structure in trained weights where they exist, such as [28] for convolutional kernels.
  2. [Section 3 (paragraph beginning "TNNs offer a richer interpretability space") and Figure 4] The claim that bond indices introduce "new latent spaces not found in conventional networks" and that intermediate bond features provide a "temporal resolution" of a layer's output is presented as a property of TNNs rather than as a research hypothesis. No experiment or external reference in the paper demonstrates that these bond features are semantically meaningful or correspond to task-relevant representations. Moreover, the gauge freedom illustrated in Figure 4(iii) means that the same MPO layer has multiple equivalent stack representations, so the "temporal" progression of bond features is not canonical without additional gauge-fixing choices. The authors should soften this to a conjecture and explicitly discuss gauge ambiguity as a challenge for interpretability.
  3. [Section 3 (Forward pass acceleration and Backward pass acceleration)] The text states that "in principle, TNNs offer the potential for substantial speedups in the forward pass" and that "when bond dimensions are small enough, TNNs could still offer faster backward passes in practice." These are plausible but unsupported by any benchmark or numerical comparison in the paper, and the preceding discussion correctly notes that the naive dense-reconstruction contraction sequence is always slower than a dense GEMM. For a position paper, such claims are acceptable only if clearly marked as open questions; as written, they risk being read as demonstrated advantages. Please rephrase these passages to indicate that actual speedups depend on contraction-path optimization, hardware, and bond dimensions, and remain to be established empirically.
minor comments (4)
  1. [Figure 1 caption] The caption says the matrix B is reshaped into an 8-index tensor with "4 output indices n1, n2, n3, n4, n5" but lists five output indices; the count should be corrected.
  2. [Section 3, first paragraph] The phrase "the weights and activations of neural networks trained on such data would inherit similar structure" extends the correlation premise to activations without any supporting argument or citation. Since the paper is about weight tensorization, restricting the claim to weights would be more defensible.
  3. [Section 3, interpretability subsection] The term "temporal resolution" is used to describe the sequence of bond-feature spaces in a stack view, but the notion of time is not defined. Replacing it with "sequential decomposition" or an explicit computational-order description would improve clarity.
  4. [References] Several of the key citations that support the strongest empirical claims, including [27], [28], [29], [34], and [50], are arXiv preprints from the authors' own research groups. This is acceptable for a position paper, but the evidence base would be more convincing if it also cited independent or peer-reviewed evaluations of tensorized compression.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: position paper relies on self-cited empirical support, but no derivation reduces to its inputs.

full rationale

This paper is a position paper and contains no fitted parameters, no quantitative predictions from calibrated inputs, and no formal derivation that could be equivalent to its assumptions by construction. The central empirical proposition—that pretrained neural-network weights often admit low-rank tensor-network decompositions—is supported by prior work, including same-author papers [27,28] and independent external works [24,30,31]; these are separate empirical results rather than restatements of this paper's definitions or assumptions. The paper's load-bearing premise that trained weights inherit the correlation structure of real-world data is explicitly framed as a reasonable expectation and is later acknowledged in Sec. 4 as an open risk, so it is a missing-evidence concern rather than a circular step. Self-citations are present (e.g., [27], [28], [29], [34], [50]), but they are not used as uniqueness theorems, nor do they smuggle in an ansatz that the paper then presents as derived. No equation in the paper equals its own input, and no renamed empirical result is presented as a prediction. Score 2 reflects minor self-citation in the evidence chain without actual circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper rests on several domain assumptions about data structure and weight statistics; none are fitted parameters. No new entities are postulated.

assumptions (3)
  • domain assumption Real-world data is highly structured and correlated
    Stated in Section 1 as the key motivation for expecting compressibility: 'real-world data is often highly structured and correlated. It is therefore reasonable to expect that the weights and activations ... inherit similar structure.' This is assumed, not demonstrated.
  • domain assumption Gradient descent tends to align the statistical structure of data, weights, and gradients
    Invoked in Section 3 to support the claim that trained weights will show tensor-friendly correlations, citing [39]. This is a hypothesis, not proven.
  • domain assumption MPO bond indices serve as meaningful feature-carrying channels
    Assumed in Section 3 (Interpretability) to argue that bond spaces develop task-relevant representations. This underpins the interpretability claims.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks." pith.science (2026). https://pith.science/paper/BJA5CWKW

@misc{pith2026250520132,
  author       = {Pith},
  title        = {Pith review of: Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BJA5CWKW}},
  note         = {Machine review of arXiv:2505.20132}
}
read the original abstract

Tensorizing a neural network involves reshaping some or all of its dense weight matrices into higher-order tensors and approximating them using low-rank tensor network decompositions. This technique has shown promise as a model compression strategy for large-scale neural networks. However, despite encouraging empirical results, tensorized neural networks (TNNs) remain underutilized in mainstream deep learning. In this position paper, we offer a perspective on both the potential and current limitations of TNNs. We argue that TNNs represent a powerful yet underexplored framework for deep learning--one that deserves greater attention from both engineering and theoretical communities. Beyond compression, we highlight the value of TNNs as a flexible class of architectures with distinctive scaling properties and increased interpretability. A central feature of TNNs is the presence of bond indices, which introduce new latent spaces not found in conventional networks. These internal representations may provide deeper insight into the evolution of features across layers, potentially advancing the goals of mechanistic interpretability. We conclude by outlining several key research directions aimed at overcoming the practical barriers to scaling and adopting TNNs in modern deep learning workflows.

Figures

Figures reproduced from arXiv: 2505.20132 by the authors.

Figure 1
Figure 1. (i) A MLP with three fully-connected layers A, B, C, depicted as rectangles, and two non-linear layers (here, ReLu and Tanh activation layers), depicted as wiggly lines. The network is tensorized as follows. (ii) First, each weight matrix (A, B and C) is reshaped into a higher-dimensional tensor. For example, the m × n matrix B can be reshaped into an 8-index tensor with 3 input indices m1, m2, m3 such that m1m2m3 =… view at source ↗
Figure 2
Figure 2. A tensor Ti1,i2,...,in is shown as a shape – like a circle or rectangle – with n lines extending from it, each line representing an index. Tensor Network.– A tensor network (TN) T is simply a set of arrays that can be contracted together – via e.g. einsum() – to obtain a dense array T. Conversely, T is represented or decomposed as the TN T . For example, consider a TN made of three tensors {P, Q, R} that are contrac… view at source ↗
Figure 3
Figure 3. Examples of tensor network layers that we consider in this paper. (i-ii) An MPO (or [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: (i) An MPO layer with 4 tensors. (ii) A “stack” view of the same MPO layer as a [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Forward pass of an MPO layer Tensorization offers a novel model compression approach, which can be seamlessly combined with traditional compression strategies.– Consider an MPO layer composed of three tensors, ai1j1χ, bχi2j2χ′ , and cχ′i3j3 . The total number of parame…
Figure 6
Figure 6. Figure 6: (i) Forward pass of activation X through a dense layer L producing Y = LX followed by non-linearity f resulting in Z = f(Y ). (ii) Corresponding operations through a fully tensorized neural network where inputs, weights, and output activations are also tensorized. Acti…
Figure 5
Figure 5. Figure 5: 3) Large and complex hyperparameter space.– TNNs introduce a wide hyperparameter space – network architecture, number of tensors, and bond dimensions. Selecting appropriate configurations is challenging and often done heuristically, as systematic exploration is costly.…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 35 canonical work pages

  1. [39]

    Mechanism for feature learning in neural networks and backpropagation-free machine learning models

    A. Radhakrishnan et al. "Mechanism for feature learning in neural networks and backpropagation-free machine learning models." Science 383.6690: 1461-1467, (2024)

    work page 2024

  2. [7]

    Analysing real world data streams with spatio-temporal correlations: Entropy vs. Pearson correlation

    M. Bermudez-Edo, P. Barnaghi, and K. Moessner. "Analysing real world data streams with spatio-temporal correlations: Entropy vs. Pearson correlation." Automation in Construction 88 (2018): 87-100

    work page 2018

  3. [8]

    Correlation analysis of invasive disease-free survival and overall survival in a real-world population of patients with HR+/HER2–early breast cancer

    S. L.,Graff et al. "Correlation analysis of invasive disease-free survival and overall survival in a real-world population of patients with HR+/HER2–early breast cancer." Cancer 131.7 (2025): e35817

    work page 2025

  4. [28]

    Tensor network compressibility of convolutional models

    S. Singh, S. S. Jahromi, and R. Orus. "Tensor network compressibility of convolutional models." arXiv:2403.14379 (2024)

    arXiv 2024

  5. [1]

    Scaling laws for neural language models

    J. Kaplan et al. "Scaling laws for neural language models." arXiv:2001.08361 (2020). 9However, naive local applications of standard activations can lead to unstable gradients and harm training—for instance, local ReLU may zero out useful negative components prematurely. 9

    arXiv 2020

  6. [2]

    A survey on model compression for large language models

    X. Zhu et al. "A survey on model compression for large language models." Transactions of the Association for Computational Linguistics 12 (2024): 1556-1577

    work page 2024

  7. [3]

    Can neural network memorization be localized?

    P. Maini et al. (2023). "Can neural network memorization be localized?." arXiv:2307.09542

    arXiv 2023

  8. [4]

    The super weight in large language models

    M. Yu et al. "The super weight in large language models." arXiv:2411.07191 (2024)

    arXiv 2024

Show all 56 references
  1. [5]

    Neuron shapley: Discovering the responsible neurons

    A. Ghorbani, and J. Y . Zou. "Neuron shapley: Discovering the responsible neurons." Advances in neural information processing systems 33 (2020): 5922-5932

    2020

  2. [6]

    DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

    V . Sanh et al. "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter." arXiv:1910.01108 (2019)

    2019 arXiv

  3. [9]

    Smoothquant: Accurate and efficient post-training quantization for large language models

    G. Xiao et al. "Smoothquant: Accurate and efficient post-training quantization for large language models." International Conference on Machine Learning. PMLR, 2023

    2023

  4. [10]

    Llm-qat: Data-free quantization aware training for large language models

    Z. Liu et al. "Llm-qat: Data-free quantization aware training for large language models." arXiv:2305.17888 (2023)

    2023 arXiv

  5. [11]

    A simple and effective pruning approach for large language models

    M. Sun et al. "A simple and effective pruning approach for large language models." arXiv:2306.11695 (2023)

    2023 arXiv

  6. [12]

    Llm-pruner: On the structural pruning of large language models

    X. Ma, G. Fang, and X. Wang. "Llm-pruner: On the structural pruning of large language models." Advances in neural information processing systems 36 (2023): 21702-21720

    2023

  7. [13]

    Distilling the Knowledge in a Neural Network

    G. Hinton. "Distilling the Knowledge in a Neural Network." arXiv:1503.02531 (2015)

    2015 arXiv

  8. [14]

    Model compression via distillation and quantization

    A. Polino, R. Pascanu, and D. Alistarh. "Model compression via distillation and quantization." arXiv:1802.05668 (2018)

    2018 arXiv

  9. [15]

    Combining weight pruning and knowledge distillation for cnn compression

    N. Aghli and E. Ribeiro. "Combining weight pruning and knowledge distillation for cnn compression." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021

    2021

  10. [16]

    A novel tensor decomposition-based efficient detector for low-altitude aerial objects with knowledge distillation scheme

    N. Zeng et al. "A novel tensor decomposition-based efficient detector for low-altitude aerial objects with knowledge distillation scheme." IEEE/CAA Journal of Automatica Sinica 11.2 (2024): 487-501

    2024

  11. [17]

    Density matrix formulation for quantum renormalization groups

    S. R. White (1992). "Density matrix formulation for quantum renormalization groups." Physical review letters, 69(19), 2863

    1992

  12. [18]

    Tensor networks for complex quantum systems

    R. Orús. (2019). "Tensor networks for complex quantum systems."" Nature Reviews Physics, 1(9), 538-550

    2019

  13. [19]

    Matrix product states and projected entangled pair

    J. I. Cirac; D. Pérez-García, N. Schuch, F. Verstraete. "Matrix product states and projected entangled pair." Rev. Mod. Phys. 9, 4, 045003 (2021)

    2021

  14. [20]

    Tensor networks meet neural networks: A survey and future perspectives

    M. Wang et al. "Tensor networks meet neural networks: A survey and future perspectives." arXiv:2302.09019 (2023)

    2023 arXiv

  15. [21]

    Efficient tree tensor network states (TTNS) for quantum chemistry: Generalizations of the density matrix renormalization group algorithm

    N. Nakatani and G. K. Chan. "Efficient tree tensor network states (TTNS) for quantum chemistry: Generalizations of the density matrix renormalization group algorithm." The Journal of chemical physics 138.13 (2013)

    2013

  16. [22]

    Tensor network factorizations: Relationships between brain structural connectomes and traits

    Z. Zhang et al. "Tensor network factorizations: Relationships between brain structural connectomes and traits." Neuroimage 197: 330-343, (2019). 10

    2019

  17. [23]

    Era of big data processing: A new approach via tensor networks and tensor decompositions

    A. Cichocki. "Era of big data processing: A new approach via tensor networks and tensor decompositions." arXiv:1403.2048, (2014)

    2014 arXiv

  18. [24]

    Tensorizing neural networks

    A. Novikov, D. Podoprikhin, A. Osokin, and D. P. Vetrov. "Tensorizing neural networks." Advances in neural information processing systems, 28, (2015)

    2015

  19. [25]

    Compressing convolutional neural networks with hierarchical Tucker-2 decomposition

    M. Gabor and R. Zdunek. "Compressing convolutional neural networks with hierarchical Tucker-2 decomposition." Applied Soft Computing 132: 109856 (2023)

    2023

  20. [26]

    Tensor rank learning in CP decomposition via convolutional neural network

    M. Zhou, Y . Liu, Z. Long, L. Chen, and C. Zhu. "Tensor rank learning in CP decomposition via convolutional neural network." Signal Processing: Image Communication, 73, 12-21, (2019)

    2019

  21. [27]

    Boosting defect detection in manufacturing using tensor convolutional neural networks

    P. Martin-Ramiro, U.S. de la Maza, S. Singh, and R. Orus. "Boosting defect detection in manufacturing using tensor convolutional neural networks." arXiv:2401.01373, (2023)

    2023 arXiv

  22. [29]

    CompactifAI: extreme compression of large language models using quantum-inspired tensor networks

    A. Tomut et al. "CompactifAI: extreme compression of large language models using quantum-inspired tensor networks." arXiv:2401.14109 (2024)

    2024 arXiv

  23. [30]

    A tensorized transformer for language modeling

    X. Ma et al. "A tensorized transformer for language modeling." Advances in neural information processing systems 32 (2019)

    2019

  24. [31]

    Tensorgpt: Efficient compression of the embedding layer in llms based on the tensor-train decomposition

    M. Xu, Y . Lei Xu, and D. P. Mandic. "Tensorgpt: Efficient compression of the embedding layer in llms based on the tensor-train decomposition." arXiv:2307.00526 (2023)

    2023 arXiv

  25. [32]

    Improving language understanding by generative pre-training

    A. Radford. "Improving language understanding by generative pre-training." (2018)

    2018

  26. [33]

    Asvd: Activation-aware singular value decomposition for compressing large language models

    Z. Yuan et al. "Asvd: Activation-aware singular value decomposition for compressing large language models." arXiv:2312.05821 (2023)

    2023 arXiv

  27. [34]

    Quantum large language models via tensor network disentanglers

    B. Aizpurua et al. "Quantum large language models via tensor network disentanglers." arXiv:2410.17397 (2024)

    2024 arXiv

  28. [35]

    Machine learning of inductive bias

    P. E. Utgoff. "Machine learning of inductive bias." V ol. 15. Springer Science & Business Media, (2012)

    2012

  29. [36]

    Inductive biases for deep learning of higher-level cognition

    A. Goyal and Y . Bengio. "Inductive biases for deep learning of higher-level cognition." Proceedings of the Royal Society A 478.2266: 20210068, (2022)

    2022

  30. [37]

    Geometric deep learning

    J. Masci et al. "Geometric deep learning." SIGGRAPH ASIA 2016 Courses, 1-50 (2016)

    2016

  31. [38]

    Learning with invariances in random features and kernel models

    S. Mei, T. Misiakiewicz, and A. Montanari. "Learning with invariances in random features and kernel models." Conference on Learning Theory. PMLR, (2021)

    2021

  32. [40]

    Incremental learning algorithms and applications

    A. Gepperth and B. Hammer. "Incremental learning algorithms and applications." European symposium on artificial neural networks (ESANN), (2016)

    2016

  33. [41]

    Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?

    Y . Tay et al. "Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?"" arXiv:2207.10551 (2022)

    2022 arXiv

  34. [42]

    Bayesian tensorized neural networks with automatic rank selection

    C. Hawkins and Z. Zhang. "Bayesian tensorized neural networks with automatic rank selection." Neurocomputing 453: 172-180, (2021)

    2021

  35. [43]

    Lightweight tensorized neural networks for hyperspectral image classification

    T.-Y . Ma et al. "Lightweight tensorized neural networks for hyperspectral image classification." IEEE Transactions on Geoscience and Remote Sensing 60: 1-16, (2022)

    2022

  36. [44]

    LLaMA: open and efficient foundation language models. arXiv

    H. Touvron et al. "LLaMA: open and efficient foundation language models. arXiv." arXiv:2302.13971 (2023)

    2023 arXiv

  37. [45]

    Speeding-up convolutional neural networks using fine-tuned cp-decomposition

    V . Lebedev et al. "Speeding-up convolutional neural networks using fine-tuned cp-decomposition." arXiv:1412.6553 (2014). 11

    2014 arXiv

  38. [46]

    On interpretability of artificial neural networks: A survey

    F.-L. Fan et al. "On interpretability of artificial neural networks: A survey." IEEE Transactions on Radiation and Plasma Medical Sciences 5.6: 741-760, (2021)

    2021

  39. [47]

    Sparse autoencoder

    A. Ng. "Sparse autoencoder." CS294A Lecture notes 72.2011: 1-19, (2011)

    2011

  40. [48]

    Sparse autoencoders find highly interpretable features in language models

    H. Cunningham et al. "Sparse autoencoders find highly interpretable features in language models." arXiv:2309.08600 (2023)

    2023 arXiv

  41. [49]

    Sparse autoencoders reveal universal feature spaces across large language models

    M. Lan et al. "Sparse autoencoders reveal universal feature spaces across large language models." arXiv:2410.06981 (2024)

    2024 arXiv

  42. [50]

    Tensor networks for explainable machine learning in cybersecurity

    B. Aizpurua, S. Palmer, and R. Orus. "Tensor networks for explainable machine learning in cybersecurity." arXiv:2401.00867 (2023)

    2023 arXiv

  43. [51]

    FPGA-based component-wise LSTM training accelerator for neural granger causality analysis

    C. Guo, Y . Chen, and Y . Fu. "FPGA-based component-wise LSTM training accelerator for neural granger causality analysis." Neurocomputing 615: 128871, (2025)

    2025

  44. [52]

    FlexCNN: An end-to-end framework for composing CNN accelerators on FPGA

    S. Basalama et al. "FlexCNN: An end-to-end framework for composing CNN accelerators on FPGA." ACM Transactions on Reconfigurable Technology and Systems 16.2: 1-32, (2023)

    2023

  45. [53]

    Neural architecture search survey: A hardware perspective

    K. T. Chitty-Venkata and A. K. Somani. "Neural architecture search survey: A hardware perspective." ACM Computing Surveys 55.4: 1-36, (2022)

    2022

  46. [54]

    Compression of deep neural networks based on quantized tensor decomposition to implement on reconfigurable hardware platforms

    A. Nekooei and S. Safari. "Compression of deep neural networks based on quantized tensor decomposition to implement on reconfigurable hardware platforms." Neural Networks 150: 350-363, (2022)

    2022

  47. [55]

    Successive randomized compression: A randomized algorithm for the compressed MPO-MPS product

    C. Camaño, E. N. Epperly, J. A. Tropp. "Successive randomized compression: A randomized algorithm for the compressed MPO-MPS product", arXiv:2504.06475 (2025)

    2025

  48. [56]

    Kan: Kolmogorov-arnold networks

    Z. Liu et al. "Kan: Kolmogorov-arnold networks." arXiv:2404.19756 (2024). 12

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.