Smaller Models, Better Generalization

Abhimanyu Dubey; Jayadeva; Mayank Sharma; Nihal Goalla; Sai Guruju; Suraj Tripathi

REVIEW 5 major objections 6 minor 43 references

Smaller Models, Better Generalization

T0 review · 5 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that minimizing a data-dependent regularizer derived from an upper bound on VC dimension yields sparser neural networks that generalize at least as well as dense models, and that quantization shrinks them further.

desk verdict The regularizer in Eq (15) is not a VC dimension bound, so the paper's central theoretical claim collapses; the experiments are extensive but do not rescue it. read the letter →

arxiv 1908.11250 v1 pith:SBWOXPFK submitted 2019-08-29 cs.LG stat.ML

Mayank Sharma , Suraj Tripathi , Abhimanyu Dubey , Jayadeva , Sai Guruju , Nihal Goalla This is my paper · ORCID

classification cs.LGstat.ML MSC 68T0768Q32

keywords VCdimensionnetworkpruningquantizationdata-dependentregularizationsparsitygeneralizationneuralnetworksCIFAR-10

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a training objective that treats network complexity as something to be minimized directly. It derives a data-dependent regularizer from an upper bound on the Vapnik-Chervonenkis (VC) dimension of a fat-margin classifier—a measure of how complex a function class is—and adds this regularizer to the hinge loss so that backpropagation can shrink the model while it learns. The central claim is that this objective produces sparser networks, with fewer non-zero weights and neurons, at accuracies comparable to dense models, and that quantizing the resulting weights can shrink models further and sometimes even improve accuracy. If this claim holds, it gives practitioners a direct route to training small, deployable networks for mobile and embedded settings rather than compressing an already-trained large model.

What carries the argument

The load-bearing object is the regularizer $\Gamma = \min\left(\sum_{i=1}^M \|w^T x_i + b\|^2 + C\|w\|^2\right)$, obtained from Vapnik's fat-margin bound $\gamma \le 1 + R^2/d_{\min}^2$ through a chain of relaxations that replace the radius-to-margin ratio $R/d$ with a convex expression in the weights and data. Applied to the classifier layer and to every hidden layer's pre-activations, it becomes a least-squares penalty pulling pre-activations toward zero, which under ReLU activations creates neuron-level sparsity. The quantization result (Theorem 1) uses the same $\Gamma$ to argue that a quantized classifier has a smaller VC bound when the magnitude of each weight does not increase under quantization.

What would settle it

Take a small linearly separable dataset in two dimensions whose points all have large norm but lie nearly parallel to the separating hyperplane, so that the maximum distance from a point to the plane is much smaller than the maximum point norm. Compute the minimizer of Eq. (15) on this data and compare the value of the regularizer against Vapnik's bound $1 + R^2/d_{\min}^2$: if the regularizer can be decreased while the VC bound increases, the claimed inequality is false. Alternatively, quantize a weight vector whose smallest component is near the rounding threshold and exhibit a sign flip, which would violate the premise of Theorem 1 and let $\|u_Q\|_2 > \|u\|_2$.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that minimizing the objective of Eq. (18)—a multiclass hinge loss combined with weight decay and the data-dependent regularizer $\Gamma$ of Eq. (15)—is a tractable way to control the VC dimension of a neural network during training. The authors argue that $\Gamma$, a sum of squared pre-activations (the linear inputs to each neuron) plus a scaled $\|w\|^2$ term, is a convex, differentiable upper bound on the VC dimension, and that applying it to hidden-layer pre-activations forces ReLU neurons toward zero, pruning synapses and neurons. They further claim (Theorem 1) that quantizing the trained weights reduces the VC bound provided quantization does not increase weight magnitudes or flip signs, which they connect to the empirical observation that quantized models often match or beat full-precision accuracy. The reported experiments on CNNs and fully connected networks are offered as evidence that the regularizer achieves higher compression ratios after pruning and comparable or better test accuracy than $L_1$ and $L_2$ regularization and dropout.

Load-bearing premise

The argument assumes that the regularizer $\Gamma$ is genuinely an upper bound on the VC dimension; specifically, that every data point's norm is no larger than its distance to the separating hyperplane, and that dropping the margin denominator while introducing a constant preserves the inequality.

Editorial extensions

If this is right

Networks trained with the regularizer can be pruned to high compression ratios (up to thousands-fold on some datasets in the reported tables) with little or no drop in test accuracy.
Quantizing the trained models yields accuracy comparable to or better than full precision, with the smallest models often performing best, supporting the idea that reduced precision acts as an additional regularizer.
Because the regularizer applies to any layer's pre-activations, it transfers across fully connected and convolutional architectures without architectural changes.
The method combines with existing regularizers—$L_1$, $L_2$, dropout—so it can be added to already-regularized training pipelines rather than replacing them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the VC-bound interpretation is set aside, the regularizer is simply a data-weighted shrinkage penalty on pre-activations, and its practical value can be tested independently of whether it truly bounds the VC dimension; a comparison against plain $L_2$ on pre-activations would settle how much the bound derivation adds.
The reported accuracy peak at intermediate bit widths suggests a testable mechanism: fixed-point rounding injects noise that may help the model escape sharp minima, a phenomenon that could be studied with loss-landscape analysis.
The paper's suggestion of per-layer hyperparameters for $C$ and $D$ is a natural extension, since a single global pair may over-regularize early layers and under-regularize the classifier.
One could test the claimed generalization benefit on data with controlled label noise or on out-of-distribution shifts, where a true complexity-control effect should show up as a larger gap between the regularized and unregularized models.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, here's my read of 1908.11250.

The paper's main claim is that it trains networks with a novel regularizer that minimizes an upper bound on VC dimension, yielding sparser models that generalize better. That theoretical claim does not hold up. The derivation in Section 3 has two fatal steps. Eq (11) asserts the max norm of data points is bounded by the max distance to the separating hyperplane, which is false in general (e.g., points at 9 and 11 with hyperplane x=10 give max norm 11 but max distance 1). Even if that were true, going from Eq (14) to Eq (15) drops the 1/||w||^2 denominator from the data term and replaces it with an arbitrary constant C. So Gamma is not an upper bound on VC dimension; it is just a regularizer. The paper itself only calls it a convex and differentiable version without proving the inequality is preserved. Theorem 1 also relies on the assumption that quantization never increases weight magnitude or flips signs, which is not guaranteed by fixed-point rounding.

What the paper does well: the empirical study is fairly extensive. They test the objective across multiple FNN and CNN architectures and datasets, and compare against L1/L2 and dropout. The final objective (multiclass hinge + L2 on weights + L2 on pre-activations) is a reasonable combination and could be useful as a heuristic. The writing is clear about the setup and notation.

The soft spots: the empirical results do not consistently support the superiority claim. In the compression tables, L2 alone often matches or exceeds LCA in compression ratio (e.g., Tables 10-12 show H+W2 and H+W2+LCA often have identical compression, meaning the data-dependent term adds nothing). The quantization curves in Section 5.3 select fraction bits based on the test set, which makes the reported robustness untrustworthy.

Overall: the central theoretical argument is unsupported and the empirical evidence is mixed. A reader looking for an off-the-shelf regularizer might try the objective, but the framing as a VC-dimension-driven method is misleading. I would not pass this to a serious referee in its current form; it needs either a correct derivation or a complete reframing as an empirical heuristic.

Referee Report

5 major / 6 minor

Summary. This manuscript proposes a data-dependent regularizer, denoted Γ in Eq. (15), derived from an upper bound on the VC dimension of a separating hyperplane. The regularizer is added to the objective for training CNNs and FNNs, and the paper further claims that quantizing weights reduces Γ, yielding smaller models that generalize better. The authors report pruning and quantization experiments on CIFAR-10, several LIBSVM datasets, and standard network architectures, comparing against L1/L2 regularization and dropout. The central theoretical claims are that Eq. (15) is a convex, differentiable upper bound on the VC dimension and that Theorem 1 establishes a quantization-induced decrease in VC complexity.

Significance. If the derivation were correct, the paper would provide a principled and easy-to-optimize regularizer for sparsity, an important practical goal. The experimental apparatus is broad: multiple datasets, architectures, and regularizers are compared, and the compression tables give a useful empirical snapshot. The paper also deserves credit for making the proposed loss simple enough to reuse. However, the main theoretical result is invalid at two load-bearing steps, and several experimental summaries do not support the claimed superiority. Consequently, the contribution as it stands does not establish a new complexity-control principle, and the practical value of the regularizer is left unconvincing.

major comments (5)

[Section 3, Eq. (11)] The inequality max_i ||x_i|| <= max_i |w^T x_i + b| / ||w|| is false in general. For example, with x1=9, x2=11, w=1, b=-10, the maximum norm is 11 while the maximum distance to the separating hyperplane x=10 is 1. Since Eq. (12) and the subsequent bound (14) rely on this inequality, the derivation of Γ as an upper bound on R^2/d_min^2 fails. This is not a minor numerical slip; it is the step that connects the data radius to the hyperplane geometry.
[Section 3, Eqs. (14)-(15)] The transition from R/d <= sum_i ||w^T x_i+b||^2 / ||w||^2 + ||w||^2 to Γ = sum_i ||w^T x_i+b||^2 + C||w||^2 drops the 1/||w||^2 factor and introduces an unconstrained constant C. The paper calls Γ the 'convex and differentiable version of the bound' but provides no proof that the inequality is preserved. For small ||w||, the true bound in (14) can be much larger than Γ, so minimizing Γ is not equivalent to minimizing the derived VC-bound expression. The central claim that the proposed loss minimizes a VC-dimension upper bound is therefore unsupported.
[Section 4, Theorem 1] The proof does not establish Γ_Q <= Γ. The condition (21), |u_j| >= |u^Q_j|, does not imply that |u^T x_i| >= |(u^Q)^T x_i| for each fixed x_i; componentwise shrinkage can increase the absolute value of a dot product (for example, u=(100,-1) and x=(1,99) give u·x=1, while shrinking the first component to 50 gives -49). The claim that (21) implies the sign condition (19), i.e., that the two classifiers make identical predictions, is also not proved. Since these implications are load-bearing for the quantization conclusion, Theorem 1 is not established.
[Section 5.3] The quantization protocol selects, for every total bit count, the number of fraction bits that yields the best test-set accuracy ('the number which amounted to best test set accuracies was selected'). This is a form of test-set selection, so the reported accuracy-versus-bits curves and the claim that the proposed regularizer is 'most robust' are not reliable evidence. The comparison should use a validation split or report all fraction-bit settings.
[Tables 3 and 6-12] The empirical summaries do not support the paper's superiority claim. In Table 3, plain L2 (S+W) achieves the highest compression ratio under pruning, and in Tables 6-8 the text itself states that L1 regularization has the best accuracies in most datasets; the data-dependent term usually matches rather than beats existing regularizers. The conclusion's phrase 'incontrovertibly demonstrate' is therefore contradicted by the manuscript's own tables.

minor comments (6)

[Section 3] The symbol γ is used both for the VC dimension in Eq. (1) and for the objective Γ in Eq. (15); using a different letter for the regularizer would avoid confusion.
[Section 4, Theorem 1] The weights are augmented from n to n+1 dimensions, but condition (21) is written 'for all j in {1,...,n}' while the preceding equations use n+1; the index range should be consistent.
[Figure 3] The caption says the dataset is 'dna', but the subfigure titles say 'dataset 7', which Table 9 identifies as 'protein'; the mismatch should be corrected.
[Section 5.3] The sentence in the text and the caption of Figure 3 describe different selection rules for fraction bits ('best test set accuracies' versus '1% error tolerance'); the protocol should be stated uniformly.
[Section 5.2.2] The text says results for CIFAR-100 and MNIST appear in the supplementary section, but no supplementary material is included; either add it or remove the reference.
[Section 3, Eq. (9)] Equation (9) is written for vectors but is applied to scalar norms; a scalar version with a short justification would be clearer.

Circularity Check

2 steps flagged · score 6.0 of 10

Theorem 1's 'smaller VC bound' is the regularizer's own decrease by definition; quantization curves are test-set-fitted.

self definitional [Section 4, Theorem 1 (Eqs. 15, 22–24)]
"Theorem 1. Consider full precision and a quantized fat margin classiﬁers with upper bounds on VC dimensions as Γ and ΓQ. ... then the quantized classiﬁer has smaller VC bound (ΓQ < Γ). ... From, eq. (15) where we deﬁne Γ = C‖u‖2 2 +‖uT xi‖2 2, analogous to it, the quantized counterpart can be deﬁned as ΓQ = C‖uQ‖2 2 +‖uQT xi‖2 2. Now, using eq. (22) and eq. (23), we have, ΓQ≤ Γ."

Eq. (15) introduces Γ as 'the convex and differentiable version of the bound on VC dimension'—i.e., the regularizer itself, not an independently established bound. The proof of Theorem 1 establishes only that ΓQ ≤ Γ whenever the quantized weights have no larger magnitude than the full-precision weights (Eqs. 22–23), which follows from the definition of Γ. The phrase 'smaller VC bound' therefore adds no information beyond 'the regularizer decreased.' Any conclusion about the hypothesis class's VC dimension is obtained by renaming the objective Γ as the VC bound, not by a separate theorem.
fitted input called prediction [Section 5.3, FNN: Quantization]
"For every value of total number of bits, the number of fraction bits were varied from 3 to 15 and the number which amounted to best test set accuracies was selected."

The quantization accuracy curves are presented as evidence about generalization ('We show the accuracy ... as the total number of bits are varied'), but the fraction-bit configuration is chosen on the test set itself. The reported accuracy at each bit count is therefore a selection result, not an out-of-sample prediction. The apparent robustness to quantization is partly forced by the test-set selection procedure, making the 'generalization' claim close to a fitted input.

full rationale

The paper's central theoretical claim is that minimizing Γ (Eq. 15) minimizes an upper bound on the VC dimension, yielding sparser and better-generalizing networks. That claim is not itself a circularity in the narrow sense of 'prediction equals input by construction': the earlier chain from Vapnik's bound to Eq. (14) is an attempted derivation, albeit one that breaks at Eq. (11) and at the transition from Eq. (14) to Eq. (15), where the 1/||w||^2 factor is dropped and replaced by an arbitrary constant C. Those are correctness defects, not circular steps, and I do not score them as circularity. The genuine circular step is Theorem 1: there Γ is simply the regularizer defined in Eq. (15), and the proof shows only that this regularizer decreases when quantized weights shrink; calling ΓQ < Γ a 'smaller VC bound' is a definitional renaming rather than an independent statement about the hypothesis class. A second circularity is in Section 5.3, where fraction bits are selected on the test set and then the resulting curves are reported as evidence of quantization robustness. The empirical comparisons to L1, L2, and dropout are independent content and not circular, even though the tables show L1 often matching or beating the proposed method in accuracy; that is a factual tension, not circularity. The self-citation to Jayadeva [18] is present, but the paper reproduces the derivation rather than relying on the citation as a black box, so I do not count it as load-bearing circularity. Overall, the main theoretical prediction reduces, at its decisive step, to a property of the objective's own definition, giving a score of 6.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim depends on Gamma being a valid upper bound on VC dimension, which is asserted rather than derived. The constants C and D are fitted to validation data, the pruning threshold is hand-chosen, and the quantization curves are fitted to the test set. The radius inequality in Eq (11) and the discarded normalization term are the two structural assumptions that, if false, invalidate the theory.

free parameters (4)

C = tuned in [1e-4, 1e-1] in multiples of 10
Global weight decay coefficient in Eq (18), tuned on validation accuracy.
D = tuned in [1e-8, 1e-4] in multiples of 10
Coefficient on the data dependent pre-activation penalty in Eq (18), tuned on validation accuracy.
Pruning threshold = 1e-3
Hand-chosen threshold for absolute weight magnitude below which weights are pruned; affects reported compression ratios.
Fraction bits in quantization = 3 to 15, selected per total bit count
For each total bit count, the number of fraction bits giving best test accuracy is selected (Section 5.3), fitting the quantization curves to the test set.

assumptions (5)

standard math Vapnik's fat margin VC bound (Eq 1)
Borrowed from Vapnik (1998) without proof; valid for linear classifiers but the n term is dropped using a linear separability assumption.
domain assumption Cover's theorem high-dimensional separability
Invoked to justify assuming linearly separable high-dimensional data when reducing the bound to R^2/d_min^2.
ad hoc to paper Eq (11) radius bound
Claims the maximum norm of data is bounded by the maximum distance to the separating hyperplane, which is false in general; used untested to simplify the bound.
ad hoc to paper Eq (15) as an upper bound
The step from Eq (14) to Eq (15) drops the 1/||w||^2 term and introduces C, so the objective is not derived as an upper bound on VC dimension.
ad hoc to paper Quantization magnitude condition (21)
Theorem 1 assumes quantization never increases weight magnitudes and does not flip signs, which is not guaranteed by fixed-point rounding.

invented entities (1)

Gamma, the 'data dependent VC dimension bound'
purpose: Acts as a regularizer claimed to control model complexity and induce sparsity.
Gamma in Eq (15) is asserted to be a convex, differentiable bound on VC dimension, but the derivation removes the 1/||w||^2 factor, so no valid connection to actual VC dimension is established; its efficacy is measured only by the paper's own experiments.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Smaller Models, Better Generalization." pith.science (2026). https://pith.science/paper/SBWOXPFK

@misc{pith2026190811250,
  author       = {Pith},
  title        = {Pith review of: Smaller Models, Better Generalization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SBWOXPFK}},
  note         = {Machine review of arXiv:1908.11250}
}

read the original abstract

Reducing network complexity has been a major research focus in recent years with the advent of mobile technology. Convolutional Neural Networks that perform various vision tasks without memory overhaul is the need of the hour. This paper focuses on qualitative and quantitative analysis of reducing the network complexity using an upper bound on the Vapnik-Chervonenkis dimension, pruning, and quantization. We observe a general trend in improvement of accuracies as we quantize the models. We propose a novel loss function that helps in achieving considerable sparsity at comparable accuracies to that of dense models. We compare various regularizations prevalent in the literature and show the superiority of our method in achieving sparser models that generalize well.

Figures

Figures reproduced from arXiv: 1908.11250 by the authors.

**Figure 2.** tSNE visualization of last layer in densenet [ [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Effect of quantization on accuracy, margin and loss function for 1,2 and 3 hidden layer FNN for dataset ’dna’. Here [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 25 canonical work pages

[1]

Abadi, A

M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, et al. Tensorﬂow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467, 2016. 4

arXiv 2016
[2]

Aghasi, N

A. Aghasi, N. Nguyen, and J. Romberg. Net-trim: A layer- wise convex pruning of deep neural networks.arXiv preprint arXiv:1611.05162, 2016. 2

arXiv 2016
[3]

Babaeizadeh, P

M. Babaeizadeh, P. Smaragdis, and R. H. Campbell. Noise- out: A simple way to prune neural networks. arXiv preprint arXiv:1611.06211, 2016. 2

arXiv 2016
[4]

Bahdanau, K

D. Bahdanau, K. Cho, and Y . Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. 1

arXiv 2014
[5]

Chang and C.-J

C.-C. Chang and C.-J. Lin. Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3):27, 2011. 6, 8

work page 2011
[6]

M. D. Collins and P. Kohli. Memory bounded deep convolu- tional networks. arXiv preprint arXiv:1412.1442, 2014. 1

arXiv 2014
[7]

T. M. Cover. Capacity problems for linear machines. pages 283–289, 1968. 2

work page 1968
[8]

A. Graves. Generating sequences with recurrent neural net- works. arXiv preprint arXiv:1308.0850, 2013. 1

arXiv 2013

Show all 43 references

[9]

M. U. Guide. The mathworks. Inc., Natick, MA, 5:333, 1998. 4

1998
[10]

S. Han, H. Mao, and W. J. Dally. Deep compres- sion: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, 2015. 2

2015 arXiv
[11]

S. Han, J. Pool, J. Tran, and W. Dally. Learning both weights and connections for efﬁcient neural network. In Advances in Neural Information Processing Systems , pages 1135–1143,
[12]

Hannun, C

A. Hannun, C. Case, J. Casper, B. Catanzaro, G. Diamos, E. Elsen, R. Prenger, S. Satheesh, S. Sengupta, A. Coates, et al. Deep speech: Scaling up end-to-end speech recogni- tion. arXiv preprint arXiv:1412.5567, 2014. 1

2014 arXiv
[13]

K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectiﬁers: Surpassing human-level performance on imagenet classiﬁcation. In Proceedings of the IEEE international con- ference on computer vision, pages 1026–1034, 2015. 1

2015
[14]

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learn- ing for image recognition. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 770–778, 2016. 1

2016
[15]

E. H. Huang, R. Socher, C. D. Manning, and A. Y . Ng. Im- proving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meet- ing of the Association for Computational Linguistics: Long Table 7: Accuracies for various methods fo...

2012
[16]

Huang, Z

G. Huang, Z. Liu, K. Q. Weinberger, and L. van der Maaten. Densely connected convolutional networks. arXiv preprint arXiv:1608.06993, 2016. 5, 7

2016 arXiv
[17]

Hubara, M

I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y . Bengio. Quantized neural networks: Training neural net- works with low precision weights and activations. arXiv Table 11: Compression ratios for various methods for 2 hid- den layer FNN H H+W2H+W1H+LCAH+W2+LCAH+W1+LCAa...
[18]

Learning a hyperplane classiﬁer by minimizing an exact bound on the vc dimensioni

Jayadeva. Learning a hyperplane classiﬁer by minimizing an exact bound on the vc dimensioni. NEUROCOMPUTING, 149:683–689, 2015. 2, 6

2015
[19]

Y . Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Gir- shick, S. Guadarrama, and T. Darrell. Caffe: Convolu- tional architecture for fast feature embedding. In Proceed- ings of the 22nd ACM international conference on Multime- dia, pages 675–678. ACM, 2014. 4, 5 2 4 6...

2014
[20]

Karpathy and L

A. Karpathy and L. Fei-Fei. Deep visual-semantic align- ments for generating image descriptions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3128–3137, 2015. 1

2015
[21]

Krizhevsky and G

A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. 2009. 5

2009
[22]

Krizhevsky, I

A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classiﬁcation with deep convolutional neural networks. In Advances in neural information processing systems , pages 1097–1105, 2012. 1

2012
[23]

LeCun, Y

Y . LeCun, Y . Bengio, and G. Hinton. Deep learning.Nature, 521(7553):436–444, 2015. 1

2015
[24]

LeCun, C

Y . LeCun, C. Cortes, and C. J. Burges. The mnist database of handwritten digits, 1998

1998
[25]

Z. Lin, M. Courbariaux, R. Memisevic, and Y . Bengio. Neural networks with few multiplications. arXiv preprint arXiv:1510.03009, 2015. 2

2015 arXiv
[26]

C. Liu, Z. Zhang, and D. Wang. Pruning deep neural net- works by optimal brain damage. In INTERSPEECH, pages 1092–1095, 2014. 2

2014
[27]

Mellempudi, A

N. Mellempudi, A. Kundu, D. Das, D. Mudigere, and B. Kaul. Mixed low-precision deep learning inference us- ing dynamic ﬁxed point. arXiv preprint arXiv:1701.08978,

arXiv
[28]

V . Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller. Play- ing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013. 1

2013 arXiv
[29]

Pennington, R

J. Pennington, R. Socher, and C. D. Manning. Glove: Global vectors for word representation. In EMNLP, volume 14, pages 1532–1543, 2014. 1

2014
[30]

Rastegari, V

M. Rastegari, V . Ordonez, J. Redmon, and A. Farhadi. Xnor- net: Imagenet classiﬁcation using binary convolutional neu- ral networks. In European Conference on Computer Vision, pages 525–542. Springer, 2016. 2

2016
[31]

Scardapane, D

S. Scardapane, D. Comminiello, A. Hussain, and A. Uncini. Group sparse regularization for deep neural networks. arXiv preprint arXiv:1607.00485, 2016. 2

2016 arXiv
[32]

Schmidhuber

J. Schmidhuber. Deep learning in neural networks: An overview. Neural networks, 61:85–117, 2015. 1

2015
[33]

Silver, A

D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V . Panneershelvam, M. Lanctot, et al. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484–489, 2016. 1

2016
[34]

Simonyan and A

K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. 1

2014 arXiv
[35]

Srinivas and R

S. Srinivas and R. V . Babu. Data-free parameter pruning for deep neural networks. arXiv preprint arXiv:1507.06149,

arXiv
[36]

Srinivas, A

S. Srinivas, A. Subramanya, and R. V . Babu. Training sparse neural networks. arXiv preprint arXiv:1611.06694, 2016. 2

2016 arXiv
[37]

R. K. Srivastava, K. Greff, and J. Schmidhuber. Training very deep networks. In Advances in neural information pro- cessing systems, pages 2377–2385, 2015. 1

2015
[38]

Y . Sun, X. Wang, and X. Tang. Sparsifying neural network connections for face recognition. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 4856–4864, 2016. 2

2016
[39]

Sutskever, O

I. Sutskever, O. Vinyals, and Q. V . Le. Sequence to sequence learning with neural networks. In Advances in neural infor- mation processing systems, pages 3104–3112, 2014. 1

2014
[40]

Y . Tang. Deep learning using linear support vector machines. arXiv preprint arXiv:1306.0239, 2013. 3

2013 arXiv
[41]

V . Vapnik. Statistical learning theory. Wiley, 1998. 2

1998
[42]

Wolfe, A

N. Wolfe, A. Sharma, L. Drude, and B. Raj. The incredi- ble shrinking neural network: New perspectives on learning representations through the lens of pruning. arXiv preprint arXiv:1701.04465, 2017. 2

2017 arXiv
[43]

H. Zhou, J. M. Alvarez, and F. Porikli. Less is more: Towards compact cnns. In European Conference on Computer Vision, pages 662–677. Springer, 2016. 2

2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Abadi, A

M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, et al. Tensorﬂow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467, 2016. 4

arXiv 2016

[2] [2]

Aghasi, N

A. Aghasi, N. Nguyen, and J. Romberg. Net-trim: A layer- wise convex pruning of deep neural networks.arXiv preprint arXiv:1611.05162, 2016. 2

arXiv 2016

[3] [3]

Babaeizadeh, P

M. Babaeizadeh, P. Smaragdis, and R. H. Campbell. Noise- out: A simple way to prune neural networks. arXiv preprint arXiv:1611.06211, 2016. 2

arXiv 2016

[4] [4]

Bahdanau, K

D. Bahdanau, K. Cho, and Y . Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. 1

arXiv 2014

[5] [5]

Chang and C.-J

C.-C. Chang and C.-J. Lin. Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3):27, 2011. 6, 8

work page 2011

[6] [6]

M. D. Collins and P. Kohli. Memory bounded deep convolu- tional networks. arXiv preprint arXiv:1412.1442, 2014. 1

arXiv 2014

[7] [7]

T. M. Cover. Capacity problems for linear machines. pages 283–289, 1968. 2

work page 1968

[8] [8]

A. Graves. Generating sequences with recurrent neural net- works. arXiv preprint arXiv:1308.0850, 2013. 1

arXiv 2013