Generalization Analysis for Deep Contrastive Representation Learning

Antoine Ledent; Cheng Yeaw Ku; Nong Minh Hieu; Yunwen Lei

REVIEW 3 major objections 4 minor 69 references

Generalization Analysis for Deep Contrastive Representation Learning

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims generalization bounds for deep contrastive learning that, up to logarithms, do not depend on the number of negative samples and avoid exponential depth dependence.

desk verdict Promising covering-number machinery for contrastive learning, but a missing 1/normalization in Dudley's integral makes every main theorem invalid as stated; a corrected resubmission would deserve serious review. read the letter →

arxiv 2412.12014 v2 pith:752OIZZG submitted 2024-12-16 stat.ML cs.LG

Nong Minh Hieu , Antoine Ledent , Yunwen Lei , Cheng Yeaw Ku This is my paper · ORCID

classification stat.MLcs.LG MSC 68T0768Q32

keywords contrastiverepresentationlearninggeneralizationboundsRademachercomplexitycoveringnumbersdeepneuralnetworkslossaugmentationsample

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that the generalization gap of deep contrastive representation learning—training an encoder to pull an anchor toward a positive sample and away from k negatives—can be bounded without a significant dependence on k: up to logarithmic factors, the bound is independent of the number of negative samples. It also aims to remove the exponential dependence on network depth that appears in the only previous bound with logarithmic k-dependence, replacing it with polynomial factors built from spectral norms and network size. The proof works by bounding covering numbers of the neural-network function class over an auxiliary dataset containing every individual vector in the training tuples, and then converting those covers into Rademacher-complexity bounds for the contrastive loss class. A companion parameter-counting bound scales with the total number of neurons, with no dependence on k at all. If the argument holds, contrastive learning with deep networks has sample-complexity behavior comparable to ordinary supervised deep learning.

What carries the argument

The machinery is the empirical covering number of the network class in the uniform vector-valued metric $\|f-g\|_{L_{\infty,2}(S)} = \max_{x\in S}\|f(x)-g(x)\|_2$, measured on the auxiliary dataset $S_2$ of all individual vectors appearing in the contrastive tuples. Covering-number bounds for regularized linear layers are composed layer by layer, then propagated through the bilinear inner-product interaction $f(x)^\top(f(x^+)-f(x^-))$, which is where the extra product of spectral norms enters; the $\ell^\infty$-Lipschitz property of the loss converts these into covers of the loss class. Dudley's entropy integral then bounds the Rademacher complexity, and loss-augmented soft indicators let the bounds substitute data-dependent quantities such as the empirical output norm or layer-wise activation bounds for worst-case products.

What would settle it

Recompute the prefactor in the paper's Dudley entropy integral (Theorem B.1) and compare with its use in the proof of Theorem 4: the theorem states $12\sqrt{n}$, while the proof substitutes $12\sqrt{W/n}$. If $12\sqrt{n}$ is the correct statement, the Rademacher bound diverges with sample size; if the intended factor is $12/\sqrt{n}$, the bounds are sane. A reader can settle this without training anything.

Watch

Extended reading notes

Core claim

The central claim is that for $\ell^\infty$-Lipschitz contrastive losses, including the hinge, logistic, and InfoNCE losses, the population-minus-empirical unsupervised risk of a deep network can be bounded as $\widetilde{O}\bigl(\eta B_x^2\sqrt{n}\,\log(W)\,\prod_{m=1}^L \rho_m^2 s_m^2 \,[\sum_{\ell=1}^L (a_\ell/s_\ell)^{2/3}]^{3/2}\bigr)$ plus the usual $O(M\sqrt{\log(1/\delta)/n})$ confidence term, with only logarithmic dependence on k. With loss augmentation, the product of squared spectral norms is replaced by a single product times an empirical output-norm bound $R$, or by intermediate activation bounds $b_\ell$, and a separate parameter-counting bound gives $O(M\sqrt{(W/n)\log(1+24\eta L n B_x^2 \prod_\ell \rho_\ell^2 s_\ell^2)})$. The paper states these results for its constrained network class $\mathcal{F}_A$ and argues they transfer post hoc to trained networks. The qualitative punchline is that negative-sample count is not a driver of the generalization gap and depth enters polynomially rather than exponentially.

Load-bearing premise

The load-bearing premise is a standard covering-number-to-Rademacher bound; as printed it carries a factor $12\sqrt{n}$, and if that factor is genuine rather than a typo, the main bounds grow with the sample size and the argument collapses.

Editorial extensions

If this is right

Larger numbers of negative samples do not degrade the predicted generalization gap; only logarithmic factors hide k, so the theory aligns with empirical findings that large batches of negatives help or do not hurt.
Depth enters with polynomial rather than exponential dependence, so the bounds are meaningful for the deep architectures used in practice.
Loss augmentation lets the bound adapt to the trained network's observed output and activation norms, so the worst-case product over layers can be replaced by measured quantities.
The same bounds transfer to the downstream average supervised risk of the mean classifier, via the unsupervised-risk-to-supervised-risk lemma.
For small networks and many negatives, the parameter-counting bound offers a k-free control that scales only with total neuron count.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the k-independence holds more generally, it suggests batch size in contrastive methods can be chosen for optimization and representation quality rather than out-of-sample generalization.
The same composition of per-layer covering numbers may extend to convolutional, graph, and residual architectures, since only per-layer Lipschitz and norm controls are used.
The printed prefactor inconsistency in the Dudley integral is the decisive check: with $12\sqrt{n}$ the bounds would grow with sample size, so a corrected $12/\sqrt{n}$ is necessary for the qualitative claims to stand.
A direct empirical test would train encoders at fixed capacity while varying k and depth, measuring validation risk after saturating training loss; the paper's bounds predict flat generalization gap in k beyond small values.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Bottom line: the paper's machinery is worth taking seriously—the auxiliary-dataset trick and the composition lemma genuinely circumvent the k-dependence and the peeling-based depth explosion from Lei et al.—but the main theorems as written are not generalization bounds. The reason is a normalization error in Theorem B.1, and it is load-bearing.

What is new and good: the construction puts the bilinear contrastive loss into a covering-number box, and the bounds reduce k to logarithmic dependence and replace Frobenius-norm products with a spectral-type term. The loss augmentation in Theorems 2 and 3 is a plausible route to data-dependent complexity. The parameter-counting theorem is standard but cleanly transported to this setting. The experimental section is illustrative—it compares bound magnitudes, not real generalization—but it is honestly presented as such.

The flaw: Theorem B.1 states Dudley's integral with a prefactor 12√n. Under the paper's own definition, the empirical Rademacher complexity is a 1/n average, and the covering number uses the normalized L2(S) metric, so the correct prefactor is 12/√n. This is not a typo in one line: the √n appears in Theorems 1, 2, 3, and 4, so all bounds grow with n and cannot vanish. The proof of Theorem 4 shows the intended scaling: it uses 12√(W/n), which only follows from a 12/√n prefactor. This internal contradiction confirms the stress-test note. With 12/√n, the bounds behave as ~√(log n / n), which is the plausible outcome.

A smaller issue: the soft-indicator λ in equation (10) is written as a ramp with threshold at 0, but the augmentation arguments in Section E effectively treat it as a threshold at −γ in places; the mismatch is patchable but should be cleaned up. I did not find circular reasoning: the covering-number lemmas from the authors' own prior work are reproduced in the appendix, and no step assumes the desired conclusion.

Recommendation: this deserves a serious referee, but not as is. The right desk action is to reject with an invitation to correct the prefactor and resubmit. If you receive a revision with 12/√n and the soft-indicator fixed, send it out; the corrected results would be a solid contribution to contrastive learning theory.

Referee Report

3 major / 4 minor

Summary. The paper derives generalization bounds for the unsupervised risk in deep contrastive representation learning. The main results (Theorems 1-4) claim bounds that, up to logarithmic factors, are independent of the number of negative samples k, and that avoid the exponential depth dependence of Lei et al. (2023) by using covering-number arguments. The proofs construct auxiliary datasets, bound covering numbers of neural network classes, and then apply Dudley's entropy integral and Rademacher complexity arguments. The paper also includes loss-augmentation variants and a parameter-counting bound, plus experiments on MNIST comparing the bounds to previous work.

Significance. If the stated bounds were correct, the paper would provide a valuable improvement over existing contrastive-learning generalization analyses: k-independence up to logs, polynomial-type depth dependence, and a parameter-counting bound with only logarithmic dependence on network size. The auxiliary-dataset construction (Appendix D) and the loss-augmentation schemes (Section 5, Appendices E) are genuinely interesting technical ideas. However, the main theorems as stated are not valid generalization bounds because they grow with the sample size n; this is a load-bearing error that currently invalidates the central claim. The intended rates appear plausible after a normalization correction, so the underlying approach is promising, but the manuscript cannot be accepted in its present form.

major comments (3)

[Appendix B, Theorem B.1 (Eq. B.4)] The stated Dudley entropy integral has a prefactor of 12√n rather than 12/√n. Even with the definition of Rademacher complexity in Eq. (B.1), which includes the factor 1/n, the correct prefactor for a bounded real-valued class is 12/√n. As stated, the right-hand side is O(√n log n) after choosing α=1/n and grows with n, so it cannot serve as a generalization bound. This is not a cosmetic constant error: it is inherited by every main theorem.
[Theorems 1-4 and Table 1] The 12√n prefactor propagates into Eq. (8), Eq. (14), Eq. (17), and Eq. (18), all of which display a √n factor outside the logarithmic terms. For a fixed input distribution and fixed function class, these bounds diverge as n grows, contradicting the usual notion of a generalization bound. Table 1 repeats the same erroneous √n factor. The claim of k-independence (Section 4) is downstream of this error and is moot until the prefactor is corrected.
[Proof of Theorem 4 (Appendix F.2)] The proof of Theorem 4 is internally inconsistent with the stated Theorem B.1. After applying 'Dudley's entropy integral,' the derivation uses the expression 12√(W/n), which corresponds to the correct 12/√n prefactor multiplied by √W, rather than the stated 12√n. This confirms that the prefactor in Theorem B.1 is a typographical or normalization error, but the inconsistency means that the technical content of the paper does not currently establish the displayed bounds.

minor comments (4)

[Abstract and Section 1] The abstract claims 'polynomial depth dependence,' but Theorems 2 and 3 still contain products of spectral norms over all layers, which can grow exponentially in L; the claim should be qualified or clarified.
[Throughout] The paper overloads the symbol W to denote both the maximum width (Section 2) and the total number of neurons (Theorem 4). This is confusing and should be disambiguated.
[Section 3, Definition 2] There is a typo in the phrase 'ℓ∞-Lipscthiz' in the problem formulation; it should read 'ℓ∞-Lipschitz.'
[Appendix E, Proposition 6] In the display after Eq. (E.2), the argument of the logarithm contains 110ηR^2, but the subsequent derivation and the final covering-number bound use 110ηR. This appears to be a typo and should be checked.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the bounds follow from covering-number lemmas that are either proved in the appendix or cited to independent external work; the only flagged issue (the 12√n prefactor in Theorem B.1) is a correctness concern, not a circular derivation.

full rationale

The derivation starts from the definition of the loss class G, bounds its L2 covering number through the auxiliary classes H and FA (Propositions 5 and 4), and then invokes Dudley's entropy integral (Theorem B.1) followed by the standard Rademacher bound. The key covering-number lemmas are either proved in the appendices (Proposition 3 is proved after being attributed to Ledent et al. 2021b; Lemma 7 is proved after being attributed to Ledent et al. 2021b; Lemma 3 is proved with an alternative reference to Bartlett et al. 2017) or cited to genuinely external sources (Zhang 2002, Long and Sedghi 2020, Bartlett et al. 2017). Thus the self-citations are not load-bearing: the arguments are reproduced and the same tools exist in the external literature. The claim that the bounds are independent of k up to logarithmic factors is a consequence of the tilde-O convention hiding polylog terms that contain n(k+2)W; it is not an assumption smuggled into the bound. The loss augmentation arguments introduce empirical counts I_A,R and I_A,B as additive slack terms via standard soft-indicator constructions; these are not fitted parameters renamed as predictions. The one substantive issue in the manuscript is the stated prefactor 12√n in Theorem B.1, which would make the displayed bounds grow with n; however this is a normalization/correctness flaw, not a step that reduces the conclusion to its own inputs by construction or by self-citation. I therefore find no circular step.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

No free parameters are fitted to data; the quantities a_l, s_l, rho_l, R, and b_l are a priori constants defining the hypothesis class or augmentation margins. The analysis relies on standard background results in empirical process theory and neural network covering numbers. No new particles, forces, or entities are introduced.

assumptions (6)

standard math Dudley's entropy integral bound (Theorem B.1) relating empirical Rademacher complexity to covering numbers.
Used in every main proof to convert covering number estimates into Rademacher complexity. The stated prefactor is inconsistent with the standard 1/sqrt(n) form.
standard math Covering number bounds for linear classes (Propositions 2 and 3, from Zhang 2002 and Ledent et al. 2021b).
Used to bound the complexity of each linear layer in the network composition. The proof of Proposition 3 is reproduced in Appendix C.
standard math Rademacher complexity uniform bound (Proposition 1).
Used to translate Rademacher complexity into a generalization gap with high probability. Standard in learning theory.
domain assumption The input tuples are drawn i.i.d. from the population distribution with k negative samples per tuple.
Defines the contrastive learning setup and is required for the Rademacher complexity bound to imply a generalization bound.
domain assumption The loss function is bounded and Lipschitz with respect to the infinity norm.
The covering number composition for the loss class relies on the Lipschitz property and boundedness. The paper shows hinge and logistic losses satisfy eta = 1.
domain assumption Network weight matrices lie in balls defined by spectral norm bounds and (2,1)-norm deviations from reference matrices, with Lipschitz activations.
Defines the hypothesis class FA and is used in the layer-wise covering number composition and in the spectral norm bounds.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Generalization Analysis for Deep Contrastive Representation Learning." pith.science (2026). https://pith.science/paper/752OIZZG

@misc{pith2026241212014,
  author       = {Pith},
  title        = {Pith review of: Generalization Analysis for Deep Contrastive Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/752OIZZG}},
  note         = {Machine review of arXiv:2412.12014}
}

abstract

In this paper, we present generalization bounds for the unsupervised risk in the Deep Contrastive Representation Learning framework, which employs deep neural networks as representation functions. We approach this problem from two angles. On the one hand, we derive a parameter-counting bound that scales with the overall size of the neural networks. On the other hand, we provide a norm-based bound that scales with the norms of neural networks' weight matrices. Ignoring logarithmic factors, the bounds are independent of $k$, the size of the tuples provided for contrastive learning. To the best of our knowledge, this property is only shared by one other work, which employed a different proof strategy and suffers from very strong exponential dependence on the depth of the network which is due to a use of the peeling technique. Our results circumvent this by leveraging powerful results on covering numbers with respect to uniform norms over samples. In addition, we utilize loss augmentation techniques to further reduce the dependency on matrix norms and the implicit dependence on network depth. In fact, our techniques allow us to produce many bounds for the contrastive learning setting with similar architectural dependencies as in the study of the sample complexity of ordinary loss functions, thereby bridging the gap between the learning theories of contrastive learning and DNNs.

Figures

Figures reproduced from arXiv: 2412.12014 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 63 canonical work pages

[1]

Alves and A

R. Alves and A. Ledent. Context-aware representation: Jointly learning item features and selection from triplets. IEEE Transactions on Neural Networks and Learning Systems, 2024

work page 2024
[2]

Neural Network Learning - Theoretical Foundations

Anthony and Bartlett. Neural Network Learning - Theoretical Foundations. Cambridge University Press, 2002

work page 2002
[3]

A theoretical analysis of contrastive unsupervised representation learning

Arora, Khandeparkar, Khodak, Plevrakis, and Saunshi. A theoretical analysis of contrastive unsupervised representation learning. In International Conference on Machine Learning, 2019

work page 2019
[4]

Awasthi, N

P. Awasthi, N. Dikkala, and P. Kamath. Do more negative samples necessarily hurt in contrastive learning? In International Conference on Machine Learning, 2022

work page 2022
[5]

H. Bao, Y. Nagano, and K. Nozawa. On the surrogate gap between contrastive and supervised losses. In International Conference on Machine Learning, pages 1585--1606. PMLR, 2022

work page 2022
[6]

Advances in Kernel Method - Support Vector Learning

Bartlett and Shawe-taylor. Advances in Kernel Method - Support Vector Learning. MIT Press, 1999

work page 1999
[7]

Spectrally-normalized margin bounds for neural networks

Bartlett, Foster, and Telgarsky. Spectrally-normalized margin bounds for neural networks. In Advances in Neural Information Processing Systems, 2017

work page 2017
[8]

P. L. Bartlett, N. Harvey, C. Liaw, and A. Mehrabian. Nearly-tight vc-dimension and pseudodimension bounds for piecewise linear neural networks. Journal of Machine Learning Research, 20 0 (63): 0 1--17, 2019

work page 2019

Show all 69 references

[9]

T. Chen, S. Kornblith, M. Norouz, and G. Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, 2020

2020
[10]

Chuang, J

C.-Y. Chuang, J. Robinson, Y.-C. Lin, A. Torralba, and S. Jegelka. Debiased contrastive learning. Advances in Neural Information Processing Systems, 33: 0 8765--8775, 2020

2020
[11]

J. Deng, J. Guo, J. Yang, N. Xue, I. Kotsia, and S. Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Conference on Computer Vision and Pattern Recognition, 2018

2018
[12]

Eldele, M

E. Eldele, M. Ragab, Z. Chen, M. Wu, C. K. Kwoh, X. Li, and C. Guan. Time-series representation learning via temporal and contextual contrasting. International Joint Conference on Artificial Intelligence, 2021

2021
[13]

Galanti, M

T. Galanti, M. Xu, L. Galanti, and T. Poggio. Norm-based generalization bounds for sparse neural networks. Advances in Neural Information Processing Systems, 36, 2024

2024
[14]

T. Gao, X. Yao, and D. Chen. Simcse: Simple contrastive learning of sentence embeddings. In Conference on Empirical Methods in Natural Language Processing, 2021

2021
[15]

Gidaris, P

S. Gidaris, P. Singh, and N. Komodakis. Unsupervised representation learning by predicting image rotation. In International Conference on Learning Representation, 2018

2018
[16]

Golowich, A

N. Golowich, A. Rakhlin, and O. Shamir. Size-independent sample complexity of neural networks. In Conference On Learning Theory, pages 297--299. PMLR, 2018

2018
[17]

F. Graf, S. Zeng, B. Rieck, M. Niethammer, and R. Kwitt. On measuring excess capacity in neural networks. In Advances in Neural Information Processing Systems, 2022

2022
[18]

J. Z. HaoChen, C. Wei, A. Gaidon, and T. Ma. Provable guarantees for self-supervised deep learning with spectral contrastive loss. Advances in Neural Information Processing Systems, 34: 0 5000--5011, 2021

2021
[19]

C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. F. del Río, M. Wiebe, P. Peterson, P. Gérard-Marchant, K. Sheppard, T....

2020 doi
[20]

Hassani and A

K. Hassani and A. H. Khasahmadi. Contrastive multi-view representation learning on graphs. In Advances in Neural Information Processing Systems, 2020

2020
[21]

K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning. In Computer Vision and Pattern Recognition, 2019

2019
[22]

O. J. Henaff, A. Srinivas, J. D. Fauw, A. Razavi, C. Doersch, S. M. A. Eslami, and A. van den Oord. Data-efficient image recognition with contrastive predictive coding. In International Conference on Machine Learning, 2020

2020
[23]

R. A. Horn. Matrix Analysis. Cambridge University Press, 2012

2012
[24]

D. Hsu, Z. Ji, M. Telgarsky, and L. Wang. Generalization bounds via distillation. In International Conference on Learning Representations, 2021

2021
[25]

J. D. Hunter. Matplotlib: A 2d graphics environment. Computing in Science & Engineering, 9 0 (3): 0 90--95, 2007. doi:10.1109/MCSE.2007.55

2007 doi
[26]

Khosla, P

P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan. Supervised contrastive learning. In Advances in Neural Information Processing Systems, 2020

2020
[27]

LeCun, C

Y. LeCun, C. Cortes, and C. Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist, 2, 2010

2010
[28]

Ledent and R

A. Ledent and R. Alves. Generalization analysis of deep non-linear matrix completion. In Proceedings of the 41st International Conference on Machine Learning, volume 235, pages 26290--26360, 2024

2024
[29]

Ledent, R

A. Ledent, R. Alves, Y. Lei, and M. Kloft. Fine-grained generalization analysis of inductive matrix completion. In Advances in Neural Information Processing Systems, volume 34, pages 25540--25552, 2021 a

2021
[30]

Ledent, W

A. Ledent, W. Mustafa, Y. Lei, and M. Kloft. Norm-based generalisation bounds for deep multi-class convolutional neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021 b

2021
[31]

Ledoux and M

M. Ledoux and M. Talagrand. Probability in Banach Spaces. Springer Berlin, Heidelberg, 2011

2011
[32]

S. Lee, T. Park, and K. Lee. Soft contrastive learning for time series. In International Conference on Learning Representation, 2024

2024
[33]

Lei, \"U

Y. Lei, \"U . Dogan, D.-X. Zhou, and M. Kloft. Data-dependent generalization bounds for multi-class classification. IEEE Transactions on Information Theory, 65 0 (5): 0 2995--3021, 2019

2019
[34]

Y. Lei, A. Ledent, and M. Kloft. Sharper generalization bounds for pairwise learning. Advances in Neural Information Processing Systems, 33: 0 21236--21246, 2020

2020
[35]

Y. Lei, M. Liu, and Y. Ying. Generalization guarantee of sgd for pairwise learning. Advances in Neural Information Processing Systems, 34: 0 21216--21228, 2021

2021
[36]

Y. Lei, T. Yang, Y. Ying, and D.-X. Zhou. Generalization analysis for contrastive representation learning. In International Conference on Machine Learning, pages 19200--19227, 2023

2023
[37]

S.-B. Lin, K. Wang, Y. Wang, and D.-X. Zhou. Universal consistency of deep convolutional neural networks. IEEE Transactions on Information Theory, 68 0 (7): 0 4610--4617, 2022

2022
[38]

Generalization bounds for deep convolutional neural networks

Long and Sedghi. Generalization bounds for deep convolutional neural networks. In International Conference on Learning Representations, 2020

2020
[39]

A. Maurer. A vector-contraction inequality for rademacher complexities. In Algorithmic Learning Theory, 2016

2016
[40]

Mitrovic, B

J. Mitrovic, B. McWilliams, J. C. Walker, L. H. Buesing, and C. Blundell. Representation learning via invariant causal mechanisms. In International Conference on Learning Representations, 2021

2021
[41]

Mohri, A

M. Mohri, A. Rostamizadeh, and A. Talwalkar. Foundations of Machine Learning. The MIT Press, 2018

2018
[42]

Mustafa, Y

W. Mustafa, Y. Lei, A. Ledent, and M. Kloft. Fine-grained generalization analysis of structured output prediction. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI). International Joint Conferences on Artificial Intelligence, 2021

2021
[43]

Nagarajan and J

V. Nagarajan and J. Z. Kolter. Deterministic pac-bayesian generalization bounds for deep networks via generalizing noise-resilience. In International Conference on Learning Representation, 2019

2019
[44]

Neyshabur, R

B. Neyshabur, R. Tomioka, and N. Srebro. Norm-based capacity control in neural networks. In Conference on Learning Theory, pages 1376--1401. PMLR, 2015

2015
[45]

Y. Nie1, N. H. Nguyen, P. Sinthong, and J. Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representation, 2023

2023
[46]

Nozawa, P

K. Nozawa, P. Germain, and B. Guedj. PAC -bayesian contrastive unsupervised representation learning. In Uncertainty in Artificial Intelligence, pages 21--30. PMLR, 2020

2020
[47]

Paszke, S

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. Pytorch: An imperative style, high-per...

2019
[48]

Maurey-Schwartz

Pisier. Remarques sur un r\'esultat non publi\'e de B. Maurey . S\'eminaire d'Analyse fonctionnelle (dit "Maurey-Schwartz"), pages 1--12, 1981. talk:5

1981
[49]

Reimers and I

N. Reimers and I. Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networkss. In Conference on Empirical Methods in Natural Language Processing, 2021

2021
[50]

Shamir and S

O. Shamir and S. Shalev-Shwartz. Matrix completion with the trace norm: Learning, bounding, and transducing. Journal of Machine Learning Research, 15: 0 3401--3423, 2014

2014
[51]

K. Sohn. Improved deep metric learning with multi-class n-pair loss objective. In Advances in Neural Information Processing Systems, 2016

2016
[52]

N. Srebro. Learning with Matrix Factorizations. PhD thesis, Massachusetts Institute of Technology, 2004

2004
[53]

Srebro, K

N. Srebro, K. Sridharan, and A. Tewari. Smoothness, low noise and fast rates. Advances in Neural Information Processing Systems, 23, 2010

2010
[54]

Y. Tian, D. Krishnan, and P. Isola. Contrastive multiview coding. In European Conference on Computer Vision, 2020 a

2020
[55]

Y. Tian, L. Yu, X. Chen, and S. Ganguli. Understanding self-supervised learning with dual deep networks. arXiv preprint arXiv:2010.00578, 2020 b

2010 arXiv
[56]

Y.-H. H. Tsai, Y. Wu, R. Salakhutdinov, and L.-P. Morency. Demystifying self-supervised learning: An information-theoretical framework. arXiv preprint arXiv:2006.05576, 2, 2020

2006 arXiv
[57]

van den Oord, Y

A. van den Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. In Advances in Neural Information Processing Systems, 2018

2018
[58]

Velickovic, W

P. Velickovic, W. Fedus, W. L. Hamilton, P. Lio, Y. Bengio, and R. D. Hjelm. Deep graph informax. In International Conference on Learning Representation, 2019

2019
[59]

Data-dependent sample complexity of deep neural networks via lipschitz augmentation

Wei and Ma. Data-dependent sample complexity of deep neural networks via lipschitz augmentation. In Advances in Neural Information Processing Systems, 2019

2019
[60]

W. Wen, H. Li, T. Gong, and H. Chen. Towards sharper generalization bounds for adversarial contrastive learning. In K. Larson, editor, International Joint Conference on Artificial Intelligence, pages 5190--5198, 8 2024

2024
[61]

J. Wu, J. Chen, J. Wu, W. Shi, X. Wang, and X. He. Understanding contrastive learning via distributionally robust optimization. Advances in Neural Information Processing Systems, 36, 2024

2024
[62]

L. Wu, A. Ledent, Y. Lei, and M. Kloft. Fine-grained generalization analysis of vector-valued learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10338--10346, 2021

2021
[63]

X. Yang, Z. Zhang, and R. Cui. Timeclr: A self-supervised contrastive learning framework for univariate time series representation. Knowledge-Based Systems, 2022

2022
[64]

Z. Yang, Y. Lei, P. Wang, T. Yang, and Y. Ying. Simple stochastic and online gradient descent algorithms for pairwise learning. Advances in Neural Information Processing Systems, 34: 0 20160--20171, 2021

2021
[65]

Covering number bounds of certain regularized linear function classes

Zhang. Covering number bounds of certain regularized linear function classes. Journal of Machine Learning Research, 2002

2002
[66]

Zhang, S.-W

D. Zhang, S.-W. Li, W. Xiao, H. Zhu, R. Nallapati, A. O. Arnold, and B. Xiang. Pairwise supervised contrastive learning of sentence representations. In Conference on Empirical Methods in Natural Language Processing, 2021

2021
[67]

Zhou and X

T.-Y. Zhou and X. Huo. Learning ability of interpolating deep convolutional neural networks. Applied and Computational Harmonic Analysis, 68: 0 101582, 2024

2024
[68]

Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang. Deep graph contrastive representation learning. In ICML Workshop on Graph Representation Learning and Beyond, 2020

2020
[69]

Zou and W

X. Zou and W. Liu. Generalization bounds for adversarial contrastive learning. Journal of Machine Learning Research, 24 0 (114): 0 1--54, 2023

2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Alves and A

R. Alves and A. Ledent. Context-aware representation: Jointly learning item features and selection from triplets. IEEE Transactions on Neural Networks and Learning Systems, 2024

work page 2024

[2] [2]

Neural Network Learning - Theoretical Foundations

Anthony and Bartlett. Neural Network Learning - Theoretical Foundations. Cambridge University Press, 2002

work page 2002

[3] [3]

A theoretical analysis of contrastive unsupervised representation learning

Arora, Khandeparkar, Khodak, Plevrakis, and Saunshi. A theoretical analysis of contrastive unsupervised representation learning. In International Conference on Machine Learning, 2019

work page 2019

[4] [4]

Awasthi, N

P. Awasthi, N. Dikkala, and P. Kamath. Do more negative samples necessarily hurt in contrastive learning? In International Conference on Machine Learning, 2022

work page 2022

[5] [5]

H. Bao, Y. Nagano, and K. Nozawa. On the surrogate gap between contrastive and supervised losses. In International Conference on Machine Learning, pages 1585--1606. PMLR, 2022

work page 2022

[6] [6]

Advances in Kernel Method - Support Vector Learning

Bartlett and Shawe-taylor. Advances in Kernel Method - Support Vector Learning. MIT Press, 1999

work page 1999

[7] [7]

Spectrally-normalized margin bounds for neural networks

Bartlett, Foster, and Telgarsky. Spectrally-normalized margin bounds for neural networks. In Advances in Neural Information Processing Systems, 2017

work page 2017

[8] [8]

P. L. Bartlett, N. Harvey, C. Liaw, and A. Mehrabian. Nearly-tight vc-dimension and pseudodimension bounds for piecewise linear neural networks. Journal of Machine Learning Research, 20 0 (63): 0 1--17, 2019

work page 2019