Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Pretraining with random noise for uncertainty calibration

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Random-noise pretraining can calibrate neural-network confidence by driving untrained networks to chance-level output, aligning confidence with accuracy during later training.

desk verdict A real, reproducible calibration effect on small MLPs, but the paper never compares to temperature scaling or other standard baselines, so the headline claim is unproven. read the letter →

arxiv 2412.17411 v2 pith:LWR33LOI submitted 2024-12-23 cs.LG cs.AIcs.NE

Jeonghwan Cheon , Se-Bum Paik This is my paper · ORCID
classification cs.LGcs.AIcs.NE
keywords uncertaintycalibrationexpectederrorrandominitializationnoisepretrainingout-of-distributiondetectionconfidencefeedforwardneuralnetworksdevelopmentalneuroscience
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that conventional random initialization is a root cause of miscalibrated confidence in deep networks: untrained networks already output overconfident, class-biased probabilities. The proposed fix is a pretraining stage on randomly sampled Gaussian noise with freshly re-sampled uniform random labels, which pushes the network to chance-level confidence before any real data are seen. After real-data training, these pre-calibrated networks show lower Expected Calibration Error across model depths and dataset sizes, and their raw confidence scores detect out-of-distribution SVHN inputs more accurately. If the claim holds, confidence calibration can be improved with no extra calibration step, only a different initialization strategy.

What carries the argument

The central mechanism is a pretraining stage in which the network is trained on Gaussian noise inputs and independent uniform random labels, with both re-sampled every iteration. This drives the SoftMax output toward the uniform distribution, so confidence over the whole input space collapses to the chance level $1/C$ and class bias disappears. The paper uses this 'pre-calibration' as a replacement for conventional random initialization; it is the object that carries the later alignment of confidence and accuracy.

What would settle it

Train the same feedforward architecture at increasing widths on random noise and record average SoftMax confidence after pretraining: the claim predicts it falls to the chance level $1/10$ for every width. A width at which average confidence stays well above chance, or at which training accuracy on random labels exceeds chance, would refute the load-bearing premise.

Watch

Extended reading notes

Core claim

Conventional random initialization leaves an untrained network's SoftMax confidence high and unevenly distributed across the input space, so when real data arrive the network starts from an overconfident prior and stays miscalibrated. The paper claims that pretraining the same network on Gaussian noise inputs with freshly re-sampled uniform random labels drives the output distribution to a uniform, chance-level solution; after that pre-calibration, subsequent training on CIFAR-10 keeps confidence and accuracy aligned throughout learning. The reported result is lower Expected Calibration Error across depths and dataset sizes, and lower confidence on SVHN out-of-distribution inputs, so raw confidence alone separates in-distribution from out-of-distribution data.

Load-bearing premise

The entire effect rests on the pretraining optimizer driving the network to a uniform-output solution instead of exploiting the randomly re-sampled noise-label pairs; if any batch-specific correlation gets memorized, confidence after pretraining would rise rather than fall.

Editorial extensions

If this is right

  • Calibration error should be lower after random-noise pretraining for feedforward networks of depths 2 to 6 and training-set sizes 500 to 32,000 on CIFAR-10.
  • Raw confidence, without temperature scaling or post-hoc calibration, should separate CIFAR-10 in-distribution from SVHN out-of-distribution inputs above chance, as measured by AUROC.
  • The benefit should persist when training uses feedback alignment instead of backpropagation, so it is not tied to a specific learning rule.
  • Untrained networks initialized by the pretraining procedure should show no class bias and no overconfidence on held-out inputs before data training.
  • Pretraining should reduce test loss during subsequent training more than data-only training at matched accuracy or matched training epochs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is a general flattening of the output prior, the same pretraining might apply to transformer-based models by feeding random token sequences and uniform labels; the paper does not test this.
  • The method could be combined with existing post-hoc calibration, but the paper's value is removing the need; whether gains would be additive or redundant is an open question.
  • The load-bearing assumption that the optimizer does not memorize noise-label pairs becomes harder to satisfy as capacity grows, so the method's applicability to very large models is an open question.
  • A practical extension would be to use the post-pretraining confidence map as a diagnostic for whether an architecture is overparameterized relative to its data, since overconfidence appears exactly in that regime.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript proposes that random weight initialization is an underlying cause of confidence miscalibration in deep networks, and that pretraining with random Gaussian noise and randomly re-sampled uniform labels drives the network's softmax outputs toward the uniform distribution, thereby pre-calibrating uncertainty. The authors report that this pretraining step reduces Expected Calibration Error (ECE) and improves OOD detection AUROC across a grid of small fully-connected networks (2-6 layers) trained on CIFAR-10 subsets, and that the benefit also appears with a biologically plausible feedback-alignment rule. The paper explicitly acknowledges that no real-world or large-scale validation was performed and that all experiments use relatively simple tasks and small-scale networks.

Significance. The empirical mapping is systematic: ECE improvements are reported across depths (2-6) and data sizes (500-32,000) with appropriate nonparametric tests, and the method is simple and requires no post-hoc calibration. The inclusion of a biologically plausible learning rule and the public code are positive aspects. However, the central mechanistic claim is substantially weakened by the fact that training against uniform random labels mathematically drives the softmax output to the uniform distribution; the observation that confidence after pretraining is near chance is a by-construction property of the loss, not an emergent discovery. In addition, the paper does not compare against standard calibration baselines such as temperature scaling, nor against other low-confidence initializations, so the specific benefit of random-noise pretraining over simply starting from a low-confidence state is not established. These gaps are load-bearing for the paper's broad conclusions.

major comments (3)
  1. [Methods: Pretraining with random noise] Because labels are sampled uniformly and re-sampled each iteration, the conditional label distribution given any input is uniform, so the population optimum of the cross-entropy objective is the constant softmax output (1/K, ..., 1/K). Consequently, the observations in Fig. 3c, Fig. 3f-h, and Supplementary Fig. 6 that confidence after pretraining is near chance are direct consequences of the loss function rather than evidence of a novel 'pre-calibration' mechanism. The manuscript should state this explicitly and add a control that starts from any other uniform-output initial condition (e.g., zero or near-zero final-layer weights) to isolate what random-noise pretraining contributes beyond minimizing entropy at initialization.
  2. [Figure 2f inset and Figure 5d] The paper's central practical claim is that pretraining removes the need for 'additional pre- or post-processing,' but all comparisons are against an uncalibrated baseline (w/o pretraining). The standard post-hoc calibration baseline, temperature scaling (Guo et al., 2017), is not included, nor are label smoothing or other simple low-confidence initializations. Without these comparisons, the reader cannot tell whether the ECE and OOD AUROC improvements are comparable to a one-temperature-parameter post-hoc calibration, which would undercut the claim of a novel no-post-processing solution. Please add these baselines to Figures 2 and 5.
  3. [Abstract and Discussion] The claim that random initialization is an 'underlying cause' of miscalibration is not supported by the experimental design. The paper shows that untrained networks have high initial confidence and that pretraining changes this, but it does not vary initialization scale or output temperature to test whether the initial peakedness is causal for final miscalibration or merely a starting point that training can overcome. A concrete test would be to train normally from an initialization with a much smaller final-layer weight scale (or with an initial logit offset) and measure whether the final ECE matches the pretrained network; if it does, the effect is attributable to starting at low confidence, not to the random-noise pretraining procedure itself.
minor comments (4)
  1. [Figure 2e legend] The text 'w/o vs. zero, Wilcoxon signed-rank test, NS, P < 10−3' is internally contradictory; if P < 10^−3, the test is significant, so 'NS' should be removed or corrected.
  2. [Methods: Subsequent training with real data] The text states that the primary analysis uses 1000 training samples, but Fig. 2g reports conditions from 500 to 32,000; please clarify which panels correspond to the primary analysis and which are the extended grid.
  3. [Eq. (4)] The class-bias measure, std([N0, N1, ..., NM]/sum Ni), is not normalized for the number of classes; for a fair comparison across different output dimensions, consider reporting a normalized entropy or dividing by the maximum possible standard deviation.
  4. [Discussion] The statement that the method 'is applicable to all types of network models' is stronger than the evidence supports; the paper tests only fully-connected networks with ReLU activations and one biologically plausible learning rule, so please temper the generality claim.

Circularity Check

1 steps flagged · score 6.0 of 10

Core pre-calibration observation is by construction of the random-label loss; downstream calibration and OOD gains remain empirical but lack the advertised mechanistic support.

  1. self definitional [Methods, 'Pretraining with random noise'; Results, 'Pre-calibration of network uncertainty across the input space using random noise' (Fig. 3d).]
    "The corresponding labels for the random noise were also randomly sampled from a uniform distribution and one-hot encoded. Both the random noise inputs and their labels were re-sampled at each iteration and were not paired with one another... the network was trained to minimize this error, similar to conventional training procedures. ... random noise pretraining significantly reduced this initial class bias, leading to a more uniform confidence distribution across the classes."

    For inputs paired with i.i.d. uniform random labels, the expected cross-entropy (or BCE) gradient points toward the uniform class distribution for every input, so at the optimum the softmax outputs are 1/K and confidence equals the chance level. Thus the reported post-pretraining confidence near chance and near-zero class bias are mathematical consequences of the training objective, not empirical discoveries about calibration. The paper presents these outcomes as evidence that pretraining 'pre-calibrates' confidence, but pre-calibration to chance is the very quantity the loss is designed to produce. The later ECE and OOD improvements during real-data training are separate empirical claims and do not inherit this by-construction status, so the circularity is partial.

full rationale

The paper's central mechanistic observation—that random-noise pretraining brings confidence down to chance and removes class bias—is a direct consequence of training with cross-entropy against labels drawn uniformly at random; the Bayes-optimal output for every input is the uniform distribution. The paper presents this convergence as a key result, but it is the training objective itself, so that part of the claimed derivation is self-definitional. However, the paper's most useful claims—improved ECE during subsequent CIFAR-10 training and improved OOD AUROC on SVHN—are measured outcomes of the full training procedure and are not fitted to or entailed by the pretraining loss alone. They could fail, and they are reported with controls (same architecture, same epochs, same data). No parameter is fitted to the target metric and then reported as a prediction. The self-citations ([34], [55-58]) are motivational or contextual and are not the load-bearing justification for the empirical improvements. The absence of a temperature-scaling or low-initial-confidence control is a validity gap, not a circularity, and belongs in a correctness review. Overall, one prominent claim reduces by construction, giving partial circularity, but the final calibration/OOD results are independent empirical findings.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central method rests on standard deep-learning assumptions (softmax-as-confidence, ECE as metric) plus a paper-specific assumption that Adam with weight decay on re-sampled random labels converges to the uniform-output solution. No new entities are invented. Hand-chosen hyperparameters, especially pretraining duration, are not reported in enough detail to fully reproduce the experiments.

free parameters (5)
  • Pretraining duration (epochs or iterations) = not specified in Methods
    Hand-chosen and not reported as a sweep; it is central to reaching a uniform-output solution without memorizing random labels.
  • Random noise standard deviation = 1.0
    Fixed to N(0,1) in Methods (Pretraining with random noise); no sensitivity analysis is given, though the calibration effect may depend on input scale.
  • Weight decay constant = 0.001
    Chosen to prevent overfitting (Methods); no sweep is reported, and it likely influences whether random labels are memorized.
  • Learning rate = between 1e-4 and 1e-5
    Adam optimizer with this range (Methods, Subsequent training with real data); no selection procedure is reported.
  • Hidden layer size = 256
    Fixed at 256 neurons (Methods, Neural network model); the paper says results are consistent across typical settings but no systematic sweep is shown.
assumptions (4)
  • domain assumption Softmax probabilities are interpreted directly as confidence and compared to accuracy via ECE.
    This is the standard calibration setup (Methods, Evaluating calibration of neural networks), but it assumes the softmax output is the quantity of interest for confidence.
  • domain assumption Adam with weight decay and finite pretraining converges to a uniform-output solution rather than memorizing re-sampled random labels.
    The method relies on this convergence, which is reported in Fig. 2d and Supplementary Fig. 2 but not proven or characterized across training durations.
  • ad hoc to paper Gaussian noise with unit variance spans the input distribution relevant for pre-calibration.
    Random Gaussian noise is the paper's chosen pretraining distribution; its adequacy is only empirically tested on CIFAR-10 and SVHN inputs.
  • standard math ECE with 10 equal-width bins is a faithful measure of calibration for comparing the two conditions.
    This is a standard metric from ref 37, but bin-count sensitivity is not reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Pretraining with random noise for uncertainty calibration." pith.science (2026). https://pith.science/paper/LWR33LOI

@misc{pith2026241217411,
  author       = {Pith},
  title        = {Pith review of: Pretraining with random noise for uncertainty calibration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LWR33LOI}},
  note         = {Machine review of arXiv:2412.17411}
}
read the original abstract

Uncertainty calibration is crucial for various machine learning applications, yet it remains challenging. Many models exhibit hallucinations - confident yet inaccurate responses - due to miscalibrated confidence. Here, we show that the common practice of random initialization in deep learning, often considered a standard technique, is an underlying cause of this miscalibration, leading to excessively high confidence in untrained networks. Our method, inspired by developmental neuroscience, addresses this issue by simply pretraining networks with random noise and labels, reducing overconfidence and bringing initial confidence levels closer to chance. This ensures optimal calibration, aligning confidence with accuracy during subsequent data training, without the need for additional pre- or post-processing. Pre-calibrated networks excel at identifying "unknown data," showing low confidence for out-of-distribution inputs, thereby resolving confidence miscalibration.

Figures

Figures reproduced from arXiv: 2412.17411 by the authors.

Figure 1
Figure 1. Confidence miscalibration in artificial neural networks. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Pretraining with random noise enables confidence calibration in neural networks. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Random noise pre-calibrates neural network uncertainty over input space. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Pre-calibration enables learning with matching confidence and accuracy. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Out-of-distribution detection using calibrated network confidence. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. One-Time Soft Alignment Enables Resilient Learning without Weight Transport

    cs.LG 2025-05 conditional novelty 5.0 of 10

    One-time soft alignment between forward and fixed feedback weights at initialization substantially improves feedback alignment training and approaches backpropagation accuracy on small image tasks.

Reference graph

Works this paper leans on

59 extracted references · 48 canonical work pages · cited by 1 Pith paper

  1. [1]

    & Hinton, G

    LeCun, Y ., Bengio, Y . & Hinton, G. Deep learning.Nature 521, 436–444 (2015)

    2015

  2. [2]

    & Haffner, P

    LeCun, Y ., Bottou, L., Bengio, Y . & Haffner, P . Gradient-based learning applied to document recognition.Proceedings of the IEEE 86, 2278–2324 (1998)

    work page 1998

  3. [3]

    & Hinton, G

    Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. In Pereira, F ., Burges, C., Bottou, L. & Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 25 (Curran Associates, Inc., 2012)

    work page 2012

  4. [4]

    & Sun, J

    He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778 (2016)

    2016

  5. [5]

    & Weinberger, K

    Huang, G., Liu, Z., Van Der Maaten, L. & Weinberger, K. Q. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 4700–4708 (2017)

    2017

  6. [6]

    & Huang, X

    Chen, Z. & Huang, X. End-to-end learning for lane keeping of self-driving cars. In 2017 IEEE intelligent vehicles symposium (IV), 1856–1860 (IEEE, 2017)

    work page 2017

  7. [7]

    & Suk, H.-I

    Shen, D., Wu, G. & Suk, H.-I. Deep learning in medical image analysis. Annual review of biomedical engineering 19, 221–248 (2017)

    work page 2017

  8. [8]

    & Krauss, C

    Fischer, T. & Krauss, C. Deep learning with long short-term memory networks for financial market predictions. European journal of operational research 270, 654–669 (2018)

    work page 2018

Show all 59 references
  1. [9]

    & Weinberger, K

    Guo, C., Pleiss, G., Sun, Y . & Weinberger, K. Q. On calibration of modern neural networks. InInternational conference on machine learning, 1321–1330 (PMLR, 2017)

    2017

  2. [10]

    W., Gorur, D

    Nalisnick, E., Matsukawa, A., Teh, Y . W., Gorur, D. & Lakshminarayanan, B. Do deep generative models know what they don’t know? In International Conference on Learning Representations (2019)

    2019

  3. [11]

    Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift

    Ovadia, Y .et al. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. In Wallach, H. et al. (eds.) Advances in Neural Information Processing Systems, vol. 32 (Curran Associates, Inc., 2019)

    2019

  4. [12]

    & Gimpel, K

    Hendrycks, D. & Gimpel, K. A baseline for detecting misclassified and out-of-distribution examples in neural networks. InInternational Conference on Learning Representations (2017)

    2017

  5. [13]

    & Srikant, R

    Liang, S., Li, Y . & Srikant, R. Enhancing the reliability of out-of-distribution image detection in neural networks. In International Conference on Learning Representations (2018)

    2018

  6. [14]

    & Davidsson, S

    Helldin, T., Falkman, G., Riveiro, M. & Davidsson, S. Presenting system uncertainty in automotive uis for supporting trust calibration in autonomous driving. In Proceedings of the 5th international conference on automotive user interfaces and interactive vehicular applications...

    2013

  7. [15]

    M., Tempany, C

    Mehrtash, A., Wells, W. M., Tempany, C. M., Abolmaesumi, P . & Kapur, T. Confidence calibration and predictive uncertainty estimation for deep medical image segmentation. IEEE transactions on medical imaging 39, 3868–3878 (2020)

    2020

  8. [16]

    Achiam, J. et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    2023 arXiv

  9. [17]

    Team, G. et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 (2023)

    2023 arXiv

  10. [18]

    & Gal, Y

    Farquhar, S., Kossen, J., Kuhn, L. & Gal, Y . Detecting hallucinations in large language models using semantic entropy.Nature 630, 625–630 (2024)

    2024

  11. [19]

    & Wang, W

    Xiao, Y . & Wang, W. Y . On hallucination and predictive uncertainty in conditional language generation.arXiv preprint arXiv:2103.15025 (2021). 17

    2021 arXiv

  12. [20]

    Geng, J. et al. A survey of confidence estimation and calibration in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 6577–6595 (2024)

    2024

  13. [21]

    & Valdenegro-Toro, M

    Groot, T. & Valdenegro-Toro, M. Overconfidence is key: Verbalized uncertainty evaluation in large language and vision-language models. arXiv preprint arXiv:2405.02917 (2024)

    2024 arXiv

  14. [22]

    & Caruana, R

    Niculescu-Mizil, A. & Caruana, R. Predicting good probabilities with supervised learning. In Proceedings of the 22nd international conference on Machine learning, 625–632 (2005)

    2005

  15. [23]

    Platt, J. et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10, 61–74 (1999)

    1999

  16. [24]

    & Elkan, C

    Zadrozny, B. & Elkan, C. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. InInternational Conference on Machine Learning, vol. 18, 609–616 (2001)

    2001

  17. [25]

    & Elkan, C

    Zadrozny, B. & Elkan, C. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, 694–699 (2002)

    2002

  18. [26]

    & Dietterich, T

    Hendrycks, D., Mazeika, M. & Dietterich, T. Deep anomaly detection with outlier exposure. In International Conference on Learning Representations (2019)

    2019

  19. [27]

    & Shin, J

    Lee, K., Lee, K., Lee, H. & Shin, J. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Bengio, S. et al. (eds.) Advances in Neural Information Processing Systems, vol. 31 (Curran Associates, Inc., 2018)

    2018

  20. [28]

    Liu, W., Wang, X., Owens, J. & Li, Y . Energy-based out-of-distribution detection. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. & Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, 21464–21475 (Curran Associates, Inc., 2020)

    2020

  21. [29]

    & Tooby, J

    Cosmides, L. & Tooby, J. Are humans good intuitive statisticians after all? rethinking some conclusions from the literature on judgment under uncertainty. cognition 58, 1–73 (1996)

    1996

  22. [30]

    Fleming, S. M. & Dolan, R. J. The neural basis of metacognitive ability. Philosophical Transactions of the Royal Society B: Biological Sciences 367, 1338–1349 (2012)

    2012

  23. [31]

    Kepecs, A., Uchida, N., Zariwala, H. A. & Mainen, Z. F . Neural correlates, computation and behavioural impact of decision confidence. Nature 455, 227–231 (2008)

    2008

  24. [32]

    & Shadlen, M

    Kiani, R. & Shadlen, M. N. Representation of confidence associated with a decision by neurons in the parietal cortex. science 324, 759–764 (2009)

    2009

  25. [33]

    & Kepecs, A

    Masset, P ., Ott, T., Lak, A., Hirokawa, J. & Kepecs, A. Behavior- and modality-general representation of confidence in orbitofrontal cortex. Cell 182, 112–126 (2020)

    2020

  26. [34]

    Cheon, J., Lee, S. W. & Paik, S.-B. Pretraining with random noise for fast and robust learning without weight transport. In Globerson, A. et al. (eds.) Advances in Neural Information Processing Systems, vol. 37, 13748–13768 (Curran Associates, Inc., 2024)

    2024

  27. [35]

    Krizhevsky, A., Hinton, G. et al. Learning multiple layers of features from tiny images. Tech. Rep., University of Toronto (2009)

    2009

  28. [36]

    DeGroot, M. H. & Fienberg, S. E. The comparison and evaluation of forecasters. Journal of the Royal Statistical Society: Series D (The Statistician) 32, 12–22 (1983)

    1983

  29. [37]

    P ., Cooper, G

    Naeini, M. P ., Cooper, G. & Hauskrecht, M. Obtaining well calibrated probabilities using bayesian binning. InProceedings of the AAAI conference on artificial intelligence, vol. 29 (2015)

    2015

  30. [38]

    & Maffei, L

    Galli, L. & Maffei, L. Spontaneous impulse activity of rat retinal ganglion cells in prenatal life. Science 242, 90–91 (1988). 18

    1988

  31. [39]

    J., Guillam´on-Vivancos, T., Moreno-Juan, V., Valdeolmillos, M

    Martini, F . J., Guillam´on-Vivancos, T., Moreno-Juan, V., Valdeolmillos, M. & L´opez-Bendito, G. Spontaneous activity in developing thalamic and cortical sensory networks. Neuron 109, 2519–2534 (2021)

    2021

  32. [40]

    B., Burbridge, T

    Ackman, J. B., Burbridge, T. J. & Crair, M. C. Retinal waves coordinate patterned activity throughout the developing visual system. Nature 490, 219–225 (2012)

    2012

  33. [41]

    Ant´on-Bola˜nos, N. et al. Prenatal activity from thalamic neurons governs the emergence of functional cortical maps in mice. Science 364, 987–990 (2019)

    2019

  34. [42]

    E., Hinton, G

    Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. Nature 323, 533–536 (1986)

    1986

  35. [43]

    P ., Cownden, D., Tweed, D

    Lillicrap, T. P ., Cownden, D., Tweed, D. B. & Akerman, C. J. Random synaptic feedback weights support error backpropagation for deep learning. Nature communications 7, 13276 (2016)

    2016

  36. [44]

    & Sun, J

    He, K., Zhang, X., Ren, S. & Sun, J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, 1026–1034 (2015)

    2015

  37. [45]

    Reading digits in natural images with unsupervised feature learning

    Netzer, Y .et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, vol. 2, 4 (Granada, 2011)

    2011

  38. [46]

    & Boudette, N

    Vlasic, B. & Boudette, N. Self-driving tesla was involved in fatal crash, u.s. says (2016)

    2016

  39. [47]

    & Lasser, T

    Wollek, A., Willem, T., Ingrisch, M., Sabel, B. & Lasser, T. Out-of-distribution detection with in-distribution voting using the medical example of chest x-ray classification. Medical Physics 51, 2721–2732 (2024)

    2024

  40. [48]

    & Yuksel, D

    Griot, M., Hemptinne, C., Vanderdonckt, J. & Yuksel, D. Large language models lack essential metacognition for reliable medical reasoning. Nature Communications 16, 642 (2025)

    2025

  41. [49]

    & Bengio, Y

    Glorot, X. & Bengio, Y . Understanding the difficulty of training deep feedforward neural networks. InProceedings of the thirteenth international conference on artificial intelligence and statistics, 249–256 (JMLR Workshop and Conference Proceedings, 2010)

    2010

  42. [50]

    & Hinton, G

    Sutskever, I., Martens, J., Dahl, G. & Hinton, G. On the importance of initialization and momentum in deep learning. In International conference on machine learning, 1139–1147 (PMLR, 2013)

    2013

  43. [51]

    M., McClelland, J

    Saxe, A. M., McClelland, J. L. & Ganguli, S. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6120 (2013)

    2013 arXiv

  44. [52]

    & Botvinick, M

    Hassabis, D., Kumaran, D., Summerfield, C. & Botvinick, M. Neuroscience-inspired artificial intelligence. Neuron 95, 245–258 (2017)

    2017

  45. [53]

    Richards, B. A. et al. A deep learning framework for neuroscience. Nature Neuroscience 22, 1761–1770 (2019)

    2019

  46. [54]

    Y amins, D. L. K. & DiCarlo, J. J. Using goal-driven deep learning models to understand sensory cortex.Nature Neuroscience 19, 356–365 (2016)

    2016

  47. [55]

    & Paik, S.-B

    Kim, G., Jang, J., Baek, S., Song, M. & Paik, S.-B. Visual number sense in untrained deep neural networks. Science advances 7, eabd6127 (2021)

    2021

  48. [56]

    & Paik, S.-B

    Baek, S., Song, M., Jang, J., Kim, G. & Paik, S.-B. Face detection in untrained deep neural networks. Nature communications 12, 7328 (2021)

    2021

  49. [57]

    & Paik, S.-B

    Cheon, J., Baek, S. & Paik, S.-B. Invariance of object detection in untrained deep neural networks. Frontiers in Computational Neuroscience 16, 1030707 (2022)

    2022

  50. [58]

    & Paik, S.-B

    Lee, H., Choi, W., Lee, D. & Paik, S.-B. Comparison of visual quantities in untrained neural networks. Cell Reports 42 (2023)

    2023

  51. [59]

    & Paik, S.-B

    Kim, J., Song, M., Jang, J. & Paik, S.-B. Spontaneous retinal waves can generate long-range horizontal connectivity in visual cortex. Journal of Neuroscience 40, 6584–6599 (2020). 19 Supplementary Materials Supplementary Fig. 1. Measuring confidence calibration of a model netw...

    2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.