Pith. sign in

REVIEW 4 major objections 5 minor 41 references

How to warm-start your unfolding network

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that wrapping an overparameterized unfolding network in a continuation loop and training it with log-cosh loss improves compressed-sensing reconstruction and generalization, producing smoother loss landscapes.

desk verdict A clear, short paper with a sensible idea, but the main empirical comparison is uncontrolled: different losses, test-set early stopping, and no variance. read the letter →

arxiv 2502.01854 v2 pith:RASCRWN6 submitted 2025-02-03 cs.LG eess.IVeess.SP

Vicky Kouni This is my paper · ORCID
classification cs.LGeess.IVeess.SP
keywords deepunfoldingnetworkcompressedsensingcontinuationwarm-startoverparameterizationlog-coshlosslandscapegeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a model-based optimization trick called continuation can be wrapped around an overparameterized deep unfolding network to improve compressed-sensing image reconstruction. The proposed network, C-DEC, replaces the decoder inside a continuation loop that repeatedly warm-starts the initial guess $x_0$, and it is trained and evaluated with the log-cosh loss instead of mean squared error. The paper reports lower test and generalization errors than the base decoder DECONET on MNIST and CIFAR10, and visualizes loss landscapes that become smoother when continuation and overparameterization are combined. If this holds, continuation would be a simple, broadly applicable way to boost existing unfolding networks without changing their architecture.

What carries the argument

The machinery is the continuation loop from the model-based solver, whose inner solver is replaced by the unfolded DECONET decoder. At each outer step the decoder produces $\hat{x}_{j+1}$, and the initial guess is updated as $x^{j+1}_0 = \hat{x}_j + \frac{j}{j+3}(\hat{x}_{j+1} - \hat{x}_j)$, so the next call starts from a warmer estimate. The second ingredient is the log-$\cosh$ loss $\mathcal{L}_{\text{train}} = \frac{1}{s}\sum_i \log\cosh(g^W_{x_0}(y_i)-x_i)$, whose quadratic/linear behavior is argued to combine MSE and MAE advantages. Overparameterization enters through the learnable redundant transform $W\in\mathbb{R}^{N\times n}$ shared across layers.

What would settle it

Train C-DEC and DECONET with the same loss function (log-cosh for both, then MSE for both) and stop both on a validation split instead of a test-derived estimate. If the continuation advantage disappears or reverses, the reported lower errors do not establish that continuation improves reconstruction or generalization.

Watch

Extended reading notes

Core claim

The central claim is that a deep unfolding network for compressed sensing inherits the benefit of continuation from its model-based parent algorithm: feeding better and better initial guesses $x_0$ through the continuation loop while the network weights are learned produces a decoder, C-DEC, whose reconstruction and generalization errors are consistently below those of the un-continued DECONET on both datasets tested. The paper further claims that training with log-cosh loss, which behaves quadratically near zero and linearly far from zero, is a better training and evaluation metric for this setting than MSE, and that the combination of continuation with the overparameterized sparsifying transform produces visibly smoother loss landscapes.

Load-bearing premise

The load-bearing premise is that the comparison is fair even though C-DEC is trained and evaluated with log-cosh while DECONET uses MSE, and early stopping uses a generalization estimate computed from the test set.

Editorial extensions

If this is right

  • Continuation can be applied to other unfolding networks as an outer loop, since it only modifies how the decoder is initialized.
  • Log-cosh loss becomes a viable default for training and evaluating deep unfolding networks, not just regression models.
  • If the smoothing effect is real, loss-landscape analysis could provide a way to predict when warm-started unfolding networks will generalize better.
  • The overparameterization of the learned transform and the number of continuation steps could be tuned jointly to control the train-test gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would replace the fixed continuation schedule with a learned schedule, letting the network decide how fast to warm-start across layers.
  • The same warm-starting idea could transfer to other inverse problems solved by unfolding, such as deblurring or inpainting, where a model-based solver with an initial guess exists.
  • If the loss-landscape smoothing is confirmed on more architectures, continuation and overparameterization together could be used as a deliberate regularizer, rather than only a reconstruction booster.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes C-DEC, an ensemble framework that wraps the overparameterized unfolding network DECONET in a continuation (warm-starting) loop and trains/evaluates it with the log-cosh loss instead of the MSE used by DECONET. The authors claim that this combination yields smoother loss landscapes and consistently improved reconstruction and generalization performance on MNIST and CIFAR10, and they provide numerical comparisons in Table I and loss-landscape visualizations in Fig. 2.

Significance. If the claims were fully supported, the contribution would be a simple, potentially portable plug-in: warm-starting an existing unfolding network with a continuation loop and swapping the loss to log-cosh. The paper also provides, to the authors' knowledge, the first loss-landscape visualizations for deep unfolding networks, which is a useful qualitative addition to the literature. However, the central empirical evidence is not currently valid because the comparison between C-DEC and DECONET changes the evaluation metric and leaks test information into model selection, so the paper does not establish its main claim as written.

major comments (4)
  1. [Section III, Table I] The head-to-head comparison in Table I is not made on a common metric: C-DEC's test error is the log-cosh loss (Eq. 4) while DECONET's is the MSE. Since log-cosh behaves as u^2/2 for small residuals u, a lower log-cosh value than an MSE value does not imply better reconstruction; for equal reconstructions the log-cosh number would be roughly half the MSE number. The reported values (e.g., 0.0299 vs 0.0560 on MNIST, a factor of ~1.9) are thus consistent with the two models having identical quality, and the claim that C-DEC 'always' achieves lower test and generalization errors is not established. Please report both losses for both models, or use a metric such as PSNR/SSIM, and re-run the comparison under identical evaluation conditions.
  2. [Section III, Settings] Early stopping is performed 'with respect to Lgen', and Lgen is defined as |Ltest - Ltrain| with Ltest computed on the test set. This makes the test set part of model selection, so the reported test errors and EGE values are optimistically biased and do not constitute a valid generalization estimate. The authors should use a separate validation set for early stopping and keep the test set completely held out until final evaluation.
  3. [Section III, Results and discussion] The improvement is attributed jointly to continuation and to the log-cosh loss, but the comparison against DECONET changes both factors at once: DECONET is trained and evaluated with MSE, while C-DEC uses continuation and log-cosh. Without ablations (e.g., DECONET with log-cosh, and C-DEC with MSE), the observed differences cannot be attributed to warm-starting. Please include such ablations to separate the effects of continuation and the loss function.
  4. [Section III, Loss landscapes] The claim that continuation and overparameterization 'smooth out' the loss landscape is based on visual inspection of Fig. 2, with no quantitative measure of smoothness (e.g., an estimate of the local Lipschitz constant or the variance of the surface). Please provide a quantitative smoothness metric or a statistical comparison to support this claim.
minor comments (5)
  1. [Section III, Settings] The notation 'W ∈ R10n×n' and 'R50n×n' is ambiguous; it should be W ∈ R^{10n × n} or the text should define N = 10n explicitly before these expressions.
  2. [Section II, Eq. (1)] The constraint ‖y − Ax‖₂ ≤ ε uses ε which is only defined later in Section III; please define ε at the point of first use or state that it is a problem parameter to be set later.
  3. [Section II, Algorithm 3] The phrase 'This tantamounts to a new unfolded architecture' should be corrected to 'This amounts to a new unfolded architecture'.
  4. [Section III, Settings] The sentence 'The rest of C-DEC's hyperparameters, e.g., µ, t1, t2, θ, are inherited by DECONET' should read 'inherited from DECONET' or 'shared with DECONET' to clarify the direction of inheritance.
  5. [Section III, Settings] The paper states that experiments are repeated at least 10 times and averaged, but no standard deviations or error bars are reported in Table I or Fig. 1; providing variability information would strengthen the reliability of the comparisons.

Circularity Check

2 steps flagged · score 6.0 of 10

C-DEC's reported advantage is partly by construction: the comparison uses different loss functions for C-DEC and DECONET, and early stopping is driven by the test set through Lgen, so the claimed consistent improvement is not attributable to continuation alone.

  1. self definitional [Section III, Eq. (4), Table I caption and Results & discussion]
    "PERFORMANCE COMPARISONS BETWEEN DECONET ( MEASURED WITH THE MSE) AND C-DEC ( MEASURED WITH THE LOG -COSH LOSS ) ... Both the test and generalization errors are always lower for our proposed DUN, consistently for both datasets."

    The comparison does not use a common metric: C-DEC's reported test error is the log-cosh loss from Eq. (4), while DECONET's reported test error is the MSE. For small residuals, log-cosh(z) = z^2/2 + O(z^4), so C-DEC's numerical loss is approximately half of its own MSE. A lower log-cosh number than an MSE number is therefore expected even when the reconstructions are of similar quality. The claimed 'always lower' test and generalization errors are thus partly produced by the definition of the evaluation metric, not by continuation. The EGE values similarly compare |logcosh_test - logcosh_train| with |MSE_test - MSE_train|, which are not commensurable quantities.

  2. fitted input called prediction [Section III, Settings (definition of Lgen and early stopping)]
    "For our experiments, we report the test log-cosh loss Ltest – which is essentially the log-cosh loss evaluated on a set of d test data not used during training – and the empirical generalization error (EGE) Lgen = |Ltest − Ltrain|, with Ltrain as in (4). We train C-DEC, on all datasets, employing an early stopping technique [40] with respect to Lgen."

    Lgen is defined using Ltest, and early stopping selects the model by tracking Lgen during training. The test set therefore participates directly in model selection, so the subsequently reported Ltest is not an independent test statistic but the value that drove early stopping. The reported generalization errors are optimistically biased by construction, making the 'always lower' comparison a fitted outcome rather than a clean prediction on unseen data.

full rationale

The construction of C-DEC itself is not circular: Algorithm 3 is a straightforward composition of a continuation outer loop with the unfolded decoder h(y), and the loss-landscape visualizations are an independent empirical observation. The circularity is concentrated in the evaluation protocol that supports the central empirical claim. First, Table I compares C-DEC under the log-cosh loss against DECONET under the MSE loss without reporting a common metric; because log-cosh is approximately half the MSE near zero, the lower numerical values are partly an artifact of the chosen loss, not evidence of better reconstruction. Second, early stopping with respect to Lgen = |Ltest - Ltrain| uses the test set during training, so the reported test and generalization errors are fitted rather than genuinely predictive. These two issues together mean the paper's headline claim that C-DEC 'always' improves reconstruction and generalization over DECONET is not established by the provided evidence. The self-citations to [11] and [12] are present, but [11] is a published peer-reviewed baseline and the direct C-DEC versus DECONET comparison does not reduce to that citation; the main defect is the uncontrolled and test-leaking evaluation, not the citation chain. A matched-loss comparison with a clean validation split could still reveal a real benefit from continuation, so the score is 6 rather than higher.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

C-DEC introduces no new physical or mathematical entities; it is an architectural combination of DECONET and continuation. The paper's central claim depends on the validity of inherited algorithms and on the fairness of the experimental protocol.

free parameters (6)
  • learning rate = 1e-3 (MNIST), 1e-4 (CIFAR10)
    Chosen per dataset; affects optimization and final performance; not derived.
  • number of layers L = 5, 10, 15, 50
    Varies across experiments; selected by hand; impacts capacity and results.
  • number of continuation steps = 1 (baseline) and 5 (C-DEC), plus varying in Fig 1
    Key component; number of warm-start iterations is a hyperparameter, not derived.
  • transform redundancy N = 10n and 50n
    Two instances of W dimension; chosen to study overparameterization.
  • batch size = 128
    Fixed for all runs; not justified.
  • W initialization = Beta distribution (parameters not specified)
    Random initialization scheme; affects training.
assumptions (5)
  • standard math Algorithm 1 from [28] solves the constrained CS problem (1) and converges.
    The paper builds on the model-based solver without reproving convergence.
  • domain assumption DECONET [11] with hyperparameters µ, t1, t2, θ is a valid and strong baseline that outperforms other state-of-the-art DUNs.
    Paper relies on [11] for architecture and baseline quality.
  • domain assumption Continuation warm-starting improves the performance of Algorithm 1 and carries over to the unfolded setting.
    Motivated by [28]; the transfer to DUNs is assumed and only empirically tested.
  • domain assumption Log-cosh loss is an appropriate training metric for DUNs because of its mixed linear/quadratic behavior.
    No proof that this translates to better reconstruction; it is a plausible heuristic.
  • standard math Loss landscape visualization framework of [35] applies to DUNs.
    Used without adaptation to measure smoothness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of How to warm-start your unfolding network." pith.science (2026). https://pith.science/paper/RASCRWN6

@misc{pith2026250201854,
  author       = {Pith},
  title        = {Pith review of: How to warm-start your unfolding network},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RASCRWN6}},
  note         = {Machine review of arXiv:2502.01854}
}
read the original abstract

We present a new ensemble framework for boosting the performance of overparameterized unfolding networks solving the compressed sensing problem. We combine a state-of-the-art overparameterized unfolding network with a continuation technique, to warm-start a crucial quantity of the said network's architecture; we coin the resulting continued network C-DEC. Moreover, for training and evaluating C-DEC, we incorporate the log-cosh loss function, which enjoys both linear and quadratic behavior. Finally, we numerically assess C-DEC's performance on real-world images. Results showcase that the combination of continuation with the overparameterized unfolded architecture, trained and evaluated with the chosen loss function, yields smoother loss landscapes and improved reconstruction and generalization performance of C-DEC, consistently for all datasets.

Figures

Figures reproduced from arXiv: 2502.01854 by the authors.

Figure 1
Figure 1. Performance plots of C-DEC, with W ∈ R10n×n, on MNIST (left) and CIFAR10 (right) datasets. training and 10000 test 32×32 coloured image examples). We transform the CIFAR10 images into grayscale ones. For both datasets, we consider the vectorized form of the images. We fix m = 0.25 · n, use a Gaussian A˜ := A/√ m ∈ R m×n, and add zero-mean Gaussian noise e with standard deviation std = 10−4 to the measurements y, so … view at source ↗
Figure 2
Figure 2. Loss landscapes of 5-layer C-DEC on (a) MNIST and (b) CIFAR10 datasets. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 38 canonical work pages

  1. [1]

    AMP- Inspired Deep Networks for Sparse Linear Inverse Prob- lems

    M. Borgerding, P. Schniter, and S. Rangan. “AMP- Inspired Deep Networks for Sparse Linear Inverse Prob- lems”. In: IEEE Trans. Signal Process. 65.16 (2017), pp. 4293–4308

    work page 2017

  2. [2]

    ISTA-Net: Interpretable optimization-inspired deep network for image com- pressive sensing

    J. Zhang and B. Ghanem. “ISTA-Net: Interpretable optimization-inspired deep network for image com- pressive sensing”. In: Proc. IEEE Comput. Vision and Pattern Recognit. 2018, pp. 1828–1837

    work page 2018

  3. [3]

    ADMM-CSNet: A deep learning ap- proach for image compressive sensing

    Y . Yang et al. “ADMM-CSNet: A deep learning ap- proach for image compressive sensing”. In: Trans. Pat- tern Anal. and Mach. Intell. 42.3 (2018), pp. 521–538

    work page 2018

  4. [4]

    Model-based deep learning

    N. Shlezinger et al. “Model-based deep learning”. In: Proc. IEEE 111.5 (2023), pp. 465–499

    work page 2023

  5. [5]

    Star DGT: a robust Gabor transform for speech denoising

    V . Kouni, H. Rauhut, and T. Theoharis. “Star DGT: a robust Gabor transform for speech denoising”. In: Sampling Theory, Signal Process., and Data Anal. 21.1 (2023), p. 14

    work page 2023

  6. [6]

    ASCONVSR: Fast and lightweight super- resolution network with assembled convolutions

    J. Guo et al. “ASCONVSR: Fast and lightweight super- resolution network with assembled convolutions”. In: Proc. IEEE/CVF Conf. Comput. Vision and Pattern Recognit. 2023, pp. 1582–1592

    work page 2023

  7. [7]

    Pansharpening method based on deep nonlocal unfolding

    X. Li et al. “Pansharpening method based on deep nonlocal unfolding”. In: IEEE Trans. Geoscience and Remote Sensing 61 (2023), pp. 1–11

    work page 2023

  8. [8]

    A Deep Proximal-Unfolding Method for Monaural Speech Dereverberation

    Meihuang Wang et al. “A Deep Proximal-Unfolding Method for Monaural Speech Dereverberation”. In: 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE. 2022, pp. 324–329

    work page 2022

Show all 41 references
  1. [9]

    Model-based compressive sens- ing

    R. G. Baraniuk et al. “Model-based compressive sens- ing”. In: IEEE Trans. Inf. Theory 56.4 (2010), pp. 1982– 2001

    2010

  2. [10]

    Spark Deficient Gabor Frame Provides a Novel Analysis Operator for Compressed Sensing

    V . Kouni and H. Rauhut. “Spark Deficient Gabor Frame Provides a Novel Analysis Operator for Compressed Sensing”. In: Int. Conf. Neural Inf. Process. Springer. 2021, pp. 700–708

    2021

  3. [11]

    DECONET: an Unfolding Network for Analysis-based Compressed Sensing with Generalization Error Bounds

    V . Kouni and Y . Panagakis. “DECONET: an Unfolding Network for Analysis-based Compressed Sensing with Generalization Error Bounds”. In: IEEE Trans. Signal Process. 71 (2023), pp. 1938–1951. DOI: 10.1109/TSP. 2023.3272286

    2023

  4. [12]

    Generalization analysis of an unfolding network for analysis-based Compressed Sensing

    V . Kouni and Y . Panagakis. “Generalization analysis of an unfolding network for analysis-based Compressed Sensing”. In: arXiv preprint arXiv:2303.05582, under review in Appl. & Comput. Harmon. Anal. (2023)

    2023 arXiv

  5. [13]

    Artificial neural networks for nonlinear regression and classification

    A. Landi et al. “Artificial neural networks for nonlinear regression and classification”. In: 10th Int. Conf. Intell. Syst. Design and Appl. IEEE. 2010, pp. 115–120

    2010

  6. [14]

    Dr2-net: Deep residual reconstruction network for image compressive sensing

    H. Yao et al. “Dr2-net: Deep residual reconstruction network for image compressive sensing”. In: Neurocom- puting 359 (2019), pp. 483–493

    2019

  7. [15]

    A residual dense u-net neural network for image denois- ing

    J. Gurrola-Ramos, O. Dalmau, and T. E. Alarcon. “A residual dense u-net neural network for image denois- ing”. In: IEEE Access 9 (2021), pp. 31742–31754

    2021

  8. [16]

    Designing interpretable recurrent neural networks for video reconstruction via deep un- folding

    H. Van Luong et al. “Designing interpretable recurrent neural networks for video reconstruction via deep un- folding”. In: IEEE Trans. Image Process. 30 (2021), pp. 4099–4113

    2021

  9. [17]

    AMP-Net: Denoising-Based Deep Unfolding for Compressive Image Sensing

    Z. Zhang et al. “AMP-Net: Denoising-Based Deep Unfolding for Compressive Image Sensing”. In: IEEE Trans. Image Process. 30 (2021), pp. 1487–1500

    2021

  10. [18]

    Dynamic path- controllable deep unfolding network for compressive sensing

    J. Song, B. Chen, and J. Zhang. “Dynamic path- controllable deep unfolding network for compressive sensing”. In: IEEE Trans. Image Process. 32 (2023), pp. 2202–2214

    2023

  11. [19]

    Gates-Controlled Deep Unfolding Network for Image Compressed Sensing

    T. Li et al. “Gates-Controlled Deep Unfolding Network for Image Compressed Sensing”. In: IEEE Trans. Com- put. Imag. (2024)

    2024

  12. [20]

    UFC-Net: Unrolling Fixed-point Continuous Network for Deep Compressive Sensing

    X. Wang and H. Gan. “UFC-Net: Unrolling Fixed-point Continuous Network for Deep Compressive Sensing”. In: Proc. IEEE/CVF Conf. Comput. Vision and Pattern Recognit. 2024, pp. 25149–25159

    2024

  13. [21]

    Deep unfolding with normalizing flow priors for inverse problems

    X. Wei et al. “Deep unfolding with normalizing flow priors for inverse problems”. In: IEEE Trans. Signal Process. 70 (2022), pp. 2962–2971

    2022

  14. [22]

    A fixed-point continuation method for l1-regularized minimization with applications to compressed sensing

    E. T. Hale, W. Yin, and Y . Zhang. “A fixed-point continuation method for l1-regularized minimization with applications to compressed sensing”. In: CAAM TR07-07, Rice University 43.44 (2007), p. 2

    2007

  15. [23]

    Fixed-point contin- uation for ℓ1-minimization: Methodology and conver- gence

    E. T. Hale, W. Yin, and Y . Zhang. “Fixed-point contin- uation for ℓ1-minimization: Methodology and conver- gence”. In: SIAM J. on Optim. 19.3 (2008), pp. 1107– 1130

    2008

  16. [24]

    NESTA: A fast and accurate first-order method for sparse recovery

    S. Becker, J. Bobin, and E. J. Candes. “NESTA: A fast and accurate first-order method for sparse recovery”. In: SIAM J. on Imag. Scie. 4.1 (2011), pp. 1–39

    2011

  17. [25]

    Fast ℓ1-Minimization Algorithms for Robust Face Recognition

    A. Y . Yang et al. “Fast ℓ1-Minimization Algorithms for Robust Face Recognition”. In: IEEE Trans. Image Process. 22.8 (2013), pp. 3234–3246

    2013

  18. [26]

    Warm-start strategies in interior-point methods for linear programming

    E. A. Yildirim and S. J. Wright. “Warm-start strategies in interior-point methods for linear programming”. In: SIAM J. Optim. 12.3 (2002), pp. 782–810

    2002

  19. [27]

    On warm starts for interior methods

    A. Forsgren. “On warm starts for interior methods”. In: System Modeling and Optimization: Proc. of 22nd IFIP TC7 Conference held from July 18–22, 2005, in Turin, Italy 22. Springer. 2006, pp. 51–66

    2005

  20. [28]

    Templates for convex cone problems with applications to sparse signal recovery

    S. R. Becker, E. J. Candes, and M. C. Grant. “Templates for convex cone problems with applications to sparse signal recovery”. In: Math. Prog. Comput. 3.3 (2011), pp. 165–218

    2011

  21. [29]

    Comprehensive Examination of Un- rolled Networks for Linear Inverse Problems

    E. Chen et al. “Comprehensive Examination of Un- rolled Networks for Linear Inverse Problems”. In: arXiv preprint arXiv:2501.04608 (2025)

    2025 arXiv

  22. [30]

    Denoiser-Regulated Deep Unfolding Compressed Sensing with Learnable Fixed-Point Pro- jections

    Yu Zhou et al. “Denoiser-Regulated Deep Unfolding Compressed Sensing with Learnable Fixed-Point Pro- jections”. In: IEEE Trans. Circuits and Syst. for Video Technol. (2024)

    2024

  23. [31]

    Generalization error bounds for deep unfolding RNNs

    B. Joukovsky et al. “Generalization error bounds for deep unfolding RNNs”. In: Uncertainty in Artif. Intell. PMLR. 2021, pp. 1515–1524

    2021

  24. [32]

    On Generaliza- tion Bounds for Deep Compound Gaussian Neural Net- works

    C. Lyons, R. G. Raj, and M. Cheney. “On Generaliza- tion Bounds for Deep Compound Gaussian Neural Net- works”. In: arXiv preprint arXiv:2402.13106 (2024)

    2024 arXiv

  25. [33]

    A comprehensive survey of regression-based loss functions for time series forecasting

    A. Jadon, A. Patil, and S. Jadon. “A comprehensive survey of regression-based loss functions for time series forecasting”. In: Int. Conf. Data Manage., Analytics & Innov. Springer. 2024, pp. 117–147

    2024

  26. [34]

    Statistical properties of the log-cosh loss function used in machine learning

    R. A. Saleh and A.K. Saleh. “Statistical properties of the log-cosh loss function used in machine learning”. In: arXiv preprint arXiv:2208.04564 (2022)

    2022 arXiv

  27. [35]

    Visualizing the loss landscape of neural nets

    H. Li et al. “Visualizing the loss landscape of neural nets”. In: Adv. Neural Inf. Process. Syst. 31 (2018)

    2018

  28. [36]

    A deep unfolding method for satellite super resolution

    J. Wang et al. “A deep unfolding method for satellite super resolution”. In: IEEE Trans. Computa. Imag. 8 (2022), pp. 933–944

    2022

  29. [37]

    Dying relu and initialization: The- ory and numerical examples

    L. Lu et al. “Dying relu and initialization: The- ory and numerical examples”. In: arXiv preprint arXiv:1903.06733 (2019)

    2019 arXiv

  30. [38]

    Introduction to pytorch

    N. Ketkar. “Introduction to pytorch”. In: Deep learning with python. Springer, 2017, pp. 195–208

    2017

  31. [39]

    Adam: A method for stochas- tic optimization

    D. P. Kingma and J. Ba. “Adam: A method for stochas- tic optimization”. In: arXiv preprint arXiv:1412.6980 (2014)

    2014 arXiv

  32. [40]

    Early stopping-but when?

    L. Prechelt. “Early stopping-but when?” In: Neural Net- works: Tricks of the trade . Springer, 1998, pp. 55–69

    1998

  33. [41]

    Loss landscapes and optimization in over-parameterized non-linear systems and neural networks

    C. Liu, L. Zhu, and M. Belkin. “Loss landscapes and optimization in over-parameterized non-linear systems and neural networks”. In: Appl. and Comput. Harmon. Anal. 59 (2022), pp. 85–116

    2022

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.