Pith. sign in

REVIEW 4 major objections 4 minor 33 references

DeepPolar+: Breaking the BER-BLER Trade-off with Self-Attention and SMART (SNR-MAtched Redundancy Technique) decoding

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DeepPolar+ sets out to prove that a neural polar code can improve block error rate and bit error rate simultaneously, ending the BER-BLER trade-off that limited DeepPolar.

desk verdict SMART gains are likely rate-driven from hidden CRC overhead; the attention decoder and joint loss are plausible but the paper's central claim needs equal-rate comparison. read the letter →

arxiv 2506.10166 v1 pith:NNQJBW7B submitted 2025-06-11 cs.IT eess.SPmath.IT

Shubham Srivastava , Adrish Banerjee This is my paper · ORCID
classification cs.ITeess.SPmath.IT MSC 94B3568T07
keywords polarcodesneuralchannelcodingself-attentiondecoderBER-BLERtrade-offsuccessivecancellationdecodingCRC-aideddeeplearningforerrorcorrectionAWGN
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the BER-BLER trade-off of DeepPolar codes is not fundamental but can be removed by changing the decoder and training objective. DeepPolar+ keeps the DeepPolar encoder and successive-cancellation-style sequential decoding, but inserts multi-head self-attention into each kernel decoder and adds a differentiable block-error term to the loss. Around this, the SMART (SNR-matched redundancy) decoder runs several models trained at different SNRs in parallel and uses CRC verification to pick a valid output. For a (256,37) code on AWGN, the authors report a 0.4 dB BER gain over SC decoding at $10^{-4}$ while slightly beating SC on BLER, with DP+SMART adding up to 0.5 dB over SC in BER and 0.6 dB over DeepPolar in BLER at $10^{-3}$. A sympathetic reader would care because it points toward learned codes that do not sacrifice block reliability for bit accuracy, with a 75% reduction in training epochs.

What carries the argument

The machinery is an attention-augmented neural successive-cancellation decoder paired with a block-aware loss and an SNR-specialized ensemble. Each kernel component network embeds the channel LLRs and previously decoded bits, applies multi-head self-attention with four heads of dimension 32 inside a residual, layer-normalized block, and then continues with feed-forward layers. The total loss is $L_{total} = L_{BCE} + L_{block}$, where $L_{block}$ is the average log-probability of the full codeword, making whole-block mistakes directly costly during training. The SMART decoder encodes a CRC with each message, decodes with M models trained at different SNR pairs, and returns the first candidate whose CRC check passes, falling back to a baseline model otherwise.

What would settle it

Recompute the BER and BLER curves for the SMART variants at their true information rate, namely k minus r information bits carried in 256 channel uses, and compare against SC decoding and DeepPolar at identical rates; if the reported gains collapse or reverse, the claimed trade-off elimination is an equal-rate artifact. Repeat the simulations at the 0.08 dB margin to see whether the apparent BLER improvement survives Monte Carlo noise.

Watch

Extended reading notes

Core claim

The central claim is that the BER-BLER trade-off observed in DeepPolar codes is an artifact of the decoder architecture and the training loss, not a property of neural polar codes. The authors show that inserting self-attention into the kernel decoders, adding a soft block-error term to the binary cross-entropy objective, and selecting among SNR-specialized decoders with CRC yields simultaneous improvement in both metrics for the (256,37) code over AWGN. They also report that the learned codewords have a pairwise-distance distribution close to that of a random Gaussian codebook, suggesting the model is using the available signal space well, and that training converges in 500 epochs versus the 2000 needed by DeepPolar.

Load-bearing premise

The comparison assumes every system sends the same number of information bits per codeword, but the SMART scheme adds CRC bits before encoding and the paper never states the resulting effective rate.

Editorial extensions

If this is right

  • If the claim holds, neural polar codes can match or exceed conventional SC decoding on block reliability while keeping their bit-level advantage, which is the property needed for practical use.
  • The SMART ensemble shows that training at several SNRs and picking a valid candidate with CRC can remove the degradation a single learned decoder suffers outside its training SNR.
  • The 500-epoch convergence result means the attention mechanism and structured loss cut training cost by about 75%, lowering the barrier to deploying learned codes.
  • Because decoding remains sequential, extending the scheme to list decoding by keeping parallel paths and CRC selection is a direct next step that the authors explicitly identify.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • At equal information rate, counting the CRC bits as overhead, the SMART gains may shrink; the paper never states the effective payload length, so a fair-rate comparison is the natural experiment to run next.
  • CRC-guided selection is functionally close to CRC-aided successive cancellation list decoding, so part of the SMART gain may come from the CRC redundancy itself; an ablation that gives the same CRC to the SC baseline would separate the effects.
  • The reported 0.08 dB BLER improvement over SC sits inside typical Monte Carlo noise at $10^{-3}$ without error bars, so independent replication with fixed codeword budgets would test whether the trade-off is truly eliminated.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes DeepPolar+, an attention-augmented neural successive cancellation decoder for DeepPolar codes, together with a loss that adds a soft block-error term to the bit-wise cross-entropy and a multi-model SNR-matched decoding framework with CRC verification (DP+SMART). For a (256,37) code on the AWGN channel, it reports BER and BLER improvements over successive cancellation and DeepPolar, including up to 0.5 dB BER gain at 10^-4 and 0.6 dB BLER gain at 10^-3 for DP+SMART, and claims that the architecture removes the BER-BLER trade-off while reducing training epochs by 75%.

Significance. If the reported gains survive equal-rate and equal-complexity comparisons, DeepPolar+ would be a meaningful advance in neural channel coding: it provides a public implementation, a detailed architecture description, and several useful analyses (training convergence, codeword-distance statistics, and SNR specialization). The paper is clearly written and the experiments are easy to reproduce in principle. However, the central quantitative claims are currently confounded by an unstated payload-rate reduction in the SMART scheme, by the absence of CRC-aided list-decoding baselines, and by test-set model selection. The contribution is therefore not yet established at the level claimed.

major comments (4)
  1. [III-C, Eq. (12)] The CRC augmentation is rate-confounding. Equation (12) writes x = g+_phi([u, CRC(u)]) with u in {0,1}^k, but the encoder g+_phi maps {0,1}^k to R^n; if the concatenated vector has k+r entries, the encoder domain is inconsistent. If k is instead the total number of encoder input bits including the CRC, then the true payload is k-r: 29 bits for the 8-bit CRC and 34 bits for the 3-bit CRC, not 37 bits. All curves in Fig. 2a are plotted against SNR (1/sigma^2), not Eb/N0, so the reported 0.5 dB BER and 0.6 dB BLER gains include the energy-per-information-bit advantage of a 21.6% lower rate. The effective information rate of every SMART variant must be stated, and equal-payload comparisons (e.g., Eb/N0 plots or (256,29) and (256,34) reference codes) are required before any gain can be attributed to the neural decoder.
  2. [IV-E, Fig. 2a] No CRC-aided successive cancellation list (CA-SCL) baseline is provided. Since DP+SMART uses CRC verification and an ensemble of decoders, the relevant comparison is CA-SCL with the same CRC polynomial and the same effective rate. Without this baseline, the reported gains over plain SC decoding could be explained by the CRC's error-detection capability and list diversity rather than by the attention-enhanced neural decoder. The paper's conclusion that DeepPolar+ is superior to existing polar decoding approaches is unsupported until such a baseline is included.
  3. [IV-E, Fig. 2d] The SMART SNR-pair set appears to be selected after inspecting Fig. 2d, which reports performance on the same SNR range used for evaluation. This constitutes model selection on the test set and inflates the reported DP+SMART gains. The authors need to specify how the five SNR pairs were chosen (e.g., on a separate validation set) and, ideally, average over multiple random choices of candidate pairs to assess selection bias.
  4. [IV-B] The key trade-off-breaking claim rests on a 0.08 dB BLER improvement at 10^-3 over SC decoding, yet no error bars, confidence intervals, or multiple-seed results are reported for any curve. Given typical Monte Carlo uncertainty at these error rates, this margin may be noise. The paper should report the number of simulated blocks, standard errors, or confidence bands for at least the headline BER and BLER values.
minor comments (4)
  1. [Throughout] The naming is inconsistent: 'DeepPolar', 'DeepPolar+', and 'Deeppolar' are used interchangeably in Fig. 2a and the text; please unify notation.
  2. [III-C] Equation (12) has a typo: 'u ∈ 0,1^k' should be 'u ∈ {0,1}^k', and the CRC polynomial should be defined with its standard generator before use.
  3. [IV-A] The empirical choice of kernel size ℓ=16 as √N is not justified; please provide an ablation or a reference for this heuristic.
  4. [IV-C] The Gaussian-distance analysis would be stronger with a quantitative divergence metric (e.g., KL divergence) and a comparison to the original DeepPolar distribution; the current visual similarity claim is qualitative.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central BER/BLER comparisons are grounded in external SC, DeepPolar, RM, KO, and DKO baselines, and the loss/SMART design choices are training choices rather than derivation inputs.

full rationale

The paper's central claim is empirical: DeepPolar+ and DP+SMART are trained and evaluated against external baselines (SC polar, DeepPolar, RM, KO, DKO) in Fig. 2a. The structured loss in Eq. (11) is a soft block-error surrogate, but optimizing a training objective that resembles the evaluation metric is standard supervised learning, not a derivation that reduces the result to its inputs. The SMART decoder's ensemble is selected using the SNR-behavior analysis in Fig. 2d, which is a model-selection (and possible overfitting-to-test) concern rather than a circular reduction; the reported curves are measurements, not predictions derived from those curves. DeepPolar [30] is external prior work (not self-citation), and the only same-author citation (DKO [25]) is a baseline, not a load-bearing premise. The CRC-payload/rate issue raised in the reader's take is a fairness-of-comparison concern (comparing at SNR rather than Eb/N0 with fewer information bits), not a circularity: the gain may be rate-driven, but it is not definitionally identical to an input. Therefore no circular step meets the evidentiary bar; score 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities. The hidden cost is in the free parameters: several architecture choices and the SMART SNR-pair set are fitted to the evaluation setting, and the CRC overhead changes the information rate without being declared. The central claims rest on the prior DeepPolar framework and on the empirically chosen hyperparameters listed above.

free parameters (5)
  • Kernel size l = 16
    Chosen as sqrt(N), stated as empirically determined; affects code structure and decoding complexity.
  • Attention heads and per-head dimension = ha=4, dk=32
    Stated as empirically determined to balance capacity and efficiency; no theoretical derivation.
  • Training SNR range = -5 dB to 0 dB
    Training distribution; evaluation spans -5 to -1 dB, so all reported gains are measured in-distribution.
  • SMART model set (SNR pairs) = {(0,-2), (-1,-3), (-3,-5), (1,-1), (-2,-4)} dB
    Selected after inspecting Fig. 2d, which is the same evaluation setting; this is a post-hoc selection that tunes the ensemble to the test range.
  • CRC polynomial length = 3-bit and 8-bit
    Choice of CRC determines overhead and false-accept probability; the payload-rate reduction is not accounted in the reported comparisons.
assumptions (4)
  • domain assumption The AWGN channel with known noise variance and LLR computation is the operating assumption.
    Eq. (3) feeds channel LLRs Ly into the decoder; all reported gains are for this channel model only.
  • domain assumption Successive cancellation decoding assumes previously decoded bits are correct when decoding later bits.
    Eq. (4) uses u_hat_0:j-1 as input; error propagation is not modeled, matching the DeepPolar architecture from [30].
  • domain assumption The DeepPolar encoder and two-phase curriculum training from [30] are taken as given.
    The paper modifies the decoder and loss but relies on the DeepPolar construction and pretrained kernels for its encoder.
  • ad hoc to paper A differentiable product of bit posterior probabilities is a valid surrogate for block error probability.
    Eq. (11) defines L_block as the log of the product of sigmoid probabilities; under dependent sequential decoding this is not the true BLER, only a surrogate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DeepPolar+: Breaking the BER-BLER Trade-off with Self-Attention and SMART (SNR-MAtched Redundancy Technique) decoding." pith.science (2026). https://pith.science/paper/NNQJBW7B

@misc{pith2026250610166,
  author       = {Pith},
  title        = {Pith review of: DeepPolar+: Breaking the BER-BLER Trade-off with Self-Attention and SMART (SNR-MAtched Redundancy Technique) decoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NNQJBW7B}},
  note         = {Machine review of arXiv:2506.10166}
}
read the original abstract

DeepPolar codes have recently emerged as a promising approach for channel coding, demonstrating superior bit error rate (BER) performance compared to conventional polar codes. Despite their excellent BER characteristics, these codes exhibit suboptimal block error rate (BLER) performance, creating a fundamental BER-BLER trade-off that severely limits their practical deployment in communication systems. This paper introduces DeepPolar+, an enhanced neural polar coding framework that systematically eliminates this BER-BLER trade-off by simultaneously improving BLER performance while maintaining the superior BER characteristics of DeepPolar codes. Our approach achieves this breakthrough through three key innovations: (1) an attention-enhanced decoder architecture that leverages multi-head self-attention mechanisms to capture complex dependencies between bit positions, (2) a structured loss function that jointly optimizes for both bit-level accuracy and block-level reliability, and (3) an adaptive SNR-Matched Redundancy Technique (SMART) for decoding DeepPolar+ code (DP+SMART decoder) that combines specialized models with CRC verification for robust performance across diverse channel conditions. For a (256,37) code configuration, DeepPolar+ demonstrates notable improvements in both BER and BLER performance compared to conventional successive cancellation decoding and DeepPolar, while achieving remarkably faster convergence through improved architecture and optimization strategies. The DeepPolar+SMART variant further amplifies these dual improvements, delivering significant gains in both error rate metrics over existing approaches. DeepPolar+ effectively bridges the gap between theoretical potential and practical implementation of neural polar codes, offering a viable path forward for next-generation error correction systems.

Figures

Figures reproduced from arXiv: 2506.10166 by the authors.

Figure 1
Figure 1. DeepPolar code architecture and training framework: (a) Hierarchical encoding structure for (16,8) code using 4×4 kernels, where each node gd,b represents a learned neural kernel at depth d and bit position b transforming input bits through a multi-layer network. (b) Sequential decoding structure with neural kernels fd,b maintaining the successive cancellation strategy while leveraging both channel information and p… view at source ↗
Figure 2
Figure 2. Comprehensive performance analysis of DeepPolar+ and its variants: (a) Performance comparison on AWGN channel showing BER (solid) and BLER (dashed) for DeepPolar+ and DeepPolar+SMART with 3-bit and 8-bit CRC against baseline approaches for (256,37) code, demonstrating consistent improvements over conventional SC decoding and DeepPolar. (b) Pairwise Euclidean distance distribution analysis comparing DeepPolar+ codewo… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 23 canonical work pages

  1. [1]

    A class of multiple-error-correcting codes and the decoding scheme,

    I. Reed, “A class of multiple-error-correcting codes and the decoding scheme,” Transactions of the IRE Professional Group on Information Theory, vol. 4, no. 4, pp. 38–49, 1954

    1954

  2. [2]

    Application of Boolean algebra to switching circuit design and to error detection,

    D. E. Muller, “Application of Boolean algebra to switching circuit design and to error detection,” Transactions of the I.R.E. Professional Group on Electronic Computers , vol. EC-3, no. 3, pp. 6–12, 1954

    work page 1954

  3. [3]

    Near Shannon limit error-correcting coding and decoding: Turbo-codes,

    C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon limit error-correcting coding and decoding: Turbo-codes,” in IEEE Interna- tional Conference on Communications , vol. 2, 1993, pp. 1064–1070 vol.2

    1993

  4. [4]

    Gallager, Low Density Parity Check Codes

    R. Gallager, Low Density Parity Check Codes . Cambridge, MA: MIT Press, 1963

    1963

  5. [5]

    Channel polarization: A method for constructing capacity- achieving codes,

    E. Arikan, “Channel polarization: A method for constructing capacity- achieving codes,” in IEEE International Symposium on Information Theory (ISIT) , 2008, pp. 1173–1177

    2008

  6. [6]

    Multiplexing and channel coding. Technical Specification 38.212, 3rd Generation Partnership Project (3GPP), 2018. Release 15

    3GPP, “Multiplexing and channel coding. Technical Specification 38.212, 3rd Generation Partnership Project (3GPP), 2018. Release 15.”

    work page 2018

  7. [7]

    Design of polar codes in 5G new radio,

    V . Bioglio, C. Condo, and I. Land, “Design of polar codes in 5G new radio,” IEEE Communications Surveys & Tutorials , vol. 23, no. 1, pp. 29–40, 2020

    work page 2020

  8. [8]

    Channel coding in 5G New Radio: A tutorial overview and performance comparison with 4G LTE,

    D. Hui, S. Sandberg, Y . Blankenship, M. Andersson, and L. Grosjean, “Channel coding in 5G New Radio: A tutorial overview and performance comparison with 4G LTE,” IEEE V ehicular Technology Magazine , vol. 13, no. 4, pp. 60–69, 2018

    work page 2018

Show all 33 references
  1. [9]

    CRC-aided decoding of polar codes,

    K. Niu and K. Chen, “CRC-aided decoding of polar codes,” IEEE communications letters , vol. 16, no. 10, pp. 1668–1671, 2012

    2012

  2. [10]

    Parity-check-concatenated polar codes,

    T. Wang, D. Qu, and T. Jiang, “Parity-check-concatenated polar codes,” IEEE Communications Letters , vol. 20, no. 12, pp. 2342–2345, 2016

    2016

  3. [11]

    Polar codes with dynamic frozen symbols and their decoding by directed search,

    P. Trifonov and V . Miloslavskaya, “Polar codes with dynamic frozen symbols and their decoding by directed search,” in 2013 IEEE Informa- tion Theory Workshop (ITW) . IEEE, 2013, pp. 1–5

    2013

  4. [12]

    Polar subcodes,

    ——, “Polar subcodes,” IEEE Journal on Selected Areas in Communi- cations, vol. 34, no. 2, pp. 254–266, 2015

    2015

  5. [13]

    Parity-check polar coding for 5G and beyond,

    H. Zhang, R. Li, J. Wang, S. Dai, G. Zhang, Y . Chen, H. Luo, and J. Wang, “Parity-check polar coding for 5G and beyond,” in 2018 IEEE International Conference on Communications (ICC) . IEEE, 2018, pp. 1–7

    2018

  6. [14]

    From sequential decoding to channel polarization and back again,

    E. Arıkan, “From sequential decoding to channel polarization and back again,” arXiv preprint arXiv:1908.09594 , 2019

    1908 arXiv

  7. [15]

    List decoding of polar codes,

    I. Tal and A. Vardy, “List decoding of polar codes,” IEEE transactions on information theory , vol. 61, no. 5, pp. 2213–2226, 2015

    2015

  8. [16]

    Polar codes: Characteri- zation of exponent, bounds, and constructions,

    S. B. Korada, E. ¸ Sa¸ so˘glu, and R. Urbanke, “Polar codes: Characteri- zation of exponent, bounds, and constructions,” IEEE Transactions on Information Theory , vol. 56, no. 12, pp. 6253–6264, 2010

    2010

  9. [17]

    Tinyturbo: Efficient turbo decoders on edge,

    S. A. Hebbar, R. K. Mishra, S. K. Ankireddy, A. V . Makkuva, H. Kim, and P. Viswanath, “Tinyturbo: Efficient turbo decoders on edge,” inIEEE International Symposium on Information Theory (ISIT) . IEEE, 2022, pp. 2797–2802

    2022

  10. [18]

    Multi-label and concatenated neural block decoders,

    C. T. Leung, M. Motani, and R. V . Bhat, “Multi-label and concatenated neural block decoders,” in IEEE International Symposium on Informa- tion Theory (ISIT) , 2020, pp. 274–279

    2020

  11. [19]

    Learning to decode linear codes using deep learning,

    E. Nachmani, Y . Be’ery, and D. Burshtein, “Learning to decode linear codes using deep learning,” in 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton) , 2016, pp. 341–346

    2016

  12. [20]

    Deep learning based communication over the air,

    S. Dörner, S. Cammerer, J. Hoydis, and S. ten Brink, “Deep learning based communication over the air,” IEEE Journal of Selected Topics in Signal Processing, vol. 12, no. 1, pp. 132–143, 2017

    2017

  13. [21]

    Hyper-graph-network decoders for block codes,

    E. Nachmani and L. Wolf, “Hyper-graph-network decoders for block codes,” Advances in Neural Information Processing Systems , vol. 32, 2019

    2019

  14. [22]

    Turbo autoencoder: Deep learning based channel codes for point-to-point communication channels,

    Y . Jiang, H. Kim, H. Asnani, S. Kannan, S. Oh, and P. Viswanath, “Turbo autoencoder: Deep learning based channel codes for point-to-point communication channels,” Advances in neural information processing systems, vol. 32, 2019

    2019

  15. [23]

    KO codes: inventing nonlinear encoding and decoding for reliable wireless communication via deep-learning,

    A. V . Makkuva, X. Liu, M. V . Jamali, H. Mahdavifar, S. Oh, and P. Viswanath, “KO codes: inventing nonlinear encoding and decoding for reliable wireless communication via deep-learning,” in 38th Interna- tional Conference on Machine Learning , ser. Proceedings of Machine Lear...

    2021

  16. [24]

    ProductAE: Towards training larger channel codes based on neural product codes,

    M. Vahid Jamali, H. Saber, H. Hatami, and J. H. Bae, “ProductAE: Towards training larger channel codes based on neural product codes,” arXiv e-prints , pp. arXiv–2110, 2021

    2021

  17. [25]

    Dense KO codes: Faster convergence and reduced complexity through dense connectivity,

    S. Srivastava and A. Banerjee, “Dense KO codes: Faster convergence and reduced complexity through dense connectivity,” in 2024 IEEE International Symposium on Information Theory (ISIT) , 2024, pp. 1415– 1420

    2024

  18. [26]

    Combin- ing belief propagation and successive cancellation list decoding of polar codes on a GPU platform,

    S. Cammerer, B. Leible, M. Stahl, J. Hoydis, and S. ten Brink, “Combin- ing belief propagation and successive cancellation list decoding of polar codes on a GPU platform,” in 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2017, p...

    2017

  19. [27]

    CRISP: Curriculum based sequential neural decoders for polar code family,

    S. A. Hebbar, V . V . Nadkarni, A. V . Makkuva, S. Bhat, S. Oh, and P. Viswanath, “CRISP: Curriculum based sequential neural decoders for polar code family,” in International Conference on Machine Learning . PMLR, 2023, pp. 12 823–12 845

    2023

  20. [28]

    Construction of polar codes with reinforcement learning,

    Y . Liao, S. A. Hashemi, J. M. Cioffi, and A. Goldsmith, “Construction of polar codes with reinforcement learning,” IEEE Transactions on Communications, vol. 70, no. 1, pp. 185–198, 2021

    2021

  21. [29]

    Nested construction of polar codes via transformers,

    S. K. Ankireddy, S. A. Hebbar, H. Wan, J. Cho, and C. Zhang, “Nested construction of polar codes via transformers,” arXiv preprint arXiv:2401.17188, 2024

    2024 arXiv

  22. [30]

    DeepPolar: Inventing nonlinear large-kernel polar codes via deep learning,

    S. A. Hebbar, S. K. Ankireddy, H. Kim, S. Oh, and P. Viswanath, “DeepPolar: Inventing nonlinear large-kernel polar codes via deep learning,” arXiv preprint arXiv:2402.08864 , 2024

    2024 arXiv

  23. [31]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017

    2017

  24. [32]

    SGDR: Stochastic gradient descent with warm restarts,

    I. Loshchilov and F. Hutter, “SGDR: Stochastic gradient descent with warm restarts,” in International Conference on Learning Representations, 2017. [Online]. Available: https://openreview.net/ forum?id=Skq89Scxx

    2017

  25. [33]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations (ICLR) , 2015

    2015

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.