REVIEW 3 major objections 6 minor 32 references
Code Rate Optimization via Neural Polar Decoders
T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A single alternating algorithm uses neural polar decoders to estimate channel capacity and to build practical polar codes for black-box channels.
desk verdict Useful integration of NPDs with input-distribution optimization and SCL decoding, but the gradient step rests on an unproven plug-in approximation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the neural polar decoder (NPD), which replaces the check-node, bit-node, embedding, and LLR functions of successive cancellation decoding with neural networks, giving a decoder whose complexity $O(LkdN\log N)$ does not depend on the channel's state size. Carrying the argument are two derived objects: the MI estimator of Eq. (23), the difference between the NPD's cross-entropy losses on real channel outputs and on constant zero outputs, whose consistency is stated in Theorem 4; and the score-function gradient of Theorem 5, implemented in Eq. (26) by using the per-sample loss difference as the plug-in estimate of $\log(P_{U|Y}/P_U)$. Code construction then uses the HY scheme (a polar-coding construction for non-uniform input distributions) with an adaptive frozen set to accommodate the optimized input, plus list decoding.
What would settle it
On a finite-state channel whose capacity-achieving input distribution is known exactly (for example a simple Markov channel), run Algorithm 3 and compute the true MI of the resulting distribution by brute-force dynamic programming. If the NPD's MI estimate continues to rise while the true MI falls below the uniform-i.i.d. value, the per-sample loss-difference surrogate is biased and the alternating loop can converge away from capacity.
Extended reading notes
Core claim
The paper's central claim is that Algorithm 3 simultaneously estimates the mutual information $I(X_\psi^N;Y^N)$ and optimizes the input distribution $P_\psi$ by alternating two steps. In the estimation step, two NPDs conditioned on the channel output and on a constant input respectively provide consistent estimates of conditional entropies, so their difference estimates the MI. In the improvement step, with the NPD fixed, the algorithm applies the score-function gradient from Theorem 5, replacing the intractable log-ratio $\log(P_{U|Y}/P_U)$ with the plug-in difference of NPD losses, and updates the LSTM parameters $\psi$. The paper argues that the optimized distribution's MI approaches the channel capacity lower bounds for the Ising and Trapdoor channels and that polar codes built from it, using the HY scheme and SCL decoding, attain lower bit error rates than uniform i.i.d. inputs for block lengths up to 1024.
Load-bearing premise
The optimization step assumes the per-sample difference of the two NPD losses is an accurate estimate of the log-ratio of posterior to prior on the transformed input, so that gradient ascent on that surrogate increases the true mutual information; only the averaged MI estimator, not this per-sample plug-in, is proved consistent in the paper.
Editorial extensions
If this is right
- For an unknown channel with memory, the same training run yields both an estimate of achievable rate and a working polar code, removing the usual separation between capacity estimation and code design.
- On channels whose capacity-achieving input is non-uniform, the optimized distribution gives higher estimated MI and lower BER than uniform i.i.d. inputs at block lengths up to 1024, so non-uniform shaping can be realized within the polar-code framework.
- Because NPD decoding complexity is $O(LkdN\log N)$ and independent of channel state size, list decoding can scale to channels where trellis-based decoders are infeasible.
- The adaptive frozen set trades a small rate loss for mitigation of the reduced polarization caused by optimized inputs, making the scheme practical for finite block lengths.
Reading between the lines
- The plug-in log-ratio gradient in Eq. (26) is a surrogate for the true density ratio; if it is biased, the loop maximizes the NPD's estimate rather than capacity, so finite-sample validation on a known channel is the decisive test.
- The same loss-difference trick could be reused as a density-ratio estimator for other neural coding or shaping schemes, since it only requires two NPD passes per sample.
- Reduced polarization under optimized inputs suggests that longer blocks or list sizes may be needed to close the gap to capacity, and that combining this method with polarization-rate-preserving shaping could improve finite-length performance.
- Treating the channel as a black box makes the procedure a candidate for adaptive link configuration in real systems, where the channel changes and the input distribution must be re-optimized from streaming samples.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes an alternating training procedure for black-box channels: a neural polar decoder (NPD) is used to estimate the mutual information between channel inputs and outputs (Eq. (23)), and a parametric LSTM input distribution is updated by gradient ascent where the log-density ratio in the exact score-function gradient is replaced by a per-sample difference of NPD losses (Eq. (26)). In the inference phase, the trained NPD is used to construct polar codes for asymmetric input distributions via the Honda-Yamamoto scheme with adaptive frozen sets and SCL decoding. Experiments on binary-input AWGN, Ising, and Trapdoor channels report MI estimates approaching published capacity bounds and BER/FER improvements over uniform i.i.d. inputs for block lengths up to 1024.
Significance. If the method works as claimed, it would be a useful step toward joint capacity estimation and code design for channels with memory, where model-based methods are often intractable. The paper's strengths include a public implementation, comparison against published capacity bounds and an SCT benchmark, an exact score-function gradient identity (Theorem 5), and a consistency result for the averaged MI estimator conditional on the assumptions of [19]. However, the theoretical results as stated do not cover the actual stage-averaged loss used in training, the coupled minimization of the two NPD embeddings, or the per-sample plug-in in the gradient update; these gaps leave the central optimization claim supported mainly by the experiments.
major comments (3)
- [III-D/E, Eq. (14), Theorems 3-4] The estimator in Eq. (23) is computed from the stage-averaged loss L(x^N,y^N;θ) = (1/(n+1)) Σ_{j=0}^n L_j(v_j,e_j;θ), where each L_j is an average cross-entropy over the stage-j bits v_j. Theorem 3, however, concerns the final-stage conditional log-loss -log σ(l_θ(y^N,u^i)) for H(U_i|U^{i-1},Y^N) and H(U_i|U^{i-1}). Minimizing a sum of stage-wise cross-entropies is a different variational problem, and the proof of Theorem 4 applies Theorem 3 directly to a quantity that is not the one being minimized. The claim in Section III-D that the guarantees 'remain intact' needs a proof; as written, consistency of bI^M_Θ is not established.
- [IV-B and Algorithm 3, Eq. (23)] Eq. (23) defines the MI estimate as the difference of two independent minimizations, min over θ_co and min over θ_ch, yet the NPD parameters F_θF, G_θG, H_θH are shared between θ_co and θ_ch, and Algorithm 3 updates a single coupled θ via the sum L(x,0;θ_co)+L(x,y;θ_ch). The two conditional-entropy terms cannot generally be minimized independently under this coupling, so the estimator actually implemented is not the estimator in Eq. (23). Theorem 4, which relies on separate consistency of the two conditional entropy estimators, therefore does not apply to the implemented algorithm without an additional argument.
- [IV-C, Eq. (26)] The gradient step replaces Q(x^N,y^N)=log(P_{U|Y}/P_U) in the exact score-function identity of Theorem 5 with the per-sample plug-in L(x,0;θ*_co)-L(x,y;θ*_ch). Theorem 4 only proves consistency of the averaged conditional MI estimate; it does not prove that this per-sample difference is a consistent estimate of the log-density ratio, nor that the expectation of the plug-in times ∇_ψ log P^ψ_X converges to the true gradient. Without such a result, Algorithm 3 may converge to a stationary point of the NPD's proxy objective rather than of the true MI. The empirical gaps in Fig. 2 (0.5415 vs the lower bound 0.5451 for Ising; 0.571 vs 0.572 for Trapdoor) are consistent with this concern, though finite-N effects are also possible.
minor comments (6)
- [Eq. (9)] The two terms in the cross-entropy loss are identical; the second term should presumably be (1-v0,i) log(1-σ(l0,i)).
- [Theorem 5 proof] The proof differentiates the expectation of Q without accounting for the ψ-dependence of Q; the missing term vanishes in expectation, but the proof should state this explicitly.
- [III-D] The claim that the fast loss computation reduces the number of computational steps from N log N to log N is unclear, since each stage computes O(N) node LLRs; please clarify what quantity is being counted.
- [Theorems 3-4 and Fig. 1] The theoretical statements are for indecomposable FSCs, while the AWGN experiment in Fig. 1 is memoryless and not covered by these statements; the paper should state the scope of the theory explicitly.
- [V-B3, Eq. (33)] The adaptive frozen set threshold t is heuristic, and no finite-block-length guarantee is given for the resulting code; this should be stated as a limitation.
- [Eq. (16)] The notation l_θ(y^N,u^i) should be defined explicitly with the sign convention used in Eq. (9), since the relationship between the LLR and the posterior probability is essential to the loss.
Circularity Check
No circularity found: the MI estimator is a proved plug-in, the optimization step uses it as an objective rather than defining the result, and the experimental claims are anchored to external capacity bounds and true BER simulations.
full rationale
No significant circularity is present. The paper's MI estimator in Eq. (23) is explicitly defined as a difference of NPD cross-entropy losses and is proved consistent in Theorem 4, which in turn relies on Theorem 3 imported from the authors' prior work [19]. That import is a stated mathematical theorem with assumptions (indecomposable FSC, compact parameter set) that do not include this paper's rate-optimization or coding conclusions, so it constitutes independent support rather than a definitional loop. The improvement step in Eq. (26) replaces the exact log-ratio Q from Theorem 5 with the plug-in difference L(x^N,0^N;theta*_co)-L(x^N,y^N;theta*_ch); this is an approximation whose per-sample unbiasedness is not delivered by Theorem 4, but it is a correctness and statistical-consistency gap, not an equivalence by construction or a fitted parameter renamed as a prediction. The reported MI values are compared against known external capacity bounds for the Ising and Trapdoor channels, and the BER experiments are true channel simulations benchmarked against the SCT decoder, so the central empirical claims are not self-defined. The observation that the final MI values sit slightly below the external lower bounds is a statistical or finite-N concern, not evidence of circularity. The self-citations [19] and [21] are used as theorem source and benchmark, respectively, and neither forces the paper's conclusion by citation alone.
Assumptions & free parameters
free parameters (5)
- NPD hidden units k and embedding dimension d =
not stated in paper
- LSTM state dimension =
not stated in paper
- learning rate gamma and iteration counts N_iters, N_warmup =
N_warmup=1000 used for AWGN; others not given
- adaptive frozen set threshold t =
t=1
- code rate settings =
0.4 and approximately 0.37-0.435 in Ising experiments
assumptions (5)
- domain assumption NPD consistency theorem from [19, Thm 4] holds for the channels and NN classes used here (indecomposable FSC, compact parameter space, universal approximation).
- domain assumption The Honda-Yamamoto scheme, with NPD-computed conditional distributions, yields a valid polar code for the optimized non-uniform input distribution.
- standard math The score-function gradient identity and the vanishing of the derivative of the normalization term in MI.
- ad hoc to paper Per-sample NPD loss differences L(x,0;theta*_co)-L(x,y;theta*_ch) are faithful plug-in estimates of log(P_U|Y/P_U) inside the gradient estimator.
- ad hoc to paper Polarization of synthetic channels under the optimized input distribution, combined with an adaptive frozen set, is sufficient for reliable finite-block-length coding.
Cite this review
Pith. "Pith review of Code Rate Optimization via Neural Polar Decoders." pith.science (2026). https://pith.science/paper/LDDTWXAF
@misc{pith2026250615836,
author = {Pith},
title = {Pith review of: Code Rate Optimization via Neural Polar Decoders},
year = {2026},
howpublished = {\url{https://pith.science/paper/LDDTWXAF}},
note = {Machine review of arXiv:2506.15836}
}
read the original abstract
This paper proposes a method to optimize communication code rates via the application of neural polar decoders (NPDs). Employing this approach enables simultaneous optimization of code rates over input distributions while providing a practical coding scheme within the framework of polar codes. The proposed approach is designed for scenarios where the channel model is unknown, treating the channel as a black box that produces output samples from input samples. We employ polar codes to achieve our objectives, using NPDs to estimate mutual information (MI) between the channel inputs and outputs, and optimize a parametric model of the input distribution. The methodology involves a two-phase process: a training phase and an inference phase. In the training phase, two steps are repeated interchangeably. First, the estimation step estimates the MI of the channel inputs and outputs via NPDs. Second, the improvement step optimizes the input distribution parameters to maximize the MI estimate obtained by the NPDs. In the inference phase, the optimized model is used to construct polar codes. This involves incorporating the Honda-Yamamoto (HY) scheme to accommodate the optimized input distributions and list decoding to enhance decoding performance. Experimental results on memoryless and finite-state channels (FSCs) demonstrate the effectiveness of our approach, particularly in cases where the channel's capacity-achieving input distribution is non-uniform. For these cases, we show significant improvements in MI and bit error rates (BERs) over those achieved by uniform and independent and identically distributed (i.i.d.) input distributions, validating our method for block lengths up to 1024. This scalable approach has potential applications in real-world communication systems, bridging theoretical capacity estimation and practical coding performance.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[19]
Data-Driven Neural Polar Codes for Unknown Channels With and Without Memory
Z. Aharoni, B. Huleihel, H. D. Pfister, and H. H. Permuter, “Data-driven neural polar codes for unknown channels with and without memory,” arXiv preprint arXiv:2309.03148, 2023
work page Pith review arXiv 2023
-
[1]
Code Rate Optimization via Neural Polar Decoders,
Z. Aharoni, B. Huleihel, H. D. Pfister, and H. H. Permuter, “Code Rate Optimization via Neural Polar Decoders,” in2024 IEEE International Symposium on Information Theory (ISIT), 2024, pp. 2424–2429
work page 2024
-
[2]
Computation of channel capacity and rate-distortion func- tions,
R. Blahut, “Computation of channel capacity and rate-distortion func- tions,”IEEE transactions on Information Theory, vol. 18, no. 4, pp. 460–473, 1972
work page 1972
-
[3]
An algorithm for computing the capacity of arbitrary dis- crete memoryless channels,
S. Arimoto, “An algorithm for computing the capacity of arbitrary dis- crete memoryless channels,”IEEE Transactions on Information Theory, vol. 18, no. 1, pp. 14–20, 1972
1972
-
[4]
A generalization of the Blahut- Arimoto algorithm to finite-state channels,
P. V ontobel, A. Kavcic, and D. Arnold, “A generalization of the Blahut- Arimoto algorithm to finite-state channels,”IEEE Trans. Inf. Theory, vol. 54, no. 5, pp. 1887–1918, 2008
work page 1918
-
[5]
Extension of the blahut–arimoto algorithm for maximizing directed information,
I. Naiss and H. H. Permuter, “Extension of the blahut–arimoto algorithm for maximizing directed information,”IEEE Transactions on Informa- tion Theory, vol. 59, no. 1, pp. 204–222, 2012
work page 2012
-
[6]
The capacity of channels with feedback,
S. Tatikonda and S. Mitter, “The capacity of channels with feedback,” IEEE Trans. Inf. Theory, vol. 55, no. 1, pp. 323–349, 2009
work page 2009
-
[7]
Capacity of the trapdoor channel with feedback,
H. H. Permuter, P. Cuff, B. V . Roy, and T. Weissman, “Capacity of the trapdoor channel with feedback,”IEEE Trans. Inf. Theory, vol. 54, no. 7, pp. 3150–3165, 2009
work page 2009
Show all 32 references
-
[8]
Universal estimation of directed information,
J. Jiao, H. H. Permuter, L. Zhao, and T. Weissman, “Universal estimation of directed information,”IEEE Trans. Inf. Theory, vol. 59, no. 10, pp. 6220–6242, 2013
2013
-
[9]
Capacity and coding for the Ising channel with feedback,
O. Elishco and H. H. Permuter, “Capacity and coding for the Ising channel with feedback,”IEEE Trans. Inf. Theory, vol. 60, no. 9, pp. 5138–5149, 2014
2014
-
[10]
Causality, feedback and directed information,
J. Massey, “Causality, feedback and directed information,”Proc. Int. Symp. Inf. Theory Applic. (ISITA-90), pp. 303–305, 1990
1990
-
[11]
Capacity estimation using machine learning,
Z. Aharoni, D. Tsur, Z. Goldfeld, and H. Permuter, “Capacity estimation using machine learning,” inMachine Learning and Wireless Communi- cation, A. Goldsmith, D. Gunduz, H. V . Poor, and Y . C. Eldar, Eds. The address of the publisher: Cambridge University Press, 02
-
[12]
Neural estimation and optimization of directed information over continuous spaces,
D. Tsur, Z. Aharoni, Z. Goldfeld, and H. Permuter, “Neural estimation and optimization of directed information over continuous spaces,”IEEE Trans. Inf. Theory, 2023
2023
-
[13]
Data-driven optimization of directed information over discrete alphabets,
——, “Data-driven optimization of directed information over discrete alphabets,”IEEE Trans. Inf. theory, 2023
2023
-
[14]
Discriminative Mutual Information Estimation for the Design of Channel Capacity Driven Autoencoders,
N. A. Letizia and A. M. Tonello, “Discriminative Mutual Information Estimation for the Design of Channel Capacity Driven Autoencoders,” in2022 International Balkan Conference on Communications and Networking (BalkanCom), 2022, pp. 41–45
2022
-
[15]
Learning Channel Capacity With Neural Mutual Information Estimator Based on Message Importance Measure,
Z. Li, R. She, P. Fan, C. Peng, and K. B. Letaief, “Learning Channel Capacity With Neural Mutual Information Estimator Based on Message Importance Measure,”IEEE Transactions on Communications, vol. 72, no. 3, pp. 1370–1384, 2024
2024
-
[16]
Channel polarization: A method for constructing capacity- achieving codes for symmetric binary-input memoryless channels,
E. Arikan, “Channel polarization: A method for constructing capacity- achieving codes for symmetric binary-input memoryless channels,”IEEE Trans. Inf. Theory, vol. 55, no. 7, pp. 3051–3073, 2009
2009
-
[17]
List decoding of polar codes,
I. Tal and A. Vardy, “List decoding of polar codes,”IEEE Trans. Inf. Theory, vol. 61, no. 5, pp. 2213–2226, 2015
2015
-
[18]
Technical specification group radio access network; nr; multiplexing and channel coding,
3rd Generation Partnership Project (3GPP), “Technical specification group radio access network; nr; multiplexing and channel coding,” 3GPP, Tech. Rep. 3GPP TS 38.212 version 16.5.0, March 2021
2021
-
[20]
Polar coding without alphabet extension for asymmetric models,
J. Honda and H. Yamamoto, “Polar coding without alphabet extension for asymmetric models,”IEEE Trans. Inf. Theory, vol. 59, no. 12, pp. 7829–7838, 2013
2013
-
[21]
Computable upper bounds on the capacity of finite-state channels,
B. Huleihel, O. Sabag, H. H. Permuter, N. Kashyap, and S. Shamai, “Computable upper bounds on the capacity of finite-state channels,” IEEE Trans. Inf. Theory, 2021
2021
-
[22]
Kramer,Directed Information for Channels with Feedback, 1998, vol
G. Kramer,Directed Information for Channels with Feedback, 1998, vol. 11
1998
-
[23]
Recurrent neural networks are universal approximators,
A. M. Fer and H. G. Zimmermann, “Recurrent neural networks are universal approximators,” inProceedings of International Conference on Artificial Neural Networks. Springer, 2006, pp. 632–640
2006
-
[24]
Construction of polar codes for channels with memory,
R. Wang, J. Honda, H. Yamamoto, R. Liu, and Y . Hou, “Construction of polar codes for channels with memory,” in2015 IEEE Information Theory Workshop-Fall (ITW). IEEE, 2015, pp. 187–191
2015
-
[25]
Beitrag zur theorie des ferromagnetismus,
E. Ising, “Beitrag zur theorie des ferromagnetismus,”Zeitschrift f ¨ur Physik, vol. 31, no. 1, pp. 253–258, 1925
1925
-
[26]
Information theory,
D. Blackwell, “Information theory,” inModern Mathematics for the Engineer: Second Series. New York, NY , USA: Dover, 1961, pp. 182– 193
1961
-
[27]
Long short-term memory,
S. Hochreiter, “Long short-term memory,”Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997
1997
-
[28]
Gradient estimation using stochastic computation graphs,
J. Schulman, N. Heess, T. Weber, and P. Abbeel, “Gradient estimation using stochastic computation graphs,”Advances in neural information processing systems, vol. 28, 2015
2015
-
[29]
Capacity and zero-error capacity of Ising channels,
T. Berger and F. Bonomi, “Capacity and zero-error capacity of Ising channels,”IEEE Trans. Inf. Theory, vol. 36, pp. 173–180, 1990
1990
-
[30]
On the capacity of generalized ising channels,
A. Sharov and R. Roth, “On the capacity of generalized ising channels,” inProc. IEEE Int. Symp. Inf. Theory (ISIT), 2015, pp. 2256–2260
2015
-
[31]
Some considerations on the trapdoor channel,
K. Kobayashi, H. Morita, and M. Hoshi, “Some considerations on the trapdoor channel,” inProceedings of the 3rd Asian-European Workshop on Information Theory, 2003, pp. 9–10
2003
-
[32]
FV polar coding for lossy compression with an improved exponent,
R. Wang, J. Honda, H. Yamamoto, and R. Liu, “FV polar coding for lossy compression with an improved exponent,” in2015 IEEE International Symposium on Information Theory (ISIT), 2015, pp. 1517– 1521
2015
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.