{"id":"cb6c2247-d273-4fc6-8d9c-2525c1a7f3da","arxiv_id":"2501.14102","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A linear-attention transformer decoder achieves bit error rate comparable to a standard transformer and better than one-iteration belief propagation on 5G NR LDPC codes, with O(n) instead of O(n^2) complexity.","lead":"This paper proposes a linear-complexity transformer decoder for 5G LDPC error correction codes and shows it matches a standard transformer while beating one iteration of belief propagation. The result matters because transformer decoders were previously limited to small code sizes by quadratic computation, and this work is a step toward making deep learning decoders scalable for practical wireless systems.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The O(n) scalability claim depends on a fixed projection dimension K, but no scaling experiment or K disclosure supports K staying constant for 5G NR LDPC block sizes beyond n=576.","rationale":"The reader's weakest assumption identifies exactly the condition on which the central claim rests: linear complexity requires K to be constant in N. I agree that this is the most load-bearing concern. The paper's own limitation statement (larger PCM shapes not tested, best performance for n=20-150) reinforces that the O(n) claim is extrapolated, not demonstrated. A secondary internal inconsistency is worth flagging: Eq. (7) defines the syndrome as sigma = H c^T with c the transmitted codeword, which is identically zero by Eq. (3); if taken literally, the syndrome input carries no information, and the claimed parity-check guidance is absent or the notation omits a hard-decision step. This should be clarified, but it does not replace the K-scaling question as the primary risk. Because the proposed concrete test can settle the main concern and the current verdict already conditions acceptance on further evidence, I recommend no change to the reader's CONDITIONAL verdict.","tokens_in":6107,"tokens_out":7138,"duration_ms":68054,"concrete_test":"Using the released code, train the linear transformer on the same 5G NR LDPC rate-1/2 family at n=576 and at n=1152 (or the largest supported base graph size) with the same K used in Section 5, reporting K and projection parameter counts. Compare BER at matched Eb/N0 values; if BER at n=1152 is materially worse than at n=576 and increasing K restores performance, the fixed-K assumption is false and the O(n) claim collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 defines linear attention by projecting keys and values into a low-rank space of dimension K and states that fixing K as a constant makes complexity O(N). The central scalability claim of the paper therefore holds only if a single K suffices as the block length n grows. The paper never reports the K values used in Figures 3-8, and all BER experiments stop at n=576; Section 5 explicitly says larger PCM shapes could not be tested due to GPU memory. Linformer's constant-K construction is not by itself sufficient evidence here: it approximates full attention under generic structural assumptions, whereas this decoder's target function is tied to a 5G NR parity-check matrix whose number of check nodes m=n-k grows with n. If K must grow with n to maintain BER, the realized complexity is O(nK), not O(n), and the headline scalability claim fails. This is the load-bearing risk; the weak one-iteration BP baseline and the absence of error bars are secondary reporting issues that do not directly attack the complexity claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes two transformer-based decoders for 5G New Radio LDPC codes: a regular softmax-attention decoder and a linear-attention decoder inspired by Linformer. Both incorporate the parity-check matrix as a mask in self-attention, following Choukroun and Wolf's error-correction transformer. The central claims are that the linear decoder reduces complexity from O(n^2) to O(n), that it matches a regular transformer decoder and surpasses one-iteration belief propagation in bit error rate, and that it is competitive in time with BP even for larger block codes. Experiments are reported for code sizes from (13,26) up to (256,576), with Sionna used for reproducible 5G NR LDPC construction. A short hyper-parameter study and timing comparison are also included.","tokens_in":6318,"tokens_out":3197,"duration_ms":32830,"significance":"If established, a learned decoder with genuinely O(n) complexity and BER competitive with multi-iteration BP would be a practically important result for neural channel decoding, since standard transformer decoders scale quadratically. The paper has concrete strengths: the code is publicly released, the implementation is built on Sionna, and the authors report a hyper-parameter study. However, the significance is presently limited by two gaps: the empirical comparison is made only against one-iteration BP, and the O(n) complexity claim is not supported by any disclosure of the low-rank dimension K or by scaling experiments beyond n=576. The central idea is plausible and the reported pipeline is reproducible, but the evidence as written does not yet establish the headline scalability or practical decoding claims.","major_comments":[{"comment":"The O(n) complexity claim is load-bearing but is not verified. Section 4.2 states that complexity becomes O(N) if K is fixed as a constant, yet the manuscript never reports the values of K used in Figures 3-8, and all BER experiments stop at n=576. Since the number of check nodes m=n-k grows with n, the input length N=n+m grows, and if K must increase with N to maintain decoding quality, the realized complexity is O(NK), not O(N). The statement in Section 5 that larger PCM shapes could not be tested due to GPU memory further suggests that the constant-K assumption has not been validated. The authors should report K for every experiment, provide BER scaling for larger n (e.g., n=1152 and beyond, possibly with memory-efficient implementations), and either demonstrate empirically that fixed K preserves BER or give a structural argument tied to the 5G NR parity-check matrix.","section":"Section 4.2 and Section 5"},{"comment":"The only BP baseline is one-iteration belief propagation, which is much weaker than the iterative BP used in 5G NR production receivers. The abstract's phrase 'surpasses one iteration BP' is accurate but the conclusion's characterization of this as 'the current production standard for 5G NR LDPC decoding' is misleading. Since the paper claims competitive performance against BP, the comparison should include BP with the iteration counts normally used in practice (for example, 5, 10, and 20 iterations) and report the corresponding BER and runtime. Without this, neither the BER advantage nor the 'competitive time performance' claim can be assessed against the actual decoding baseline in 5G systems.","section":"Section 5, Figures 3-8"},{"comment":"No error bars, confidence intervals, or number of independent training runs are reported. For small block sizes such as (13,26), the apparent margin over one-iteration BP could be within run-to-run variation, especially given the modest training budgets described. The authors should report statistics over multiple seeds or at least provide the number of runs and the observed variance. This is a reporting issue that directly affects the strength of the central empirical claim.","section":"Section 5, Figures 3-8"},{"comment":"The manuscript does not specify whether the learned projection matrices P_K and P_V are trained separately for each block length or whether a single trained model generalizes across n. Since 5G NR uses many different lifted base graphs and block sizes, the practical contribution depends on this point. The authors should state clearly whether each BER curve corresponds to a model trained on that exact code shape, and if so, whether the O(n) claim refers only to inference for a fixed trained model or also to the training procedure across sizes.","section":"Section 4, Figure 2 and Section 5"}],"minor_comments":[{"comment":"There is a typo in the abstract: 'surpases' should be 'surpasses'.","section":"Abstract"},{"comment":"The text contains typos: 'recieved' should be 'received' and 'conatenation' should be 'concatenation'.","section":"Section 2.2, Eq. (8)"},{"comment":"Figure 2 is referenced as 'illustrated below' but the figure is not present in the provided text; the figure placement needs to be corrected.","section":"Section 4, Figure 2"},{"comment":"The conclusion says performance is 'comparable to one-iteration belief propagation', while the abstract claims it 'surpasses one iteration BP'. These statements should be reconciled.","section":"Section 6, Conclusion"},{"comment":"The time comparison in Figure 5 needs details on hardware, batch size, GPU versus CPU execution for BP, and whether the reported times include training or only inference; otherwise the timing comparison is difficult to interpret.","section":"Section 5, Figure 5"},{"comment":"The text says models were trained for approximately 6 hours for each PCM shape and later says all models were trained for 1000 iterations around 8 minutes per model; the relationship between iterations, epochs, and wall-clock time should be clarified.","section":"Section 5, hyper-parameter study"}],"recommendation":"major_revision","confidential_remarks":"The paper is more of a short workshop-style empirical report than a full journal paper, but the core idea is legitimate and the code release is a positive feature. The missing K values and the absence of a multi-iteration BP baseline are fixable with additional experiments, so I do not see grounds for rejection. The stress-test concern about K scaling is real and should be addressed head-on by the authors."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a straightforward mashup of the masked attention from Choukroun and Wolf and Linformer-style linear attention, applied to 5G NR LDPC codes. That combination is genuinely new as far as I know, and the authors are honest that it is not the full ECCT architecture. Credit is due for shipping code in Sionna, reporting BER and time results, and not pretending the masked attention is invented here. There is no circular reasoning, no relabeled fitted constants, and the limitation about GPU memory is stated plainly.\n\nThe soft spots are real but not disqualifying. The O(n) complexity claim is the load-bearing risk. The paper never reports the projection dimension K used in any experiment, and all block lengths stop at 576. Since the sequence length here is n+m and m = n-k grows with n, a fixed K must keep enough of the parity-check structure to preserve BER as n grows. Linformer guarantees low-rank approximation only under generic assumptions; nothing in the paper argues that LDPC decoding attention stays low-rank with constant K. If K must scale, the realized complexity is O(nK), not O(n). This should have been tested or at least discussed. The baseline is also generous: one-iteration BP is far weaker than the iterative BP used in production 5G, and the paper's own figures show the transformer roughly matching one iteration, not beating full BP. Missing error bars and incomplete hyperparameter disclosure are minor but should be fixed.\n\nNone of this suggests bad faith. The idea is plausible and the code is usable. But the current evidence only supports a conditional conclusion for short-to-moderate block lengths against a weak baseline. A serious referee could turn this into a solid workshop paper or a shorter journal contribution by asking for K values, a scaling experiment, and a 10-iteration BP comparison.\n\nI would send it to peer review rather than desk reject, because the reproducibility and the clear, testable claim deserve referee time even if the outcome is major revision. I would not cite it in its current state, but I would read a revised version.","headline":"A plausible but thinly evaluated application of Linformer to LDPC decoding, where the headline O(n) claim rests on an unexamined constant-rank assumption.","tokens_in":6820,"tokens_out":1996,"would_cite":false,"duration_ms":19775,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A transformer decoder with linear attention decodes 5G NR LDPC codes at $O(n)$ cost while matching a regular transformer and beating one-iteration belief propagation.","keywords":["5G New Radio","LDPC codes","channel decoding","transformer","linear attention","belief propagation","neural decoding","bit error rate"],"falsifier":"Evaluate the same linear transformer with the same fixed bottleneck width $K$ on 5G NR LDPC codes with block lengths well beyond $n = 576$, for example $n$ from $1000$ to $2000$, at matched $E_b/N_0$ values, and compare bit error rate with the regular transformer and with belief propagation. If the best $K$ needed to match BER grows with $n$, or the linear decoder's advantage over one-iteration BP disappears at these sizes, the fixed-$K$ $O(n)$ claim is refuted; a complementary check is to measure per-codeword decoding time and see whether it grows linearly in $n$.","tokens_in":5921,"feed_emoji":"📡","tokens_out":11675,"duration_ms":94127,"temperature":0.7,"pith_summary":"This paper tries to establish that transformer-based neural decoders can scale to practical 5G New Radio LDPC codes if the attention mechanism is replaced by a linear one. The authors build a decoder whose attention is masked by the code's parity-check matrix, then swap quadratic self-attention for a low-rank key/value projection so that per-codeword complexity drops from $O(n^2)$ to $O(n)$. Their bit-error-rate results match a regular transformer decoder and beat one-iteration belief propagation on code lengths up to $n = 576$, with linear-attention models training about three times faster. If the scaling holds, it removes the main obstacle to learned decoding of the larger block codes used in wireless systems.","feed_headline":"O(n) transformer decoder beats one-iteration BP on 5G LDPC","feed_subtitle":"Matches a regular transformer and beats one-iteration BP in tests up to n=576","key_machinery":"The load-bearing mechanism is low-rank projected self-attention, applied inside an error-correction-code transformer. In standard attention the score tensor has shape $B \\times H \\times N \\times N$; projecting the keys and values through matrices $P_K, P_V \\in \\mathbb{R}^{N \\times K}$ with $K \\ll N$ yields score tensors of shape $N \\times K$, and the weighted sum then costs $O(NK)$, which is $O(N)$ when $K$ is a fixed constant. The companion mechanism is the parity-check-matrix mask from [3]: the decoder input concatenates channel log-likelihood ratios with the syndrome $Hc^T$, and the attention mask encodes the nonzero entries of $H$, guiding the model toward codewords that satisfy the parity constraints. Together these components keep code structure in the model while avoiding quadratic scaling.","core_discovery":"The paper's central claim is that a fully differentiable transformer decoder with linear attention can decode 5G New Radio LDPC codes at $O(n)$ cost per codeword, while standard self-attention would cost $O(n^2)$. The decoder follows the error-correction-code transformer design: its input is the vector of channel log-likelihood ratios concatenated with the syndrome $\\sigma = Hc^T$, and the parity-check matrix $H$ is folded into the attention mask so the model is guided by the code's constraints. The linear variant replaces ordinary attention with the low-rank projection from [19], compressing keys and values through a fixed dimension $K$ so the attention score tensor has shape $N \\times K$ rather than $N \\times N$. With $K$ held constant, the attention cost is linear in the sequence length. The paper reports that this linear decoder matches the bit error rate of a regular transformer decoder, surpasses one-iteration belief propagation, and trains about three times faster on the block sizes tested.","pith_inferences":["An untested implication of the paper's constant-$K$ assumption is that the bottleneck width does not need to grow with code length; a direct sweep of block sizes beyond $n = 576$ with fixed $K$ would determine whether the $O(n)$ claim survives at production 5G NR lengths.","Because linear attention compresses all key/value context through a fixed low-rank bottleneck, the architecture may act as a regularizer as well as a speedup; separating that effect from the extra training budget (for example by equalizing wall-clock training time) would identify what drives the reported bit error rate gains.","The comparison against only one-iteration BP leaves open how the decoder would fare against multi-iteration BP as deployed in practice; extending the benchmark to five or ten BP iterations would place the result in a production context."],"forward_implications":["Deep-learning decoding becomes a realistic option for 5G NR LDPC codes at block lengths where regular transformer decoders are computationally prohibitive.","A linear-attention decoder can beat one-iteration belief propagation on 5G NR LDPC codes, indicating that learned decoders capture useful structure beyond a single message-passing pass.","Linear-attention decoders train about three times faster than regular transformers at the tested sizes, so for a fixed compute budget they can train longer and reach lower bit error rates.","Because the parity-check matrix enters through the attention mask, the same architecture can be applied to regular LDPC and 5G NR LDPC codes without a code-specific redesign.","The reported quick convergence to one-iteration belief propagation suggests transformer decoders could serve as cheap learned replacements for a single BP pass in iterative receivers."],"supporting_citations":[{"why":"Supplies the error-correction-code transformer architecture and the parity-check-matrix attention mask that the paper adapts, and is the regular-transformer baseline matched in the BER comparisons.","marker":"[3]"},{"why":"Supplies the low-rank key/value projection that turns $O(N^2)$ attention into $O(NK)$, the mechanism underlying the paper's linear-complexity claim.","marker":"[19]"},{"why":"Defines the standard transformer self-attention that both the regular and linear decoders build on.","marker":"[18]"},{"why":"Provides the 5G NR LDPC code construction, rate matching, modulation, and simulation environment used for the bit error rate and timing results.","marker":"[11]"},{"why":"Grounds the description of 5G NR LDPC design and rate matching that determines the parity-check matrices used in the experiments.","marker":"[16]"}],"fun_headline_variants":["O(n) transformer decoder beats 1-iter BP on 5G LDPC","Linear attention for 5G LDPC decoding matches BP and is faster","Beat one-iteration BP with a linear transformer for 5G LDPC","Transformer with O(n) attention surpasses single-pass BP on 5G LDPC","Fast 5G LDPC decoding: linear transformer outperforms 1-iter BP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument rests on the assumption that the fixed small width of the linear-attention bottleneck, chosen for the tested block sizes, keeps working without loss as the code length grows; if the bottleneck has to widen with $n$, the claimed $O(n)$ scaling and the scalability conclusion both fail.","fun_headline_variants_meta":{"raw":{"variants":["O(n) transformer decoder beats 1-iter BP on 5G LDPC","Linear attention for 5G LDPC decoding matches BP and is faster","Beat one-iteration BP with a linear transformer for 5G LDPC","Transformer with O(n) attention surpasses single-pass BP on 5G LDPC","Fast 5G LDPC decoding: linear transformer outperforms 1-iter BP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000773,"raw_usage":{"total_tokens":3385,"prompt_tokens":874,"completion_tokens":2511,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":2400}},"tokens_in":490,"tokens_out":2511,"duration_ms":15186,"temperature":1.0,"reasoning_tokens":2400,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:21:37.405425+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the same linear transformer with the same fixed bottleneck width $K$ on 5G NR LDPC codes with block lengths well beyond $n = 576$, for example $n$ from $1000$ to $2000$, at matched $E_b/N_0$ values, and compare bit error rate with the regular transformer and with belief propagation. If the best $K$ needed to match BER grows with $n$, or the linear decoder's advantage over one-iteration BP disappears at these sizes, the fixed-$K$ $O(n)$ claim is refuted; a complementary check is to measure per-codeword decoding time and see whether it grows linearly in $n$.","supporting_citations":[{"cited_title":"Error Correction Code Transformer","cited_arxiv_id":"2203.14966","evidence_quote":"Supplies the error-correction-code transformer architecture and the parity-check-matrix attention mask that the paper adapts, and is the regular-transformer baseline matched in the BER comparisons."},{"cited_title":"Richardson and S","cited_arxiv_id":null,"evidence_quote":"Grounds the description of 5G NR LDPC design and rate matching that determines the parity-check matrices used in the experiments."}],"review_version":1}