Computationally Efficient Neural Receivers via Axial Self-Attention

Atchutaram K. Kocharlakota; Esa Ollila; M\'ario Costa; SaiKrishna Saketh Yellapragada; Sergiy A. Vorobyov

REVIEW 3 major objections 4 minor 21 references

Computationally Efficient Neural Receivers via Axial Self-Attention

T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A transformer neural receiver that factorizes self-attention along time and frequency axes achieves near-identical or better block error rates than global self-attention at 2.8x lower computational cost.

desk verdict Solid complexity story, shaky BLER gains: axial attention for neural receivers is a sensible engineering idea that needs more rigorous empirical validation. read the letter →

arxiv 2510.12941 v3 pith:YE6BHONV submitted 2025-10-14 eess.SP

SaiKrishna Saketh Yellapragada , Atchutaram K. Kocharlakota , Mário Costa , Esa Ollila , Sergiy A. Vorobyov This is my paper · ORCID

classification eess.SP

keywords neuralreceiveraxialattentiontransformerOFDMblockerrorratecomputationalcomplexity6Gself-attention

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a transformer neural receiver for OFDM can be made dramatically cheaper without sacrificing detection quality by replacing global self-attention with axial self-attention—attention computed separately along the time axis and the frequency axis of the resource grid. The claimed result is a reduction in computational complexity from O((TF)^2) to O(T^2F+TF^2), which for a 14×128 resource grid amounts to a 12.6× theoretical reduction and a measured 2.81× reduction in FLOPs versus global attention. Sympathetically read, the paper intends to establish that factorized attention is a practical design choice for 6G neural receivers, retaining long-range dependency modeling while making edge deployment feasible. The reported BLER curves show small gains (0.12–0.40 dB) over global attention and over CNN baselines, plus robustness at 1% BLER under high mobility where classical LS-LMMSE fails.

What carries the argument

Axial self-attention: a factorization of multi-head self-attention in which attention is computed independently along the time axis (for each fixed subcarrier, a T×T attention matrix) and along the frequency axis (for each fixed OFDM symbol, an F×F attention matrix), with separate query/key/value projections for each axis and residual connections around each. It replaces the single (TF)×(TF) attention matrix of global MHSA with F matrices of size T×T and T matrices of size F×F, reducing per-block complexity from O(T^2F^2 D) to O(TF D(T+F)).

What would settle it

Train both axial and global MHSA receivers on the same hardware with identical hyperparameters and data, repeat with at least five random seeds, and compare BLER distributions at 10% and 1% BLER under CDL-C at 30 m/s. If the axial receiver is not at least as good as global MHSA within one standard deviation across seeds, the performance-equivalence part of the claim fails, even though the 2.8× FLOP reduction would still hold.

Watch

Extended reading notes

Core claim

The central claim is that factorizing the self-attention operation along the two axes of an OFDM time-frequency grid—first attending across OFDM symbols for each subcarrier, then across subcarriers for each OFDM symbol—reduces the quadratic cost of global attention to a sum of two quadratic terms, while preserving the model's ability to capture long-range temporal and spectral dependencies. On a 14-symbol, 128-subcarrier grid with 128-dimensional embeddings, the paper reports 3.34 GFLOPs per inference for the axial receiver versus 9.40 for global multi-head self-attention and 11.81 for a CNN-ResNet baseline, with the axial model achieving 0.12–0.40 dB better BLER than global attention at 10%

Load-bearing premise

The reported performance advantage over global self-attention rests on a single set of BLER curves without error bars or repeated training runs, so the measured 0.12–0.40 dB differences could lie within run-to-run variance; the complexity reduction, however, does not depend on these curves.

Editorial extensions

If this is right

Neural receiver inference cost drops to roughly a third of global-attention and CNN baselines, making transformer receivers more plausible in latency- and power-constrained edge hardware.
The theoretical complexity reduction scales with grid size: for larger bandwidths (hundreds of subcarriers), the axial receiver's advantage over global attention grows as TF/(T+F).
At 1% BLER and high user speeds (40 m/s), the axial receiver keeps working where classical LS-LMMSE equalization fails to converge, supporting ultra-reliable low-latency applications.
The architectural change is a drop-in modification to the attention block; the rest of the receiver (input projection, positional encoding, output projection) is unchanged, so existing training pipelines and quantization techniques likely transfer.
The paper claims a favorable efficiency-performance trade-off: 64% fewer FLOPs than global MHSA for 33% more parameters, and 1.6M parameters versus 9.7M for CNN-ResNet.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The performance equivalence between axial and global attention on small grids suggests that time-frequency correlations in OFDM channels are nearly separable; if so, even cheaper factorized variants (e.g., sharing projections between axes or using fewer heads) might retain most of the gain.
A direct test of the separability assumption would be to measure the performance gap as F grows toward 5G full bandwidths (e.g., 512–3200 subcarriers); the paper only tests F=128, so the claimed 12.6× reduction may understate or overstate real gains at scale.
Because the paper reports no variance or multiple seeds, the 0.12–0.40 dB gains over global MHSA may partly be training noise; a replication with repeated runs would separate architectural benefit from run-to-run fluctuation.
The method is not limited to receivers: the same axial factorization could apply to other 2D wireless processing tasks (channel prediction, beam management) where global attention is currently too costly.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

You should know this paper does two very different things. The complexity analysis is clean: factorizing MHSA along time and frequency cuts the attention FLOPs from quadratic in T*F to O(TF(T+F)), and their Table III shows a real 2.8x inference FLOP reduction (3.34 vs 9.40 GFLOPs) at a modest 1.3x parameter increase. That half of the paper is sound and useful.

The other half — the claim that axial attention matches or beats global MHSA and CNN-ResNet in BLER — is not established. The 0.12–0.40 dB gains in Fig. 3 come from single unlabeled curves with no error bars, no seeds, and no training hyperparameters. At that effect size, run-to-run variance could flip the ordering. The authors also compare to LS-LMMSE, where the gains are large, but that doesn't validate neural-vs-neural superiority. They also omit the most relevant transformer-based receiver baseline from the efficiency comparison.

The novelty is modest. Axial attention is known (Ho et al. 2019), and this is a direct application to the OFDM resource grid. Still, the idea is well-motivated for 6G edge deployment, and the complexity derivation is correct. The paper is clearly written and the architecture is easy to follow. I'd give credit for the honest 'work in progress' note on code, but the promise of code isn't the same as having it.

For peer review: I would not desk-reject this. The core engineering contribution — factorized attention for neural receivers — is worth refereeing. But the empirical section needs a major revision: multiple seeds, confidence intervals, full hyperparameter disclosure, and a more relevant set of baselines. If the BLER differences vanish under proper evaluation, the paper still stands as an efficiency contribution, just with weaker language about performance.

I'd bring it to a reading group if anyone in the group works on neural receivers (otherwise it's a bit niche). I wouldn't cite it yet until the code and seeded runs appear.

Recommendation: send to peer review with the expectation of significant empirical revision.

Referee Report

3 major / 4 minor

Summary. The paper proposes an axial self-attention transformer for OFDM neural receivers. Instead of applying multi-head self-attention to the flattened time–frequency grid of TF elements, the architecture applies attention separately along the time axis (F independent sequences of length T) and frequency axis (T independent sequences of length F), with residual connections and layer normalization. The authors claim this reduces asymptotic attention complexity from O((TF)^2) to O(T^2F + TF^2) and, in simulations under 3GPP CDL-C/D channels, yields BLER performance comparable to or better than global-MHSA and CNN-ResNet baselines while reducing measured inference FLOPs from 9.40 to 3.34 GFLOPs. The paper concludes that axial attention is a practical efficiency–accuracy trade-off for AI-native 6G receivers.

Significance. Strengths: the factorized-complexity derivation in Sections III–V follows directly from the definitions and contains no fitted parameters; Table III provides concrete parameter and FLOP counts; the architecture is clearly described, and comparison to a global-MHSA receiver is a natural ablation. If the reported BLER gains are reproducible, the two-axis factorization is a useful design for high-dimensional resource grids. However, the experimental section does not currently support the accuracy half of the central claim: single curves, no variance estimates, no training hyperparameters, and no comparison to the most relevant efficient transformer baselines. The efficiency half is plausible but overstated by the advertised 12.6x complexity-reduction factor.

major comments (3)

[§VI, Fig. 3] All BLER comparisons rest on one set of curves. No error bars, no retraining over seeds, and no learning-rate/batch-size/epoch/regularization details are reported. The claimed gains over global MHSA are 0.12–0.40 dB, and over CNN-ResNet about 0.1–0.2 dB; these are small enough to be explained by run-to-run variance. The phrases 'consistently outperforms' (Abstract, §VI) and 'state-of-the-art' are therefore not yet justified. Please provide at least three independent training runs with mean and spread, plus the full training protocol. The large LS-LMMSE gap does not fix this, since that is a neural-versus-model-based comparison.
[§V, Eq. (17); Table III] The asymptotic complexity is correct for the attention matrices alone, but Eq. (17) omits the QKV and output projection cost O(T F D^2), and the separate time/frequency projections double this cost in the axial block. This omission matters at the evaluated sizes: with T=14, F=128, D=128, the linear projection terms are comparable to or larger than the factorized attention terms, which is why Table III shows a measured reduction of only 2.81x (9.40/3.34 GFLOPs), not the advertised 12.6x. Please report the full FLOP count and present the 12.6x as the attention-only asymptotic ratio under the assumption D << (T+F), or revise the claim.
[§VI, Table I/II; Ref. [11]] The paper claims the axial receiver achieves state-of-the-art BLER, but evaluates only global MHSA, a CNN-ResNet, and LS-LMMSE. The closest prior transformer receiver [11], which applies tiled MHSA on resource blocks, is cited but not compared. No windowed, linear, or other efficient attention baseline is included, so the efficiency–accuracy trade-off against existing efficient attention mechanisms is unknown. Please add at least one relevant efficient-attention baseline or restrict the claims to 'versus global MHSA and the DeepRx-style CNN'.

minor comments (4)

[Footnote 1] The footnote states 'Work in Progress: Code will be available...' Since the code is not available at submission, this should be removed or replaced with a clear statement of when and where the code will be released; an unavailable link is not reproducible.
[References] Reference [17] contains a typo: 'K. He aand X. Zhang' should be 'K. He, X. Zhang, S. Ren, and J. Sun'.
[Table I] Minor formatting: 'SNR0–15 dB' should be 'SNR 0–15 dB'; 'DMRS Configuration 3 rd, 12 th symbol' should be '3rd and 12th symbols'.
[Fig. 3] The figure refers to 'upper panel' and 'lower panel' but has no subplot labels or described axes. Add (a)/(b), axis labels, and BLER target markers so the reading is unambiguous.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: complexity reduction follows from the architecture definitions, BLER results are external simulations, and self-citations are background only.

full rationale

The paper's central complexity claim is derived directly from the definitions of global and axial attention: global MHSA computes an N×N attention matrix with N=TF (Eqs. 14–16), while axial attention computes F matrices of size T×T and T matrices of size F×F (Eqs. 6–13), yielding O(T F D(T+F)) in Eq. (17). This is exact arithmetic on the architecture's definitions, with no fitted parameter, empirical constant, or external result needed. The BLER comparisons in Section VI are empirical evaluations of trained networks under 3GPP CDL-C/D channels using Sionna; they are not quantities fitted to the model, and no parameter is set so as to force a particular BLER outcome. The self-citations [8], [9], [12]–[14] appear only as background or motivation in the introduction and are not load-bearing for the architecture, the complexity reduction, or the simulation results. Reproducibility concerns such as missing seeds or error bars for Fig. 3 affect the strength of the empirical performance claims, but they are not circularity. The score is 2 rather than 0 only because the paper contains several non-load-bearing self-citations; no actual circular step was identified.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical entities or fitted constitutive laws. It relies on standard attention math, the OFDM/3GPP simulation stack, and design choices for model size.

free parameters (1)

Model hyperparameters (D, H, #blocks, kernel sizes) = D=128, H=4, 6 blocks
Chosen by hand, not tuned systematically. They determine the FLOP counts in Table III and the network capacity, so the reported efficiency and BLER numbers depend on them.

assumptions (5)

domain assumption OFDM system model with received signal y_n,k = h_n,k x_n,k + n_n,k
Section II-A; standard model for single-carrier-per-subcarrier OFDM with AWGN.
domain assumption 3GPP CDL channel models accurately represent wireless propagation in the tested scenarios
Section II-A; training and test channels are generated per TR 38.901 CDL models; the paper does not validate against real measurements.
domain assumption Binary cross-entropy loss between predicted LLRs and ground-truth bits is an appropriate surrogate for BLER
Section II-B, based on bit-metric decoding rate [16]; the paper does not study the effect of this choice.
domain assumption Sionna's simulation of 5G NR PHY is correct and matches 3GPP-compliant behavior
Section VI; all results rely on Sionna for channel generation, modulation, and decoding.
standard math Scaled dot-product attention (Vaswani et al.) is a valid building block
Equations (5)-(8); standard transformer machinery.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Computationally Efficient Neural Receivers via Axial Self-Attention." pith.science (2026). https://pith.science/paper/YE6BHONV

@misc{pith2026251012941,
  author       = {Pith},
  title        = {Pith review of: Computationally Efficient Neural Receivers via Axial Self-Attention},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YE6BHONV}},
  note         = {Machine review of arXiv:2510.12941}
}

abstract

Deep learning-based neural receivers offer promising physical-layer solutions for next-generation wireless systems. We propose an axial self-attention transformer neural receiver that achieves state-of-the-art Block Error Rate (BLER) performance with significantly improved computational efficiency during inference and large-scale training. By factorizing attention operations along temporal and spectral axes, the proposed architecture reduces computational complexity from $O((TF)^2)$ to $O(T^2F+TF^2)$, yielding substantially fewer floating-point operations and attention matrix multiplications per transformer block. Experimental validation under 3GPP Clustered Delay Line (CDL) channels demonstrates consistent performance gains across varying mobility scenarios. Under non-line-of-sight conditions, our proposed axial neural receiver outperforms global self-attention and convolutional neural receiver baselines at 10% BLER and 1% BLER respectively, with reduced computational complexity.

Figures

Figures reproduced from arXiv: 2510.12941 by the authors.

**Figure 3.** BLER performance under CDL-C Non-LoS (NLoS) and CDL-D Line-of-Sight (LOS) channels at user velocities 10– [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 6 linked inside Pith

[11]

A Unified Transformer Architecture for Low-Latency and Scalable Wireless Signal Processing,

Y Kawai and R Koodli, “A Unified Transformer Architecture for Low-Latency and Scalable Wireless Signal Processing,”arXiv preprint arXiv:2508.17960, 2025

arXiv 2025
[1]

Axial Attention in Multidimensional Transformers,

J. Ho, N. Kalchbrenner, D. Weissenborn, and T. Sali- mans, “Axial Attention in Multidimensional Transformers,” https://arxiv.org/abs/1912.12180, 2019

arXiv 1912
[2]

A Tale of Two Mobile Generations: 5G-Advanced and 6G in 3GPP Release 20,

X Lin, “A Tale of Two Mobile Generations: 5G-Advanced and 6G in 3GPP Release 20,”IEEE Communications Standards Magazine, pp. 1–9, 2025

2025
[3]

Industrial Viewpoints on RAN Technologies for 6G,

M Shafi, E. Larsson, X Lin, D Panaitopol, S Parkvall, F Ronteix-Jacquet, and A Toskala, “Industrial Viewpoints on RAN Technologies for 6G,” https://arxiv.org/pdf/2508.08225, 2025

arXiv 2025
[4]

DeepRx: Fully Convolutional Deep Learning Receiver,

M. Honkala, D. Korpi, and J. Huttunen, “DeepRx: Fully Convolutional Deep Learning Receiver,”IEEE Transactions on Wireless Communica- tions, vol. 20, no. 6, pp. 3925–3940, 2021

2021
[5]

End-to-End Learning for OFDM: From Neural Receivers to Pilotless Communication,

F. Ait Aoudia and J. Hoydis, “End-to-End Learning for OFDM: From Neural Receivers to Pilotless Communication,”IEEE Transactions on Wireless Communications, vol. 21, no. 2, pp. 1049–1063, 2022

2022
[6]

A Neural Receiver for 5G NR Multi-User MIMO,

S. Cammerer, F. A. Aoudia, J. Hoydis, A. Oeldemann, A. Roessler, T. Mayer, and A. Keller, “A Neural Receiver for 5G NR Multi-User MIMO,” in2023 IEEE Globecom Workshops (GC Wkshps), 2023

2023
[7]

DeepRx MIMO: Convolutional MIMO Detection with Learned Multiplicative Transfor- mations,

D Korpi, M Honkala, J. Huttunen, and V Starck, “DeepRx MIMO: Convolutional MIMO Detection with Learned Multiplicative Transfor- mations,” inICC 2021 - IEEE International Conference on Communi- cations, 2021, pp. 1–7

2021

Show all 21 references

[8]

Efficient Quantization- Aware Neural Receivers: Beyond Post-Training Quantization,

S. S. Yellapragada, E. Ollila, and M. Costa, “Efficient Quantization- Aware Neural Receivers: Beyond Post-Training Quantization,”arXiv preprint arXiv:2509.13786, 2025

2025
[9]

Efficient Deep Neural Receiver with Post-Training Quantization,

S. S. Yellapragada, E. Ollila, and M. Costa, “Efficient Deep Neural Receiver with Post-Training Quantization,”arXiv:2508.06275, Accepted for IEEE 59th Asilomar Conference on Signals, Systems, and Computers, October, 2025

2025
[10]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” inProceedings of the 31st International Conference on Neural Information Processing Systems, 2017, pp. 6000–6010

2017
[12]

Attention neural network for downlink cell-free massive mimo power control,

A. K. Kocharlakota, S. A. V orobyov, and R. W. Heath, “Attention neural network for downlink cell-free massive mimo power control,” in2022 56th Asilomar Conference on Signals, Systems, and Computers, 2022

2022
[13]

Pilot contamina- tion aware transformer for downlink power control in cell-free massive mimo networks,

A. K. Kocharlakota, S. A. V orobyov, and R. W. Heath, “Pilot contamina- tion aware transformer for downlink power control in cell-free massive mimo networks,”https://arxiv.org/abs/2411.19020, 2024

2024 arXiv
[14]

Pilot contamination-aware graph attention network for power control in cfm- mimo,

T Zhang, S. A. V orobyov, D. J. Love, T. Kim, and K. Dong, “Pilot contamination-aware graph attention network for power control in cfm- mimo,”https://arxiv.org/abs/2506.00967, 2025

2025
[15]

Study on channel model for frequencies from 0.5 to 100 GHz,

3GPP, “Study on channel model for frequencies from 0.5 to 100 GHz,” Tech. Rep. TR 38.901, 3rd Generation Partnership Project (3GPP), 2020, version 16.1.0

2020
[16]

Bit-Metric Decoding Rate in Multi-User MIMO Systems: Theory,

K.P. Srinath and J. Hoydis, “Bit-Metric Decoding Rate in Multi-User MIMO Systems: Theory,”IEEE Transactions on Wireless Communica- tions, vol. 22, no. 11, pp. 7961–7974, 2023

2023
[17]

Deep Residual Learning for Image Recognition,

K. He aand X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,”Conference on Computer Vision and Pattern Recognition, CVPR, 2016

2016
[18]

Layer normalization,

J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,”arXiv preprint arXiv:1607.06450, 2016

2016 arXiv
[19]

Hoydis, S

J. Hoydis, S. Cammerer, F. Ait Aoudia, M. Nimier-David, L. Maggi, G. Marcus, A. Vem, and A. Keller, “Sionna,” 2022, https://nvlabs.github.io/sionna/

2022
[20]

Bishop,Deep Learning F oundations and Concepts, Springer, 2023

Christopher M. Bishop,Deep Learning F oundations and Concepts, Springer, 2023

2023
[21]

Adam: A method for stochastic optimization,

Diederik P Kingma and Jimmy Ba, “Adam: A method for stochastic optimization,”arXiv preprint arXiv:1412.6980, 2014

2014 arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [11]

A Unified Transformer Architecture for Low-Latency and Scalable Wireless Signal Processing,

Y Kawai and R Koodli, “A Unified Transformer Architecture for Low-Latency and Scalable Wireless Signal Processing,”arXiv preprint arXiv:2508.17960, 2025

arXiv 2025

[2] [1]

Axial Attention in Multidimensional Transformers,

J. Ho, N. Kalchbrenner, D. Weissenborn, and T. Sali- mans, “Axial Attention in Multidimensional Transformers,” https://arxiv.org/abs/1912.12180, 2019

arXiv 1912

[3] [2]

A Tale of Two Mobile Generations: 5G-Advanced and 6G in 3GPP Release 20,

X Lin, “A Tale of Two Mobile Generations: 5G-Advanced and 6G in 3GPP Release 20,”IEEE Communications Standards Magazine, pp. 1–9, 2025

2025

[4] [3]

Industrial Viewpoints on RAN Technologies for 6G,

M Shafi, E. Larsson, X Lin, D Panaitopol, S Parkvall, F Ronteix-Jacquet, and A Toskala, “Industrial Viewpoints on RAN Technologies for 6G,” https://arxiv.org/pdf/2508.08225, 2025

arXiv 2025

[5] [4]

DeepRx: Fully Convolutional Deep Learning Receiver,

M. Honkala, D. Korpi, and J. Huttunen, “DeepRx: Fully Convolutional Deep Learning Receiver,”IEEE Transactions on Wireless Communica- tions, vol. 20, no. 6, pp. 3925–3940, 2021

2021

[6] [5]

End-to-End Learning for OFDM: From Neural Receivers to Pilotless Communication,

F. Ait Aoudia and J. Hoydis, “End-to-End Learning for OFDM: From Neural Receivers to Pilotless Communication,”IEEE Transactions on Wireless Communications, vol. 21, no. 2, pp. 1049–1063, 2022

2022

[7] [6]

A Neural Receiver for 5G NR Multi-User MIMO,

S. Cammerer, F. A. Aoudia, J. Hoydis, A. Oeldemann, A. Roessler, T. Mayer, and A. Keller, “A Neural Receiver for 5G NR Multi-User MIMO,” in2023 IEEE Globecom Workshops (GC Wkshps), 2023

2023

[8] [7]

DeepRx MIMO: Convolutional MIMO Detection with Learned Multiplicative Transfor- mations,

D Korpi, M Honkala, J. Huttunen, and V Starck, “DeepRx MIMO: Convolutional MIMO Detection with Learned Multiplicative Transfor- mations,” inICC 2021 - IEEE International Conference on Communi- cations, 2021, pp. 1–7

2021