{"id":"d6bb8c7a-a48c-4636-bf42-1c56f5051753","arxiv_id":"2501.05190","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"RMTransformer, a U-Net-style model with a multi-scale MaxViT transformer encoder, reports a 31.7 percent RMSE reduction over PMNet on the USC radio map dataset.","lead":"A new hybrid transformer-convolution neural network, RMTransformer, predicts radio signal coverage maps from building layouts and base station locations. On a public ray-tracing dataset it reports roughly 32 percent lower root mean square error than the previous best method, PMNet.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Single-run comparison without matched training budgets or error bars leaves the 31.66% gain over PMNet unsupported as an architectural benefit.","rationale":"The reader's weakest assumption identifies exactly the load-bearing weak point of the paper: the headline RMSE improvement over PMNet rests on a single experimental comparison with unclear control of training budget and split. My stress-test pass converges on the same concern. The paper provides no code, no error bars, no multi-seed statistics, and no independent transformer-based baseline, so the numerical claim is plausible but not robustly supported. However, this is a matter of missing evidence rather than a demonstrated internal inconsistency or an obviously wrong computation. The architecture is described well enough to be reproduced, and the reported improvement is not implausible. Therefore the reader's CONDITIONAL verdict is appropriate: the paper should be accepted only if the authors release code and add matched-budget multi-seed comparisons. My concern does not move the verdict, so I mark it UNCHANGED.","tokens_in":7090,"tokens_out":3927,"duration_ms":41820,"concrete_test":"Retrain PMNet and RMTransformer from five random seeds on exactly the same fixed 90/10 split of the USC dataset, using the identical optimization budget (Adam, 50 epochs, batch size 8, learning rate 1e-4 decaying to 1e-5) and identical normalization and metric computation. Report mean and 95% confidence interval of test RMSE over seeds, and compute the paired per-sample RMSE difference between the two models on the same test set. If the 95% CI of the difference includes 0, or the relative reduction is no longer above 10%, the claimed 31.66% architectural gain is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central quantitative claim is the 31.66% RMSE reduction over PMNet (0.01046 vs 0.007148) in Table II, attributed to the hybrid transformer-CNN design. This inference depends on all models being compared under equivalent conditions: the same data split, the same normalization, and comparable optimization budgets. The manuscript does not establish that equivalence. Section IV-A gives training details only for 'the model' (50 epochs, Adam, batch size 8, LR 1e-4 decaying to 1e-5), while Section IV-C says the baselines were run 'using their originally reported network settings.' It is therefore ambiguous whether RadioUnet and PMNet received the same number of epochs, learning-rate schedule, initializations, and the exact same 90/10 split as RMTransformer. Table II reports a single RMSE value per method with no standard deviations, no multiple seeds, and no statistical test. Because the absolute RMSE gap is only 0.0033 on a normalized 0-1 scale, run-to-run variation from random initialization and data split could plausibly account for a large fraction of the reported gain. Without matched-budget, multi-seed evaluation, the headline 'over 30% reduction' is not yet established as a consequence of the transformer encoder rather than of evaluation protocol.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RMTransformer, a hybrid multi-scale transformer-CNN architecture for radio map (pathloss map) prediction. The encoder uses four MaxViT blocks to extract multi-scale features from a 2D geographical map (building/blockage map plus base-station location), and the CNN decoder reconstructs the pixel-level radio map via skip connections. The method is evaluated on the USC dataset with 19,016 ray-tracing samples and compared against RadioUnet and PMNet. The paper reports an RMSE of 0.007148 for RMTransformer, which is a 31.66% reduction over PMNet (0.01046) and a 62.46% reduction over RadioUnet (0.01904), along with channel prediction error and coverage prediction error results.","tokens_in":7397,"tokens_out":5369,"duration_ms":51049,"significance":"If the reported comparison is robust, applying a multi-scale MaxViT encoder to radio map regression is a useful, incremental contribution to data-driven pathloss prediction and digital-twin channel modeling. The paper is clearly written, uses a public benchmark, presents the architecture in detail, and includes qualitative visualizations that support the plausibility of improved boundary prediction. However, the central quantitative claim currently rests on single-run measurements with no variance reporting, no matched-training-budget information for the baselines, and no ablations isolating the transformer contribution. The core idea is not conceptually new beyond adapting an existing vision transformer to radio maps, but the empirical result could be valuable if the evaluation is made statistically sound.","major_comments":[{"comment":"The central claim of a 31.66% RMSE reduction over PMNet rests on one RMSE value per method, with no standard deviations, no number of seeds, and no statistical test. Because the RMSE is reported on a normalized 0-1 scale and the absolute gap is only 0.003318, run-to-run variation from random initialization and the 90/10 data split could account for a substantial fraction of the reported improvement. Please report mean ± standard deviation over at least 5-10 independent training runs (ideally with a paired evaluation on the same test split), or provide a bootstrap confidence interval or p-value; otherwise the headline reduction is not established as an architectural effect.","section":"Section IV-C (Table II)"},{"comment":"The comparison does not currently establish matched training conditions across methods. Section IV-A specifies 50 epochs, Adam, batch size 8, and a learning-rate schedule from 1e-4 to 1e-5 for RMTransformer, while Section IV-C states only that RadioUnet and PMNet were run using their originally reported network settings. The manuscript must state explicitly whether all three models used the same random 90/10 split, the same input normalization and preprocessing, the same number of epochs and learning-rate schedule, the same random seeds, and the same batch size. If the baselines were not retrained under matched conditions, the reported gain over PMNet is confounded by evaluation protocol rather than attributable to the architecture.","section":"Section IV-A and IV-C"},{"comment":"The improvement is attributed to the multi-scale transformer encoder and the hybrid transformer-CNN design, but no ablations are provided to separate these factors. I request at least two ablations under the same training budget: (i) replacing the MaxViT blocks with convolutional blocks of comparable capacity, and (ii) using a single-scale transformer encoder without the multi-scale skip connections. Without such experiments, the mechanistic claim that the transformer and multi-scale features drive the improvement is not supported by the data.","section":"Section III-B and IV-C"}],"minor_comments":[{"comment":"In the last paragraph, 'each feature map Xn is then propose into the (n-1)-th decoder CNN block' should read 'propagated into' or 'passed to'.","section":"Section III-C"},{"comment":"The coverage prediction error equation and the surrounding text should define Pthres in physical dBm as well as in normalized units, and the summation indices should be made consistent with the H × W grid description (the sums are written over i = 1,...,W and j = 1,...,H).","section":"Equation (7) and Section IV-C"},{"comment":"The red rectangles mentioned in the text are not clearly visible in the figure or described in the caption; please add zoomed panels or otherwise mark and explain the regions of interest.","section":"Figure 2"},{"comment":"There are minor typesetting issues that should be cleaned up, including a broken 'H' in Section II and 'inProc' in reference [10]; a careful proofread is recommended.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the main weakness is empirical rigor rather than the plausibility of the proposed architecture. The paper is within the journal's scope, and the central idea is defensible, but the headline quantitative claim needs matched-budget, multi-seed evaluation and ablations before it can be accepted. I saw no evidence of citation manipulation; the self-citations appear in the digital-twin introduction and are topically relevant."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a clean, straightforward application of a MaxViT encoder paired with a CNN decoder to radio map regression. It reports a 31.66% RMSE reduction over PMNet on the USC dataset. The architecture itself is not conceptually new—it borrows a well-known image-to-image recipe—but the specific multi-scale transformer-CNN combination for radio maps is a legitimate adaptation, and the measured numbers in Table II are new.\n\nWhat the paper does well: it addresses a meaningful problem, uses a public dataset, and reports three relevant metrics (RMSE, channel prediction error, coverage prediction error). The architecture is described clearly in Table I and Section III, and the visualizations in Fig. 2 give a useful qualitative sense of where the gains come from. The writing is concise and honest; the authors do not overclaim conceptual novelty.\n\nThe soft spots are exactly where the reader and stress-test note land. The main issue is the evaluation protocol. Section IV-C says the baselines were run using their \"originally reported network settings,\" which leaves it ambiguous whether they received the same data split, normalization, epochs, and learning-rate schedule as RMTransformer. Table II reports one RMSE value per method with no standard deviations and no multiple seeds. The absolute gap is only 0.0033 on a normalized 0–1 scale, so run-to-run variation from random initialization or data split could account for a meaningful chunk of the reported gain. There is also no comparison against any other transformer-based radio map baseline, and no code is released. These are fixable with a moderate revision.\n\nThe citation pattern is fine. Self-citations appear in the digital twin context and do not prop up the radio map result. No circular reasoning or missing reference jumps out.\n\nOverall, the central claim is plausible—transformers often help image-to-image regression—but it is not robustly established. The paper is a solid incremental contribution within an established research program, not a breakthrough.\n\nWho is this for? Researchers in radio map prediction or digital twin channel modeling who want a transformer baseline. They will get a clear architecture description and a starting point, but they should not take the 31.66% at face value until the authors provide multi-seed statistics, matched training budgets, and ideally code.\n\nRecommendation: engage with it. Send it to peer review, but require the authors to add error bars, match training conditions across all models, and release code. If the numbers hold, it is a useful contribution; if they do not, the architecture description alone still has some value.","headline":"Useful transformer-CNN baseline for radio map prediction, but the headline RMSE gain is not yet robust because the comparison is single-run and training budgets aren't matched.","tokens_in":7834,"tokens_out":2050,"would_cite":false,"duration_ms":21131,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A multi-scale transformer encoder cuts radio map error by 31.7% over PMNet and 62.5% over RadioUnet.","keywords":["radio map prediction","pathloss prediction","vision transformer","multi-scale feature extraction","coverage prediction","digital twin","deep learning"],"falsifier":"Re-train RMTransformer, PMNet, and RadioUnet on the same dataset with identical random seeds, epoch counts, and hyperparameter search budgets, repeated over several independent splits, and compare the RMSE distributions. If the 31.66% gap over PMNet shrinks to within one standard deviation under matched resources, the claimed architectural advantage would not be established.","tokens_in":6826,"feed_emoji":"📡","tokens_out":7483,"duration_ms":65160,"temperature":0.7,"pith_summary":"The paper sets out to show that a hybrid transformer-convolution network, RMTransformer, predicts radio maps (pixel-level received power, equivalently pathloss, derived from a geographical map and a base station location) more accurately than the convolutional state of the art. The proposed architecture uses a multi-scale vision-transformer encoder for feature extraction and a convolutional decoder for pixel reconstruction, with skip connections between scales. On the public ray-tracing dataset used by the prior radio-map challenge, RMTransformer reports an RMSE of 0.007148, compared with 0.01046 for PMNet and 0.01904 for RadioUnet, a 31.66% and 62.46% reduction respectively. It also reports lower channel prediction error and coverage prediction error. If the comparison holds, transformer feature extraction is a better backbone than pure CNN encoders for this image-regression task, with direct consequences for digital-twin-based network planning and coverage decisions.","feed_headline":"Transformer encoder cuts radio map error 31.7% over PMNet","feed_subtitle":"Hybrid vision-transformer network also beats RadioUnet by 62.5%, sharpening coverage predictions for digital-twin planning.","key_machinery":"The engine of the result is the multi-scale transformer encoder built from MaxViT blocks, a vision-transformer variant that combines local block attention and global grid attention in each stage. Each of the N stages halves the spatial resolution and doubles the channel count, producing feature maps at resolutions H/2 x W/2 through H/2^(N+1) x W/2^(N+1). These multi-resolution features are joined to the CNN decoder by concat[Xn, Yn] skip connections, so the decoder can use both coarse context and fine detail when reconstructing the 256 x 256 radio map. The design is what differentiates RMTransformer from the single-scale, CNN-only encoders of the baselines.","core_discovery":"The central claim is that replacing the convolutional encoder in radio-map regression with a multi-scale transformer encoder, while keeping a convolution-based decoder, yields substantially more accurate pathloss maps. RMTransformer takes a two-channel input (building/blockage map plus base-station location), downsamples it through four multi-axis vision transformer (MaxViT) blocks that alternate block attention and grid attention, and forwards the resulting multi-resolution feature maps into four convolutional decoder blocks via concatenation skip connections. This multi-scale design is tailored to pixel-level image regression, where both fine details and global context matter. Evaluated under the same 90/10 split and normalization protocol used by the baselines, the model reaches $10^{-3}$-level RMSE on normalized dB maps and recovers sharp boundaries between regions with different propagation characteristics, which is where the prior CNN models lose accuracy.","pith_inferences":["A natural next test would be to ablate the multi-scale skip connections against feeding only the final transformer feature map, to isolate how much of the gain comes from the transformer backbone versus the multi-scale fusion.","The same hybrid encoder-decoder recipe may transfer to other radio-environment regression tasks, such as estimating channel characteristics or electromagnetic field coverage from sparse measurements, but the paper does not test this.","Because the comparison reports a single split without confidence intervals, the precise percentage gains should be treated as uncertain; a multi-seed evaluation could change the ranking even if the architecture is genuinely better.","If transformer encoders scale more favourably with data and compute than CNNs, the gap over convolutional baselines may widen on larger datasets, while on very small datasets the CNN baselines could remain competitive."],"forward_implications":["If the reported result is right, transformer-based encoders can outperform convolutional encoders in radio-map regression without sacrificing pixel-level reconstruction accuracy.","The multi-scale skip-connection design is a concrete recipe: feeding downsampled feature maps at several resolutions into a CNN decoder improves prediction near boundaries between propagation regions.","Lower coverage prediction error (0.01236 vs 0.01774 for PMNet) means deployment decisions about whether a location is covered become more reliable.","Because the method only needs a geographical map and a base-station location as input, it can serve as a low-cost component of a digital twin for network planning and resource management.","The 10^-3-level RMSE on normalized dB maps suggests the predicted maps are accurate enough to use as channel state information for pre-validation in 6G network management."],"supporting_citations":[{"why":"Defines the RadioUnet CNN encoder-decoder baseline, whose reported RMSE (0.01904) RMTransformer claims to reduce by 62.46%.","marker":"[18]"},{"why":"The challenge entry that established PMNet as the strongest convolutional baseline, with reported RMSE 0.01046.","marker":"[19]"},{"why":"Journal version of PMNet; provides the public dataset, the 90/10 split, the normalization, and the three evaluation metrics used in Table II.","marker":"[20]"},{"why":"Introduces MaxViT, the multi-axis vision transformer blocks (block attention plus grid attention) used as the encoder's core.","marker":"[22]"},{"why":"The model-driven pathloss formula in Eq. (2), whose simplified statistical assumptions motivate a data-driven alternative.","marker":"[17]"},{"why":"The ray-tracing emulation used to generate the ground-truth radio maps in the public dataset.","marker":"[23]"},{"why":"The competition that set the radio-map prediction benchmark, making PMNet the baseline to beat.","marker":"[21]"}],"fun_headline_variants":["RMTransformer: 31.7% lower RMSE in radio map prediction","Hybrid transformer-conv network cuts pathloss error 30%","Multi-scale transformer sharpens coverage prediction accuracy","RMTransformer beats SOTA radio map error by 31.7%","Transformer encoder reduces radio map RMSE 31.7%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reported error reductions reflect the architecture rather than the particular random train/test split or unequal training budgets: the paper reports one RMSE per model with no standard deviations and does not state whether the baselines received the same seed, epoch count, and hyperparameter selection.","fun_headline_variants_meta":{"raw":{"variants":["RMTransformer: 31.7% lower RMSE in radio map prediction","Hybrid transformer-conv network cuts pathloss error 30%","Multi-scale transformer sharpens coverage prediction accuracy","RMTransformer beats SOTA radio map error by 31.7%","Transformer encoder reduces radio map RMSE 31.7%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000355,"raw_usage":{"total_tokens":1881,"prompt_tokens":849,"completion_tokens":1032,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":465,"completion_tokens_details":{"reasoning_tokens":944}},"tokens_in":465,"tokens_out":1032,"duration_ms":9625,"temperature":1.0,"reasoning_tokens":944,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:13:03.905757+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-train RMTransformer, PMNet, and RadioUnet on the same dataset with identical random seeds, epoch counts, and hyperparameter search budgets, repeated over several independent splits, and compare the RMSE distributions. If the 31.66% gap over PMNet shrinks to within one standard deviation under matched resources, the claimed architectural advantage would not be established.","supporting_citations":[{"cited_title":"Radiounet: Fast radio map estimation with convolutional neural networks,","cited_arxiv_id":null,"evidence_quote":"Defines the RadioUnet CNN encoder-decoder baseline, whose reported RMSE (0.01904) RMTransformer claims to reduce by 62.46%."},{"cited_title":"PMNet: Large-scale channel prediction system for ICASSP 2023 first pathloss radio map prediction challenge,","cited_arxiv_id":null,"evidence_quote":"The challenge entry that established PMNet as the strongest convolutional baseline, with reported RMSE 0.01046."},{"cited_title":"A scalable and generalizable pathloss map prediction,","cited_arxiv_id":null,"evidence_quote":"Journal version of PMNet; provides the public dataset, the 90/10 split, the normalization, and the three evaluation metrics used in Table II."},{"cited_title":"MaxViT: Multi-axis vision transformer,","cited_arxiv_id":null,"evidence_quote":"Introduces MaxViT, the multi-axis vision transformer blocks (block attention plus grid attention) used as the encoder's core."},{"cited_title":"Study on channel model for frequencies from 0.5 to 100 GHz,","cited_arxiv_id":null,"evidence_quote":"The model-driven pathloss formula in Eq. (2), whose simplified statistical assumptions motivate a data-driven alternative."},{"cited_title":"[Online]","cited_arxiv_id":null,"evidence_quote":"The ray-tracing emulation used to generate the ground-truth radio maps in the public dataset."},{"cited_title":"The first pathloss radio map prediction challenge,","cited_arxiv_id":null,"evidence_quote":"The competition that set the radio-map prediction benchmark, making PMNet the baseline to beat."}],"review_version":1}