{"id":"64f229c0-a5e6-49d4-ad05-754b83c137fc","arxiv_id":"2411.18277","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Spatial CSI prediction experiments on simulated data show a VAE outperforms GPT-2, Transformer, and MLP, but the claimed benefit of fusing environmental features is not tested against a no-feature baseline.","lead":"A single-author preprint proposes using GPT-2, VAE, Transformer, and diffusion models to predict wireless channel state information at new locations from simulated ray-traced maps of a university campus. The paper reproduces an earlier LLM4CP time-series experiment, then reports that a VAE, not the large language models, achieves the best spatial prediction error.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim requires predicting CSI at unmeasured positions, but the paper never states a spatially disjoint train/validation split; UEs sit on a 0.1 m grid (Sec. VII.A), so Table IV's VAE NMSE of 0.0445 may measure interpolation among near-duplicate samples, not spatial generalization.","rationale":"The paper's central claim is that large models, aided by physical environment features, can predict spatial CSI at unmeasured locations. For that claim to hold, the validation set must contain positions that are genuinely not measured during training. The manuscript never states such a split. Section VII.A says UEs are placed on a grid with 0.1 m spacing; Section VII.C describes feature extraction and training but omits the train/validation partitioning rule. With dense spatial sampling and a random point-wise split, the holdout samples are not independent of training samples; the channel at adjacent points is strongly correlated, so a model that memorizes the training grid can report low validation NMSE. This is not a purely theoretical worry: Table IV shows GPT-2 and Transformer at NMSE around 0.49, barely below the MLP's 0.4833 and far above the VAE's 0.0445. If the validation were a true spatial extrapolation, the simultaneous failure of all sequence models and the success of the VAE would need an explanation; the more economical explanation is that the VAE overfits the point distribution and the validation protocol does not expose it. The conclusion's claim that geometric path information 'reliably raises the accuracy' is similarly unsupported, because no ablation removes that information. The absence of a second map and real measurements (the reader's weakest assumption) is real, but it is downstream of the internal evaluation problem: if spatial generalization is not demonstrated inside the simulator, transfer to a new map cannot be assessed. I therefore agree with the reader's rejection but ground it in the missing spatial holdout rather than in the simulation-to-reality gap. A strict contiguous-region holdout would settle this: if VAE NMSE remains low, the rejection should be reconsidered; if it collapses to ~0.49, the central claim is unsupported. Because the paper also lacks code/data and contains no external validation, the current REJECT verdict remains appropriate.","tokens_in":8663,"tokens_out":7649,"duration_ms":68266,"concrete_test":"Re-run Table IV with a strict contiguous-region spatial holdout: designate one contiguous region of the CUHK(SZ) ray-tracing area (e.g., a 10 m x 10 m zone) as validation and use all other grid points for training, ensuring no training UE lies within, say, 5 m of any validation UE. Report validation NMSE for VAE, GPT-2, Transformer, and Diffusion on this held-out region. If VAE NMSE rises from 0.0445 toward the ~0.49 level of the other models, the spatial-generalization claim is unsupported; if it stays below ~0.1, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is that a map-aware ML pipeline can predict MIMO-OFDM CSI at unmeasured positions accurately enough for proactive beamforming. The paper's internal evaluation does not establish this. In Section VII.A, UE positions are placed on a uniform grid with 0.1 m spacing; in Section VII.C the authors describe feature extraction and model training, but they never describe how training and validation sets were split. If the split is random over individual UE grid points, each validation point has training points only 0.1 m away. At typical 5G NR sub-6 GHz carrier frequencies, the channel coherence distance is of a similar order, so those near-duplicate samples make the regression task close to interpolation. Under that protocol, the low validation NMSE of the VAE (0.0445, Table IV) does not demonstrate prediction at unmeasured positions; the poor results of GPT-2 and Transformer (~0.49) and the high training--validation gap in Fig. 11 suggest those models are not learning the spatial function at all. The paper also never tests the 'physical characteristics reduce uncertainty' claim with an ablation (e.g., with vs. without map features), and it provides no second map or real measurement. But the more fundamental problem is the missing spatial holdout: without it, even the in-simulation claim is unverified. This concern is independent of the simulation-to-reality transfer issue: if the evaluation cannot separate memorization from generalization, external transfer is moot.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a machine-learning framework for spatial CSI prediction in MIMO-OFDM systems, combining ray-tracing data from a realistic map (CUHK(SZ) via WinProp) with a feature-fusion architecture that feeds environmental, positional, and CSI information into several models (GPT-2, VAE, Transformer, Diffusion, MLP). It also reproduces a temporal CSI prediction experiment (LLM4CP) using QuaDRiGa. The central claim is that large models, augmented with physical environment features, can predict spatial CSI at unmeasured positions accurately enough to support proactive beamforming and resource allocation.","tokens_in":8993,"tokens_out":2897,"duration_ms":27943,"significance":"If the central claim were established, the work would be a useful step toward environment-aware CSI prediction for 5G/6G. The paper has some strengths: it uses a deterministic ray-tracing simulator (WinProp) on a real OpenStreetMap layout, reports a standard NMSE metric, and compares several architectures. The reproduction of the LLM4CP temporal prediction experiment is a reasonable benchmarking exercise. However, the primary scientific claim is not supported by the paper's own data. The VAE (which is not a 'large model' in the language-model sense) is the only model that achieves low validation NMSE, while GPT-2 and Transformer are essentially at the level of the MLP baseline. Moreover, the evaluation protocol does not establish spatial generalization because no spatially disjoint train/validation split is described, and the proposed feature fusion is never tested with an ablation. These issues are load-bearing for the paper's stated contribution.","major_comments":[{"comment":"The central 'large models' claim is contradicted by the paper's own results. GPT-2 and Transformer validation NMSE remain near 0.49 across all epochs, whereas the VAE reaches 0.0445 and the Diffusion model reaches 0.125. The VAE is not a language model, and the GPT-2/Transformer results are no better than the MLP baseline. The conclusion in §VIII that 'the use of large models like GPT-2 and VAE provide an effective solution' is therefore not supported by Table IV. The authors need to either reframe the claim to 'variational autoencoders' or provide evidence that a language model can actually perform spatial CSI prediction.","section":"§VII.D, Table IV"},{"comment":"The paper never states how the training and validation sets were split. Section VII.A places UEs on a uniform grid with 0.1 m spacing. If the split is random over individual grid points, then every validation point has training points only 0.1 m away, making the regression problem essentially interpolation at sub-6 GHz coherence distances. Under that protocol, the VAE's low validation NMSE (Table IV) does not demonstrate prediction at unmeasured positions; it may reflect memorization of near-duplicate samples. The authors must describe the split explicitly and, ideally, use a spatially disjoint holdout (e.g., a contiguous geographic zone or a coarser grid) to demonstrate genuine spatial generalization.","section":"§VII.A, VII.C"},{"comment":"The claim in §VIII that incorporating physical characteristics of signal paths 'reduces the uncertainty of the function to be learned' is not tested. No ablation compares the full pipeline with a version that omits the environmental features, or compares the 'top 5 nearest walls' representation with the CNN-based map feature extraction. Without such an ablation, the reported gains cannot be attributed to the proposed feature fusion rather than to the choice of model architecture or data representation.","section":"§VII.C, VIII"},{"comment":"The method is evaluated on a single simulated environment (CUHK(SZ) via WinProp). Section VIII asserts the method 'is believed to be adaptive to totally different and complex urban maps,' but no second map, no real-world channel measurement, and no out-of-distribution test is provided. The paper should either add such a test or clearly state that cross-environment generalization is an open question. The simulation-to-reality transfer is assumed, not demonstrated.","section":"§VII.A, VIII"}],"minor_comments":[{"comment":"Several references are cited with bare arXiv links and without venue or year; e.g., [5], [6], [8], [9], [13], [15]. The citation style should be made consistent, and missing bibliographic details should be added.","section":"§II"},{"comment":"The channel matrix notation in Eq. (1) is confusing: the entries h11, h12, ..., hn1, hnm suggest a rectangular arrangement that does not match the standard MIMO channel matrix notation h_{ij} with i as the receive index and j as the transmit index. Please correct the indices.","section":"Eq. (1)"},{"comment":"The sentence 'Both frequency domain and delay domain information are feeded into the model' contains a typo ('feeded') and is vague about the actual input representation. Please specify the tensor shapes and the preprocessing steps.","section":"§V.C"},{"comment":"The reproduced QuaDRiGa spatial prediction results report NMSE values (0.17 and 0.035) without an accompanying table or figure. Please add the corresponding learning curves or at least a table with standard deviations.","section":"§VI.B"},{"comment":"The initial approach using 'top 5 nearest walls' is described but no results are shown for this variant. Either present a comparison or remove the description to avoid an untested claim.","section":"§VII.C"},{"comment":"The figure captions contain typos: 'The learning cutve of Diffusion model' should be 'curve', and the text mentions Fig. 9, Fig. 10, and Fig. 11 but the figures are not included in the manuscript text as submitted.","section":"§VII.D"},{"comment":"The abstract says 'language models' but the best-performing model in the spatial prediction task is a VAE, which is not a language model. Please align the abstract terminology with the actual models used.","section":"Abstract"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an extended student report rather than a finished research paper. The experimental evaluation does not support the central claim, and the missing spatial holdout is a fundamental methodological flaw that cannot be fixed without re-running the experiments. I would not encourage resubmission unless the authors substantially redesign the evaluation and reframe the contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read: this is a student report with a real direction in it and an evaluation that does not support its claims. The core idea—spatial CSI prediction using OSM map features—is a sensible extension of the digital twin and LLM4CP line, and the preprocessing fusion of geometric wall features with amplitude-phase CSI decomposition is worth trying. The LLM4CP reproduction in Section V is diligent and the time-series results look consistent. The paper is also honest about the limitations of QuaDRiGa and moves to ray tracing for physical plausibility.\n\nBut the central claims fail on the evidence. Table IV shows the VAE winning at NMSE 0.0445 while GPT-2 and Transformer sit near 0.49, so the 'large models enabled' title is contradicted by your own data. More seriously, you never state a spatially disjoint train/validation split; UEs are on a 0.1 m grid (Section VII.A), so a random split makes the VAE's low error close to interpolation, not prediction at unmeasured positions. That is a load-bearing flaw: without a spatial holdout, even the in-simulation claim is unverified.\n\nThe other soft spots are of a piece. There is no ablation to isolate the contribution of the map features, no second map or real measurement, no code or data, and the conclusion extrapolates to arbitrary urban maps without evidence. The paper reads as a report, not a validated research claim. The missing spatial holdout is the kind of thing a serious referee would demand immediately; the rest are fixable but the evaluation design is not.\n\nWho is this for? As a workshop entry or a term project it is fine and shows a student grappling with real tools. As a research paper it is not ready. I would not send it to a top venue, and I would desk reject it for a conference if the split issue is present. But the direction has legs: re-run with a proper spatial holdout, add an ablation, and release the dataset and code, and there could be a solid workshop paper.\n\nMy recommendation: do not engage further until the evaluation is fixed. Not worth referee time in its current form.","headline":"A plausible map-aware CSI prediction idea is undermined by an evaluation that never separates interpolation from generalization, and the paper's own numbers contradict the 'large models' framing.","tokens_in":9486,"tokens_out":3580,"would_cite":false,"duration_ms":32642,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that feeding a model ray-traced building geometry, not just raw channel data, makes spatial CSI prediction accurate enough for proactive beamforming.","keywords":["CSI","MIMO-OFDM","spatial prediction","large models","wireless communication","machine learning","data-driven approaches","5G communication"],"falsifier":"A direct test: train the VAE-based predictor on the ray-traced campus dataset and evaluate it on real measured CSI taken at the same grid of locations, or on a second city's ray-traced map. If validation NMSE jumps toward the no-prediction level around 0.5, the transfer claim fails. A second, already available check is the paper's Table IV: GPT-2 and Transformer validation NMSE around 0.49 means the large-model claim is false unless those models are evaluated on a task that plays to sequence modeling.","tokens_in":8428,"feed_emoji":"📡","tokens_out":8471,"duration_ms":79412,"temperature":0.7,"pith_summary":"This paper tries to establish that spatial channel state information (CSI) in MIMO-OFDM systems, the matrix describing how signals travel from base station to user, can be predicted at unmeasured locations by combining a machine-learning predictor with physical knowledge of the environment. The route is to simulate realistic urban channels with ray tracing over a digital city map, then train models to map a 3D position to the channel matrix. The paper reports that a variational autoencoder reaches a validation NMSE of 0.0445, well below the no-prediction baseline around 0.5, while GPT-2, Transformer, and MLP baselines stay near 0.49. The author concludes that incorporating physical path characteristics reduces the uncertainty of the learned mapping and that the methodology should transfer to other urban maps. If this works in the field, base stations could anticipate channel conditions at locations users have not yet occupied, enabling proactive beamforming and resource allocation.","feed_headline":"City-map geometry predicts wireless channels at new locations","feed_subtitle":"A VAE trained on ray-traced urban geometry beats GPT-2 and Transformer baselines by an order of magnitude.","key_machinery":"The load-bearing mechanism is the map-aware feature-fusion preprocessing pipeline: raw geographic layout and TX/RX positions are turned into aligned tensors by embedding positions into the map, extracting hierarchical features with a CNN, concatenating multiscale intermediate features, encoding RX positions with a positional encoding module, and decomposing CSI into amplitude and phase. The predictor then learns the mapping from these aligned features to the channel matrix, trained with a smooth L1 loss. The paper argues that this physical grounding shrinks the function the model must learn, which is why accuracy improves: the environment determines the multipath geometry, so the model does not have to infer it from CSI statistics alone.","core_discovery":"On its own terms, the paper's central discovery is that a spatial CSI prediction framework can be built by fusing geographic features with channel data: transmitter and receiver positions are embedded into a map, the map is processed by a CNN to extract multiscale environmental features, receiver positions are positionally encoded, and CSI is decomposed into amplitude and phase before being fed together with the encoded positions into the predictor. Under this pipeline, the variational autoencoder predicts CSI across a ray-traced urban area with validation NMSE as low as 0.0445, an order of magnitude better than the large-sequence models (GPT-2 and Transformer) that plateau around 0.49 and the MLP baseline near 0.49. The paper also reports that the diffusion model reaches 0.125 with early stopping. The stated conclusion is that large models, when supplied with physical path characteristics, provide an effective solution for spatial CSI prediction, and that the likelihood of generalization to different and more complex urban maps is high.","pith_inferences":["The paper's own Table IV makes an implicit point the author does not draw out: the two models the conclusion credits as 'large models' (GPT-2 and Transformer) perform barely above a constant predictor, so the demonstrated success is specifically the VAE and diffusion model, not large language models as a class.","Because the ray-tracing simulation covers a single campus map with no real-world measurements, the strong generalization claim is untested; a natural next experiment is to train on one city map and evaluate on a second map or on measured channels, comparing NMSE to the no-prediction level.","The static ray-tracing setup removes the temporal dimension entirely, while the earlier QuaDRiGa section shows multi-time-slice inputs improved NMSE from 0.17 to 0.035; feeding time-sliced CSI into the map-aware spatial model may combine the two gains.","The wall-distance-and-orientation feature representation was abandoned for the full-map CNN; an ablation comparing those two feature sets would isolate how much geometric detail actually drives the VAE's accuracy."],"forward_implications":["If spatial CSI prediction at unmeasured points is accurate, a network can form beams toward a user's future location before the user sends a pilot, reducing feedback overhead in 5G and 6G systems.","A model trained once on one ray-traced urban map could, under the transfer claim, be applied to new cities without per-site channel measurement campaigns.","The large gap between VAE and GPT-2 or Transformer indicates that for this static spatial task, generative latent-variable models are better suited than sequence-oriented large language models.","The success of the fused-feature pipeline suggests that the main gain comes from environmental feature engineering, so further gains may come from richer geometry such as materials, vegetation, and 3D building shapes.","Combining the spatial predictor with the time-series predictor from the LLM4CP reproduction, which reaches NMSE near 0.02 at low speeds, could give joint spatial-temporal CSI prediction."],"supporting_citations":[{"why":"Defines the spatial CSI prediction task via the digital radio twin and U-Net, the prior work this paper extends.","marker":"[1]"},{"why":"The time-series channel prediction benchmark the paper reproduces, establishing the GPT-2-based prediction setup and NMSE evaluation.","marker":"LLM4CP"},{"why":"Supplies the stochastic channel generation used for the time-series and early spatial experiments.","marker":"[16]"},{"why":"Motivates applying large language models to telecommunications tasks, including CSI prediction.","marker":"[15]"},{"why":"The 3D CNN baseline for future CSI prediction in mobile scenarios that the paper contrasts with its own spatial approach.","marker":"[8]"}],"fun_headline_variants":["Geometry-aware VAE beats GPT-2 for channel prediction","City maps outperform LLMs in wireless channel forecasting","VAE with map context wins over large models for CSI","Map-based VAE trumps GPT-2 in spatial CSI prediction","Environmental geometry boosts VAE over large language models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that ray-traced simulated channels from a single digital urban map are a faithful stand-in for real physical channels, so that a model trained on those simulated maps will generalize to other maps and to real deployments.","fun_headline_variants_meta":{"raw":{"variants":["Geometry-aware VAE beats GPT-2 for channel prediction","City maps outperform LLMs in wireless channel forecasting","VAE with map context wins over large models for CSI","Map-based VAE trumps GPT-2 in spatial CSI prediction","Environmental geometry boosts VAE over large language models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000191,"raw_usage":{"total_tokens":1283,"prompt_tokens":824,"completion_tokens":459,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":440,"completion_tokens_details":{"reasoning_tokens":380}},"tokens_in":440,"tokens_out":459,"duration_ms":5498,"temperature":1.0,"reasoning_tokens":380,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:21:10.213683+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test: train the VAE-based predictor on the ray-traced campus dataset and evaluate it on real measured CSI taken at the same grid of locations, or on a second city's ray-traced map. If validation NMSE jumps toward the no-prediction level around 0.5, the transfer claim fails. A second, already available check is the paper's Table IV: GPT-2 and Transformer validation NMSE around 0.49 means the large-model claim is false unless those models are evaluated on a task that plays to sequence modeling.","supporting_citations":[{"cited_title":"Spatial Channel State Information Prediction with Generative AI: Towards Holographic Communication and Digital Radio Twin","cited_arxiv_id":"2401.08023","evidence_quote":"Defines the spatial CSI prediction task via the digital radio twin and U-Net, the prior work this paper extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the stochastic channel generation used for the time-series and early spatial experiments."}],"review_version":1}