pith. sign in

arxiv: 2606.28274 · v1 · pith:7CR4IP2Unew · submitted 2026-06-26 · 💻 cs.LG · cs.AI

Parameter Efficient Hybrid Transformer (PEHT) for Network Traffic Prediction via Dynamic Urban Congestion Integration

Abdolazim Rezaei , Mehdi Sookhak , Mahboobeh Haghparast This is my paper

Pith reviewed 2026-06-29 04:16 UTC · model grok-4.3

classification 💻 cs.LG cs.AI
keywords network traffic predictionhybrid transformerLoRAurban mobilitycongestion integrationmultimodal fusionparameter efficientcellular networks
0
0 comments X

The pith

PEHT improves network traffic forecasts by fusing urban mobility and congestion data into a LoRA-efficient Transformer.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces PEHT to predict network traffic more accurately by accounting for how urban mobility and congestion affect demand. It separates core network features from mobility data, applies LoRA to the Transformer encoder to limit trainable parameters, and uses multimodal fusion to incorporate the external information in the decoder. A reader would care because better predictions support efficient resource allocation in cellular networks facing variable urban patterns. Experiments on the Milan dataset and synthetic cases show gains in standard error metrics over existing methods.

Core claim

PEHT separates primary network communication features from secondary urban mobility features, incorporates LoRA into the Transformer encoder, and injects mobility and congestion features via multimodal fusion into the decoder, resulting in lower RMSE and MAE and higher R² than state-of-the-art baselines on the Telecom Italia Milan dataset and synthetic scenarios.

What carries the argument

The multimodal fusion strategy that injects external mobility and congestion features into the LoRA-adapted Transformer decoder after separating primary network features.

Load-bearing premise

That separating network features from urban mobility features and fusing them multimodally will yield predictive improvements rather than just capturing dataset-specific patterns.

What would settle it

A test on new data where adding the mobility fusion step fails to improve or worsens the RMSE compared to the base Transformer without it.

Figures

Figures reproduced from arXiv: 2606.28274 by Abdolazim Rezaei, Mahboobeh Haghparast, Mehdi Sookhak.

Figure 1
Figure 1. Figure 1: PEHT: The proposed framework initially applies LoRA on [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: NVIDIA Jetson AGX Orin Development Kit and supporting equipment [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Ablation comparison of the Full Model against variants excluding [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
read the original abstract

Accurate network traffic prediction is a critical element for efficient resource allocation in dynamic urban cellular networks. However, prediction remains challenging because network demand is influenced by complex mobility patterns, congestion dynamics, and heterogeneous user behavior. This paper introduces the Parameter-Efficient Hybrid Transformer (PEHT), a network traffic prediction framework that integrates urban mobility and congestion information into a Transformer-based architecture. PEHT separates primary network communication features from secondary urban mobility features and incorporates Low-Rank Adaptation (LoRA) into the Transformer encoder to reduce the number of trainable parameters while maintaining high predictive accuracy. A multimodal fusion strategy then injects external mobility and congestion features into the decoder to improve traffic forecasting. Experiments on the Telecom Italia Milan dataset and multiple synthetic congestion scenarios show that PEHT outperforms state-of-the-art baselines in terms of RMSE, MAE, and $R^2$. The implementation is available in the GitHub repository.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper introduces the Parameter-Efficient Hybrid Transformer (PEHT) for network traffic prediction. It separates primary network communication features from secondary urban mobility and congestion features, applies LoRA to the Transformer encoder for parameter reduction, and uses a multimodal fusion strategy to inject mobility features into the decoder. Experiments on the Telecom Italia Milan dataset and synthetic congestion scenarios are reported to show outperformance over state-of-the-art baselines on RMSE, MAE, and R², with code released on GitHub.

Significance. If the claimed gains are shown to be robust and attributable to the fusion mechanism rather than capacity or overfitting, the work could provide a useful template for parameter-efficient multimodal integration in time-series forecasting for network management. The explicit use of LoRA and public code release are concrete strengths that support reproducibility.

major comments (3)
  1. [Experiments] The experimental section provides no ablation that removes the multimodal fusion module or compares PEHT against a capacity-matched plain LoRA-Transformer baseline without mobility features. This is load-bearing for the central claim that the urban congestion integration produces genuine predictive gains rather than dataset-specific correlations.
  2. [Experiments] No error bars, standard deviations across runs, or statistical significance tests are reported for the RMSE/MAE/R² improvements on the Milan dataset or synthetic scenarios, preventing assessment of whether the outperformance is reliable.
  3. [Experiments] The manuscript lacks an out-of-distribution evaluation (e.g., on a different city or held-out congestion regime) to test whether the reported gains generalize beyond the training distribution, which directly addresses the risk that fusion captures spurious correlations.
minor comments (2)
  1. [Abstract] The abstract states that synthetic scenarios are used but the main text should explicitly describe their generation process and parameter settings to allow replication.
  2. [Methodology] Notation for the multimodal fusion operation (e.g., how secondary features are combined with decoder hidden states) could be clarified with a precise equation or diagram.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive comments, which highlight important aspects of experimental rigor. We agree that additional analyses will strengthen the manuscript and address each point below with plans for revision.

read point-by-point responses

  1. Referee: [Experiments] The experimental section provides no ablation that removes the multimodal fusion module or compares PEHT against a capacity-matched plain LoRA-Transformer baseline without mobility features. This is load-bearing for the central claim that the urban congestion integration produces genuine predictive gains rather than dataset-specific correlations.

    Authors: We agree that an ablation isolating the multimodal fusion is necessary to substantiate the central claim. In the revised manuscript, we will add a direct comparison of PEHT against a capacity-matched LoRA-Transformer baseline that excludes the urban mobility and congestion features, while keeping parameter counts equivalent. This will clarify the contribution of the fusion mechanism. revision: yes

  2. Referee: [Experiments] No error bars, standard deviations across runs, or statistical significance tests are reported for the RMSE/MAE/R² improvements on the Milan dataset or synthetic scenarios, preventing assessment of whether the outperformance is reliable.

    Authors: We acknowledge this limitation in the current reporting. We will rerun all experiments across multiple random seeds (at least 5), report means with standard deviations, and include statistical significance tests (e.g., paired t-tests with p-values) for the reported metrics on both the Milan dataset and synthetic scenarios. revision: yes

  3. Referee: [Experiments] The manuscript lacks an out-of-distribution evaluation (e.g., on a different city or held-out congestion regime) to test whether the reported gains generalize beyond the training distribution, which directly addresses the risk that fusion captures spurious correlations.

    Authors: We agree that OOD testing is important for assessing generalization and ruling out spurious correlations. We will add an evaluation on held-out synthetic congestion regimes with different parameters from the training distribution. We will also discuss the challenges of cross-city evaluation given dataset constraints and note this as a direction for future work. revision: partial

Circularity Check

0 steps flagged

No circularity in derivation chain

full rationale

The paper proposes an empirical ML architecture (PEHT) combining LoRA-adapted Transformer with multimodal fusion of network and urban mobility features, then reports RMSE/MAE/R² gains on Telecom Italia Milan and synthetic data. No mathematical derivation, first-principles result, or predictive claim is presented that reduces by construction to fitted inputs or self-citations. The load-bearing elements are experimental comparisons, which remain externally falsifiable and do not match any enumerated circularity pattern.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, axioms, or invented entities; standard Transformer and LoRA hyperparameters are presumed but unspecified, and no new entities are introduced in the summary.

pith-pipeline@v0.9.1-grok · 5695 in / 1078 out tokens · 63415 ms · 2026-06-29T04:16:54.474398+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references

  1. [1]

    Graph neural networks: foundation, frontiers and applications,

    L. Wu, P. Cui, J. Pei, L. Zhao, and X. Guo, “Graph neural networks: foundation, frontiers and applications,” inProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, 2022, pp. 4840–4841

    2022

  2. [2]

    Towards deeper graph neural networks,

    M. Liu, H. Gao, and S. Ji, “Towards deeper graph neural networks,” inProceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, 2020, pp. 338–348

    2020

  3. [3]

    Kolmogorov–arnold graph neural networks for molecular property prediction,

    L. Li, Y . Zhang, G. Wang, and K. Xia, “Kolmogorov–arnold graph neural networks for molecular property prediction,”Nature Machine Intelligence, vol. 7, no. 8, pp. 1346–1354, 2025

    2025

  4. [4]

    Interaction- aware trajectory prediction for safe motion planning in autonomous driving: A transformer-transfer learning approach,

    J. Liang, C. Tan, L. Yan, J. Zhou, G. Yin, and K. Yang, “Interaction- aware trajectory prediction for safe motion planning in autonomous driving: A transformer-transfer learning approach,”IEEE Transactions on Intelligent Transportation Systems, 2025

    2025

  5. [5]

    Self-supervised transformer for trajectory prediction using noise imputed past trajectory,

    V . Bharilya, A. Arora, and N. Kumar, “Self-supervised transformer for trajectory prediction using noise imputed past trajectory,”IEEE Transactions on Intelligent Transportation Systems, 2025

    2025

  6. [6]

    Tailored meta-learning for dual trajectory transformer: advancing generalized trajectory prediction,

    F. Huang, Z. Fan, X. Li, W. Zhang, P. Li, Y . Geng, and K. Zhu, “Tailored meta-learning for dual trajectory transformer: advancing generalized trajectory prediction,”Complex & Intelligent Systems, vol. 11, no. 3, p. 174, 2025

    2025

  7. [7]

    A novel cellular network traffic prediction algorithm based on graph convolution neural networks and long short-term memory through extraction of spatial-temporal characteristics,

    G. Chen, Y . Guo, Q. Zeng, and Y . Zhang, “A novel cellular network traffic prediction algorithm based on graph convolution neural networks and long short-term memory through extraction of spatial-temporal characteristics,”Processes, vol. 11, no. 8, p. 2257, 2023

    2023

  8. [8]

    Multi- representation spatial-temporal graph convolutional networks for net- work traffic prediction,

    Y . Yang, Y . He, B. Zhao, C. Wu, Z. Gao, and L. Rui, “Multi- representation spatial-temporal graph convolutional networks for net- work traffic prediction,”IEEE Internet of Things Journal, 2025

    2025

  9. [9]

    A dynamic bernstein graph recurrent network for wireless cellular traffic prediction,

    A. Mehrabian, S. Bahrami, and V . W. Wong, “A dynamic bernstein graph recurrent network for wireless cellular traffic prediction,” inICC 2023- IEEE International Conference on Communications. IEEE, 2023, pp. 3842–3847

    2023

  10. [10]

    Cellular network traffic prediction with hybrid graph convolutional recurrent network,

    M. Zhang, H. Zhou, K. Yu, and X. Wu, “Cellular network traffic prediction with hybrid graph convolutional recurrent network,”Wireless Personal Communications, vol. 138, no. 3, pp. 1867–1892, 2024

    2024

  11. [11]

    Capturing spatial–temporal cor- relations with attention based graph convolutional network for network traffic prediction,

    Y . Guo, Y . Peng, R. Hao, and X. Tang, “Capturing spatial–temporal cor- relations with attention based graph convolutional network for network traffic prediction,”Journal of Network and Computer Applications, vol. 220, p. 103746, 2023

    2023

  12. [12]

    St-tran: Spatial-temporal transformer for cellular traffic prediction,

    Q. Liu, J. Li, and Z. Lu, “St-tran: Spatial-temporal transformer for cellular traffic prediction,”IEEE Communications Letters, vol. 25, no. 10, pp. 3325–3329, 2021

    2021

  13. [13]

    Sttf: A spatiotemporal transformer framework for multi-task mobile network prediction,

    J. Gong, Y . Liu, T. Li, J. Ding, Z. Wang, and D. Jin, “Sttf: A spatiotemporal transformer framework for multi-task mobile network prediction,”IEEE Transactions on Mobile Computing, 2025

    2025

  14. [14]

    St-dcan: Spatio-temporal dual compression attention network for traffic prediction,

    T. Guan, J. Peng, Y . Zhan, and J. Liang, “St-dcan: Spatio-temporal dual compression attention network for traffic prediction,” in2024 China Automation Congress (CAC). IEEE, 2024, pp. 5048–5053

    2024

  15. [15]

    Transformer based traffic flow forecasting in sdn- vanet,

    A. A. Shuvro, M. S. Khan, M. Rahman, F. Hussain, M. Moniruzzaman, and M. S. Hossen, “Transformer based traffic flow forecasting in sdn- vanet,”IEEE Access, vol. 11, pp. 41 816–41 826, 2023

    2023

  16. [16]

    Transformer-based wireless traffic prediction and network optimization in o-ran,

    M. A. Habib, P. E. I. Rivera, Y . Ozcan, M. Elsayed, M. Bavand, R. Gaigalas, and M. Erol-Kantarci, “Transformer-based wireless traffic prediction and network optimization in o-ran,” in2024 IEEE Inter- national Conference on Communications Workshops (ICC Workshops). IEEE, 2024, pp. 1–6

    2024

  17. [17]

    St2t: A spatio-temporal transformer for cellular traffic prediction in digital twin systems,

    Z. Zhang, L. Yan, and Y . Gu, “St2t: A spatio-temporal transformer for cellular traffic prediction in digital twin systems,” in2023 IEEE 6th International Conference on Electronic Information and Communication Technology (ICEICT). IEEE, 2023, pp. 1112–1117

    2023

  18. [18]

    Citywide cellular traffic prediction based on densely connected convolutional neural networks,

    C. Zhang, H. Zhang, D. Yuan, and M. Zhang, “Citywide cellular traffic prediction based on densely connected convolutional neural networks,” IEEE Communications Letters, vol. 22, no. 8, pp. 1656–1659, 2018

    2018

  19. [19]

    Deep transfer learning for intelligent cellular traffic prediction based on cross-domain big data,

    C. Zhang, H. Zhang, J. Qiao, D. Yuan, and M. Zhang, “Deep transfer learning for intelligent cellular traffic prediction based on cross-domain big data,”IEEE Journal on Selected Areas in Communications, vol. 37, no. 6, pp. 1389–1401, 2019

    2019

  20. [20]

    Mvstgn: A multi-view spatial- temporal graph network for cellular traffic prediction,

    Y . Yao, B. Gu, Z. Su, and M. Guizani, “Mvstgn: A multi-view spatial- temporal graph network for cellular traffic prediction,”IEEE Transac- tions on Mobile Computing, vol. 22, no. 5, pp. 2837–2849, 2021

    2021

  21. [21]

    A multi-source dataset of urban life in the city of milan and the province of trentino,

    G. Barlacchi, M. De Nadai, R. Larcher, A. Casella, C. Chitic, G. Torrisi, F. Antonelli, A. Vespignani, A. Pentland, and B. Lepri, “A multi-source dataset of urban life in the city of milan and the province of trentino,” Scientific data, vol. 2, no. 1, pp. 1–15, 2015

    2015