Pith. sign in

REVIEW 3 major objections 7 minor 38 references

AE-PSL preserves downstream accuracy at about 10.2x communication reduction in parallel split learning by aligning a frozen autoencoder to the pre-trained model's feature manifold before fine-tuning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 16:37 UTC pith:YKEW5Q2L

load-bearing objection A genuinely useful alignment recipe for inserting frozen autoencoder compressors into pre-trained vision transformers, with meticulous ablations, but the headline trade-off numbers rest on underspecified interpolation. the 3 major comments →

arxiv 2607.17913 v1 pith:YKEW5Q2L submitted 2026-07-20 cs.DC cs.LG

AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuning

Bas Meuwissen , Vasileios Tsouvalas , Nirvana Meratnia This is my paper
classification cs.DC cs.LG
keywords parallel split learningautoencoder compressiondistributed fine-tuningfoundation modelscommunication efficiencyfeature alignmentparameter-efficient fine-tuningvision transformers
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that learnable compression of intermediate activations and gradients can replace heuristic sparsification and quantization in split learning without hurting fine-tuning accuracy, provided the compressor is aligned to the pre-trained model's feature manifold before training begins. It proposes AE-PSL, which inserts a lightweight autoencoder at the cut layer — encoder on the client, decoder on the server — and a two-stage alignment procedure (general alignment on public data, then client-specific alignment on local data) that adapts the autoencoder without centralizing private data. Across four vision datasets, AE-PSL matches or slightly exceeds the uncompressed baseline at roughly 5x and 10x communication reduction, and stays within a few points at 20x, while heuristic baselines degrade sharply. The reason to care: if true, it removes the main communication bottleneck for fine-tuning large models on edge devices without changing the model or the training algorithm.

Core claim

The central discovery is that a randomly initialized autoencoder inserted into a pre-trained model destroys accuracy, but the same autoencoder becomes nearly lossless in terms of downstream task performance after a two-stage alignment: first training it to reconstruct split-layer activations on public data with the client-side sub-model frozen, then locally adapting each client's encoder while aggregating only the decoders at the server. Once aligned and frozen, the autoencoder compresses both forward activations and backward gradients, giving about 10.2x communication reduction for the same accuracy as uncompressed fine-tuning, and reaching 99% of peak accuracy with 12.4x less communication

What carries the argument

The load-bearing object is the frozen autoencoder (a token-wise MLP bottleneck, i.e., a small feedforward network that compresses each token's hidden vector) placed exactly at the split layer, combined with the design that the encoder lives on each client, the decoder is aggregated across clients on the server, and the whole module is aligned before fine-tuning rather than co-trained with it. The alignment protocol — General Alignment (GA) on public data followed by Client-Specific Alignment (CSA) on local data with decoder weight averaging — is what makes learnable compression compatible with off-the-shelf pre-trained models; the paper's ablation shows that removing GA or unfreezing the AE

Load-bearing premise

The entire result rests on the premise that an autoencoder trained only to minimize reconstruction error (on public data plus one local epoch) captures the information the server-side sub-model needs during fine-tuning, and that freezing it keeps reconstruction error low while client-side activations shift; the paper's own Fig. 7 shows reconstruction error rising when the AE is not frozen.

What would settle it

Run the same AE-PSL protocol with the [CLS] token also compressed (compensating by raising the latent dimension to hold R fixed); Table 7 in the paper already shows accuracy dropping from 82.4 to 73.9 at low reduction, so a full sweep across R and datasets that reproduces this collapse would falsify the claim that learnable compression alone preserves accuracy — the preserved accuracy may depend on the uncompressed global token rather than on the alignment mechanism.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • AE-PSL can be dropped into any split-learning or split-federated-learning setup that uses a pre-trained transformer, since GA depends only on the model's activation manifold and CSA only aggregates decoders.
  • At 10x compression, fine-tuning quality is statistically indistinguishable from uncompressed fine-tuning on the four vision datasets tested, so communication can be cut by an order of magnitude for free.
  • Because the AE is frozen, backward gradients are also compressed, so the savings apply to both directions of the per-iteration exchange.
  • The method converges faster in terms of total bytes: it reaches 99% of peak accuracy with 12.4x less communication than the heuristic baselines.
  • The framework inherits the scalability of aggregated split learning (server-side backprop stays O(1)) while adding only about 1.6% client-side parameters.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 5.2 pp edge over Rand-Top-K at equal communication budget suggests that the bottleneck's regularization — suppressing task-irrelevant noise in activations — may itself be a source of generalization gain, not just a compression trick; a test would be whether AE-PSL also improves accuracy on out-of-distribution data.
  • The reliance on leaving the [CLS] token uncompressed bounds the achievable compression at roughly 20x-25x for ViT-style models; extending token-adaptive importance weighting (the paper's stated future direction) would remove the hard ceiling and is the natural next experiment.
  • Because CSA aggregates decoders by simple weight averaging, the framework assumes client feature shifts are small enough that one shared decoder can serve all clients; under heavy label skew this assumption may break, and a per-client decoder or clustered aggregation would be the test.
  • The findings are demonstrated only for vision transformers and a fixed split layer (s=5); transferring the same two-stage alignment to text or multimodal models is untested, and the non-vision case is the likeliest place to find a failure mode.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes AE-PSL, a parallel split learning framework that inserts a lightweight autoencoder at the split layer to compress intermediate activations and gradients during fine-tuning of a pre-trained ViT. To make the randomly initialized AE compatible with an off-the-shelf foundation model, the method introduces a two-stage alignment procedure: General Alignment (GA) trains the AE on public data to reconstruct the frozen client-side activations, and Client-Specific Alignment (CSA) adapts the encoder locally on each client before aggregating only the decoders at the server. The AE is then frozen during distributed fine-tuning. The paper evaluates AE-PSL on CIFAR-100, Food101, SUN397, and FEMNIST with N=5 and N=25 clients at three compression levels (R≈5, 10, 20), comparing against Rand-Top-K, C3-SL, and ADC. The central claims are that AE-PSL preserves downstream accuracy at 10.2x communication reduction and that the strongest heuristic baseline trails it by 5.2 percentage points at the same communication budget.

Significance. If the claims hold, AE-PSL would be a useful contribution to communication-efficient fine-tuning of pre-trained models in split learning: it extends learnable compression to off-the-shelf FMs without joint training, and the evaluation is reasonably broad by SL standards. The paper ships a public code repository, reports three seeds, includes both global and local evaluation modes, and provides component ablations (AE, +GA, +FZ, +CSA), alternative alignment strategies, split-layer sensitivity, and CLS-token compression experiments. These are genuine strengths. However, the headline numerical claims are currently not reproducible from the reported data because the fixed-budget interpolation and the communication accounting are not specified, and the per-setting tables show non-negligible accuracy degradation that is averaged away. The core idea is plausible and the empirical core is salvageable, but the central quantitative claims need to be made precise before the paper can be recommended for publication.

major comments (3)
  1. [§5.3.1, Fig. 2, Fig. 5, Appx. A.2] The headline numbers '10.2x reduction' and '5.2 pp lead' are the result of an interpolation procedure that is never described. Only three discrete compression levels (R≈5, 10, 20) are measured per method, yet figures 2 and 5 draw smooth accuracy-vs-communication curves and report x-axis labels such as '9.7x' and '15.0x'. The text does not state (i) what interpolation is used (linear in R, linear in log-GB, or something else), (ii) whether the x-axis 'total communication (GB)' includes the one-time GA and CSA communication costs in addition to per-iteration activation/gradient traffic, (iii) how bidirectional compressed activation/gradient exchanges are accounted for, or (iv) how per-client totals are aggregated. The thresholds are also inconsistent: Fig. 2's caption says markers indicate 'closest (within 99%)' of no-compression DFT, while Fig. 5's caption says '>95% of peak performance';
  2. [Abstract; §5.3.1, Table 1] The statement that AE-PSL 'preserves downstream accuracy at 10.2x reduction' is stronger than what Table 1 shows. At R≈20, Food101 drops by 3.8 pp (N=5) and 3.2 pp (N=25) relative to the no-compression baseline, and CIFAR-100 drops by 1.3 pp (N=5) and 0.9 pp (N=25). The text itself acknowledges 'the largest drop is 3.8 pp', so 'without accuracy degradation' is only true if the claim is an average or is defined with an explicit tolerance. The abstract and §5.3.1 should state the exact criterion (e.g., 'within 99% of the no-compression baseline on average', or 'within 1.0 pp for R≤10') and should report the worst-case degradation alongside the average. As written, a reader cannot tell whether the claim is about a mean over settings or about every setting.
  3. [§4.1.2, Table 2, Appx. A.5] The CSA design has a potential mismatch that is not directly tested. Each client locally adapts both encoder and decoder on its own data, but then only the decoder weights are sent to the server and averaged; each client keeps its own encoder. The aggregated decoder is thus an average of decoders that were trained against different encoders, and it is never trained jointly with any client encoder. Table 2 shows only a small difference between AE+GA+FZ and AE+GA+CSA+FZ (≤1.1 pp in most cells), so the contribution of the CSA decoder-aggregation step is not convincingly isolated. Appx. A.5 shows that encoder-only alignment (EO-CSA) has no effect, which supports the need to update the decoder, but it does not test whether the averaged decoder is better than, say, keeping the GA decoder or using per-client decoders. Please add an ablation that separates the effect of client-specific encoders
minor comments (7)
  1. [Appx. A.1] Typo: 'using Adam with with learning rate' should be 'using Adam with learning rate'.
  2. [Fig. 2 and Fig. 5] The caption of Fig. 2 says 'within 99%' while Fig. 5 says '>95% of peak performance' for the same marker semantics in the main text. Please harmonize the thresholds and define what 'peak performance' refers to (no-compression accuracy or per-method peak).
  3. [Table 1 vs Table 2, FEMNIST] The no-compression FEMNIST baselines differ between tables for nominally identical settings: Table 1 reports 87.9 (N=5) and 85.8 (N=25), while Table 2 reports 87.6 (N=5) and 85.4 (N=25). Please reconcile these values or clarify why the no-compression baseline differs across runs.
  4. [Table 1 and throughout] The tables describe deltas as 'relative performance' to the no-compression baseline, but the values are absolute percentage-point differences. Please use the term 'absolute difference'.
  5. [Table 3] For ADC, the parameter entries '1/√5', '1/√10', '1/√20' are ambiguous: it is not clear whether these refer to batch-wise compression, token-wise compression, or both. Please define the meaning of these ratios in the caption.
  6. [§5.3.1] The abstract says the strongest baseline trails by 5.2 pp, while §5.3.1 says 5.3 pp. Please make the numbers consistent.
  7. [§5.3.3, Fig. 4] The AE architecture choice (2-layer MLP) is based on a single-client Food101 experiment (N=1), while the main claims use N=5 and N=25. It would be helpful to state explicitly whether the architecture ranking was checked under multi-client settings or why this is not necessary.

Circularity Check

0 steps flagged

No significant circularity: the AE is trained on an MSE reconstruction objective independent of downstream labels, and the accuracy–communication trade-off is measured against external baselines rather than derived from fitted parameters.

full rationale

The paper's central derivation is self-contained in the relevant sense. The AE encoder/decoder are aligned by Eq. (1), an MSE reconstruction loss on public ImageNet100 data and then locally on client data, with the pre-trained model frozen; the downstream accuracy used to validate AE-PSL comes from an external benchmark evaluation (CIFAR-100, Food101, SUN397, FEMNIST) and is not used to fit the compressor. The claimed 10.2x/5.2pp trade-off numbers are empirical summaries of measured accuracy at discrete compression levels; although the interpolation/threshold procedure behind the fixed-budget curves is under-specified (99% vs 95% thresholds in Fig. 2 vs Fig. 5), that is a reproducibility/correctness concern, not a definitional reduction. The only overlapping-author reference ([11], Fudala/Tsouvalas/Meratnia) is cited in Related Work alongside external work [36,30] for the innocuous point that transformer SL exchange costs are communication-dominated; no uniqueness theorem or prior result by the authors is invoked to force the AE design. The limitations section candidly notes non-vision domains are unevaluated and the [CLS] token is transmitted uncompressed, which further weakens any impression that the framework's success is built into its definition. No equation in the paper is equal to its inputs by construction; the AE is not trained on labels or on the accuracy metric it is later used to predict.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The method introduces no ontologically new entities; the autoencoder is a standard learned module. The free parameters are hyperparameters and design choices, none fitted to the final accuracy metric. The key assumptions are the availability of representative public data and the sufficiency of a one-epoch client-side alignment, plus the dependence on an uncompressed CLS token.

free parameters (5)
  • AE latent dimension dz = 140 (Low), 64 (Mid), 24 (High)
    Set by the target communication reduction level R; controls the compression ratio and is a design choice, not fitted to the test set.
  • AE architecture = 2-layer-MLP-(d-d-dz) with GELU
    Selected after comparing MLP vs convolutional AEs and different depths on Food101 (Sec. 5.3.3); a model-selection choice on the task.
  • Split layer s = s=5
    Chosen after evaluating split layers 0-6 (Appx. A.6); picked to balance client compute, stability, and accuracy.
  • GA training epochs = 30 on ImageNet100
    Hyperparameter of the alignment protocol; not fitted to the evaluated datasets but tuned for the public data.
  • CSA training epochs = 1 epoch locally
    One local epoch per client; ablations show 1-CSA nearly matches 2- and 3-CSA, so the choice is justified.
axioms (4)
  • standard math Backpropagation and MSE reconstruction behave as standard in neural network training.
    Invoked throughout Sec. 4 and the experimental setup.
  • domain assumption Public data (ImageNet100) is available and its split-layer activations are representative of the pre-trained model's feature manifold.
    GA (Eq. 1) trains the AE on ImageNet100; the method assumes this aligns the AE to the model that will be fine-tuned on other datasets.
  • domain assumption Client-specific distributions are close enough to the general manifold that a single local CSA epoch suffices.
    CSA warms up the AE for only one epoch on local data; the paper's ablation shows this helps, but its sufficiency for larger distribution shifts is not established.
  • domain assumption The [CLS] token carries enough global information that leaving it uncompressed preserves downstream classification accuracy.
    The framework transmits the [CLS] token at full dimension throughout; Appx. A.8 shows compressing it degrades accuracy sharply, so the method structurally depends on this assumption.

pith-pipeline@v1.3.0-alltime-deepseek · 20178 in / 10789 out tokens · 104024 ms · 2026-08-01T16:37:06.793488+00:00 · methodology

0 comments
read the original abstract

Distributed Fine-Tuning (DFT) of large-scale Foundation Models (FMs) on resource-constrained edge devices is limited by local compute constraints and communication overhead. Parallel Split Learning (PSL) reduces client-side computation by keeping few model layers on each client and offloading the remaining computation to the server; however, clients must exchange intermediate activations and gradients with the server at every training step. Existing SL communication-compression methods mainly rely on task-agnostic heuristics, such as sparsification and quantization. While learnable SL compressors can better adapt to intermediate representations, they require co-training with the target model. Therefore, directly inserting them into off-the-shelf FMs introduces feature-distribution misalignment and degrades DFT performance. To address this, we propose AE-PSL, a communication-efficient PSL framework that compresses intermediate activations and gradients using a lightweight AutoEncoder (AE) placed at the split layer. To ensure compatibility of AE compression with pre-trained FMs, AE-PSL introduces a novel two-stage alignment mechanism, which adapts the AE to the pre-trained model's feature manifold and client-specific feature distributions before DFT.

Figures

Figures reproduced from arXiv: 2607.17913 by Bas Meuwissen, Nirvana Meratnia, Vasileios Tsouvalas.

Figure 1
Figure 1. Figure 1: Overview of AE-PSL, consisting of three sequential stages: (1) GA trains the AE on public data to match the pre-trained model’s feature manifold. (2) CSA locally adapts the AE to client-specific data, aggregating only the decoders at the server. (3) DFT executes distributed training with frozen AEs, minimizing communication overhead by transmitting compressed activations and gradients. an AE-based mechanis… view at source ↗
Figure 2
Figure 2. Figure 2: Accuracy–communication trade-off using ViT-B/32 on CIFAR-100. We [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Accuracy–client-side compute trade-off using ViT-B/32 for [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Effect of AE architecture in AE-PSL across communication reduction levels (R). Experiments are conducted with ViT-B/32 [9] on Food101 with N = 1. We report average (a) MSE reconstruction loss during GA, and (b) downstream DFT accuracy over 3 seeds. Rand-Top-K, trails AE-PSL by 5.3 pp. on average. Furthermore, to reach within 99% of no-compression DFT AE-PSL needs 12.4× less communication budget, with Rand-… view at source ↗
Figure 5
Figure 5. Figure 5: Accuracy–communication trade-off using ViT-B/32 for [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Accuracy–client-side compute trade-off using ViT-B/32 for [PITH_FULL_IMAGE:figures/full_fig_p019_6.png] view at source ↗
Figure 6
Figure 6. Figure 6: Accuracy–client-side compute trade-off using ViT-B/32 for [PITH_FULL_IMAGE:figures/full_fig_p020_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Per-client averaged reconstruction error ( [PITH_FULL_IMAGE:figures/full_fig_p022_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Per-client averaged reconstruction error ( [PITH_FULL_IMAGE:figures/full_fig_p023_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Per-client averaged reconstruction error ( [PITH_FULL_IMAGE:figures/full_fig_p024_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Accuracy of DFT with AE compression at split layer [PITH_FULL_IMAGE:figures/full_fig_p025_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Evaluation of AE architectures varying communication reduction levels [PITH_FULL_IMAGE:figures/full_fig_p026_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Evaluation of encoder parameter counts for AE architectures across [PITH_FULL_IMAGE:figures/full_fig_p027_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 2 canonical work pages

  1. [1]

    arXiv preprint arXiv:2509.15058 (2025)

    Alvetreti, F., Pomponi, J., Di Lorenzo, P., Scardapane, S.: Communication efficient split learning of vits with attention-based double compression. arXiv preprint arXiv:2509.15058 (2025)

    arXiv 2025

  2. [2]

    In: 2021 IEEE Global Communications Conference (GLOBECOM)

    Ayad, A., Renner, M., Schmeink, A.: Improving the communication and com- putation efficiency of split learning for iot applications. In: 2021 IEEE Global Communications Conference (GLOBECOM). pp. 01–06 (2021).https://doi.org/ 10.1109/GLOBECOM46510.2021.9685493

    arXiv 2021

  3. [3]

    In: European conference on computer vision

    Bossard, L., Guillaumin, M., Van Gool, L.: Food-101–mining discriminative com- ponents with random forests. In: European conference on computer vision. pp. 446–461. Springer (2014)

    2014

  4. [4]

    CoRR (2018),http://arxiv.org/abs/ 1812.01097

    Caldas, S., Wu, P., Li, T., Konečný, J., McMahan, H.B., Smith, V., Talwalkar, A.: LEAF: A benchmark for federated settings. CoRR (2018),http://arxiv.org/abs/ 1812.01097

    Pith/arXiv arXiv 2018

  5. [5]

    In: International conference on machine learning

    Castiglia, T.J., Das, A., Wang, S., Patterson, S.: Compressed-vfl: Communication- efficient learning with vertically partitioned data. In: International conference on machine learning. pp. 2738–2766. PMLR (2022)

    2022

  6. [6]

    IEEE Internet of Things Journal (99) (2025).https://doi.org/10.1109/JIOT.2025.3600269

    Chen, X., Wu, W., Ji, F., Lu, Y., Li, L.: Privacy-Aware Split Federated Learning for LLM Fine-Tuning over Internet of Things. IEEE Internet of Things Journal (99) (2025).https://doi.org/10.1109/JIOT.2025.3600269

    arXiv 2025

  7. [7]

    IEEE Internet of Things Journal 12(15), 30460–30474 (2025)

    Chen, X., Li, J., Fan, D., Chakrabarti, C.: HeteroSFL: Split Federated Learning With Heterogeneous Clients and Non-IID Data. IEEE Internet of Things Journal 12(15), 30460–30474 (2025)

    2025

  8. [8]

    In: 2020 IEEE International Conference on Multimedia and Expo (ICME)

    Cohen, R.A., Choi, H., Bajić, I.V.: Lightweight compression of neural network feature tensors for collaborative intelligence. In: 2020 IEEE International Conference on Multimedia and Expo (ICME). pp. 1–6 (2020).https://doi.org/10.1109/ ICME46284.2020.9102797

    arXiv 2020

  9. [9]

    In: International Conference on Learning Representations (ICLR) (2021), https://openreview.net/forum?id=YicbFdNTTy

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In: International Conference on Learning Representations (ICLR) (2021), https://openreview.net/forum?id=YicbFdNTTy

    2021

  10. [10]

    In: 2019 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED)

    Eshratifar, A.E., Esmaili, A., Pedram, M.: Bottlenet: A deep learning architecture for intelligent mobile cloud computing services. In: 2019 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED). pp. 1–6. IEEE (2019)

    2019

  11. [11]

    arXiv preprint arXiv:2502.06355 (2025)

    Fudala, T., Tsouvalas, V., Meratnia, N.: Fine-tuning multimodal transformers on edge: A parallel split learning approach. arXiv preprint arXiv:2502.06355 (2025)

    Pith/arXiv arXiv 2025

  12. [12]

    In: 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

    Guo, S., Lu, Z., Lu, S., Cui, Y., Tang, X., Wu, J.: Split learning optimized for the medical field: Reducing communication overhead. In: 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). pp. 3226–3231 (2024). https://doi.org/10.1109/BIBM62325.2024.10822194

    arXiv 2024

  13. [13]

    Journal of Network and Computer Applications116, 1–8 (2018) 14 B

    Gupta, O., Raskar, R.: Distributed learning of deep neural network over multiple agents. Journal of Network and Computer Applications116, 1–8 (2018) 14 B. Meuwissen et al

    2018

  14. [14]

    In: 2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing (MLSP)

    Hsieh, C.Y., Chuang, Y.C., Wu, A.Y.: C3-sl: Circular convolution-based batch- wise compression for communication-efficient split learning. In: 2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing (MLSP). pp. 1–6. IEEE (2022)

    2022

  15. [15]

    ICLR1(2), 3 (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. ICLR1(2), 3 (2022)

    2022

  16. [16]

    In: International Conference on Learning Representations (ICLR)

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: International Conference on Learning Representations (ICLR). San Diego, CA, USA (2015)

    2015

  17. [17]

    Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)

    2009

  18. [18]

    IEEE Networking Letters (2025)

    Li, Z., Wu, S., Li, L., Zhang, S.: Energy-efficient split learning for fine-tuning large language models in edge networks. IEEE Networking Letters (2025)

    2025

  19. [19]

    arXiv preprint arXiv:2407.00952 (2024)

    Lin, Z., Hu, X., Zhang, Y., Chen, Z., Fang, Z., Chen, X., Li, A., Vepakomma, P., Gao, Y.: Splitlora: A split parameter-efficient fine-tuning framework for large language models. arXiv preprint arXiv:2407.00952 (2024)

    Pith/arXiv arXiv 2024

  20. [20]

    IEEE Transactions on Mobile Computing (01), 1–17 (Apr 5555)

    Lin, Z., Zhang, Y., Chen, Z., Fang, Z., Chen, X., Vepakomma, P., Ni, W., Luo, J., Gao, Y.: HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models . IEEE Transactions on Mobile Computing (01), 1–17 (Apr 5555). https://doi.org/10.1109/TMC.2026.3680521, https:// doi.ieeecomputersociety.org/10.1109/TMC.2026.3680521

    arXiv 2026

  21. [21]

    IEEE Internet of Things Magazine6(4), 124–129 (2023)

    Lyu, X., Liu, S., Liu, J., Ren, C.: Scalable aggregated split learning for data-driven edge intelligence on internet-of-things. IEEE Internet of Things Magazine6(4), 124–129 (2023)

    2023

  22. [22]

    IEEE Communications Magazine64(3) (2026).https://doi

    Ma, J., Lyu, X., Jiang, J., Cui, Q., Yao, H., Tao, X.: Splitfrozen: Split learn- ing with device-side model frozen for fine-tuning llm on heterogeneous resource- constrained devices. IEEE Communications Magazine64(3) (2026).https://doi. org/10.48550/arxiv.2503.18986

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.18986 2026

  23. [23]

    In: 2022 IEEE 23rd International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM)

    Matsubara, Y., Callegaro, D., Singh, S., Levorato, M., Restuccia, F.: Bottlefit: Learning compressed representations in deep neural networks for effective and efficient split computing. In: 2022 IEEE 23rd International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM). pp. 337–346 (2022). https://doi.org/10.1109/WoWMoM54355.2022.00032

    arXiv 2022

  24. [24]

    Artifi- cial Intelligence and Statistics (AISTATS)54, 1273–1282 (2017)

    McMahan, H.B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.y.: Communication-efficient learning of deep networks from decentralized data. Artifi- cial Intelligence and Statistics (AISTATS)54, 1273–1282 (2017)

    2017

  25. [25]

    ACM Trans

    Mudvari,A.,Vainio,A.,Ofeidis,I.,Tarkoma,S.,Tassiulas,L.:Adaptivecompression- aware split learning and inference for enhanced network efficiency. ACM Trans. Internet Technol.24(4) (Nov 2024).https://doi.org/10.1145/3687471, https: //doi.org/10.1145/3687471

    work page doi:10.1145/3687471 2024

  26. [26]

    IEEE Transactions on Neural Networks and Learning Systems pp

    Oh, Y., Lee, J., Brinton, C., Jeon, Y.S.: Communication-efficient split learning via adaptive feature-wise compression. IEEE Transactions on Neural Networks and Learning Systems pp. 1–15 (01 2025).https://doi.org/10.1109/TNNLS.2025. 3526227

    doi:10.1109/tnnls.2025 2025

  27. [27]

    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M.S., Berg, A.C., Fei-Fei, L.: Imagenet large scalevisualrecognitionchallenge.CoRR(2014), http://arxiv.org/abs/1409.0575

    Pith/arXiv arXiv 2014

  28. [28]

    2020 IEEE International Confer- ence on Communications Workshops (ICC Workshops) pp

    Shao, J., Zhang, J.: Bottlenet++: An end-to-end approach for feature com- pression in device-edge co-inference systems. 2020 IEEE International Confer- ence on Communications Workshops (ICC Workshops) pp. 1–6 (2019),https: //api.semanticscholar.org/CorpusID:207757421 AutoEncoder-Compressed Parallel Split Learning 15

    2020

  29. [29]

    Shiranthika, C., Hadizadeh, H., Saeedi, P., Bajić, I.V.: SplitFedZip: Learned com- pression for data transfer reduction in split-federated learning (2024), https: //arxiv.org/abs/2412.17150

    Pith/arXiv arXiv 2024

  30. [30]

    arXiv preprint arXiv:1909.09145 (2019)

    Singh, A., Vepakomma, P., Gupta, O., Raskar, R.: Detailed comparison of com- munication efficiency of split learning and federated learning. arXiv preprint arXiv:1909.09145 (2019)

    Pith/arXiv arXiv 1909

  31. [31]

    In: 2010 IEEE computer society conference on computer vision and pattern recognition

    Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: Sun database: Large-scale scene recognition from abbey to zoo. In: 2010 IEEE computer society conference on computer vision and pattern recognition. pp. 3485–3492. IEEE (2010)

    2010

  32. [32]

    In: International conference on machine learning

    Yurochkin, M., Agarwal, M., Ghosh, S., Greenewald, K., Hoang, N., Khazaeni, Y.: Bayesian nonparametric federated learning of neural networks. In: International conference on machine learning. pp. 7252–7261. PMLR (2019)

    2019

  33. [33]

    In: 2024 IEEE Globecom Workshops (GC Wkshps)

    Zhang, S., Cheng, G., Li, Z., Wu, W.: Splitllm: Hierarchical split learning for large language model over wireless network. In: 2024 IEEE Globecom Workshops (GC Wkshps). pp. 1–6. IEEE (2024)

    2024

  34. [34]

    1–16 (01 2025).https://doi.org/10.1109/JSTSP.2025.3581484

    Zhang, S., Cheng, G., Wu, W., Xinyu, H., Song, L., Shen, X.: Split fine-tuning for largelanguagemodelsinwirelessnetworks.IEEEJournalofSelectedTopicsinSignal Processing pp. 1–16 (01 2025).https://doi.org/10.1109/JSTSP.2025.3581484

    arXiv 2025

  35. [35]

    Zhao, K., Zhu, C., Chen, M., Chongwen, H., Yang, Z., Zhang, Z.: Sflllm: Efficient split federated learning for large language model over wireless networks. pp. 1835– 1840 (12 2025).https://doi.org/10.1109/GLOBECOM59602.2025.11432069

    arXiv 2025

  36. [36]

    Frontiers of Computer Science20(12), 2012627 (2026)

    Zhao, W.X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., Wang, X., Du, Y., Yang, C., Chen, Y., Chen, Z., Jiang, J., Ren, R., Li, Y., Tang, X., Liu, Z., Hu, Y., Nie, J.Y., Wen, J.R.: A survey of large language models. Frontiers of Computer Science20(12), 2012627 (2026). https://doi.org/10.1007/s11704-026-60308-3

    doi:10.1007/s11704-026-60308-3 2026

  37. [37]

    CoRR (2018),http://arxiv.org/abs/1806.00582

    Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., Chandra, V.: Federated learning with non-iid data. CoRR (2018),http://arxiv.org/abs/1806.00582

    Pith/arXiv arXiv 2018

  38. [38]

    In: Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence

    Zheng, F., Chen, C., Lyu, L., Yao, B.: Reducing communication for split learning by randomized top-k sparsification. In: Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence. IJCAI ’23 (2023).https://doi.org/10. 24963/ijcai.2023/519,https://doi.org/10.24963/ijcai.2023/519 16 B. Meuwissen et al. A Appendix A.1 Detailed...

    work page doi:10.24963/ijcai.2023/519 2023