Pith. sign in

REVIEW 3 major objections 8 minor 35 references

Edge devices fine-tune vision transformers by sending only semantically important tokens

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-05 07:38 UTC pith:UFBWKSPQ

load-bearing objection Sound wireless optimization wrapped around a token selection metric that doesn't actually optimize anything semantic the 3 major comments →

arxiv 2605.26120 v1 pith:UFBWKSPQ submitted 2026-04-21 cs.DC

Semantic-aware Token Selection and Resource Optimization for Communication-efficient Split Federated Fine-tuning in Edge Intelligence

Xianke Qiang , Zheng Chang , Geyong Min This is my paper
classification cs.DC
keywords edgefederatedfine-tuningresourcesemanticsplitcommunicationdevices
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes ST-SFLora, a framework for fine-tuning large Vision Transformer models across resource-constrained mobile edge devices. The central mechanism is semantic-aware token selection: instead of transmitting the full set of intermediate activation tokens from client to server (which can exceed 37 MB per batch), the framework uses the Transformer's own [CLS] attention scores to identify and retain only the top-K most task-relevant tokens, merging the rest into a single summary token. This token-level compression is coupled with a joint optimization of uplink bandwidth and transmit power under latency and energy constraints. The authors introduce a metric called Semantic Transmission Efficiency (STE), defined as the total retained semantic information divided by the bottleneck uplink latency, and formulate a mixed-integer nonconvex program to maximize it. An alternating optimization algorithm decomposes the problem into power control (solved via bisection), bandwidth allocation (solved via nested bisection), and token selection (solved in closed form). Experiments on ImageNet100, Oxford Flowers-102, and CUB-200-2011 with ViT-S/B/L backbones show that ST-SFLora reduces client-side GPU memory to 1.4 GB and cuts communication payload proportionally to K, while retaining competitive accuracy against full-token split federated baselines.

Core claim

The paper's central claim is that the [CLS] token's attention weights from the client-side (frozen) Transformer blocks serve as a reliable proxy for the semantic importance of patch tokens, enabling aggressive token-level compression of uplink activations in split federated learning without disproportionate accuracy loss. By coupling this semantic selection with joint bandwidth and power optimization under a new efficiency metric (STE), the framework achieves the lowest client-side resource consumption among compared baselines while maintaining a favorable accuracy-efficiency trade-off. The optimization itself decomposes cleanly: power control reduces to a per-client bisection search for the

What carries the argument

ST-SFLora framework; Semantic Transmission Efficiency (STE) metric; [CLS] attention-based token selection and merging; alternating optimization over token budget K, bandwidth W, and power p; mixed-integer nonconvex program decomposed into three subproblems

Load-bearing premise

The framework assumes that the [CLS] token's attention scores from the frozen client-side Transformer blocks reliably indicate which patch tokens matter for the server-side LoRA fine-tuning task. If these early-layer attention patterns do not correlate well with the final task-discriminative features, the token selection and merging process could discard critical information, undermining the accuracy-efficiency trade-off.

What would settle it

If a strong negative correlation were found between [CLS] attention scores at the cut layer and the gradient norms of the corresponding patch tokens at the server-side loss function, the token selection mechanism would be selecting the wrong tokens and the STE optimization would be maximizing a metric uncorrelated with actual learning utility.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Token-level semantic compression could replace bit-level quantization as the primary communication bottleneck reduction strategy in split learning for Transformer architectures, shifting the optimization from signal processing to attention-based information filtering.
  • The STE metric framework could generalize beyond Vision Transformers to any split learning scenario where intermediate representations have identifiable importance scores, including NLP Transformer models with token importance from attention.
  • The coupling of semantic-layer decisions (token count) with physical-layer resources (bandwidth, power) suggests that future edge intelligence protocols may need cross-layer designs where model-level and network-level parameters are co-optimized per round.
  • The finding that K>=128 tokens approximate full-token accuracy on most benchmarks implies that a large fraction of patch activations are semantically redundant for downstream fine-tuning, which has implications for efficient model design beyond communication.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reliance on early-layer [CLS] attention as a proxy for downstream task relevance may break down when the client-side cut layer is very shallow, since attention patterns in early layers may not yet reflect task-discriminative features. A learnable or layer-adaptive importance scoring mechanism could improve robustness.
  • The framework's synchronous aggregation bottleneck (max client latency) suggests that asynchronous or hierarchical aggregation could further improve STE by removing the straggler constraint, though this would require handling staleness in LoRA updates.
  • The token merging strategy (attention-weighted average of discarded tokens) may lose spatial locality information that matters for fine-grained tasks like CUB-200-2011, where the accuracy gap is largest, suggesting that spatially-aware merging could recover some of the lost performance.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. This paper proposes ST-SFLora, a split federated LoRA fine-tuning framework for Vision Transformers on resource-constrained edge devices. The key idea is to use the [CLS] token's attention scores from a frozen client-side backbone to select the most semantically important patch tokens before uplink transmission, thereby reducing communication overhead. A new metric, Semantic Transmission Efficiency (STE), is defined as the ratio of cumulative attention scores to bottleneck uplink latency. The authors formulate a mixed-integer nonconvex optimization problem to jointly optimize token selection, bandwidth allocation, and transmit power, and solve it via alternating optimization with bisection-based subproblem solvers. Experiments on ImageNet100, Oxford Flowers-102, and CUB-200-2011 compare ST-SFLora against several FL and SFL baselines, showing reduced client-side resource consumption at the cost of some accuracy.

Significance. The paper addresses a timely problem at the intersection of split federated learning and communication-efficient fine-tuning of large Transformer models. The system architecture — one-way uplink SFL with frozen client-side backbone and server-side LoRA — is practical and well-motivated. The joint optimization formulation coupling token selection with wireless resource allocation is a reasonable contribution to the MEC literature. The alternating optimization algorithm is clearly described with per-subproblem complexity analysis. However, the central claim that attention-based token selection provides a favorable accuracy-efficiency trade-off is not adequately supported by the experimental evidence, and the STE metric has a structural issue that undermines the optimization formulation.

major comments (3)
  1. Section VI.C, Eq. (42)–(43): The token selection subproblem SUBP3 maximizes f_m(K_m) subject to upper bounds on K_m. Since f_m(K_m) is proven monotonically increasing (Lemma 1), the optimal solution is trivially K*_m = K^max_m. This means the STE metric's numerator is maximized by always selecting as many tokens as constraints allow, and the optimization never actually trades off semantic retention against latency — it simply pushes K to its constraint boundary. The paper claims (Section V-A, Fig. 6) that STE exhibits a peak demonstrating an optimal trade-off, but the formulation as written cannot produce an interior optimum for K_m. The trade-off is entirely enforced by external constraints (C5, C6), not by the STE objective itself. This undercuts the claimed contribution of a principled semantic-aware selection mechanism.
  2. Table I, CUB-200-2011 rows: ST-SFLora consistently and substantially underperforms ST-SFLora-Full (the same architecture without token selection). For ViT-L/16 Non-IID, the gap is 84.11% vs. 75.83% — an 8.3 percentage-point drop attributable solely to token selection. For ViT-B/16 Non-IID, the gap is 6.2 points (79.91% vs. 73.69%). These are not marginal degradations for fine-grained classification, yet Section VII-B characterizes them as 'moderate' and 'within a tolerable range.' The paper's central claim of a 'favorable trade-off' needs to be reconciled with these results, or the characterization should be revised. At minimum, the paper should discuss why CUB-200-2011 degrades much more than ImageNet100 or Flowers-102 and whether the [CLS] attention proxy is adequate for fine-grained tasks.
  3. Section IV-B, Eq. (12): The token selection uses [CLS] attention scores from the frozen client-side backbone. Since the backbone is never updated during fine-tuning, these attention patterns reflect pre-trained saliency rather than task-adapted discriminative features. For fine-grained datasets like CUB-200-2011, where discriminative features are often subtle local patches (beak shape, plumage pattern), a pre-trained [CLS] token focused on global object structure may not prioritize the correct regions. The paper should either empirically validate that pre-trained [CLS] attention correlates with task-discriminative patches (e.g., via a correlation analysis between selected tokens and class-discriminative regions), or acknowledge this as a fundamental limitation. The conclusion mentions 'learnable token selection strategies' as future work, but the severity of the accuracy drops on CUB-200
minor comments (8)
  1. Section I, first sentence: 'Aritifical' should be 'Artificial'.
  2. Section III-B, Eq. (4): The payload size formula S_m = B × (K*_m + 2) × D × q_0 includes a '+2' for the [CLS] token and the merged token, but this is only explained later in Section IV-B. A forward reference or brief note here would improve readability.
  3. Section V-A, Eq. (18): The batch-level token importance sums attention scores across the batch after per-sample sorting. This means the n-th ranked position aggregates scores from potentially different patch tokens across samples. The text should clarify that K_m selects the top-K positions in this batch-level ranking, not the same spatial tokens across images.
  4. Section VII-A: The coverage radius is stated as '5 to 500m' — this should be clarified as a range for the radius, or if it refers to the deployment area dimensions.
  5. Fig. 8(b): The y-axis label and the legend labels are not clearly described. It is unclear whether 'STE' is normalized or in raw units, and what 'No Power Control', 'No Bandwidth Allocation', and 'No Token Selection' correspond to in terms of the algorithm.
  6. Table II: The communication cost entry '3/16 × N' and '3/16 × (K+2)' would benefit from units (MB) in the table cells or a clearer note, as the explanation appears only in the text below.
  7. Section VI-D: The complexity analysis states the total complexity but does not discuss convergence guarantees for the alternating algorithm. A brief remark on whether the algorithm converges to a stationary point would strengthen the analysis.
  8. References: Several arXiv preprints are cited (e.g., [5], [13], [21]) without publication status. Where peer-reviewed versions are available, they should be cited instead.

Simulated Author's Rebuttal

3 responses · 2 unresolved

We thank the referee for the careful and constructive review. The referee raises three substantive points: (1) the STE formulation yields a trivial boundary solution for K_m, undermining the claimed interior optimum; (2) the accuracy drops on CUB-200-2011 are too large to be called 'moderate'; and (3) the use of frozen pre-trained [CLS] attention may be inadequate for fine-grained tasks. We address each below.

read point-by-point responses

  1. Referee: Section VI.C, Eq. (42)-(43): SUBP3 maximizes f_m(K_m) subject to upper bounds. Since f_m is monotonically increasing (Lemma 1), K*_m = K^max_m trivially. The STE metric's numerator is maximized by always selecting as many tokens as constraints allow. The optimization never trades off semantic retention against latency; the trade-off is enforced by external constraints (C5, C6), not by the STE objective. Fig. 6 claims a peak but the formulation cannot produce an interior optimum for K_m.

    Authors: The referee is correct that, as formulated, SUBP3 admits the trivial solution K*_m = K^max_m because f_m(K_m) is monotonically increasing. We acknowledge this as a genuine structural issue in the current formulation. The STE metric as defined in Eq. (20) does not by itself create an interior optimum for the token selection subproblem; the trade-off is entirely mediated by the latency and energy constraints (C5, C6, C7). Fig. 6 illustrates the STE curve as a function of K_m for fixed (W_m, p_m), and the peak arises because the denominator (bottleneck latency) grows with K_m while the numerator saturates — but this peak occurs at the boundary imposed by constraints, not at an unconstrained interior point of the STE objective. We will revise the manuscript to state this explicitly and honestly: the STE metric captures a system-level efficiency ratio, but the token selection trade-off is constraint-driven rather than objective-driven. We will also explore reformulating the objective to incorporate a latency penalty directly into the numerator (e.g., a weighted or regularized form) so that an interior optimum can emerge from the objective itself. At minimum, we will correct the characterization in Section V-A and Fig. 6 to accurately describe the mechanism by which the trade-off arises. revision: yes

  2. Referee: Table I, CUB-200-2011 rows: ST-SFLora substantially underperforms ST-SFLora-Full. ViT-L/16 Non-IID: 84.11% vs. 75.83% (8.3 pp drop). ViT-B/16 Non-IID: 79.91% vs. 73.69% (6.2 pp drop). These are not marginal for fine-grained classification. Section VII-B calls them 'moderate' and 'within a tolerable range.' The paper should reconcile the 'favorable trade-off' claim with these results, or revise the characterization. At minimum, discuss why CUB-200-2011 degrades more than ImageNet100 or Flowers-102 and whether [CLS] attention is adequate for fine-grained tasks.

    Authors: We agree that the characterization of the CUB-200-2011 degradation as 'moderate' and 'within a tolerable range' is not adequately supported by the data. The 6-8 percentage-point drops on CUB-200-2011 are substantial, particularly for fine-grained classification where subtle local features are critical. We will revise Section VII-B to remove the 'moderate' characterization and instead present the CUB results as a clear limitation of the current approach. We will also add a dedicated discussion of why CUB-200-2011 degrades more than ImageNet100 and Flowers-102. The key reason is that CUB-200-2011 requires discrimination based on subtle local patches (e.g., beak shape, plumage patterns), while the pre-trained [CLS] attention tends to focus on global object structure and salient regions that may not coincide with these fine-grained discriminative features. In contrast, ImageNet100 classes are often distinguishable from global structure, and Flowers-102 exhibits strong color and texture saliency that aligns well with pre-trained attention patterns. We will make this analysis explicit in the revised manuscript. revision: yes

  3. Referee: Section IV-B, Eq. (12): Token selection uses [CLS] attention from the frozen client-side backbone. Since the backbone is never updated, attention patterns reflect pre-trained saliency rather than task-adapted discriminative features. For fine-grained datasets like CUB-200-2011, discriminative features are subtle local patches. Pre-trained [CLS] attention focused on global structure may not prioritize correct regions. The paper should empirically validate correlation between selected tokens and class-discriminative regions, or acknowledge this as a fundamental limitation.

    Authors: The referee's observation is correct and well-taken. The frozen client-side backbone produces attention patterns that reflect pre-trained saliency rather than task-adapted discriminative features. This is a fundamental limitation of the current design, and the severity of the accuracy drops on CUB-200-2011 directly evidences it. We do not currently have a correlation analysis between selected tokens and ground-truth class-discriminative regions (e.g., Grad-CAM or bounding-box-based evaluation), and we acknowledge that such an analysis would strengthen the paper. In the revision, we will: (1) explicitly acknowledge this as a fundamental limitation in both the experimental discussion and the conclusion; (2) add qualitative analysis using Grad-CAM visualizations to illustrate where pre-trained [CLS] attention focuses versus where task-discriminative regions actually lie for CUB-200-2011 samples; and (3) strengthen the future-work discussion on learnable or task-adaptive token selection strategies, noting that the CUB results specifically motivate this direction. We agree that without such validation or acknowledgment, the paper overstates the adequacy of the pre-trained [CLS] attention proxy. revision: partial

standing simulated objections not resolved
  • The STE formulation as written does not produce an interior optimum for K_m from the objective alone; the trade-off is constraint-driven. A full reformulation that embeds the latency trade-off into the objective (rather than relying on constraints) would require redesigning the optimization framework, which we can begin but may not fully resolve within a single revision cycle.
  • We do not currently have a quantitative correlation analysis between pre-trained [CLS] attention maps and ground-truth class-discriminative regions for CUB-200-2011. We can add Grad-CAM-based qualitative analysis, but a rigorous quantitative validation study is beyond what we can complete in the revision timeframe.

Circularity Check

1 steps flagged

Mild self-referential structure in the STE metric and token selection, but the joint resource optimization has independent content; self-citations are not load-bearing for the mathematical derivation.

specific steps
  1. self definitional [Section V-A (Eq. 19-20) and Section IV-B (Eq. 13)]
    "f_m(K_m) = Σ_{n=1}^{K_m} ᾱ_{m,n} ... E = Σ_{m∈U} f_m(K_m) / max_{m∈U} {T^U_m} ... A^sel_m = {a^{(b)}_{m,n} | n ∈ Top-K_m(α^{(b)}_{m,1},...,α^{(b)}_{m,N})}"

    The STE metric's numerator f_m(K_m) is defined as the cumulative sum of the top-K attention scores. The token selection strategy (Eq. 13) selects tokens by exactly the same attention scores. Therefore, for any fixed K, the top-K selection is optimal for the STE metric by construction — the metric is defined in terms of the selection criterion. However, this circularity is partial and mild: the optimization problem P0 jointly optimizes K (how many tokens), bandwidth W, and power p, and the solutions for power (bisection, Theorem 1) and bandwidth (nested bisection, Algorithm 3) are genuine non-trivial results. The token-selection subproblem SUBP3 reduces to the trivial solution K*_m = K^max_m (Eq. 43) because f_m is monotonically increasing, which the paper openly states. The 'semantic-aware

full rationale

The paper's core mathematical derivation is largely self-contained. Lemma 1 (concavity of f_m), Theorem 1 (monotonicity of energy in power), and the bisection-based solutions for power and bandwidth subproblems are all proven within the paper without relying on external or self-cited results. The self-citations (refs [12], [13], [20], [32]) appear in the introduction and related work for context, not as load-bearing mathematical premises. The mild circularity is that the STE metric (Eq. 20) uses cumulative attention scores as 'semantic information,' and the token selection (Eq. 13) selects tokens by those same scores, making the selection strategy optimal for the metric by definition. However, this only affects the token-selection subproblem (SUBP3), which the paper itself shows has the trivial solution K* = K^max (Eq. 43). The joint optimization of bandwidth and power — the paper's main technical contribution — has independent, non-circular content. The accuracy results (Table I) are empirically measured against external benchmarks, not derived from the STE metric. Score 2 reflects the mild self-referential structure without it undermining the central claims.

Axiom & Free-Parameter Ledger

5 free parameters · 3 axioms · 1 invented entities

The axiom ledger reveals that the paper relies on the unverified assumption that [CLS] attention scores are a good proxy for task relevance in the split learning setting. The STE metric is an invented entity that serves as the optimization target, but its correlation with actual model accuracy is only empirically shown to be weak (accuracy drops with token selection).

free parameters (5)
  • K_min
    Minimum number of tokens to select, a system parameter constraint.
  • E_max
    Maximum energy consumption constraint per client.
  • p_max = 0.2 W
    Maximum transmit power constraint.
  • W_tot = 50 MHz
    Total system uplink bandwidth.
  • LoRA rank = 16
    Hyperparameter for the LoRA adapters.
axioms (3)
  • domain assumption The [CLS] token's attention scores reliably indicate task-relevant semantic importance of patch tokens.
    Section IV-B: The paper assumes that attention weights from the [CLS] token are a 'reliable proxy for task relevance' without proving that early-layer attention correlates with final task performance, especially when the client-side model is frozen.
  • domain assumption Control signaling and optimization computation latency are negligible.
    Section III-B: The paper assumes that the latency for client selection, CSI collection, and the joint optimization algorithm on the server is negligible compared to training and activation transmission.
  • domain assumption Client velocity remains constant during one communication round.
    Section IV-A: Used to calculate standing time and mobility-aware client selection.
invented entities (1)
  • Semantic Transmission Efficiency (STE) no independent evidence
    purpose: A metric to quantify the trade-off between semantic information retention and communication latency.
    STE is defined as the ratio of summed attention scores to max latency. It is an invented system-level metric used to formulate the optimization objective. Its validity as a measure of 'semantic throughput' is assumed, not independently verified against actual task performance.

pith-pipeline@v1.1.0-glm · 26047 in / 2564 out tokens · 208630 ms · 2026-07-05T07:38:37.129650+00:00 · methodology

0 comments
read the original abstract

Deploying large Transformer-based vision models on resource-limited mobile devices at network edge is severely constrained by hardware limitations and dynamic wireless environments. While federated learning (FL) enables collaborative training without sharing raw data, strictly local fine-tuning of such massive models remains computationally prohibitive for edge devices. Split federated learning (SFL) alleviates this burden by offloading deep layers to the edge server, yet it suffers from heavy communication overhead when transmitting high-dimensional activation tokens. To address this bottleneck, we propose ST-SFLora, a semantic token-based split federated LoRA fine-tuning framework. We introduce a new metric, \emph{Semantic Transmission Efficiency} (STE), to balance semantic retention and transmission cost. Based on STE, we formulate a joint resource optimization problem that dynamically determines token selection, uplink bandwidth allocation, and transmit power under latency and energy constraints. The resulting mixed-integer nonconvex problem is efficiently solved via an alternating algorithm. Experiments on multiple benchmarks demonstrate that ST-SFLora achieves the lowest client-side resource consumption among baselines while delivering a favorable trade-off between communication efficiency and model performance.

Figures

Figures reproduced from arXiv: 2605.26120 by Geyong Min, Xianke Qiang, Zheng Chang.

Figure 1
Figure 1. Figure 1: Attention-based semantic-aware token selection for [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of Distributed Learning Architectures. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The workflow of the ST-SFLora system. have not exploited the semantic structure of Transformer activations—whose representation is naturally tokenized and semantically structured—for selective activation transmission or communication optimization under mobility-induced channel variation. Existing SL and SFL frameworks struggle to support efficient fine-tuning of large Transformer models on mobile edge devi… view at source ↗
Figure 4
Figure 4. Figure 4: The timeline of ST-SFLora. energy consumption of this phase are considered negligible and are ignored in the subsequent analysis. 4) Optimization: Utilizing the received CSI and the batch￾level token importance vectors, the server performs joint optimization of the token budget 𝑲, uplink transmit power 𝒑, and bandwidth allocation 𝑾. Upon solving the optimization prob￾lem, the server distributes the optimal… view at source ↗
Figure 5
Figure 5. Figure 5: The cumulative semantic information retention function [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: STE versus selected token number 𝐾𝑚, illustrating the existence of an optimal operating point. effective semantic information delivery. A higher value of E indicates that the system achieves greater learning contribu￾tion under lower latency, thereby facilitating more efficient distributed training. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Relationship between 𝜙 and 𝜅 in the bisection algorithm. where𝜓(𝑥) = ln(1+𝑥)− 𝑥 1+𝑥 for 𝑥 > 0. Since𝜓 ′ (𝑥) = 𝑥 (1+𝑥) 2 > 0 and 𝜓(0) = 0, we have 𝜓(𝑥) > 0 for 𝑥 > 0. Thus 𝐺˜ ′ (𝑝) > 0 for 𝑝 > 0, proving that 𝐸 U 𝑚(𝐾𝑚, 𝑊𝑚, 𝑝𝑚) = 𝜒 𝐺˜ (𝑝𝑚) is strictly increasing on (0, ∞). This completes the proof. □ Since 𝑇 U 𝑚 (𝑝𝑚) is strictly decreasing and 𝐸 U 𝑚(𝑝𝑚) is strictly increasing with respect to 𝑝𝑚, the optimal … view at source ↗
Figure 8
Figure 8. Figure 8: Optimization Algorithm Performance [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualization of proposed tokens selection strategy. [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Testing Accuracy on different datasets with varing selected token number. [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 35 canonical work pages · 5 internal anchors

  1. [1]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” inPro- ceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), 2019, pp. 4171–4186

    work page 2019

  2. [2]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

    A. Dosovitskiy, “An image is worth 16x16 words: Transformers for image recognition at scale,” 2020,arXiv:2010.11929

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  3. [3]

    GPT-4 Technical Report

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “Gpt-4 technical report,” 2023,arXiv:2303.08774

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  4. [4]

    Federated fine-tuning of large language models under heterogeneous tasks and client resources,

    J. Bai, D. Chen, B. Qian, L. Yao, and Y. Li, “Federated fine-tuning of large language models under heterogeneous tasks and client resources,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 37, Vancouver, Canada, Dec. 2024, pp. 14 457–14 483. 13 (a) ImagNet100 (b) Oxford Flowers-102 (c) CUB-200-2011 Fig. 10: Testing Accuracy on different datasets ...

    work page 2024

  5. [5]

    SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models

    Z. Lin, X. Hu, Y. Zhang, Z. Chen, Z. Fang, X. Chen, A. Li, P. Vepakomma, and Y. Gao, “Splitlora: A split parameter-efficient fine-tuning framework for large language models,” 2024,arXiv:2407.00952

    work page Pith review arXiv 2024

  6. [6]

    Mobile edge intelligence for large language models: A contemporary survey,

    G. Qu, Q. Chen, W. Wei, Z. Lin, X. Chen, and K. Huang, “Mobile edge intelligence for large language models: A contemporary survey,” IEEE Commun. Surv. Tutorials, pp. 1–1, early access, Jan. 09 2025, doi:10.1109/COMST.2025.3527641

    work page doi:10.1109/comst.2025.3527641 2025

  7. [7]

    Efficient federated learning for modern nlp,

    D. Cai, Y. Wu, S. Wang, F. X. Lin, and M. Xu, “Efficient federated learning for modern nlp,” inProc. 29th Annu. Int. Conf. Mobile Comput. Netw. (MobiCom ’23), Madrid, Spain, Oct. 2023, pp. 1–16

    work page 2023

  8. [8]

    Communication-efficient learning of deep networks from decentralized data,

    H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Ag¨ uera y Arcas, “Communication-efficient learning of deep networks from decentralized data,” inProc. 20th Int. Conf. Artif. Intell. Stat. (AISTATS), vol. 54. Fort Lauderdale, FL, USA: PMLR, Apr. 2017, pp. 1273–1282

    work page 2017

  9. [9]

    Mobilora: Accelerating lora-based llm inference on mobile devices via context-aware kv cache optimization,

    B. Li, Y. Wang, H. Ma, L. Chen, J. Xiao, and S. Wang, “Mobilora: Accelerating lora-based llm inference on mobile devices via context-aware kv cache optimization,” inProc. 63rd Annu. Meeting Assoc. Comput. Linguist. (ACL) (Vol. 1: Long Papers), Vienna, Austria, Jul. 2025, pp. 23 400–23 410

    work page 2025

  10. [10]

    Split learning for health: Distributed deep learning without sharing raw patient data

    P. Vepakomma, O. Gupta, T. Swedish, and R. Raskar, “Split learning for health: Distributed deep learning without sharing raw patient data,” 2018, arXiv:1812.00564

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  11. [11]

    Splitfed: When federated learning meets split learning,

    C. Thapa, P. C. Mahawaga Arachchige, S. Camtepe, and L. Sun, “Splitfed: When federated learning meets split learning,” inProc. AAAI Conf. Artif. Intell. (AAAI), vol. 36, no. 8, Vancouver, BC, Canada, Jun. 2022, pp. 8485–8493

    work page 2022

  12. [12]

    Split federated learning empowered vehicular edge intelligence: Concept, adaptive de- sign, and future directions,

    X. Qiang, Z. Chang, C. Ye, T. H ¨am¨al¨ainen, and G. Min, “Split federated learning empowered vehicular edge intelligence: Concept, adaptive de- sign, and future directions,”IEEE Wireless Commun., vol. 32, no. 4, pp. 90–97, 2025

    work page 2025

  13. [13]

    Deploying Large AI Models on Resource-Limited Devices with Split Federated Learning

    X. Qiang, H. Liu, X. Zhang, Z. Chang, and Y.-C. Liang, “Deploying large ai models on resource-limited devices with split federated learning,” 2025, arXiv:2504.09114

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  14. [14]

    Quantized federated learning under transmission delay and outage constraints,

    Y. Wang, Y. Xu, Q. Shi, and T.-H. Chang, “Quantized federated learning under transmission delay and outage constraints,”IEEE J. Sel. Areas Commun., vol. 40, no. 1, pp. 323–341, Nov. 2022

    work page 2022

  15. [15]

    Communication-efficient federated learning: A variance-reduced stochastic approach with adaptive sparsification,

    B. Wang, J. Fang, H. Li, and B. Zeng, “Communication-efficient federated learning: A variance-reduced stochastic approach with adaptive sparsification,”IEEE Trans. Signal Process., vol. 71, pp. 3562–3576, Sep. 2023

    work page 2023

  16. [16]

    On the road to 6G: Visions, requirements, key technologies, and testbeds,

    C.-X. Wang, X. You, X. Gao, X. Zhu, Z. Li, C. Zhang, H. Wang, Y. Huang, Y. Chen, H. Haas, J. S. Thompson, E. G. Larsson, M. D. Renzo, W. Tong, P. Zhu, X. Shen, H. V. Poor, and L. Hanzo, “On the road to 6G: Visions, requirements, key technologies, and testbeds,”IEEE Commun. Surveys Tuts., vol. 25, no. 2, pp. 905–974, Feb. 2023

    work page 2023

  17. [17]

    Not all tokens are equal: Human-centric visual analysis via token clustering transformer,

    W. Zeng, S. Jin, W. Liu, C. Qian, P. Luo, W. Ouyang, and X. Wang, “Not all tokens are equal: Human-centric visual analysis via token clustering transformer,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), New Orleans, LA, USA, Jun. 2022, pp. 11 101–11 111

    work page 2022

  18. [18]

    Wireless distributed learning: A new hybrid split and federated learning approach,

    X. Liu, Y. Deng, and T. Mahmoodi, “Wireless distributed learning: A new hybrid split and federated learning approach,”IEEE Trans. Wireless Commun., vol. 22, no. 4, pp. 2650–2665, Oct. 2023

    work page 2023

  19. [19]

    Split learning over wireless networks: Parallel design and resource management,

    W. Wu, M. Li, K. Qu, C. Zhou, X. Shen, W. Zhuang, X. Li, and W. Shi, “Split learning over wireless networks: Parallel design and resource management,”IEEE J. Sel. Areas Commun., vol. 41, no. 4, pp. 1051– 1066, Feb. 2023

    work page 2023

  20. [20]

    Adaptive and parallel split federated learning in vehicular edge computing,

    X. Qiang, Z. Chang, Y. Hu, L. Liu, and T. H ¨am¨al¨ainen, “Adaptive and parallel split federated learning in vehicular edge computing,”IEEE Internet Things J., vol. 12, no. 5, pp. 4591–4604, Mar. 2025

    work page 2025

  21. [21]

    Splitfrozen: Split learning with device-side model frozen for fine-tuning llm on heterogeneous resource-constrained devices,

    J. Ma, X. Lyu, J. Jiang, Q. Cui, H. Yao, and X. Tao, “Splitfrozen: Split learning with device-side model frozen for fine-tuning llm on heterogeneous resource-constrained devices,” 2025,arXiv:2503.18986

    work page arXiv 2025

  22. [22]

    Design and analysis of uplink and downlink communications for federated learning,

    S. Zheng, C. Shen, and X. Chen, “Design and analysis of uplink and downlink communications for federated learning,”IEEE J. Sel. Areas Commun. s, vol. 39, no. 7, pp. 2150–2167, Jul 2021

    work page 2021

  23. [23]

    Communication-efficient federated learning for heterogeneous edge devices based on adaptive gradient quantization,

    H. Liu, F. He, and G. Cao, “Communication-efficient federated learning for heterogeneous edge devices based on adaptive gradient quantization,” inProc. IEEE Conf. Comput. Commun. (INFOCOM). New York City, NY, USA: IEEE, May 2023, pp. 1–10

    work page 2023

  24. [24]

    Service delay minimization for federated learning over mobile devices,

    R. Chen, D. Shi, X. Qin, D. Liu, M. Pan, and S. Cui, “Service delay minimization for federated learning over mobile devices,”IEEE J. Sel. Areas Commun., vol. 41, no. 4, pp. 990–1006, Feb. 2023

    work page 2023

  25. [25]

    Fed-cvlc: Compressing federated learning communications with variable-length codes,

    X. Su, Y. Zhou, L. Cui, J. C. S. Lui, and J. Liu, “Fed-cvlc: Compressing federated learning communications with variable-length codes,” inProc. IEEE Conf. Comput. Commun. (INFOCOM), Vancouver, Canada, May 2024, pp. 601–610

    work page 2024

  26. [26]

    Joint accuracy and latency optimization for quantized federated learning in vehicular networks,

    X. Zhang, W. Chen, H. Zhao, Z. Chang, and Z. Han, “Joint accuracy and latency optimization for quantized federated learning in vehicular networks,”IEEE Internet Things J., vol. 11, no. 17, pp. 28 876–28 890, Sept. 2024

    work page 2024

  27. [27]

    Joint gradient sparsification and device scheduling for federated learning,

    X. Lin, Y. Liu, F. Chen, X. Ge, and Y. Huang, “Joint gradient sparsification and device scheduling for federated learning,”IEEE Trans. Green Commun. Networking, vol. 7, no. 3, pp. 1407–1419, Jan. 2023

    work page 2023

  28. [28]

    Federated split learning with model pruning and gradient quantization in wireless networks,

    J. Zhang, W. Ni, and D. Wang, “Federated split learning with model pruning and gradient quantization in wireless networks,”IEEE Trans. Veh. Technol., vol. 74, no. 4, pp. 6850–6855, 2025

    work page 2025

  29. [29]

    Reducing Communication for Split Learning by Randomized Top-k Sparsification

    F. Zheng, C. Chen, L. Lyu, and B. Yao, “Reducing communication for split learning by randomized top-k sparsification,”arXiv:2305.18469, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  30. [30]

    Adaptive resource allocation for semantic communication networks,

    L. Wang, W. Wu, F. Zhou, Z. Yang, Z. Qin, and Q. Wu, “Adaptive resource allocation for semantic communication networks,”IEEE Trans. Commun., vol. 72, no. 11, pp. 6900–6916, May. 2024

    work page 2024

  31. [31]

    Energy-efficient federated edge learning with streaming data: A lyapunov optimization approach,

    C.-H. Hu, Z. Chen, and E. G. Larsson, “Energy-efficient federated edge learning with streaming data: A lyapunov optimization approach,”IEEE Transactions on Communications, vol. 73, no. 2, pp. 1142–1156, Feb. 2025

    work page 2025

  32. [32]

    Aigc-assisted federated learning for vehicular edge intelligence: Vehicle selection, resource allocation and model augmentation,

    X. Qiang, Z. Chang, and G. Min, “Aigc-assisted federated learning for vehicular edge intelligence: Vehicle selection, resource allocation and model augmentation,”IEEE Trans. Mob. Comput., vol. 24, no. 11, pp. 11 896–11 909, Jun. 2025

    work page 2025

  33. [33]

    Imagenet large scale visual recognition challenge,

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “Imagenet large scale visual recognition challenge,”Int. J. Comput. Vis., vol. 115, no. 3, pp. 211–252, Dec. 2015

    work page 2015

  34. [34]

    Automated flower classification over a large number of classes,

    M.-E. Nilsback and A. Zisserman, “Automated flower classification over a large number of classes,” inProc. 6th Indian Conf. Comput. Vis., Graph. Image Process. (ICVGIP). Bhubaneswar, India: IEEE, Dec. 2008, pp. 722–729

    work page 2008

  35. [35]

    The caltech-ucsd birds-200-2011 dataset,

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The caltech-ucsd birds-200-2011 dataset,” California Institute of Technology, Pasadena, CA, USA, Tech. Rep. CNS-TR-2011-001, 2011

    work page 2011