Pith. sign in

REVIEW 4 major objections 5 minor 43 references

ACME: Adaptive Customization of Large Models via Distributed Systems

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ACME's central claim is that a cloud–edge–device pipeline can customize Transformer models per device, cutting data transmission to 6% while improving trade-off metrics by nearly 30%.

desk verdict Plausible system design for distributed model customization, but the headline 6% transmission claim is not supported by the paper's own accounting. read the letter →

arxiv 2507.14802 v1 pith:XMJKWP3Q submitted 2025-07-20 cs.DC cs.AI

Ziming Dai , Chao Qiu , Fei Gao , Yunfeng Zhao , Xiaofei Wang This is my paper · ORCID
classification cs.DCcs.AI
keywords distributedmodelcustomizationTransformeredgecomputingneuralarchitecturesearchParetofrontpersonalizedaggregationWassersteindistancedeployment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ACME is a distributed system that customizes pre-trained Transformer models for individual edge devices by splitting the work across a cloud server, edge servers, and the devices themselves. The paper's central claim is that this division makes model customization cost-efficient: compared with centralized customization, it reduces data transmission to about 6%, raises average accuracy by about 10% over baseline lightweight models, and improves a combined accuracy–energy–size trade-off metric by nearly 30%. The system first matches a backbone architecture to each device cluster's storage and compute limits using a Pareto-front search, then designs and refines a task-specific header using neural architecture search and data-distribution-aware aggregation. If correct, ACME would let privacy-sensitive applications deploy large models on heterogeneous local hardware without uploading local data.

What carries the argument

The load-bearing machinery is the bidirectional single-loop distributed system: a cloud layer exchanges statistical parameters with edge servers to generate and select backbone architectures, and each edge server runs an iterative loop with its devices to refine header architectures. Within that system, three components carry the argument: (1) a grid-based Pareto-front construction (borrowed from constrained decomposition) that maps each device cluster's storage and energy limits to a small set of candidate width–depth pairs; (2) a progressive neural architecture search with an LSTM controller that generates a coarse header architecture for each backbone; and (3) an importance-set mechanism that quantifies each header parameter's contribution via a first-order Taylor expansion and aggregates importance sets across devices with weights derived from Wasserstein distances between local feature distributions.

What would settle it

Measure, on the same GPU type used in the evaluation, the actual power draw and per-epoch latency of ViT backbones across a grid of widths and depths, and compare these measurements with Eq. (1)–(2). If the measured values do not follow the linear scaling assumed there, then the Pareto-front selections made by ACME and the nearly 30% trade-off improvement, both computed from that model, would need to be re-derived.

Watch

Extended reading notes

Core claim

ACME's core claim is that a Transformer model can be customized for a heterogeneous device fleet without paying the usual communication and search costs of centralized customization. The paper argues that by treating the backbone (the layer stack that extracts general features) and the header (the task-specific output network) as separately optimizable pieces, and by running their optimization at different levels of a cloud–edge–device hierarchy, one can hit a better accuracy–energy–size trade-off than any single centralized process. In experiments with ViT-B on CIFAR-100, ACME reports a 10% accuracy gain over lightweight baselines at similar model sizes, a 6% data-transmission volume relative to centralized systems, and a nearly 30% improvement in the combined trade-off metric.

Load-bearing premise

The cost-efficiency and Pareto-front claims rest on the assumption that a model's power consumption and per-epoch latency scale linearly with its number of Transformer layers and its width–depth product; if real hardware follows a different scaling law, the models ACME selects may not be the most cost-efficient and the reported trade-off improvements would need to be recomputed.

Editorial extensions

If this is right

  • Privacy-sensitive edge deployments become feasible because raw local data never leaves devices, so models can be customized on data that cannot be uploaded to the cloud.
  • Communication savings scale with the number of clusters: the 6% data-transmission figure follows from uploading only statistical parameters and importance sets rather than local datasets.
  • After the Pareto-front grid is built, selecting a model for a device cluster is fast, cutting selection latency by about 71% compared with greedy methods.
  • The method transfers to other datasets and Transformer variants: on Stanford Cars, ACME reports an average accuracy improvement of 3.94% under the same storage constraints, and 14.43% on header redesign.
  • The two-phase decomposition lets simple backbones pair with complex headers and vice versa, improving accuracy on both small and large backbone architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the reported trade-off gains are computed with the same approximate energy model used to select the Pareto front, a direct measurement of GPU power and latency across the width–depth grid could show whether the ~30% improvement is preserved under real hardware scaling.
  • The communication-saving design could be reused outside Transformer customization: any federated system that can summarize local knowledge as importance vectors or gradients rather than data might achieve similar reductions.
  • The Wasserstein similarity step assumes that a pre-trained feature extractor makes local distributions comparable; testing ACME with different feature extractors or distribution distances would reveal whether this assumption is essential.
  • The efficiency gains may depend on the fraction of data each edge server holds as a shared dataset (10–20%); ablating that fraction could show a privacy–performance trade-off not explored in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. ACME proposes a hierarchical cloud-edge-device framework for customizing Transformer-based large models (instantiated with ViT) under heterogeneous device constraints. The method decomposes customization into two phases: (1) backbone generation and Pareto-front selection under model size and energy constraints (Phase 1, Eqs. (4), (10)-(13)), and (2) header customization via edge-side NAS followed by device-side importance-set refinement with Wasserstein-distance-based personalized aggregation (Phase 2, Eqs. (14)-(21)). The paper reports on CIFAR-100 and Stanford Cars that ACME reduces data transmission to 6% of a centralized system, improves average accuracy by about 10%, and improves a composite trade-off metric by nearly 30%.

Significance. If the quantitative claims were supported, ACME would be a practically useful contribution: it targets a real deployment problem, combines backbone and header customization in a distributed setting, and explicitly addresses device and data heterogeneity. The strengths are the clear decomposition of the optimization problem and the inclusion of two datasets and several lightweight-ViT baselines. However, the headline numbers currently rest on an unvalidated energy model, an incomplete communication-cost accounting, and an evaluation without error bars or the most closely related federated-NAS baselines. These issues are load-bearing for the central claims, so the current evidence is not yet sufficient to establish the paper's main conclusions.

major comments (4)
  1. [§IV.B.3 and Eq. (1)-(2)] The 'Trade-off Score' used to claim a 28.9% improvement is defined as Ln(θn, Dn) + En(θn) + ζ(θn), where En(θn) is exactly the paper's own approximate energy model from Eqs. (1)-(2). The same En is used as an optimization objective in P1 and Eq. (10). Therefore the reported trade-off improvement is largely a self-consistency result: the optimizer selects models that score well on the very metric being reported. To support the cost-efficiency claim, the energy model should be validated against hardware measurements (or at least against a held-out measured energy/latency dataset), and the trade-off comparison should be recomputed with measured rather than modeled energy.
  2. [§IV.B.1, Table I, and §IV.A] The claim that ACME reduces data transmission volume to 6% of a centralized system is not supported by the reported accounting. Section IV.A states that each edge server stores 10%-20% of its devices' data as a shared dataset, and Phase 2-1 uses this data at the edge. Phase 2-2 additionally requires each device to provide a sampled subset \tilde{D}_i for Wasserstein-distance computation, and Algorithm 2 has every device upload its importance set Q_n at every iteration for T iterations and download Q'_n. None of these transfers appear in Table I's 'Upload Data' column. If the shared subsets must first be uploaded from devices, the true ACME upload volume is at least roughly 10%-20% of each local dataset plus T importance-set exchanges, which would likely be far above 6%. If the shared subsets are assumed to already reside at the edge, then the statement in §III-D that ACME 'avoid[s] uploading local data' is contradicted and the privacy motivation is weakened. The 6% claim needs a complete communication budget that includes all of these transfers, with the assumptions stated explicitly.
  3. [§IV.B.2 and Fig. 7] The accuracy comparison omits the most relevant baselines from the paper's own related-work section. Federated NAS methods such as FedNAS and CFDNAS [42], [43] are cited in §V as existing approaches for model customization on heterogeneous devices, but they are not included in any experiment. Without these comparisons, the claim that ACME achieves better accuracy than existing distributed customization approaches is not established. The authors should either add these baselines (or a clear explanation of why they cannot be run) and report accuracy with multiple independent runs, standard deviations, or confidence intervals, since the current figures show single-point comparisons.
  4. [Eq. (2) and §IV.A] The energy model itself is an assumption that power and latency scale linearly with the number of Transformer layers and with w_B d_B, with constants proportional to base GPU power and latency. No empirical validation of this model is provided, and the experimental setup does not report any energy or latency measurements on the actual devices. Since this model underpins both the Pareto-front selection in Phase 1 and the reported trade-off improvements, its accuracy on real hardware is a correctness risk. A concrete test would be to measure energy and latency for a few selected architectures on the deployed VMs and compare them with Eq. (1)-(2) predictions.
minor comments (5)
  1. [§II.A and Eq. (3)] The notation is ambiguous: S denotes both the set of edge servers and its cardinality, and N denotes both the set of devices and its size; Eq. (3) uses 1/S and 1/|N_s| without making these distinctions explicit.
  2. [§III.B.1] The phrase 'importance of headers' in the text should read 'importance of heads'; the paper elsewhere uses 'header' for the classification module, so the terminology should be consistent.
  3. [Algorithm 1] The formula for K on line 6 references f^l_s(\tilde{\theta}^*_s) and f^l_s(\tilde{\theta}^-_s), but these ideal and worst-case points are not explicitly defined before this line; their definitions should be stated.
  4. [§IV.B.3] The Trade-off Score is defined as a sum of loss, energy, and model size, so lower values are presumably better, but the text says 'improves the final Trade-off Score by at least 28.9%' without specifying the direction; please state whether lower is better and clarify the sign convention.
  5. [§IV.A] The paper reports 'vCPUs configured from 3 to 7' and storage capacities in MB, but it does not specify the GPU configuration of the edge servers and devices; since energy and latency are central to the method, the hardware setup should be described in more detail.

Circularity Check

1 steps flagged · score 6.0 of 10

Trade-off metric reuses the optimization objective, so the 28.9% improvement is partly self-consistency; accuracy results are external.

  1. self definitional [Section III.B.2 (Eq. 10) and Section IV.B.3 (Trade-off Score definition); energy model Section II.B (Eqs. 1-2)]
    "Lastly, we define the Trade-off Score as Ln(θn, Dn) + En(θn) + ζ(θn) to assess the overall cost-efficiency of the models [40]. ... fs(˜θs) = h Ls(˜θs, ˜Dc), Es(˜θs), ζ(˜θs) i ... Es(˜θs) = max n∈Ns En(˜θs)."

    The headline trade-off improvement ('improves the final Trade-off Score by at least 28.9%') is measured with Ln+En+ζ, while Phase-1's objective in Eq. (10) is exactly the vector [Ls, Es, ζ], with Es = max_n En. The same approximate energy model En from Eqs. (1)-(2) and the same size term ζ enter both the backbone-selection rule and the evaluation metric. Consequently, a method that explicitly optimizes E and ζ during selection is almost guaranteed to score better on L+E+ζ than baselines that ignore energy and size; the reported gain is largely a self-consistency result, not independent evidence that the approximate energy model reflects real hardware. The independent portion of the evaluation is the accuracy comparison against external lightweight-ViT baselines.

full rationale

The core derivation chain of ACME - backbone importance pruning, knowledge distillation, grid-based Pareto-front construction, header NAS, and Wasserstein-based importance aggregation - is not circular: each step is defined from external components and is checked against external baselines. The accuracy improvements (~10% in Fig. 7a and ~4% in Fig. 7b) are measured against published lightweight ViT models (Efficient-ViT, MobileViT, Twins-SVT, DeViT family), so those results are independent evidence. The self-citations [13], [14], and [24] appear only in system-model definitions and LSTM controller configuration; they are not load-bearing uniqueness claims and do not import an unverified theorem. The principal circularity is in the trade-off metric: the Trade-off Score Ln+En+ζ is the same objective vector that P1/Eq. (10) minimizes, and Eq. (13) selects the backbone closest to the ideal point in that same grid, so a large share of the reported 28.9% trade-off improvement is self-consistency rather than external validation. The energy model in Eqs. (1)-(2) is an unvalidated approximation, and because it is used on both the optimization and evaluation sides, any error in its constants affects the trade-off comparison in a direction that favors the method. The 6% data-transmission claim is a separate completeness concern: Section IV.A says each edge server stores 10-20% of device data as a shared dataset, and Algorithm 2 has every device upload its importance set Q_n each iteration; if these transfers are not counted in Table I's 'Upload Data' column, the 6% ratio is unsupported. This is a bookkeeping/correctness issue rather than circularity, so it does not raise the score further. Overall, the central accuracy results are externally grounded, but the headline trade-off metric reduces partly to its own objective, giving a partial circularity score of 6.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper rests on several unstated or underspecified modeling choices: the energy model is an unvalidated linear approximation, the importance-based pruning assumes first-order Taylor accuracy, the IID assumption is invoked explicitly, and multiple hyperparameters (γ_p, λ1, λ2, discarded-neuron count, B, U) are not reported. The method introduces no new physical entities.

free parameters (6)
  • Performance window gamma_p
    Controls the number of intervals K in the Pareto Front grid (Eq. 10-11) and therefore the granularity of the backbone selection; its value is not reported.
  • Distillation coefficients lambda1, lambda2
    Balance logit, embedding, and hidden-state distillation losses in Eq. (9); values are not provided.
  • Number of discarded neurons in Phase 2-2
    The device discards a 'preset number of neurons' based on the aggregated importance set; the preset count is not defined.
  • Header blocks B and repeats U
    The header search space uses B blocks repeated U times (Fig. 5); the paper studies their impact but does not state the values used in the main experiments.
  • Wasserstein sample size and exponent p
    The similarity matrix uses 'a tiny portion of data' sampled from each device and an L1 norm with exponent p in Eq. (19); the sample size and p are not specified.
  • Energy model coefficients (delta_G, G_beta, delta_L scaling)
    Eq. (2) assumes delta_G, G_beta scale with G_n and delta_L scales with L_n, but the actual coefficients are not specified; they determine the energy values used in both optimization and evaluation.
assumptions (5)
  • domain assumption The energy consumption of a Transformer is dominated by the backbone and scales linearly with layer count and width-depth product (Eq. 1-2).
    This is the foundation of the cost-efficiency objective in P1; it is taken from prior work [15] but not validated on the test hardware.
  • domain assumption The first-order Taylor expansion approximation for removing heads/neurons is valid (remainder term negligible), Eq. (7)-(8).
    This is a standard pruning assumption but the remainder can be significant when removing many components.
  • domain assumption Data samples on each device are independent and identically distributed for the purpose of importance estimation, as stated in Section III-D1.
    The paper explicitly invokes an IID assumption when deriving the importance error in Eq. (16), even though data across devices is non-IID.
  • standard math The constrained header search space with element-wise addition as the combination function preserves sufficient expressive power, following [23].
    This is a design constraint copied from the NAS literature to keep the search tractable.
  • domain assumption Wasserstein distance computed on a small random sample of features extracted by a pre-trained model is a good measure of data distribution similarity.
    This underpins the personalized aggregation matrix in Eq. (19)-(21); the paper does not analyze the sample size or the feature extractor choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ACME: Adaptive Customization of Large Models via Distributed Systems." pith.science (2026). https://pith.science/paper/XMJKWP3Q

@misc{pith2026250714802,
  author       = {Pith},
  title        = {Pith review of: ACME: Adaptive Customization of Large Models via Distributed Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XMJKWP3Q}},
  note         = {Machine review of arXiv:2507.14802}
}
read the original abstract

Pre-trained Transformer-based large models have revolutionized personal virtual assistants, but their deployment in cloud environments faces challenges related to data privacy and response latency. Deploying large models closer to the data and users has become a key research area to address these issues. However, applying these models directly often entails significant difficulties, such as model mismatching, resource constraints, and energy inefficiency. Automated design of customized models is necessary, but it faces three key challenges, namely, the high cost of centralized model customization, imbalanced performance from user heterogeneity, and suboptimal performance from data heterogeneity. In this paper, we propose ACME, an adaptive customization approach of Transformer-based large models via distributed systems. To avoid the low cost-efficiency of centralized methods, ACME employs a bidirectional single-loop distributed system to progressively achieve fine-grained collaborative model customization. In order to better match user heterogeneity, it begins by customizing the backbone generation and identifying the Pareto Front under model size constraints to ensure optimal resource utilization. Subsequently, it performs header generation and refines the model using data distribution-based personalized architecture aggregation to match data heterogeneity. Evaluation on different datasets shows that ACME achieves cost-efficient models under model size constraints. Compared to centralized systems, data transmission volume is reduced to 6 percent. Additionally, the average accuracy improves by 10 percent compared to the baseline, with the trade-off metrics increasing by nearly 30 percent.

Figures

Figures reproduced from arXiv: 2507.14802 by the authors.

Figure 1
Figure 1. Investigating the relationship among model size, fine-grained archi [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The challenges brought by traditional large model deployment. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The bidirectional single-loop distributed system structure. edge-device bidirectional single-loop interaction, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Overall description of the proposed ACME process. optimized backbone architecture within each device cluster managed by the edge server. III. THE DESIGN OF ACME A. ACME Overview This section provides an overview of the ACME process. As illustrated in [PITH_FULL_IMAGE:…
Figure 5
Figure 5. Figure 5: The search space and the architecture that might be searched. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Importance set generation and personalized architecture aggregation. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Performance comparison with baselines on CIFAR-100. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Accuracy of different headers applied to varying backbones. [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 12
Figure 12. Figure 12: Impact of block and layer count in header architecture. [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 38 canonical work pages

  1. [42]

    Resource-aware federated neural architecture search over heteroge- neous mobile devices,

    J. Yuan, M. Xu, Y . Zhao, K. Bian, G. Huang, X. Liu, and S. Wang, “Resource-aware federated neural architecture search over heteroge- neous mobile devices,” IEEE Trans. Big Data , 2022

    work page 2022

  2. [43]

    Toward tailored models on private aiot devices: Federated direct neural architecture search,

    C. Zhang, X. Yuan, Q. Zhang, G. Zhu, L. Cheng, and N. Zhang, “Toward tailored models on private aiot devices: Federated direct neural architecture search,” IEEE Internet Things J., vol. 9, no. 18, pp. 17 309– 17 322, 2022

    work page 2022

  3. [1]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in ICLR, 2020

    work page 2020

  4. [2]

    BERT: Pre- training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- training of deep bidirectional transformers for language understanding,” in NAACL-HLT, 2019, pp. 4171–4186

    work page 2019

  5. [3]

    Recent advances in natural language processing via large pre-trained language models: A survey,

    B. Min, H. Ross, E. Sulem, A. P. B. Veyseh, T. H. Nguyen, O. Sainz, E. Agirre, I. Heintz, and D. Roth, “Recent advances in natural language processing via large pre-trained language models: A survey,” ACM Comput. Surv., vol. 56, no. 2, pp. 1–40, 2023

    2023

  6. [4]

    Large language models and future of information retrieval: Opportunities and challenges,

    C. Zhai, “Large language models and future of information retrieval: Opportunities and challenges,” in ACM SIGIR, 2024, pp. 481–490

    work page 2024

  7. [5]

    Edgeshard: Efficient llm inference via collaborative edge computing,

    M. Zhang, J. Cao, X. Shen, and Z. Cui, “Edgeshard: Efficient llm inference via collaborative edge computing,” arXiv preprint arXiv:2405.14371, 2024

    arXiv 2024

  8. [6]

    MobileLLM: Optimizing sub-billion parameter language models for on-device use cases,

    Z. Liu, C. Zhao, F. Iandola, C. Lai, Y . Tian, I. Fedorov, Y . Xiong, E. Chang, Y . Shi, R. Krishnamoorthi et al. , “MobileLLM: Optimizing sub-billion parameter language models for on-device use cases,” in ICML, 2024, pp. 32 431–32 454

    work page 2024

Show all 43 references
  1. [7]

    To talk or to work: Flexible communication compression for energy efficient federated learning over heterogeneous mobile edge devices,

    L. Li, D. Shi, R. Hou, H. Li, M. Pan, and Z. Han, “To talk or to work: Flexible communication compression for energy efficient federated learning over heterogeneous mobile edge devices,” in IEEE INFOCOM, 2021, pp. 1–10

    2021

  2. [8]

    Dependency-aware microservice deployment for edge computing: A deep reinforcement learning approach with network representation,

    C. Wang, H. Yu et al., “Dependency-aware microservice deployment for edge computing: A deep reinforcement learning approach with network representation,” IEEE Trans. Mob. Comput. , 2024

    2024

  3. [9]

    Finch: Enhancing federated learning with hierarchical neural architecture search,

    J. Liu, J. Yan, H. Xu, Z. Wang, J. Huang, and Y . Xu, “Finch: Enhancing federated learning with hierarchical neural architecture search,” IEEE Trans. Mob. Comput., 2023

    2023

  4. [10]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” 2009

    2009

  5. [11]

    Distributed pruning towards tiny neural networks in federated learning,

    H. Huang, L. Zhang, C. Sun, R. Fang, X. Yuan, and D. Wu, “Distributed pruning towards tiny neural networks in federated learning,” in IEEE ICDCS, 2023, pp. 190–201

    2023

  6. [12]

    Scalable federated learning with system heterogeneity,

    F. Ilhan, G. Su, Q. Wang, and L. Liu, “Scalable federated learning with system heterogeneity,” in IEEE ICDCS, 2023, pp. 1037–1040

    2023

  7. [13]

    Cur- CoEdge: Curiosity-driven collaborative request scheduling in edge-cloud systems,

    Y . Zhao, C. Qiu, X. Shi, X. Wang, D. Niyato, and V . C. Leung, “Cur- CoEdge: Curiosity-driven collaborative request scheduling in edge-cloud systems,” in IEEE INFOCOM, 2024

    2024

  8. [14]

    MG²FL: Multi- granularity grouping-based federated learning in green edge computing systems,

    Z. Dai, Y . Zhao, C. Qiu, X. Wang, and F. R. Yu, “MG²FL: Multi- granularity grouping-based federated learning in green edge computing systems,” in IEEE GLOBECOM, 2023, pp. 152–157

    2023

  9. [15]

    Energy-efficient inference ser- vice of transformer-based deep learning models on gpus,

    Y . Wang, Q. Wang, and X. Chu, “Energy-efficient inference ser- vice of transformer-based deep learning models on gpus,” in IEEE iThings/GreenCom/CPSCom/SmartData/Cybermatics, 2020, pp. 323– 331

    2020

  10. [16]

    EfficientNet: Rethinking model scaling for convo- lutional neural networks,

    M. Tan and Q. Le, “EfficientNet: Rethinking model scaling for convo- lutional neural networks,” in ICML, 2019, pp. 6105–6114

    2019

  11. [17]

    Dyn- aBERT: Dynamic bert with adaptive width and depth,

    L. Hou, Z. Huang, L. Shang, X. Jiang, X. Chen, and Q. Liu, “Dyn- aBERT: Dynamic bert with adaptive width and depth,” in NeurIPS, vol. 33, 2020, pp. 9782–9793

    2020

  12. [18]

    A constrained decomposition approach with grids for evolutionary multiobjective optimization,

    X. Cai, Z. Mei, Z. Fan, and Q. Zhang, “A constrained decomposition approach with grids for evolutionary multiobjective optimization,” IEEE Trans. Evol. Comput. , vol. 22, no. 4, pp. 564–577, 2017

    2017

  13. [19]

    A pareto front grid guided multi-objective evolutionary algorithm,

    Y . Xu, H. Zhang, L. Huang, R. Qu, and Y . Nojima, “A pareto front grid guided multi-objective evolutionary algorithm,” Appl. Soft Comput., vol. 136, p. 110095, 2023

    2023

  14. [20]

    Progressive neural architecture search,

    C. Liu, B. Zoph, M. Neumann, J. Shlens, W. Hua, L.-J. Li, L. Fei-Fei, A. Yuille, J. Huang, and K. Murphy, “Progressive neural architecture search,” in ECCV, 2018, pp. 19–34

    2018

  15. [21]

    LGViT: Dynamic early exiting for accelerating vision transformer,

    G. Xu, J. Hao, L. Shen, H. Hu, Y . Luo, H. Lin, and J. Shen, “LGViT: Dynamic early exiting for accelerating vision transformer,” in ACM MM, 2023, pp. 9103–9114

    2023

  16. [22]

    Single-layer vision trans- formers for more accurate early exits with less overhead,

    A. Bakhtiarnia, Q. Zhang, and A. Iosifidis, “Single-layer vision trans- formers for more accurate early exits with less overhead,” Neural Netw., vol. 153, pp. 461–473, 2022

    2022

  17. [23]

    Learning transferable architectures for scalable image recognition,

    B. Zoph, V . Vasudevan, J. Shlens, and Q. V . Le, “Learning transferable architectures for scalable image recognition,” in IEEE CVPR, 2018, pp. 8697–8710

    2018

  18. [24]

    ENASFL: A federated neural architecture search scheme for heterogeneous deep models in distributed edge computing systems,

    Z. Zhang, Z. Liu, Y . Zhao, C. Qiu, C. Zhang, and X. Wang, “ENASFL: A federated neural architecture search scheme for heterogeneous deep models in distributed edge computing systems,” IEEE Trans. Netw. Sci. Eng., 2023

    2023

  19. [25]

    Efficient neural architecture search via parameter sharing,

    H. Pham, M. Guan, B. Zoph, Q. Le, and J. Dean, “Efficient neural architecture search via parameter sharing,” in ICML, 2018, pp. 4095– 4104

    2018

  20. [26]

    Simple statistical gradient-following algorithms for connectionist reinforcement learning,

    R. J. Williams, “Simple statistical gradient-following algorithms for connectionist reinforcement learning,” Mach. Learn. , vol. 8, pp. 229– 256, 1992

    1992

  21. [27]

    Importance estimation for neural network pruning,

    P. Molchanov, A. Mallya, S. Tyree, I. Frosio, and J. Kautz, “Importance estimation for neural network pruning,” in IEEE CVPR , 2019, pp. 11 264–11 272

    2019

  22. [28]

    Data valuation and detections in federated learning,

    W. Li, S. Fu, F. Zhang, and Y . Pang, “Data valuation and detections in federated learning,” in IEEE CVPR, 2024, pp. 12 027–12 036

    2024

  23. [29]

    Once-for-all: Train one network and specialize it for efficient deployment,

    H. Cai, C. Gan, T. Wang, Z. Zhang, and S. Han, “Once-for-all: Train one network and specialize it for efficient deployment,” in ICLR, 2020

    2020

  24. [30]

    BERxiT: Early exiting for BERT with better fine-tuning and extension to regression,

    J. Xin, R. Tang, Y . Yu, and J. Lin, “BERxiT: Early exiting for BERT with better fine-tuning and extension to regression,” in EACL, 2021, pp. 91–104

    2021

  25. [31]

    EE-Tuning: An economical yet scalable solution for tuning early-exit large language models,

    X. Pan, Y . Chen, Y . Li, B. Ding, and J. Zhou, “EE-Tuning: An economical yet scalable solution for tuning early-exit large language models,” arXiv preprint arXiv:2402.00518 , 2024

    2024 arXiv

  26. [32]

    A survey of visual transformers,

    Y . Liu, Y . Zhang, Y . Wang, F. Hou, J. Yuan, J. Tian, Y . Zhang, Z. Shi, J. Fan, and Z. He, “A survey of visual transformers,”IEEE Trans. Neural Networks Learn. Syst. , 2023

    2023

  27. [33]

    Efficient-ViT: A light-weight classification model based on CNN and ViT,

    Y . Xie and Y . Liao, “Efficient-ViT: A light-weight classification model based on CNN and ViT,” in ICIGP, 2023, pp. 64–70

    2023

  28. [34]

    MobileViT: Light-weight, general-purpose, and mobile-friendly vision transformer,

    S. Mehta and M. Rastegari, “MobileViT: Light-weight, general-purpose, and mobile-friendly vision transformer,” in ICLR, 2022

    2022

  29. [35]

    Twins: Revisiting the design of spatial attention in vision transformers,

    X. Chu, Z. Tian, Y . Wang, B. Zhang, H. Ren, X. Wei, H. Xia, and C. Shen, “Twins: Revisiting the design of spatial attention in vision transformers,” in NeurIPS, vol. 34, 2021, pp. 9355–9366

    2021

  30. [36]

    DeViT: Decomposing vision transformers for collaborative inference in edge devices,

    G. Xu, Z. Hao, Y . Luo, H. Hu, J. An, and S. Mao, “DeViT: Decomposing vision transformers for collaborative inference in edge devices,” IEEE Trans. Mob. Comput., 2023

    2023

  31. [37]

    Multi-exit vision trans- former for dynamic inference,

    A. Bakhtiarnia, Q. Zhang, and A. Iosifidis, “Multi-exit vision trans- former for dynamic inference,” in BMVC, 2021, p. 81

    2021

  32. [38]

    MobileNets: Efficient convo- lutional neural networks for mobile vision applications,

    A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient convo- lutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017

    2017 arXiv

  33. [39]

    Morphnet: Fast & simple resource-constrained structure learn- ing of deep networks,

    A. Gordon, E. Eban, O. Nachum, B. Chen, H. Wu, T.-J. Yang, and E. Choi, “Morphnet: Fast & simple resource-constrained structure learn- ing of deep networks,” in ECCV, 2018, pp. 1586–1595

    2018

  34. [40]

    Adaptive weighted sum method for multiobjective optimization: a new method for pareto front generation,

    I. Y . Kim and O. L. de Weck, “Adaptive weighted sum method for multiobjective optimization: a new method for pareto front generation,” Struct Multidiscipl Optim. , vol. 31, no. 2, pp. 105–116, 2006

    2006

  35. [41]

    3d object representations for fine-grained categorization,

    J. Krause, M. Stark, J. Deng, and L. Fei-Fei, “3d object representations for fine-grained categorization,” in IEEE ICCV Workshops , 2013, pp. 554–561

    2013

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.