Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

DAF: An Efficient End-to-End Dynamic Activation Framework for on-Device DNN Training

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DAF makes dynamic activation quantization practical on phones and embedded devices by adding a system layer that removes the framework's real-world overheads.

desk verdict Engineering is real, but the 22.9x memory claim compares activation-only memory to total training memory and is not supported as stated. read the letter →

arxiv 2507.07149 v1 pith:WYX5CXB3 submitted 2025-07-09 cs.NI cs.LG

Renyuan Liu , Yuyang Leng , Kaiyan Liu , Shaohan Hu , Chun-Fu (Richard) Chen , Peijun Zhao , Heechul Yun , Shuochao Yao This is my paper · ORCID
classification cs.NIcs.LG
keywords MobilecomputingOn-devicetrainingDynamicactivationquantizationMemory-efficientDNNEdgeAIBitpackingMemoryfragmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the theoretical memory savings promised by dynamic activation quantization have not translated to practice because of three system bottlenecks: the collective reduction operations needed to pick bit widths, the cost of bit-packing quantized values, and memory fragmentation from frequent allocation and deletion. DAF addresses each with system-level mechanisms—hybrid parallel/atomic reductions, CPU–GPU bit packing under a unified layout, and a page-based red-black-tree memory manager—without changing the forward/backward computation. Evaluated on ResNet-18, RoBERTa-base, and GPT-2 Medium across two Jetson boards and two Android phones, DAF reports up to 22.9x lower memory and 3.2x faster training than baseline methods while keeping accuracy loss under 1%. The point of the paper, if correct, is that activation compression algorithms need a companion system layer before their savings become real on resource-constrained devices.

What carries the argument

The load-bearing pieces are four. Hybrid collective reduction combines tree-based parallel reduction inside blocks with atomic compare-and-swap reduction across blocks, implemented for floats by reinterpreting their bits as integers, so the framework can pick the cheaper strategy per memory hierarchy. The bit-conversion trick turns a uint4 value into a float32 by OR-ing with the constant 0x4B000000 and subtracting $2^{23}$, replacing slow bit-conversion instructions with a fused elementwise computation. The unified CPU–GPU layout makes CPU SIMD packing and GPU tile-based unpacking compatible, so packed activations never need a separate unpacked copy in DRAM. Finally, the page-based memory manager preallocates a contiguous activation block and uses red-black trees keyed by importance over size and over time to choose evictions and insertions under first-fit page allocation.

What would settle it

Run the CIFAR-10 ResNet-18 experiment on the same Jetson AGX Xavier with DAF's system support intact but its greedy bit-width controller replaced by ActNN's published bit-width allocation rule; if memory savings fall to ActNN's level or accuracy drop exceeds 1%, then the system layer is not the main source of the 22.9x gain.

Watch

Extended reading notes

Core claim

DAF's central claim is that dynamic activation quantization can be made practical end-to-end by treating the system support as part of the algorithm. The framework decouples activation storage from computation: during the forward pass, activations are quantized to bit widths chosen by an importance-based greedy policy and stored in a preallocated page pool; during the backward pass they are dequantized for gradient computation. To remove the overhead that makes such schemes impractical, DAF fuses collective reductions into the forward kernels, switches between parallel and atomic reduction according to the memory hierarchy, packs quantized values using the CPU while the GPU computes, and resolves the resulting layout mismatch with a unified CPU–GPU bit layout. Its page manager uses two red-black trees to enforce time and memory budgets and handles dynamically changing budgets in fixed increments. With these pieces, DAF reports 11.2x–22.9x memory reduction and 1.5x–3.2x speedup over baselines while holding accuracy within 1%.

Load-bearing premise

The headline savings assume that DAF and every baseline measured the same quantity—runtime training memory under a <1% accuracy drop—and that DAF's extra savings come from its system optimizations rather than from a differently tuned bit-width selection policy.

Editorial extensions

If this is right

  • The memory budgets in Figure 8 imply that DAF can hold under-1% accuracy with activation memory budgets around 50–450 MB for these models, where the same training originally needed roughly 1–8 GB.
  • Because activations dominate training memory, DAF's reductions let users raise batch size or model size under the same memory cap rather than simply shrink total memory.
  • The speedups come from the system layer, not from skipping computation: DAF reports up to 3.2x faster training while keeping accuracy within 1%, and its accuracy under a fixed time budget beats Fixed 8 Bits and ElasticTrainer at their own budgets.
  • The red-black-tree page manager removes the need for repetitive empty_cache calls, so fragmentation is handled automatically rather than through periodic stalls.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because DAF's system components are mostly independent of the specific importance metric and bit-width allocator, future dynamic quantization policies with cheaper or better importance estimates should inherit the same memory and time benefits; the paper tests one greedy policy but does not tie the framework to it.
  • The 0x4B000000 prefix trick and the unified CPU–GPU layout are not specific to training: they could also speed up bit-packing and unpacking in on-device inference with low-bit weights and activations on the same ARM SoCs.
  • A decisive experiment would reuse ActNN's published bit-width allocator inside DAF's system; if the 22.9x savings persist, the system support is the main source, and if not, the headline comparison embeds a policy difference as well as a system difference.
  • The dynamic memory budget design suggests a broader principle: instead of asking the algorithm to fit a fixed cap, the system can grow and shrink the activation pool incrementally as other workloads claim memory, which is relevant for concurrent apps on mobile devices.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DAF, a system-level framework for on-device DNN training that dynamically quantizes activations to variable bit widths and adds three system optimizations: hybrid parallel/atomic reduction kernels, collaborative CPU-GPU bit packing with a unified layout, and a red-black-tree-based paging memory manager with dynamic budget support. The authors evaluate DAF on ResNet-18, RoBERTa-base, and GPT-2 Medium across Jetson and mobile platforms, reporting up to 22.9× lower memory usage and up to 3.2× faster training than baseline methods, with accuracy loss kept below 1%. The contribution is framed as making dynamic activation quantization practical by addressing the system overheads that prior algorithmic work ignores.

Significance. If the reported results hold, DAF would be a valuable systems contribution: it targets real, well-identified bottlenecks (reduction overhead, bit-layout conversion, and memory fragmentation) that existing dynamic-activation-quantization methods leave unresolved. The component studies, especially the reduction-operation speedups (11.2×–41.0× over CUB) and the red-black-tree paging speedups (~3.5×), are concrete and suggestive. However, the central memory claim is not verifiable as stated: the memory metric appears to exclude weights, gradients, and optimizer states, the importance metric is never defined, and no error bars, repeated runs, or artifact are provided. These issues prevent the reader from assessing whether the headline 22.9× memory reduction is an end-to-end training-memory saving or an activation-only compression ratio.

major comments (4)
  1. [§4.1, Table 1] The headline 22.9× memory-reduction claim is not supported because the memory metric is undefined or inconsistent. The metric definition in §4.1 says only that runtime memory footprint includes "stored activations and the memory allocated but not actively utilized, such as fragmentation"; it does not state whether weights, gradients, optimizer states, or the CUDA/OpenCL context are included. DAF's reported ResNet-18/CIFAR-10 value of 52 MB is smaller than the ~135 MB required for fp32 weights (≈45 MB), gradients (≈45 MB), and one momentum buffer (≈45 MB) alone, so this number cannot represent total training memory. The Original baseline's 1196 MB evidently does include those components, making the 22.9× ratio a comparison of non-commensurate quantities. The Fixed 8 Bits row (670 MB) highlights the same inconsistency: quantizing weights and activations to 8 bits should compress all tensors, yet DAF's activation-only value is still 12.9× smaller. Please provide a per-component memory breakdown (weights, gradients, optimizer states, activations, fragmentation) for every method, and either state that the headline numbers are activation-storage-only or give correct end-to-end totals.
  2. [§3.4.1, Algorithm 3] The importance metric that drives dynamic bit-width allocation is never specified. The text refers to a "sensitivity or importance value" for each activation, and Algorithm 3 uses "moving average importance" as the key for both red-black trees, but no formula, default configuration, or concrete choice (e.g., quantization error, variance, or tensor magnitude) is given. All memory-accuracy results in Table 1 and Figures 8–11 depend on this metric. Without it, the experiments are not reproducible, and the comparison with ActNN may confound DAF's system-level optimizations with a different, unspecified allocation policy. Please define the metric used in the evaluation or state that DAF accepts it as a plug-in parameter and report the exact value chosen.
  3. [§4.1, Figures 8–10, Table 1] No error bars or repeated runs are reported; every accuracy, memory, and time point appears to come from a single run. Several reported margins are small, e.g., Table 1 shows accuracy differences of roughly 0.2–0.5 percentage points at the 1% drop boundary, so seed and platform variance could affect which method meets the 1% constraint and change the reported savings or speedups. Please provide means and standard deviations over at least three seeds (or a clearly justified alternative), and state whether the memory and time numbers are measured at peak or averaged over the full training run.
  4. [§4.1, mobile baselines] The mobile baseline results are based on author-written OpenCL/TVM reimplementations with no validation against the original implementations or published numbers. For iQOO Neo3 and RedMagic 7, the paper states that Fixed 8 Bits, ElasticTrainer, and ActNN are "implemented" by the authors; the measured 1.5×–3.2× speedups on those platforms may therefore reflect implementation quality rather than DAF's system contributions. The reimplementations should be validated, e.g., by reproducing the original methods' reported accuracy and memory numbers on at least one common setting, and the validation should be described.
minor comments (5)
  1. [Throughout] There are several typos and label issues: Table 1 says "Train-from-scatch" instead of "Train-from-scratch"; Figure 1 says "Theoritical" instead of "Theoretical"; Figure 10 says "BLUE Score" instead of "BLEU"; and the conclusion says "STOAs" instead of "SOTAs".
  2. [§2.2] The sentence about quantization complexity contains a duplicated clause: "and the added complexity of quantization calculations, and the added complexity of quantization computations, including determining appropriate bit widths" should be reworded to remove the repetition.
  3. [§3.4.4] The choice of 100 MB as the memory-allocation step size is justified only by "Our experiments indicate that a step size of 100 MB is a reasonable and practical choice," and Figure 13 examines a single training iteration. A short sensitivity analysis over step sizes, or a more explicit argument for why the result generalizes across models and platforms, would be helpful.
  4. [Evaluation] No artifact or code is released. Given the empirical nature of the paper and the ambiguity in metrics, releasing the implementation (or at least a detailed reproducibility appendix) would substantially increase confidence in the reported numbers.
  5. [Figure 12] The annotations in Figure 12 are visually dense and the speedup labels are hard to read; larger fonts and clearer separation of the three device groups would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DAF's headline memory and speedup results are empirical measurements against external baselines, not derivations from equivalent assumptions.

full rationale

DAF is an empirical systems paper, not a derivation paper. The central claims (up to ightarrow22.9x lower memory usage and 3.2x speedup) are reported measurements of a concrete implementation compared with external baselines (Original, Fixed 8 Bits, ActNN, ElasticTrainer) on specific platforms and models. The dynamic quantization equations (Eq. 1) and the greedy bit-width selection described in Sec. 3.4.1 are standard algorithmic components; they do not make the measured memory savings true by definition. The paper does not fit a parameter to a subset of data and then present a closely related quantity as a prediction. It invokes no load-bearing uniqueness theorem, and there is no self-citation chain that supplies the paper's central premise. The only self-citation among the authors (DynaSPA, reference [26]) concerns dynamic inference rather than activation-compression training and is not used to justify DAF's design or evaluation. Concerns about which memory components are included in Table 1's memory footprint, or which importance metric was used in the experiments, are legitimate measurement and reproducibility questions, not indicators of circularity. Therefore, no circular step can be exhibited from the paper's text, and the appropriate score is 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The ledger is small because DAF is a systems paper: it introduces no new particles or physical entities. Its load-bearing assumptions are about the transferability of importance metrics, the efficiency of unified-memory collaboration, and the accuracy of greedy knapsack approximation. The main free parameter is the 100 MB memory step size, which is hand-chosen based on experiments.

free parameters (1)
  • Memory increment step size = 100 MB
    Sec. 3.4.4 states that DAF adjusts activation storage in 100 MB increments because experiments indicate this step size is practical; the choice affects memory adjustment overhead and activation eviction behavior.
assumptions (4)
  • domain assumption All dynamic activation quantization algorithms can be unified under a bi-objective knapsack objective, and a greedy algorithm approximates it well enough to preserve accuracy.
    Sec. 3.4.1 formulates dynamic quantization as a knapsack problem and uses a greedy algorithm for speed, but no suboptimality bound or evidence that the greedy choice preserves accuracy is provided.
  • domain assumption The sensitivity or importance metrics from prior work (quantization error, quantization variance, tensor magnitude) transfer to the mobile and embedded SoCs used in the evaluation.
    Sec. 3.2 says all existing methods use such metrics, but DAF never specifies which metric it actually uses in the experiments, making the accuracy-memory trade-off depend on an unstated assumption.
  • domain assumption Unified memory on mobile SoCs allows efficient CPU-GPU collaborative packing and unpacking without significant data transfer overhead.
    Sec. 3.3 relies on this premise to justify the CPU packing and GPU unpacking design; the paper does not report CPU-GPU transfer costs separately.
  • domain assumption The greedy page allocation scheme with first-fit placement maintains the same accuracy as exact memory budget optimization.
    Sec. 3.4.2 and Algorithm 3 use first-fit page allocation and greedy eviction; no guarantee is given that this matches the accuracy of an optimal or exact allocator.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DAF: An Efficient End-to-End Dynamic Activation Framework for on-Device DNN Training." pith.science (2026). https://pith.science/paper/WYX5CXB3

@misc{pith2026250707149,
  author       = {Pith},
  title        = {Pith review of: DAF: An Efficient End-to-End Dynamic Activation Framework for on-Device DNN Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WYX5CXB3}},
  note         = {Machine review of arXiv:2507.07149}
}
abstract

Recent advancements in on-device training for deep neural networks have underscored the critical need for efficient activation compression to overcome the memory constraints of mobile and edge devices. As activations dominate memory usage during training and are essential for gradient computation, compressing them without compromising accuracy remains a key research challenge. While existing methods for dynamic activation quantization promise theoretical memory savings, their practical deployment is impeded by system-level challenges such as computational overhead and memory fragmentation. To address these challenges, we introduce DAF, a Dynamic Activation Framework that enables scalable and efficient on-device training through system-level optimizations. DAF achieves both memory- and time-efficient dynamic quantization training by addressing key system bottlenecks. It develops hybrid reduction operations tailored to the memory hierarchies of mobile and edge SoCs, leverages collaborative CPU-GPU bit-packing for efficient dynamic quantization, and implements an importance-aware paging memory management scheme to reduce fragmentation and support dynamic memory adjustments. These optimizations collectively enable DAF to achieve substantial memory savings and speedup without compromising model training accuracy. Evaluations on various deep learning models across embedded and mobile platforms demonstrate up to a $22.9\times$ reduction in memory usage and a $3.2\times$ speedup, making DAF a scalable and practical solution for resource-constrained environments.

Figures

Figures reproduced from arXiv: 2507.07149 by the authors.

Figure 1
Figure 1. (a) Memory footprint during RoBERTa-base LoRA train￾ing. (b) Comparison of time and memory usage for various methods under a 1% accuracy drop, relative to the default setup. Original: Default setup with 1% accuracy drop; Fixed 8 Bits: All weights and ac￾tivations quantized to fixed 8 bits; ElasticTrainer [29] and ActNN [19]: Different activation compression training algorithms. methods at both block and thread level… view at source ↗
Figure 2
Figure 2. An illustration of practical gaps for dynamic activation quantization training. (a) Runtime overhead due to additional com￾putation. (b) Different bit package strategies. (c) Page management system. • Dynamic Quantization Optimization: We present and inte￾grate two complementary strategies to accelerate frequently used collective reduction operations across a variety of edge and mo￾bile platforms. • Bit Layout Conve… view at source ↗
Figure 3
Figure 3. DAF overview. (a) Default DAF computation flow. (b) DAF library. fast indexing. To address this, we introduce a CPU-assisted page management system that leverages a red-black tree to efficiently organize and manage activations. 2.3 Quantization Calculation Quantization involves mapping a range of values to a limited set of levels, typically defined by a bit-width 𝑏. For instance, the quan￾tization process for an act… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Collective reduction operation example. isolation significantly hampers the efficiency of reduction oper￾ations, which require traversing and aggregating data across the entire tensor, limiting the GPU’s ability to fully leverage parallelism. To address this challenge,…
Figure 5
Figure 5. Figure 5: Unified Bit Layout for efficient CPU packing and GPU unpacking. by the GPU with minimal overhead. Our implementation utilizes a CPU-based approach with SIMD, specifically ARM’s NEON in￾struction set, for efficient extraction and packing operations. By integrating SIMD’…
Figure 7
Figure 7. Figure 7: Red-Black Tree API example. 3.4.2 Page Allocation. To optimize memory usage, DAF is inspired by TensorFlow’s static memory design and incorporates offline anal￾ysis to evaluate the lifecycle and reuse patterns of non-activation memory. For activations, DAF allocates a …
Figure 8
Figure 8. Figure 8: Comparison of accuracy under varying memory budgets. Original ActNN DAF Runtime Overhead Activations ResNet-18 RoBERTa-base Batchsize Batchsize 22.9× 18.4× 18.3× 18.3× 20.8× 19.8× Memory budget (MB) [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Memory budget with varying batch sizes for ResNet-18 and RoBERTa-base using Jetson AGX Xavier, under an accuracy drop of <1%. standard training, implemented using PyTorch on the Jetson plat￾forms and OpenCL on the mobile platforms. Fixed 8 Bits stores both weights and …
Figure 10
Figure 10. Figure 10: Comparison of accuracy under varying time budgets. Memory budget ratio Time budget ratio ResNet-18 Time budget ratio RoBERTa-base Default Original (92.1%) Original (91.7%) Fixed 8 Bits (92.1%) Fixed 8 Bits (91.7%) ElasticTrainer (92.1%) ElasticTrainer (91.7%) ActNN (9…
Figure 11
Figure 11. Figure 11: Comparison of time and memory usage for various meth￾ods under different levels of accuracy drop, relative to the default setup, on Jetson AGX Xavier. for actual memory optimization. Instead, ElasticTrainer focuses solely on skipping computations to accelerate trainin…
Figure 13
Figure 13. Figure 13: Dynamic memory budget and DAF strategy on Jetson AGX Xavier with ResNet-18 for one training iteration. Energy (J) 20 90 91 92 93 40 60 x10^4 Acc. (%) Fixed 8 Bits Original ActNN ElasticTrainer DAF [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]
Figure 14
Figure 14. Figure 14: Energy consumption comparison of ResNet-18 training on Jetson AGX Xavier. For a fair comparison, the memory overhead is defined as the temporary memory usage incurred solely by the packing and un￾packing operations. As shown in [PITH_FULL_IMAGE:figures/full_fig_p011_…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FBLayout: Optimizing Memory Layout for Efficient LLM Finetuning on Mobile GPUs

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A tile-based memory layout for mobile GPUs that unifies forward and backward data access, eliminating most transpose/reshape overhead and speeding LLM fine-tuning 2.2–5.7× in the paper's measurements.

Reference graph

Works this paper leans on

48 extracted references · 41 canonical work pages · cited by 1 Pith paper

  1. [1]

    Vi-map: Infrastructure-assisted real-time hd mapping for autonomous driving

    Yuze He, Chen Bian, Jingfei Xia, Shuyao Shi, Zhenyu Yan, Qun Song, and Guoliang Xing. Vi-map: Infrastructure-assisted real-time hd mapping for autonomous driving. In Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, pages 1–15, 2023

    work page 2023

  2. [2]

    Robust real-time multi-vehicle collaboration on asynchronous sensors

    Qingzhao Zhang, Xumiao Zhang, Ruiyang Zhu, Fan Bai, Mohammad Naserian, and Z Morley Mao. Robust real-time multi-vehicle collaboration on asynchronous sensors. In Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, pages 1–15, 2023

    work page 2023

  3. [3]

    Autofed: Heterogeneity-aware federated multimodal learning for robust autonomous driving

    Tianyue Zheng, Ang Li, Zhe Chen, Hongbo Wang, and Jun Luo. Autofed: Heterogeneity-aware federated multimodal learning for robust autonomous driving. In Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, pages 1–15, 2023

    work page 2023

  4. [4]

    Scaleflow: Efficient deep vision pipeline with closed-loop scale-adaptive inference

    Yuyang Leng, Renyuan Liu, Hongpeng Guo, Songqing Chen, and Shuochao Yao. Scaleflow: Efficient deep vision pipeline with closed-loop scale-adaptive inference. In Proceedings of the 31st ACM International Conference on Multimedia , pages 1698–1706, 2023

    work page 2023

  5. [5]

    Speech understanding on tiny devices with a learning cache

    Afsara Benazir, Zhiming Xu, and Felix Xiaozhu Lin. Speech understanding on tiny devices with a learning cache. InProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services , pages 425–437, 2024

    work page 2024

  6. [6]

    Autodroid: Llm-powered task automation in android

    Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu, Yaqin Zhang, and Yunxin Liu. Autodroid: Llm-powered task automation in android. In Proceedings of the 30th Annual International Conference on Mobile Computing and Networking , pages 543–557, 2024

    work page 2024

  7. [7]

    Mobilegpt: Augmenting llm with human-like app memory for mobile task automation

    Sunjae Lee, Junyoung Choi, Jungjae Lee, Munim Hasan Wasi, Hojun Choi, Steve Ko, Sangeun Oh, and Insik Shin. Mobilegpt: Augmenting llm with human-like app memory for mobile task automation. ACM MobiCom ’24, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400704895. doi: 10.1145/3636534.3690682. URL https://doi.org/10.1145/3636534.3690682

    arXiv 2024

  8. [8]

    Experience: Practical challenges for indoor ar applications

    Shunpei Yamaguchi, Aditya Arun, Takuya Fujiwara, Misaki Sakuta, Ryotaro Hada, Takuya Fujihashi, Takashi Watanabe, Dinesh Bharadia, and Shunsuke Saruwatari. Experience: Practical challenges for indoor ar applications. ACM MobiCom ’24, page 1030–1044, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400704895

    work page 2024

Show all 48 references
  1. [9]

    Arise: High-capacity ar offloading inference serving via proactive scheduling

    Z Jonny Kong, Qiang Xu, and Y Charlie Hu. Arise: High-capacity ar offloading inference serving via proactive scheduling. In Proceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services , pages 451– 464, 2024

    2024

  2. [10]

    Band: coordinated multi-dnn inference on heterogeneous mobile processors

    Joo Seong Jeong, Jingyu Lee, Donghyun Kim, Changmin Jeon, Changjin Jeong, Youngki Lee, and Byung-Gon Chun. Band: coordinated multi-dnn inference on heterogeneous mobile processors. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and S...

    2022

  3. [11]

    Theia: Gaze-driven and perception-aware volumetric content delivery for mixed reality headsets

    Nan Wu, Kaiyan Liu, Ruizhi Cheng, Bo Han, and Puqi Zhou. Theia: Gaze-driven and perception-aware volumetric content delivery for mixed reality headsets. In Proceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services, pages 70–84, 2024

    2024

  4. [12]

    Zero-effort cross-domain gesture recognition with wi-fi

    Yue Zheng, Yi Zhang, Kun Qian, Guidong Zhang, Yunhao Liu, Chenshu Wu, and Zheng Yang. Zero-effort cross-domain gesture recognition with wi-fi. In Proceed- ings of the 17th annual international conference on mobile systems, applications, and services, pages 313–325, 2019

    2019

  5. [13]

    Rf genesis: Zero-shot generalization of mmwave sensing through simulation-based data synthesis and generative diffusion models

    Xingyu Chen and Xinyu Zhang. Rf genesis: Zero-shot generalization of mmwave sensing through simulation-based data synthesis and generative diffusion models. In Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems , pages 28–42, 2023

    2023

  6. [14]

    Cosmo: contrastive fusion learning with small data for multimodal human activity recognition

    Xiaomin Ouyang, Xian Shuai, Jiayu Zhou, Ivy Wang Shi, Zhiyuan Xie, Guoliang Xing, and Jianwei Huang. Cosmo: contrastive fusion learning with small data for multimodal human activity recognition. In Proceedings of the 28th Annual International Conference on Mobile Computing And...

    2022

  7. [15]

    In USENIX Workshop on Hot Topics in Edge Computing (HotEdge 18), Boston, MA, July 2018

    A Privacy-Preserving deep learning approach for face recognition with edge computing. In USENIX Workshop on Hot Topics in Edge Computing (HotEdge 18), Boston, MA, July 2018. USENIX Association. URL https://www.usenix.org/ conference/hotedge18/presentation/mao

    2018

  8. [16]

    On-device training under 256kb memory

    Ji Lin, Ligeng Zhu, Wei-Ming Chen, Wei-Chen Wang, Chuang Gan, and Song Han. On-device training under 256kb memory. Advances in Neural Information Processing Systems, 35:22941–22954, 2022

    2022

  9. [17]

    Zerofl: Efficient on-device training for federated learning with local sparsity

    Xinchi Qiu, Javier Fernandez-Marques, Pedro PB Gusmao, Yan Gao, Titouan Parcollet, and Nicholas Donald Lane. Zerofl: Efficient on-device training for federated learning with local sparsity. arXiv preprint arXiv:2208.02507, 2022

    2022 arXiv

  10. [18]

    Spotlight: Optimizing device place- ment for training deep neural networks

    Yuanxiang Gao, Li Chen, and Baochun Li. Spotlight: Optimizing device place- ment for training deep neural networks. In International Conference on Machine Learning, pages 1676–1684. PMLR, 2018

    2018

  11. [19]

    Actnn: Reducing training memory footprint via 2-bit activation compressed training

    Jianfei Chen, Lianmin Zheng, Zhewei Yao, Dequan Wang, Ion Stoica, Michael Mahoney, and Joseph Gonzalez. Actnn: Reducing training memory footprint via 2-bit activation compressed training. In International Conference on Machine Learning, pages 1803–1813. PMLR, 2021. DAF MobiSys...

    2021

  12. [20]

    Division: memory efficient training via dual activation precision

    Guanchu Wang, Zirui Liu, Zhimeng Jiang, Ninghao Liu, Na Zou, and Xia Hu. Division: memory efficient training via dual activation precision. In International Conference on Machine Learning , pages 36036–36057. PMLR, 2023

    2023

  13. [21]

    Flexpoint: An adaptive numerical format for efficient training of deep neural networks

    Urs Köster, Tristan Webb, Xin Wang, Marcel Nassar, Arjun K Bansal, William Constable, Oguz Elibol, Scott Gray, Stewart Hall, Luke Hornof, et al. Flexpoint: An adaptive numerical format for efficient training of deep neural networks. Advances in neural information processing sy...

    2017

  14. [22]

    Fractrain: Fractionally squeezing bit savings both temporally and spatially for efficient dnn training

    Yonggan Fu, Haoran You, Yang Zhao, Yue Wang, Chaojian Li, Kailash Gopalakr- ishnan, Zhangyang Wang, and Yingyan Lin. Fractrain: Fractionally squeezing bit savings both temporally and spatially for efficient dnn training. Advances in Neural Information Processing Systems , 33:1...

    2020

  15. [23]

    Gact: Activation com- pressed training for generic network architectures

    Xiaoxuan Liu, Lianmin Zheng, Dequan Wang, Yukuo Cen, Weize Chen, Xu Han, Jianfei Chen, Zhiyuan Liu, Jie Tang, Joey Gonzalez, et al. Gact: Activation com- pressed training for generic network architectures. In International Conference on Machine Learning, pages 14139–14152. PMLR, 2022

    2022

  16. [24]

    Pact: Parameterized clipping activation for quantized neural networks.arXiv preprint arXiv:1805.06085, 2018

    Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. Pact: Parameterized clipping activation for quantized neural networks.arXiv preprint arXiv:1805.06085, 2018

    2018 arXiv

  17. [25]

    Fixed-point back-propagation training

    Xishan Zhang, Shaoli Liu, Rui Zhang, Chang Liu, Di Huang, Shiyi Zhou, Jiaming Guo, Qi Guo, Zidong Du, Tian Zhi, et al. Fixed-point back-propagation training. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2330–2338, 2020

    2020

  18. [26]

    Dynaspa: Exploiting spatial sparsity for efficient dynamic dnn inference on devices

    Renyuan Liu, Yuyang Leng, Shilei Tian, Shaohan Hu, Chun-Fu Chen, and Shuochao Yao. Dynaspa: Exploiting spatial sparsity for efficient dynamic dnn inference on devices. In Proceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems, pages 422–435, 2024

    2024

  19. [27]

    Memory-efficient dnn training on mobile devices

    In Gim and JeongGil Ko. Memory-efficient dnn training on mobile devices. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services, pages 464–476, 2022

    2022

  20. [28]

    Poet: Training neural networks on tiny devices with integrated rematerialization and paging

    Shishir G Patil, Paras Jain, Prabal Dutta, Ion Stoica, and Joseph Gonzalez. Poet: Training neural networks on tiny devices with integrated rematerialization and paging. In International Conference on Machine Learning , pages 17573–17583. PMLR, 2022

    2022

  21. [29]

    Elastictrainer: Speeding up on-device training with runtime elastic tensor selection

    Kai Huang, Boyuan Yang, and Wei Gao. Elastictrainer: Speeding up on-device training with runtime elastic tensor selection. In Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services, pages 56–69, 2023

    2023

  22. [30]

    Nvidia cub, 2024

    NVIDIA. Nvidia cub, 2024. URL https://nvidia.github.io/cccl/cub/

    2024

  23. [31]

    Identity mappings in deep residual networks

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14 , pages 630–645. Springer, 2016

    2016

  24. [32]

    Roberta: A robustly optimized bert pretraining approach

    Yinhan Liu. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 364, 2019

    1907 arXiv

  25. [33]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  26. [34]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  27. [35]

    Throughput of native arithmetic instructions, 2024

    NVIDIA. Throughput of native arithmetic instructions, 2024. URL https://docs.nvidia.com/cuda/cuda-c-programming-guide/#arithmetic- instructions-throughput-native-arithmetic-instructions

    2024

  28. [36]

    Instruction throughput and latency, 2024

    ARM. Instruction throughput and latency, 2024. URL https://developer. arm.com/documentation/100400/0002/floating-point-unit-programmers- model/instruction-throughput-and-latency

    2024

  29. [37]

    Tensorflow: Large-scale machine learning on heterogeneous distributed systems

    Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467, 2016

    2016 arXiv

  30. [38]

    The multiobjective multidimensional knapsack problem: a survey and a new approach

    Thibaut Lust and Jacques Teghem. The multiobjective multidimensional knapsack problem: a survey and a new approach. International Transactions in Operational Research, 19(4):495–520, 2012

    2012

  31. [39]

    Canadian institute for advanced research, 10 classes,

    CIFAR-10 Benchmark. Canadian institute for advanced research, 10 classes, . URL https://paperswithcode.com/dataset/cifar-10

  32. [40]

    Canadian institute for advanced research, 100 classes,

    CIFAR-100 Benchmark. Canadian institute for advanced research, 100 classes, . URL https://paperswithcode.com/dataset/cifar-100

  33. [41]

    Imagenet: A large-scale hierarchical image database,

    ImageNet1K Benchmark. Imagenet: A large-scale hierarchical image database, . URL https://paperswithcode.com/dataset/imagenet

  34. [42]

    Semantic textual similarity,

    STS Benchmark. Semantic textual similarity, . URL https://paperswithcode.com/ dataset/sts-benchmark

  35. [43]

    Microsoft research paraphrase corpus,

    MRPC Benchmark. Microsoft research paraphrase corpus, . URL https:// paperswithcode.com/dataset/mrpc

  36. [44]

    The stanford sentiment treebank,

    SST-2 Benchmark. The stanford sentiment treebank, . URL https:// paperswithcode.com/dataset/sst-2

  37. [45]

    End-to-end nlg challenge,

    E2E Benchmark. End-to-end nlg challenge, . URL https://paperswithcode.com/ dataset/e2e

  38. [46]

    Creating training corpora for nlg micro-planners,

    WebNLG Benchmark. Creating training corpora for nlg micro-planners, . URL https://paperswithcode.com/dataset/webnlg

  39. [47]

    Towards unified int8 training for convolutional neural network

    Feng Zhu, Ruihao Gong, Fengwei Yu, Xianglong Liu, Yanfei Wang, Zhelong Li, Xiuqi Yang, and Junjie Yan. Towards unified int8 training for convolutional neural network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1969–1979, 2020

    1969

  40. [48]

    Re- source management with deep reinforcement learning

    Hongzi Mao, Mohammad Alizadeh, Ishai Menache, and Srikanth Kandula. Re- source management with deep reinforcement learning. In Proceedings of the 15th ACM workshop on hot topics in networks , pages 50–56, 2016

    2016

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.