Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read On-device LLM decoding can cut energy use by 23% without slowing down by adaptively selecting fewer, smaller CPU cores, because the memory-bound decode phase tolerates fewer cores while the OS lowers idle-core frequency.

desk verdict A real engine-level energy optimization for on-device LLM decode, with a believable 23% average saving; the main caveats are missing variance reporting, no tuning/eval separation shown, and device-dependent governor behavior that the paper itself documents. read the letter →

arxiv 2506.19884 v1 pith:I56OC7JB submitted 2025-06-24 cs.OS cs.AIcs.PFcs.SE

Zhengxiang Huang , Chaoyue Niu , Zhaode Wang , Jiarui Xue , Hanming Zhang , Yugang Wang , Zewei Xin , Xiaotang Jiang
show 3 more authors
Chengfei Lv Fan Wu Guihai Chen
This is my paper · ORCID
classification cs.OScs.AIcs.PFcs.SE
keywords on-deviceLLMinferenceenergyoptimizationCPUcoreselectionaffinityDVFSmemory-bounddecodingmobilesystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models running on phones spend most of their energy not on reading the prompt but on generating tokens one at a time, and the paper claims this decode phase is memory-bound: the CPU cores mostly wait on the memory bus. Because decode tolerates less parallelism, the authors argue that binding decoding to fewer, smaller CPU cores leaves other cores idle, which lets the operating system's frequency governor lower their clock speed and cut power. They build this into MNN, an on-device inference engine, as a tool called AECS that searches once per device for the best core selection, then uses it for all future decoding. Across seven phones and several small LLMs, the integrated engine reports 23% lower energy than the original engine at the same speed, and 39-78% less energy than four other engines while running faster. The relevance is practical: it is an engine-level change, requiring no root access or OS modification, so it could ship in ordinary apps.

What carries the argument

The carrying mechanism is the power heuristic h(I) of Eq. (9), $$h(I)=\sum_{i=0}^{n-1} a_i(|I_i|+(|C_i|-|I_i|)b)(f_{\max,i}\cdot s_I)^2+P_s,$$ which estimates decode power from the number of selected cores in each cluster, a per-cluster type factor, an idle-core factor b<1, and a governor-modeled frequency f_i = f_max,i * s_I, where s_I is the ratio of the selected biggest core's capacity to the biggest capacity on the chip. This heuristic is used in a two-stage search: stage 1 greedily builds the fastest selection starting from one prime core, and stage 2 grows a small candidate tree via transformations (remove one or two smallest cores, swap a big core for a smaller one, switch to a smaller cluster) and evaluates each candidate with a speed constraint, returning the selection with the lowest heuristically-regularized energy. The search is run once per device at tuning time, and the chosen selection is then applied to the thread pool for all future decode calls, with prefill and decode allowed different core selections.

What would settle it

Force the CPUFreq governor on one of the paper's Android devices into 'performance' mode, so idle cores stay at maximum frequency, then re-run the MNN-AECS versus MNN comparison on the same datasets; if the 23% energy saving persists, the claimed idle-frequency mechanism is not the cause, and if it collapses, the governor is load-bearing.

Watch

Extended reading notes

Core claim

The paper's central claim is that the energy bottleneck of on-device LLM inference is the decode phase, not the prefill phase, and that this bottleneck can be attacked by choosing the right CPU core selection. It asserts that during memory-bound decoding, using all available CPU cores wastes energy because additional cores contribute little speed while keeping every cluster's frequency high. AECS therefore solves an optimization: minimize per-token energy E(I)=P(I)t(I) subject to speed(I) being at least 92% of the fastest selection, over core binding plans on Android and thread counts on iOS. The authors report that the resulting selection uses at most two cores on all seven test devices, cuts CPU utilization by 50-75%, reduces energy by 23% on average over the base engine without slowdown, and beats four other engines by 39-78% in energy while also being 12-363% faster. The energy reduction is presented as the product of leaving cores idle and the OS governor scaling down their frequency, and it is verified on 5 Android and 2 iOS devices across 5 LLMs and 4 datasets.

Load-bearing premise

The energy savings depend on the operating system's frequency governor actually lowering the clock speed of CPU cores that are left idle; if a device's governor keeps idle clusters at high frequency, the 23% average saving shrinks, as the paper's own Meizu 21 result (10% instead of 18-42%) shows.

Editorial extensions

If this is right

  • Energy savings persist across prompt lengths from 64 to 1024 tokens, decode lengths from 128 to 512, five different LLMs, and four datasets, indicating the effect is structural rather than dataset-specific.
  • The tuned selection never uses more than two CPU cores on any of the seven test devices, cutting CPU utilization by 50-75%, and on six of seven devices it is as fast or faster than the base engine, with at most 7% slowdown elsewhere.
  • The once-and-for-all AECS search takes 1-2 minutes and matches exhaustive traversal in the chosen optimum, making per-device tuning practical enough for real deployments.
  • Because the change lives entirely in the inference engine, it requires no root access or OS modification and is orthogonal to quantization or sparsity methods, so it can combine with algorithmic optimizations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • On devices whose governor keeps idle-cluster frequency high, the same mechanism should deliver smaller savings; forcing or emulating idle frequency scaling on such devices would test whether the 23% average is recoverable across the full hardware landscape.
  • The same core-undersubscription logic might apply to other memory-bound mobile workloads, such as on-device speech recognition or embedding search; a natural test is to run an AECS-style per-device tuning pass on those workloads and measure energy per inference.
  • From the paper's own GPU discussion, a GPU or NPU that exposes shader-core selection or frequency control could extend the energy savings beyond the CPU path, potentially compounding the reported reductions.
  • Since the mechanism is orthogonal to model compression, combining AECS with smaller or more aggressively quantized models should multiply per-token energy savings; direct measurement of energy per token on an AECS-tuned device at 2-bit or sparse precision would settle how the savings add.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents MNN-AECS, an extension of the MNN on-device LLM inference engine that performs a once-per-device offline search over CPU core selections (core affinity on Android, thread count on iOS) to minimize the energy consumed during the memory-bound decode phase, subject to a decode-speed constraint of at most epsilon=8% slowdown relative to the fastest selection. The search (AECS) has two stages: Stage 1 greedily constructs the fastest core selection from CPU-cluster information; Stage 2 generates a small candidate tree by heuristic transformations and profiles each candidate, using an objective that blends measured energy with a power heuristic. The power heuristic models CPUFreq governor behavior via f_i = f_max,i * s_I and includes cluster scaling factors a_i, an idle-core factor b, and static power P_s. The authors integrate AECS into MNN with separate prefill/decode thread-pool configurations and build a cross-platform energy testbed that reads OS battery interfaces on Android and Xcode energy gauge on iOS. They report results on 5 Android and 2 iOS devices with 5 LLMs over 4 datasets: a 23% average energy reduction relative to MNN (versus 18-42% on individual devices, except 9.7% on Meizu 21), and 39-78% energy reduction and 12-363% speedup relative to llama.cpp, executorch, mllm, and MediaPipe. An ablation claims that AECS matches exhaustive search in 100% of cases while being about 10x faster.

Significance. If the measured savings are robust, the paper demonstrates a practical, root-free engine-level lever for reducing LLM decoding energy by roughly a fifth on average, a meaningful improvement for battery-constrained mobile inference. The design is well-motivated by the memory-bound nature of decode, and the use of a cross-platform testbed and a comparison against four other engines is a definite strength. The ablation against exhaustive search, if correctly reported, provides direct evidence that the heuristic search does not sacrifice solution quality. The main value is as an empirical systems contribution; the power heuristic itself is auxiliary, with the energy savings coming from the measured selection of fewer/smaller cores. However, the paper's statistical basis is thin, the heuristic parameters are undisclosed, and the central mechanism depends on OS governor behavior that is not under the engine's control, so the headline numbers should be read as device-dependent.

major comments (4)
  1. [Section 5.3/5.4, Tables 9-10, Figures 11-13] All energy and speed results in Tables 9 and 10 and Figures 11-13 are reported as single averages over datasets with no standard deviation, confidence interval, or number of repetitions. This matters because Section 3.3 explicitly states that empirically measured energy suffers from 'intolerable system fluctuation ... as high as 5%'. Without per-condition variance data, the 23% average energy saving over MNN cannot be statistically distinguished from run-to-run variation on individual devices, and the comparison against other engines lacks a noise floor. Please add repetition counts and variance/confidence intervals for at least the headline comparisons, or explicitly state if each reported number is a single measurement.
  2. [Section 3.3, Eq. (9)] The power heuristic in Eq. (9) depends on unspecified constants: the cluster scaling factors a_i, the idle-core reduction factor b, the static power P_s, and the weight alpha in the objective E_h(I) = (1-alpha)E(I) + alpha h(I)t(I). No values, fitting procedure, or sensitivity analysis is provided. Consequently AECS cannot be reimplemented from the paper, and the ablation in Table 11 that attributes better optimality to the heuristic cannot be reproduced. In addition, Section 3.3 states that 'AECS search aborts energy profiling on iOS, using only the heuristics for optimization', but the paper never says what objective is used in that case (presumably alpha=1) nor how the exhaustive-search comparison in Table 11 was conducted on iOS; please specify the exact configuration and report the parameter values.
  3. [Section 5.3, Figure 11(a)] The energy saving over MNN is only 9.7% on Meizu 21, versus 18-42% on the other six devices, and the paper attributes this to the OS not scaling down the idle-cluster frequency. Since the design's power-reduction mechanism relies on the CPUFreq governor lowering the frequency of idle cores (Eq. (9) and Section 2.4), the 23% average reported in the abstract and conclusion is a property of the test sample, not a device-independent property of AECS. Please report the per-device savings distribution explicitly, state the governor-behavior condition under which the method is effective, and discuss how widely walt/schedutil devices can be expected to exhibit it.
  4. [Table 11] Table 11, as presented, has columns 'exhaustive', 'AECS', and 'AECS(optimal) w/o heuristic' with optimality rates 100%, 60%-90%, and 100%, respectively; this conflicts with the text stating that 'removing the power heuristic leads to fluctuation in searching and consequently lower optimality rate'. The column layout and labeling need to be fixed so that the reader can tell which variant achieves 60%-90% and which achieves 100%; otherwise the ablation evidence for the power heuristic is ambiguous.
minor comments (6)
  1. [Abstract and Section 1.3] The claim that energy is reduced 'without slowdown' is stronger than what the speed constraint permits; Figure 11(b) reports a 6.8% slowdown on V30 Pro. Suggest rephrasing to 'with average speed preserved' or 'within an 8% slowdown bound'.
  2. [Tables 9 and 10] The energy columns have no stated unit; for iOS the values come from Xcode energy gauge and are relative, not mJ/token, so the tables should label the metric accordingly.
  3. [Table 11] There is a typo: 'serach time' should be 'search time'.
  4. [References] Reference [53] lists the year as '5555' and month 'Feb. 5555', which appears to be a citation-generation error that should be corrected.
  5. [Section 3.3] The phrase 'The tree depth is limited less than 2' should be 'limited to at most 2' (or 'less than or equal to 2').
  6. [Section 5.5] The phrase 'search space scales up to 20-71' should specify that these are counts of candidate core selections, not a time or size unit.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: AECS selects cores by direct energy measurement, and the reported savings are independently measured system outcomes rather than constructed from the heuristic.

full rationale

The paper's central energy-saving claims are empirical results of a once-and-for-all on-device search (Algorithm 1) that directly measures energy and speed for candidate core selections under a speed constraint. The reported numbers (23% vs. MNN, 39–78% vs. other engines) are independent re-measurements of the selected configuration against baselines, not quantities derived from the power heuristic by construction. The heuristic h(I) in Eq. (9), including the CPUFreq governor term f_i = f_max,i * s_I, is used only to generate and prune candidates and to form the modified objective E_h(I) = (1-alpha)E(I) + alpha*h(I)*t(I); the exhaustive-traversal ablation in Table 11 confirms that AECS reaches the same selection as exhaustive search, so the chosen configuration is not forced by the heuristic. The acknowledged Meizu 21 limitation (Section 5.3: only 10% energy saving because the OS does not scale down idle-cluster frequency) is a device-level generalizability caveat, not evidence that the claimed saving reduces to an input of the derivation. The base engine MNN is cited from the authors' prior work, but it is used as an open-source baseline and the comparison is a measured benchmark, not an unverified load-bearing citation. No equation is defined in terms of the target result, and no fitted parameter is renamed as a prediction. Therefore, no circular step meeting the quoted-evidence standard is present.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on empirical search and a few OS-behavior assumptions rather than on a mathematical derivation. The main free parameters are the speed constraint epsilon, the heuristic weight alpha, and the uncalibrated constants in the power heuristic (a_i, b, P_s). No new physical or conceptual entities are postulated; the power heuristic is a model, not an entity.

free parameters (5)
  • speed constraint epsilon = 8%
    Chosen by hand as an unnoticeable slowdown in Section 3.1. It defines the feasible region for the search and therefore which core selections are accepted.
  • heuristic weight alpha = not reported
    The objective in Section 3.3 is E_h = (1-alpha) E + alpha h t, but the paper never reports alpha's value, range, or tuning procedure. The ablation shows the heuristic mix is load-bearing for search optimality.
  • cluster power scaling factors a_i = not reported
    These factors in Eq. (9) are meant to distinguish CPU types in the power heuristic. No values or calibration procedure are given.
  • idle core reduction factor b = not reported
    Appears in Eq. (9) to discount idle cores. No value or calibration is given.
  • static power P_s = not reported
    Appears in Eq. (9) as a static power term. No value or measurement procedure is reported.
assumptions (5)
  • domain assumption Decode phase is memory-bound, so using fewer cores incurs little speed loss.
    Section 2.1 derives the GEMV shape and argues memory-boundedness from single-token processing. The entire speed-constrained energy optimization rests on this premise.
  • domain assumption CPU power is approximately quadratic in frequency plus a static term.
    This is used in the power heuristic Eq. (9), citing Dou et al. [14]. No per-device verification is presented.
  • domain assumption The CPUFreq governor sets cluster frequency to f_max * s_I, where s_I is the ratio of selected biggest capacity to biggest capacity.
    Section 3.3 derives this from reading Android 12.1 kernel source. Different devices and kernels can deviate, and the Meizu 21 result in Section 5.3 is a partial counterexample because its OS does not scale down idle cluster frequency.
  • domain assumption OS energy interfaces provide sufficiently precise energy for the comparisons.
    Android BatteryManager updates every 250 ms and iOS Xcode energy gauge gives relative power only. The paper assumes that 50 ms polling and 50-token decodes make the measurements precise enough.
  • domain assumption The once-and-for-all tuned core selection transfers to all evaluation datasets.
    Section 5.2 states tuned results are used for all experiments, but no explicit separation of tuning data from evaluation data is described. This is load-bearing for the 23% average claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection." pith.science (2026). https://pith.science/paper/I56OC7JB

@misc{pith2026250619884,
  author       = {Pith},
  title        = {Pith review of: MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I56OC7JB}},
  note         = {Machine review of arXiv:2506.19884}
}
read the original abstract

As the demand for on-device Large Language Model (LLM) inference grows, energy efficiency has become a major concern, especially for battery-limited mobile devices. Our analysis shows that the memory-bound LLM decode phase dominates energy use, and yet most existing works focus on accelerating the prefill phase, neglecting energy concerns. We introduce Adaptive Energy-Centric Core Selection (AECS) and integrate it into MNN to create the energy-efficient version, MNN-AECS, the first engine-level system solution without requiring root access or OS modifications for energy-efficient LLM decoding. MNN-AECS is designed to reduce LLM decoding energy while keeping decode speed within an acceptable slowdown threshold by dynamically selecting low-power CPU cores. MNN-AECS is evaluated across 5 Android and 2 iOS devices on 5 popular LLMs of various sizes. Compared to original MNN, MNN-AECS cuts down energy use by 23% without slowdown averaged over all 7 devices and 4 datasets. Against other engines, including llama.cpp, executorch, mllm, and MediaPipe, MNN-AECS delivers 39% to 78% energy saving and 12% to 363% speedup on average.

Figures

Figures reproduced from arXiv: 2506.19884 by the authors.

Figure 1
Figure 1. (a) MNN-AECS workflow. (b) MNN-AECS geometric mean performance over 5 baseline engines across 7 [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Prefill length and decode length distributional [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 2
Figure 2. Comparison of prefill and decode speed, CPU use, power, and energy of Qwen2.5-1.5B across 4 datasets on Xiaomi 15 Pro. Decode phase takes much longer time, in that decode speed is slower and decode length is longer. 1) Decode speed is 3× slower than prefill on Xiaomi 15 Pro CPU as an example in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: CPU frequency curve of 3 core selections of [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Stage 1 process on Mate 40 Pro: Starting from [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Stage 2 heuristic tree on Mate 40 Pro: trans [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: MNN-AECS energy profiling module. Implementations. For Android, we pool current and voltage in a separate profiling thread to calculate energy, which is then passed to MNN-AECS over JNI (Java-Native Interface), illustrated in [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Normalized energy and decode speed com￾parison between MNN-AECS and baselines across 7 devices and 5 models. 64 256 1024 0 200 400 600 energy (J) Mate 40 Pro 64 256 1024 0 200 400 V30 Pro 64 256 1024 0 200 400 Galaxy A56 64 256 1024 0 200 400 Meizu 21 64 256 1024 0 100…
Figure 9
Figure 9. Figure 9: MNN-AECS energy reduction over MNN un￾der different (a) prompt and (b) decode length. For decode speed, MNN-AECS only slows down for 6% on V30 Pro and speeds up 0 to 20% on all the rest, thanks to MNN￾AECS’s core affinity and less cores which contributes to less conges…
Figure 10
Figure 10. Figure 10: MNN-AECS decode speed compared to MNN under different (a) prompt and (b) decode length [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: (a) Energy and (b) decode speed compari [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: Energy and decode speed in dataset experiments, geometrically averaged across 5 LLMs on 7 devices. [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 13
Figure 13. Figure 13: (a) Energy and (b) decode speed compari [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Is Your NPU Ready for LLMs? Dissecting the Hidden Efficiency Bottlenecks in Mobile LLM Inference

    cs.AR 2026-07 conditional novelty 7.0 of 10

    Cross-layer measurements of five mobile LLM frameworks on CPU/GPU/NPU reveal amplified NPU framework gaps, a prefill–decode backend phase split, and up to ~55% NPU energy savings from scheduling fixes.

Reference graph

Works this paper leans on

55 extracted references · 38 canonical work pages · cited by 1 Pith paper

  1. [1]

    [n. d.]. ARM idle states binding description — ARM Linux Kernel doc- umentation. https://www.kernel.org/doc/Documentation/devicetree/ bindings/arm/idle-states.txt. [Accessed 03-05-2025]

    work page 2025

  2. [2]

    [n. d.]. Device Frequency Scaling — The Linux Kernel documentation — docs.kernel.org. https://docs.kernel.org/driver-api/devfreq.html. [Ac- cessed 03-05-2025]

    work page 2025

  3. [3]

    GitHub - doronz88/pymobiledevice3: Pure python3 imple- mentation for working with iDevices (iPhone, etc...)

    2013-2025. GitHub - doronz88/pymobiledevice3: Pure python3 imple- mentation for working with iDevices (iPhone, etc...). — github.com. https://github.com/doronz88/pymobiledevice3. [Accessed 09-05-2025]

    work page 2013

  4. [4]

    kernel/sched/cpufreq_schedutil.c

    2022. kernel/sched/cpufreq_schedutil.c. https://android.googlesource. com/kernel/msm/+/refs/tags/android-12.1.0_r0.32/kernel/sched/ cpufreq_schedutil.c. [Accessed 03-05-2025]

    work page 2022

  5. [5]

    kernel/sched/sched.h

    2022. kernel/sched/sched.h. https://android.googlesource.com/kernel/ msm/+/refs/tags/android-12.1.0_r0.32/kernel/sched/sched.h. [Ac- cessed 03-05-2025]

    work page 2022

  6. [6]

    kernel/sched/walt.c

    2022. kernel/sched/walt.c. https://android.googlesource.com/kernel/ msm/+/refs/tags/android-12.1.0_r0.32/kernel/sched/walt.c. [Accessed 03-05-2025]

    work page 2022

  7. [7]

    Llama 3.2: Revolutionizing edge AI and vision with open, cus- tomizable models — ai.meta.com

    2024. Llama 3.2: Revolutionizing edge AI and vision with open, cus- tomizable models — ai.meta.com. https://ai.meta.com/blog/llama-3-2- connect-2024-vision-edge-mobile-devices/. [Accessed 19-04-2025]

    work page 2024

  8. [8]

    GitHub - pytorch/executorch: On-device AI across mobile, embedded and edge for PyTorch — github.com

    2025. GitHub - pytorch/executorch: On-device AI across mobile, embedded and edge for PyTorch — github.com. https://github.com/ pytorch/executorch. [Accessed 21-04-2025]

    work page 2025

Show all 55 references
  1. [9]

    Gulavani, and Ramachandran Ramjee

    Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhar- gav S. Gulavani, and Ramachandran Ramjee. 2023. SARATHI: Effi- cient LLM Inference by Piggybacking Decodes with Chunked Prefills. arXiv:2308.16369 [cs.LG] https://arxiv.org/abs/2308.16369

    2023 arXiv

  2. [10]

    Karen Khatam- ifard, Minsik Cho, Carlo C Del Mundo, Mohammad Rastegari, and Mehrdad Farajtabar

    Keivan Alizadeh, Iman Mirzadeh, Dmitry Belenko, S. Karen Khatam- ifard, Minsik Cho, Carlo C Del Mundo, Mohammad Rastegari, and Mehrdad Farajtabar. 2024. LLM in a Flash: Efficient Large Language Model Inference with Limited Memory. InACL. https://arxiv.org/pdf/ 2312.11514

    2024 arXiv

  3. [11]

    Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019. MathQA: Towards Inter- pretable Math Word Problem Solving with Operation-Based For- malisms. arXiv:1905.13319 [cs.CL] https://arxiv.org/abs/1905.13319

    2019 arXiv

  4. [12]

    Apple. 2024. Introducing Apple’s On-Device and Server Founda- tion Models. https://machinelearning.apple.com/research/introducing- apple-foundation-models. [Accessed 11-05-2025]

    2024

  5. [13]

    Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Meghan Cowan, Haichen Shen, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. TVM: an automated end-to-end optimizing compiler for deep learning. InProceedings of the 13th USE...

    2018

  6. [14]

    Xinglei Dou, Lei Liu, and Limin Xiao. 2025. An Intelligent Scheduling Approach on Mobile OS for Optimizing UI Smoothness and Power. ACM Trans. Archit. Code Optim.22, 1, Article 12 (March 2025), 27 pages. https://doi.org/10.1145/3674910

    2025 doi

  7. [15]

    Google AI Edge. 2025. LiteRT overview — ai.google.dev. https://ai. google.dev/edge/litert. [Accessed 19-04-2025]

    2025

  8. [16]

    Google AI Edge. 2025. MediaPipe Solutions guide — ai.google.dev. https://ai.google.dev/edge/mediapipe/solutions/guide. [Accessed 19- 04-2025]. 13 Conference’17, July 2017, Washington, DC, USA Huang et al

    2025

  9. [17]

    Shiwei Gao, Youmin Chen, and Jiwu Shu. 2025. Fast State Restora- tion in LLM Serving with HCache. InProceedings of the Twentieth European Conference on Computer Systems(Rotterdam, Netherlands) (EuroSys ’25). Association for Computing Machinery, New York, NY, USA, 128–143. http...

    2025

  10. [18]

    Georgi Gerganov. 2024. GGUF. https://github.com/ggml-org/ggml/ blob/master/docs/gguf.md. [Accessed 11-05-2025]

    2024

  11. [19]

    Georgi Gerganov. 2025. GitHub - ggml-org/llama.cpp: LLM infer- ence in C/C++ — github.com. https://github.com/ggml-org/llama.cpp. [Accessed 19-04-2025]

    2025

  12. [20]

    Google. 2025. Gemini Nano with the Google AI Edge SDK | Android Developers — developer.android.com. https://developer.android.com/ ai/gemini-nano. [Accessed 11-05-2025]

    2025

  13. [21]

    Huawei. 2025. Product List | HUAWEI Support Global — consumer.huawei.com. https://consumer.huawei.com/en/support/ product/. [Accessed 03-05-2025]

    2025

  14. [22]

    Apple Inc. 2024. Apple Intelligence. https://www.apple.com/apple- intelligence/. [Accessed 03-05-2025]

    2024

  15. [23]

    Apple Inc. 2025. iPhone — apple.com. https://www.apple.com/iphone/. [Accessed 03-05-2025]

    2025

  16. [24]

    Qualcomm Technologies Inc. 2023. Qualcomm Snapdragon Mo- bile Platform OpenCL General Programming and Optimization. https://docs.qualcomm.com/bundle/publicresource/80-NB295- 11_REV_C_Qualcomm_Snapdragon_Mobile_Platform_Opencl_ General_Programming_and_Optimization.pdf. [Accesse...

    2023

  17. [25]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Sto- ica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. InProceedings of the 29th Symposium on Operating Systems Princi...

    2023

  18. [26]

    Arm Limited. 2020. The Bifrost Shader Core Version 1.0 — de- veloper.arm.com. https://developer.arm.com/documentation/102546/ 0100/?lang=en. [Accessed 03-05-2025]

    2020

  19. [27]

    Arm Limited. 2020. Principles of High Performance guide Version 1.0 — developer.arm.com. https://developer.arm.com/documentation/ 102544/0100/?lang=en. [Accessed 03-05-2025]

    2020

  20. [28]

    Chengdong Lin, Kun Wang, Zhenjiang Li, and Yu Pu. 2023. A Workload- Aware DVFS Robust to Concurrent Tasks for Mobile Devices. InPro- ceedings of the 29th Annual International Conference on Mobile Comput- ing and Networking(Madrid, Spain)(ACM MobiCom ’23). Association for Compu...

    2023

  21. [29]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. AWQ: Activation-aware Weight Quantization for On- Device LLM Compression and Acceleration. InProceedings of Machine Learning and Systems, ...

    2024

  22. [30]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Truth- fulQA: Measuring How Models Mimic Human Falsehoods. arXiv:2109.07958 [cs.CL] https://arxiv.org/abs/2109.07958

    2022 arXiv

  23. [31]

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. 2025. SpinQuant: LLM Quantiza- tion with Learned Rotations. InThe Thirteenth International Confer- ence on Learning Represe...

    2025

  24. [32]

    Arm Ltd. 2025. big.LITTLE: Balancing Power Efficiency and Perfor- mance — arm.com. https://www.arm.com/technologies/big-little. [Ac- cessed 03-05-2025]

    2025

  25. [33]

    Chengfei Lv, Chaoyue Niu, Renjie Gu, Xiaotang Jiang, Zhaode Wang, Bin Liu, Ziqi Wu, Qiulin Yao, Congyu Huang, Panos Huang, Tao Huang, Hui Shu, Jinde Song, Bin Zou, Peng Lan, Guohuan Xu, Fei Wu, Shaojie Tang, Fan Wu, and Guihai Chen. 2022. Walle: An End-to-End, General- Purpose...

    2022

  26. [34]

    Meizu. 2025. AI Phones - Meizu Global — meizu.com. https://www. meizu.com/global/product-list/phones. [Accessed 03-05-2025]

    2025

  27. [35]

    2023-2025.MLC-LLM

    MLC team. 2023-2025.MLC-LLM. https://github.com/mlc-ai/mlc-llm

    2023

  28. [36]

    Wei Niu, Md Musfiqur Rahman Sanim, Zhihao Shu, Jiexiong Guan, Xipeng Shen, Miao Yin, Gagan Agrawal, and Bin Ren. 2024. Smart- Mem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobile. InProceedings of the 29th ACM Interna- tional Conference o...

    2024

  29. [37]

    Tom Olson. 2013. How low can you go? Building low-power, low-bandwidth ARM Mali GPUs — community.arm.com. https://community.arm.com/arm-community-blogs/b/mobile- graphics-and-gaming-blog/posts/how-low-can-you-go-building- low-power-low-bandwidth-arm-mali-gpus. [Accessed 22-04-2025]

    2013

  30. [38]

    Samsung. 2025. All New Samsung Mobiles Prices & models | Sam- sung Jordan — samsung.com. https://www.samsung.com/levant/ smartphones/all-smartphones/. [Accessed 03-05-2025]

    2025

  31. [39]

    shareAI. 2023. ShareGPT-Chinese-English-90k Bilingual Human- Machine QA Dataset. https://huggingface.co/datasets/shareAI/ ShareGPT-Chinese-English-90k

    2023

  32. [40]

    Gemma Team. 2024. Gemma 2: Improving Open Language Models at a Practical Size. arXiv:2408.00118 [cs.CL] https://arxiv.org/abs/2408. 00118

    2024 arXiv

  33. [41]

    Llama Team. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https://arxiv.org/abs/2407.21783

    2024 arXiv

  34. [42]

    Qwen Team. 2025. Qwen2.5 Technical Report. arXiv:2412.15115 [cs.CL] https://arxiv.org/abs/2412.15115

    2025 arXiv

  35. [43]

    Apple Device Support Tutorial. 2024. Using Sysdiagnose to Trou- bleshoot iOS or iPadOS. https://it-training.apple.com/tutorials/ support/sup075/. [Accessed 09-05-2025]

    2024

  36. [44]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. InAdvances in Neural Information Pro- cessing Systems, I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. ...

    2017

  37. [45]

    Manni Wang, Shaohua Ding, Ting Cao, Yunxin Liu, and Fengyuan Xu. 2021. AsyMo: scalable and efficient deep-learning inference on asymmetric mobile CPUs. InProceedings of the 27th Annual Interna- tional Conference on Mobile Computing and Networking(New Orleans, Louisiana)(MobiCo...

    2021

  38. [46]

    Zekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhu Chen, Jie Fu, and Junran Peng. 2023. RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing...

    2023 arXiv

  39. [47]

    Rafael J. Wysocki. [n. d.]. CPU Performance Scaling — The Linux Ker- nel documentation — docs.kernel.org. https://docs.kernel.org/admin- guide/pm/cpufreq.html. [Accessed 03-05-2025]

    2025

  40. [48]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. SmoothQuant: accurate and efficient post-training quantization for large language models. InProceedings of the 40th International Conference on Machine Learning(Honolulu, Hawaii, USA) (ICML’23)...

    2023

  41. [49]

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. Efficient Streaming Language Models with Attention Sinks.ICLR(2024)

    2024

  42. [50]

    Xiaomi. 2025. Xiaomi Global Home — mi.com. https://www.mi.com/ global/product-list/phone/. [Accessed 03-05-2025]

    2025

  43. [51]

    Daliang Xu, Hao Zhang, Liming Yang, Ruiqi Liu, Gang Huang, Meng- wei Xu, and Xuanzhe Liu. 2025. Fast On-device LLM Inference with NPUs. InProceedings of the 30th ACM International Confer- ence on Architectural Support for Programming Languages and Op- erating Systems, Volume 1...

    2025

  44. [52]

    Zhenliang Xue, Yixin Song, Zeyu Mi, Xinrui Zheng, Yubin Xia, and Haibo Chen. 2024. PowerInfer-2: Fast Large Language Model Inference on a Smartphone. arXiv:2406.06282 [cs.LG] https://arxiv.org/abs/2406. 06282

    2024 arXiv

  45. [53]

    Rongjie Yi, Liwei Guo, Shiyun Wei, Ao Zhou, Shangguang Wang, and Mengwei Xu. 5555. EdgeMoE: Empowering Sparse Large Language Models on Mobile Devices .IEEE Transactions on Mobile Computing 01 (Feb. 5555), 1–16. https://doi.org/10.1109/TMC.2025.3546466

  46. [54]

    Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. 2022. Orca: A Distributed Serving System for Transformer-Based Generative Models. In16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). USENIX Asso- ciation, Carlsbad, CA...

    2022

  47. [55]

    Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xu- anzhe Liu, Xin Jin, and Hao Zhang. 2024. DistServe: disaggregating prefill and decoding for goodput-optimized large language model serv- ing. InProceedings of the 18th USENIX Conference on Operating Systems Design...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.