Pith. sign in

REVIEW 3 major objections 7 minor 45 references

On edge devices, VLM energy is set by how many tokens the model generates, not by what it sees.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 02:25 UTC pith:7T4G4SUJ

load-bearing objection Solid first energy profile of edge VLMs: power is basically a model constant, decode is 86–97% of energy, and the pruning-vs-output ranking is real under fixed Nout—but that fixed-decode assumption is the load-bearing soft spot. the 3 major comments →

arxiv 2607.09520 v1 pith:7T4G4SUJ submitted 2026-07-10 cs.CV cs.AI

Seeing is Free, Speaking is Not: Uncovering the True Energy Bottleneck in Edge VLM Inference

Junfei Zhan , Haoxun Shen , Mingang Guo , Zixuan Huang , Tengjiao He This is my paper
classification cs.CV cs.AI
keywords Vision-Language ModelsEnergy ProfilingEdge InferenceOn-Device AIEmbodied AIPower MeasurementPrefill-Decode AsymmetryVisual Token Pruning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Vision-language models are the eyes of robots and other edge agents, but their battery cost has been poorly measured. Most efficiency work prunes visual tokens, as if looking at the image were the expensive step. This paper profiles five models from three architecture families, at four resolutions, on a laptop GPU and a Jetson Orin NX, and finds the opposite pattern. Average power draw is almost a model fingerprint: it barely moves with resolution, scene complexity, or prompt. Energy therefore tracks wall-clock time, and time is dominated by the decode phase that emits output tokens one by one. Each output token costs roughly 11 to 39 times more wall time than each input token, so output length explains nearly all energy variation—including the up-to-4.1× gap between simple and cluttered scenes at fixed resolution. Even deleting every visual token can save at most about 10% of energy for fixed-token models, while controlling how much the model is allowed to say can save up to about 97%. The practical message is that edge VLM energy budgets should be spent on output length, not only on vision-side compression.

Core claim

Average inference power is model-intrinsic (under 5% variation across resolution, content, and prompt), so energy equals constant power times latency. Prefill is compute-bound and cheap per token; decode is memory-bound and 11–39× more expensive per token, accounting for 86–97% of total energy. Image complexity changes energy only by changing how much the model says. Therefore visual-token pruning is capped at a small fraction of total energy under fixed decoding behavior, while output-length control is the dominant lever.

What carries the argument

The constant-power identity E = P̄ × t, combined with the two-phase latency model t ≈ α_p N_in + α_d N_out + β where α_d / α_p is 11–39×. This reduces energy analysis to counting tokens and yields the visual-pruning upper bound ΔE/E ≤ η · (E_prefill / E_total).

Load-bearing premise

The claim that pruning visual tokens can save only a small share of energy assumes that removing those tokens does not systematically change how long the model’s answer is.

What would settle it

Measure total energy and output length for the same images before and after aggressive visual-token pruning (or zero visual tokens); if energy falls far more than the prefill share because answers shorten, or falls far less because answers lengthen, the fixed-decoding upper bound and the “speaking not seeing” ranking fail.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Setting max_tokens is a first-order energy control on edge VLMs; halving a typical output budget can save on the order of 40–50% energy.
  • Fixed-token vision architectures avoid the resolution-dependent prefill penalty that dynamic-token models incur at high resolution.
  • Battery budgeting for continuous perception-action loops should use worst-case max_tokens, not average caption length.
  • At larger model scale the decode share of energy grows, so output-side methods become relatively more valuable than vision pruning.
  • A five-feature linear model of size, input tokens, output tokens, and two size×token interactions predicts energy with ~98.6% R² without per-model calibration.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Energy-aware VLA agents could treat remaining battery as a hard constraint on max_tokens and route hard scenes to shorter templates.
  • Speculative decoding and early-exit generation may deliver larger edge-energy wins than further visual-token compressors, once measured in joules.
  • If pruning methods also shorten captions by removing detail the model would have verbalized, their real energy savings could exceed the paper’s fixed-decode bound—worth re-measuring jointly.
  • Content-driven output length is a cheap proxy for scene complexity; predicting N_out from image statistics before inference could enable proactive energy scheduling.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper presents a systematic energy profiling study of on-device VLM inference across five models (1B–4B, with additional 7B/8B checks), four resolutions, and two edge platforms (RTX 3070 and Jetson Orin NX). It reports three main empirical findings: (i) average inference power is effectively a model-intrinsic constant (CV < 5%) across resolution, content, and prompt; (ii) decode is 11–39× more expensive per token than prefill, so output length dominates latency and energy (decode 86–97% of energy in typical settings); (iii) image complexity drives large energy variation (up to ~4.1×) via longer outputs, not via visual compute. From this the authors argue that visual-token pruning has a low energy ceiling (≤~10% for fixed-token models under fixed decoding), while controlling output length can save up to ~97%, and they fit a simple cross-model energy predictor (R² ≈ 0.986).

Significance. If the measurements hold under the stated serving regime, the paper is a useful corrective for edge VLM efficiency work that has focused almost exclusively on visual tokens and FLOPs. The power-fingerprint result, the prefill/decode energy split, and the demonstration that content-driven energy variance is largely an output-length effect are concrete, actionable, and well aligned with embodied/edge deployment constraints. Strengths include multi-model and two-platform coverage, transparent latency fits (α_d/α_p), an explicit pruning energy bound, and a low-dimensional universal predictor validated on a large run set. These are genuine contributions even if some intervention comparisons remain partly theoretical.

major comments (3)
  1. §5.2, Eq. (11), Table 5, and the abstract: the headline claim that removing all visual tokens saves at most ~10% (fixed-token) while output control saves up to ~97% is derived under fixed decoding behavior (N_out held constant). The manuscript states this proviso in the body, but the abstract/intro present the ranking as an empirical energy fact about pruning. Without at least one real pruning/zero-vision run that reports both energy and realized N_out (and preferably quality), the intervention ranking can shift if pruning systematically shortens or lengthens answers. Please either (a) measure energy under a standard pruning method or a no-vision ablation, or (b) reframe abstract/Table 5 so the 10% figure is clearly a prefill-share upper bound, not measured pruning savings.
  2. §3 power methodology and cross-platform aggregation: laptop power is HWiNFO64 total system power, while Jetson uses VDD_IN. Absolute watts and the power–size fit (Eq. 4) are therefore not strictly comparable across platforms, and total-system power can include non-inference components. Relative constancy (CV < 5%) can still support E ≈ P̄ × t within a platform, but claims that treat P̄ as a clean model fingerprint and pool energy decompositions should state the rail difference, report idle/baseline subtraction if any, and avoid over-interpreting absolute joules across devices.
  3. Scope of the serving stack (§3, §7): all primary results use batch-1 greedy decoding via llama.cpp. Decode dominance is expected to be strongest in this regime; batched or vendor-optimized engines can raise prefill arithmetic intensity and change KV-cache traffic. The conclusion notes batch size one as a limit, but the central “speaking not seeing” claim is marketed more broadly. Add a short sensitivity discussion (or one alternate backend/batch setting if feasible) clarifying which quantitative ratios are expected to move under concurrent serving.
minor comments (7)
  1. Abstract vs §5.2: insert the fixed-decoding qualifier next to the “at most 10%” pruning statement so the abstract matches the derivation.
  2. Table 1 / token counts: InternVL3 text says 265 tokens after pixel-shuffle, while §4.3 table text sometimes says 259; reconcile reported visual token counts.
  3. Figure 1 caption and body: energy percentages (86–97%) are easy to misread as universal; note resolution/prompt conditions (e.g., Describe @ 448) in the caption.
  4. §4.2 estimation of α_p for fixed-token models via multimodal vs text-only intercepts is reasonable but underspecified; state sample sizes and whether system prompts match.
  5. Table 5 includes 7B/8B short-answer savings; briefly state whether those runs used the same measurement pipeline and hardware as the 1–4B Jetson results.
  6. Related work is solid; a sentence contrasting FLOPs-reported pruning papers with joule accounting would help non-systems readers.
  7. Minor polish: “mm_proj” notation, occasional spacing in “4 .1×” / “76 .8”, and ACM reference placeholders should be cleaned for camera-ready.

Circularity Check

0 steps flagged

Empirical energy profiling study; headline ratios and pruning bound come from measurements and a fixed-Nout upper bound, not from self-definitional or fitted-as-prediction circularity.

full rationale

The paper’s load-bearing chain is measurement-first, not derivation-from-fitted-target. Average power is reported as an observed model fingerprint (CV < 5% across resolution, content, and prompt; §3), so E = P̄ × t is the physical product of measured power and measured wall time, not a tautology that defines the headline claims. Prefill/decode asymmetry (α_d/α_p ∈ [11, 39]×; Table 2) and decode energy share (86–97%) are fitted or timed from inference runs and phase timers, then used to explain energy; they are not redefined as the quantities they “predict.” The visual-token pruning ceiling (§5.2, Eq. 11, Table 5) is an explicit upper bound ΔE/E ≤ η·E_prefill/E_total under fixed decoding behavior—i.e., a logical consequence of the measured energy split with N_out held constant—not a circular claim that pruning savings equal the prefill fraction by redefining either side. Short-answer prompt ablations measure output-side savings independently. The per-model and universal energy predictors (§5.3) are ordinary linear fits validated with MAPE/R² and leave-one-resolution-out CV; they support the low-dimensional decomposition but do not force the central “speaking not seeing” ranking by construction. No self-citation uniqueness theorem, ansatz smuggled via overlapping authors, or renaming of a known result as a forced derivation underpins the claims. Fixed-Nout when pruning is a scope/assumption risk for the bound’s applicability, not circularity of the derivation chain.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 2 invented entities

The central claim rests on measured energy/time/power under a standard two-phase transformer inference model, not on new physics. Free parameters are regression coefficients used for prediction and ratio reporting; axioms are standard systems assumptions (constant average power under saturation, roofline prefill/decode asymmetry, E=P×t). Named constructs (power fingerprint, OLS) label measurements rather than postulate new entities. The pruning ceiling additionally assumes fixed decode length when visual tokens change.

free parameters (4)
  • Per-model latency coefficients α_p, α_d, β
    Fitted from wall-clock vs token counts (Table 2); ratios α_d/α_p are empirical, not first-principles constants.
  • Power–size linear fit P̄=12.1S+42.2
    Regression across five models on RTX 3070 (R²=0.918); slope/intercept are data-fit, not universal constants.
  • Universal energy predictor weights w0…w5
    Single linear model with size×token interactions fit to 1,680 runs; used for MAPE/R² claims, not for defining the bottleneck.
  • Output Length Sensitivity OLS=P̄·α_d
    Composite of fitted power fingerprint and fitted decode cost; reported as J/token for interpretation.
axioms (5)
  • domain assumption Average inference power is effectively constant for a model–hardware pair under sequential llama.cpp serving, so energy variation reduces to time variation (E=P̄×t).
    Supported empirically in §3 but assumed when decomposing energy into prefill/decode joules and when bounding pruning savings.
  • domain assumption Autoregressive inference splits into compute-bound prefill and memory-bound decode (roofline / standard LLM systems model).
    §2.2 and §4.2; explains α_d≫α_p but is imported from prior systems literature.
  • ad hoc to paper Visual token pruning upper bound assumes fixed decoding behavior (output length unchanged by token removal).
    Explicit in §5.2 inequality ΔE/E ≤ η·E_prefill/E_total; load-bearing for “pruning illusion” headline.
  • domain assumption Image complexity can be ordered by COCO object-count tiers for content ablations.
    §3.2 / §4.3; proxy may not capture all drivers of caption length.
  • standard math Linear wall-clock model t≈α_p N_in+α_d N_out+β adequately captures phase costs.
    Definition 4.1 and Eq. 7; standard first-order approximation used throughout.
invented entities (2)
  • Power fingerprint (P̄) no independent evidence
    purpose: Name the observed model-intrinsic average power used to reduce energy analysis to latency.
    Not a new physical entity; a label for measured constancy. independent_evidence false as a novel object, but the constancy is multi-condition empirical.
  • Output Length Sensitivity (OLS) no independent evidence
    purpose: Marginal energy per generated token (P̄·α_d) for deployment intuition.
    Derived metric from measured power and fitted decode cost; not an independent ontological claim.

pith-pipeline@v1.1.0-grok45 · 22397 in / 3568 out tokens · 40407 ms · 2026-07-13T02:25:18.439136+00:00 · methodology

0 comments
read the original abstract

Vision-Language Models (VLMs) are the perceptual backbone of embodied AI, but their energy footprint on edge hardware remains poorly understood. Existing efficiency efforts focus predominantly on reducing visual tokens, implicitly treating visual processing as the dominant energy cost. We overturn this implicit assumption through the first systematic energy profiling of on-device VLM inference, spanning five models across three architecture families, four input resolutions, and two hardware platforms (NVIDIA RTX 3070 and Jetson Orin NX). Our analysis yields three findings. First, average inference power is a model-intrinsic constant, invariant to input resolution, image complexity, and prompt type, with less than 5% variation across all conditions. This means that all energy variation across inputs must arise from variation in inference time, not from variation in power draw. Second, each output token costs 11 to 39x more wall-clock time than each input token due to the compute-bound and memory-bound asymmetry between prefill and decode, making output token count the dominant driver of both latency and energy. Third, image complexity, measured by the number of objects in an image, induces up to 4.1x energy differences at identical resolution. This variation arises not from increased visual processing cost, but from differences in output length. These findings expose a fundamental limitation of visual token pruning: even removing all visual tokens saves at most 10% of total energy for fixed-token models. Across models spanning 1 billion to 8 billion parameters, controlling output length saves up to 97% of total energy, with the energy dominance of decoding growing stronger at larger model scale. In short, the true energy bottleneck in edge VLM inference is not what the model sees, but how much it says.

Figures

Figures reproduced from arXiv: 2607.09520 by Haoxun Shen, Junfei Zhan, Mingang Guo, Tengjiao He, Zixuan Huang.

Figure 1
Figure 1. Figure 1: Energy anatomy of VLM inference on the RTX 3070 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: VLM inference pipeline. An input image and text prompt are encoded separately into visual and text tokens, concate [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Per-inference average power on a laptop GPU [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Per-inference average power on a Jetson Orin NX (15W mode). (a) Power across content complexity, from single-object [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Predicted vs. measured energy on the RTX 3070 [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 10 linked inside Pith

  1. [1]

    Maximilian Abstreiter, Sasu Tarkoma, and Roberto Morabito. 2025. Sometimes Painful but Certainly Promising: Feasibility and Trade-offs of Language Model Inference at the Edge.ACM Transactions on Embedded Computing Systems(2025). doi:10.1145/3788870

    doi:10.1145/3788870 2025

  2. [2]

    Saeed Ranjbar Alvar, Gursimran Singh, Mohammad Akbari, and Yong Zhang

  3. [3]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 9392–9401

  4. [4]

    Ander Alvarez, Alessandro Genuardi, Nilotpal Sinha, Antonio Tiene, Mikail Okyay, Bakbergen Ryskulov, David Montero, Samuel Mugel, and Román Orús

  5. [5]

    Scaling Laws for Energy Efficiency of Local LLMs.arXiv preprint arXiv:2512.16531(2025)

    arXiv 2025

  6. [6]

    Nikolopoulos, Hans Vandierendonck, Deepu John, and Bo Ji

    Kazi Hasan Ibn Arif, JinYi Yoon, Dimitrios S. Nikolopoulos, Hans Vandierendonck, Deepu John, and Bo Ji. 2025. HiRED: Attention-Guided Token Dropping for Efficient Inference of High-Resolution Vision-Language Models. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 1773–1781

    2025

  7. [7]

    Mayank Arya and Yogesh Simmhan. 2025. Understanding the Performance and Power of LLM Inferencing on Edge Accelerators. In7th Workshop on Parallel AI and Systems for the Edge (PAISE), co-located with IEEE IPDPS

    2025

  8. [8]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2.5-VL Technical Report.arXiv preprint arXiv:2502.13923(2025)

    Pith/arXiv arXiv 2025

  9. [9]

    2025.𝜋 0: A Vision-Language-Action Flow Model for General Robot Control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. 2025.𝜋 0: A Vision-Language-Action Flow Model for General Robot Control. InProceedings of Robotics: Science and Systems (RSS)

    2025

  10. [10]

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, et al . 2023. RT-2: Vision- Language-Action Models Transfer Web Knowledge to Robotic Control. InPro- ceedings of the 7th Conference on Robot Learning (CoRL)

    2023

  11. [11]

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. 2024. An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models. InProceedings of the European Conference on Computer Vision (ECCV). 19–35

    2024

  12. [12]

    Xiangxiang Chu, Limeng Qiao, Xinyu Zhang, Shuang Xu, Fei Wei, et al . 2024. MobileVLM V2: Faster and Stronger Baseline for Vision Language Model.arXiv preprint arXiv:2402.03766(2024)

    Pith/arXiv arXiv 2024

  13. [13]

    Ma, and Mosharaf Chowdhury

    Jae-Won Chung, Ruofan Wu, Jeff J. Ma, and Mosharaf Chowdhury. 2026. Where Do the Joules Go? Diagnosing Inference Energy Consumption.arXiv preprint arXiv:2601.22076(2026)

    arXiv 2026

  14. [14]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. InProceedings of the International Conference on Learning Repres...

    2021

  15. [15]

    Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al

  16. [16]

    InProceedings of the 40th International Conference on Machine Learning (ICML)

    PaLM-E: An Embodied Multimodal Language Model. InProceedings of the 40th International Conference on Machine Learning (ICML)

  17. [17]

    Zico Kolter, and An- drew Gordon Wilson

    Marc Finzi, Shikai Qiu, Yiding Jiang, Pavel Izmailov, J. Zico Kolter, and An- drew Gordon Wilson. 2026. From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence.arXiv preprint arXiv:2601.03220(2026)

    arXiv 2026

  18. [18]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieil- lard, Ramona Merhej, et al . 2025. Gemma 3 Technical Report.arXiv preprint arXiv:2503.19786(2025)

    Pith/arXiv arXiv 2025

  19. [19]

    Erik Johannes Husom, Arda Goknil, Mustafa Astekin, Lwin Khin Shar, et al. 2025. Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency.ACM Transactions on Internet of Things6, 4 (2025)

    2025

  20. [20]

    Hemang Jain, Shailender Goyal, Divyansh Pandey, and Karthik Vaidhyanathan

  21. [21]

    Dissecting Transformers: A CLEAR Perspective towards Green AI.arXiv preprint arXiv:2510.02810(2025)

    Pith/arXiv arXiv 2025

  22. [22]

    SiYoung Jang and Roberto Morabito. 2025. Edge-First Language Model Inference: Models, Metrics, and Tradeoffs. InProceedings of the 45th IEEE International Conference on Distributed Computing Systems (ICDCS)

    2025

  23. [23]

    Zhiwei Jin, Xiaohui Song, Nan Wang, Yafei Liu, Chao Li, et al. 2025. AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Model. arXiv preprint arXiv:2510.11496(2025)

    arXiv 2025

  24. [24]

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. 2024. OpenVLA: An Open-Source Vision-Language-Action Model. InProceedings of the 8th Conference on Robot Learning (CoRL)

    2024

  25. [25]

    Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C Lawrence Zitnick, and Piotr Dollár

  26. [26]

    InProceedings of the European Conference on Computer Vision (ECCV)

    Microsoft COCO: Common Objects in Context. InProceedings of the European Conference on Computer Vision (ECCV). 740–755

  27. [27]

    Yuchen Liu et al. 2025. HyperVL: An Efficient and Dynamic Multimodal Large Language Model for Edge Devices.arXiv preprint arXiv:2512.14052(2025)

    arXiv 2025

  28. [28]

    Yueen Ma, Zixing Song, Yuzheng Zhuang, Jianye Hao, and Irwin King. 2024. A Survey on Vision-Language-Action Models for Embodied AI.arXiv preprint arXiv:2405.14093(2024)

    Pith/arXiv arXiv 2024

  29. [29]

    Andrés Marafioti, Orr Zohar, Miquel Farré, Merve Noyan, Elie Bakouch, et al

  30. [30]

    SmolVLM: Redefining Small and Efficient Multimodal Models.arXiv preprint arXiv:2504.05299(2025)

    Pith/arXiv arXiv 2025

  31. [31]

    Roxy, and Md Rubel Ahmed

    Alyssa Pinnock, Shakya Jayakody, Kawsher A. Roxy, and Md Rubel Ahmed. 2025. EdgeProfiler: A Fast Profiling Framework for Lightweight LLMs on Edge Using Analytical Model.arXiv preprint arXiv:2506.09061(2025)

    arXiv 2025

  32. [32]

    Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Brad- bury, Anselm Levskaya, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. 2023. Efficiently Scaling Transformer Inference. InProceedings of Machine Learning and Systems (MLSys), Vol. 5. MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil Junfei Zhan et al., Junfei Zhan, H...

    2023

  33. [33]

    Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. 2025. LLaVA- PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 22857–22867

    2025

  34. [34]

    Kele Shao, Keda Tao, Kejia Zhang, Sicheng Feng, Mu Cai, Yuzhang Shang, Haoxuan You, Can Qin, Yang Sui, and Huan Wang. 2025. A Survey of Token Compression for Efficient Multimodal Large Language Models.arXiv preprint arXiv:2507.20198(2025)

    arXiv 2025

  35. [35]

    Pavan Kumar Anasosalu Vasu, Fartash Faghri, Chun-Liang Li, Cem Koc, Nate True, Albert Antony, Gokul Santhanam, James Gabriel, Peter Grasch, Oncel Tuzel, and Hadi Pouransari. 2025. FastVLM: Efficient Vision Encoding for Vision Language Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2025

  36. [36]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any Resolution.arXiv preprint arXiv:2409.12191(2024)

    Pith/arXiv arXiv 2024

  37. [37]

    Patterson

    Samuel Williams, Andrew Waterman, and David A. Patterson. 2009. Roofline: An Insightful Visual Performance Model for Multicore Architectures.Commun. ACM52, 4 (2009), 65–76. doi:10.1145/1498765.1498785

    doi:10.1145/1498765.1498785 2009

  38. [38]

    Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, et al. 2025. Pyra- midDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2025

  39. [39]

    Hao Xu, Long Peng, Shezheng Song, Xiaodong Liu, Ma Jun, Shasha Li, Jie Yu, and Xiaoguang Mao. 2025. Camel: Energy-Aware LLM Inference on Resource- Constrained Devices.arXiv preprint arXiv:2508.09173(2025)

    Pith/arXiv arXiv 2025

  40. [40]

    Senqiao Yang, Yukang Chen, Zhuotao Tian, et al. 2025. VisionZip: Longer is Better but Not Necessary in Vision Language Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2025

  41. [41]

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al . 2025. Efficient GPT-4V Level Multimodal Large Language Model for Deployment on Edge Devices.Nature Communications16, 1 (2025), 5509. doi:10.1038/s41467-025-61040-5

    doi:10.1038/s41467-025-61040-5 2025

  42. [42]

    Yuan Zhang, Chun-Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis A Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, et al

  43. [43]

    InProceedings of the 42nd International Conference on Machine Learning (ICML)

    SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference. InProceedings of the 42nd International Conference on Machine Learning (ICML)

  44. [44]

    Baichuan Zhou, Ying Hu, Xi Weng, Junlong Jia, Jie Luo, Xien Liu, Ji Wu, and Lei Huang. 2024. TinyLLaVA: A Framework of Small-scale Large Multimodal Models.arXiv preprint arXiv:2402.14289(2024)

    Pith/arXiv arXiv 2024

  45. [45]

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. 2025. InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models. arXiv preprint arXiv:2504.10479(2025)

    Pith/arXiv arXiv 2025