REVIEW 4 major objections 7 minor 70 references
Robot motion state can drive dynamic vector quantization and centroid reuse so VLA inference runs several times faster with only a small drop in task success.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-31 22:32 UTC pith:KR6FZ5LI
load-bearing objection Solid VLA co-design paper: motion-gated dual VQ plus centroid-reuse GEMM hardware is real novelty; headline speedups ride on a simple motion threshold that is only weakly stress-tested across action spaces. the 4 major comments →
A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inference
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
VQVLA shows that coupling motion-aware dual-precision vector quantization with a codebook–index GEMM that spatially aggregates and temporally reuses centroids, realized on a specialized accelerator, yields roughly 6.5×, 2.8×, 1.9×, 3.3×, and 4.3× speedup over A100, Dadu-Corki, LUT-DLA, CodeGEMM, and ShiftAddLLM with about 2.5% average success-rate loss on standard VLA benchmarks.
What carries the argument
MotionVQ plus merged-centroid vectorized GEMM: classify each step as execution or transition from motion magnitude D = √(A_x²+A_y²+A_z²) versus threshold T_d, load the matching codebook/index set, then replace dense multiplies by input sums for repeated centroids within a column and cached products for hot centroids across columns.
Load-bearing premise
A single scalar motion size compared with one fixed threshold is enough to tell when the robot can safely use the cheaper low-precision codebook without spoiling the task.
What would settle it
On held-out LIBERO or ManiSkill tasks, force low-precision (#Centroid=64) on every step whose true motion exceeds T_d=0.8 and measure whether success rate collapses far beyond the reported ~2.5% drop; if it does, the motion proxy does not justify dynamic precision.
If this is right
- Per-inference VLA latency can fall enough that end-to-end per-action times reach the 30–60 ms range reported for the evaluated models.
- Memory traffic for transformer weights can drop by roughly four-fifths when transition phases dominate, easing edge deployment.
- About half of the multiplies in the backbone can be removed by centroid reuse without reconstructing full-precision weights.
- The same stack can be stacked with action-chunking controllers such as Dadu-Corki for further end-to-end gains.
- Hardware need only store codebooks, indices, and a modest result cache rather than full FP weights and large all-pair LUTs.
Where Pith is reading between the lines
- If motion magnitude is only a weak proxy on some embodiments, richer state signals (gripper contact, visual proximity) could extend the same dual-codebook idea without changing the GEMM engine.
- Centroid-reuse PE arrays may transfer to other VQ’d transformers outside robotics wherever index heatmaps are similarly skewed.
- The reported GPU slowdown when running the algorithm without custom hardware implies that irregular index gather/compare, not just arithmetic volume, is the real barrier on general-purpose chips.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents VQVLA, an algorithm–hardware co-design for accelerating Vision-Language-Action model inference. The algorithmic component, MotionVQ, classifies each robot control step as "execution" or "transition" by comparing the scalar magnitude D=√(Ax²+Ay²+Az²) of the most recent action against a fixed threshold T_d=0.8, and selects between two offline-trained VQ configurations (VQ[256,2,256] vs. VQ[128,2,64]) accordingly. The compute component, merged-centroid vectorized GEMM, eliminates redundant multiplications by aggregating inputs that share a centroid within a column (spatial merging) and caching products for hot centroids reused across columns (temporal reusing). A custom accelerator realizing both is evaluated in a cycle-level simulator with Ramulator, synthesized at 28nm and scaled to 7nm. Reported results: 6.5×/2.8×/1.9×/3.3×/4.3× speedups over A100, Dadu-Corki, LUT-DLA, CodeGEMM, and ShiftAddLLM, 79.4% weight-memory-access reduction, ~2.5% average success-rate loss across five VLA models on LIBERO and ManiSkill, and 75.5× energy reduction vs. A100.
Significance. If the results hold, the contribution is meaningful: VLA inference latency is a recognized deployment bottleneck, and the paper offers (i) a concrete, checkable hypothesis about execution-state-dependent quantization sensitivity, (ii) a reuse-based VQ GEMM paradigm with ablations isolating memory, spatial, and temporal contributions (Fig. 19), (iii) design-space sweeps for cache size, adder-tree depth, PE ratio, and T_d, and (iv) a demonstration of orthogonality to the strongest prior baseline (VQVLA-Corki, 6.0× over Dadu-Corki). Evaluation spans five models and two simulators, and GPU baselines are run with official implementations, Tensor Cores, and FlashAttention 2. Weaknesses limiting significance: all architecture numbers come from the authors' own cycle-level simulator with 28nm→7nm scaling and re-implemented baselines, and no code/artifact release is mentioned, so independent verification is currently not possible.
major comments (4)
- [§4 and §7.4 (threshold T_d)] Both headline numbers depend on one classification mechanism — D=√(Ax²+Ay²+Az²) vs. a fixed T_d=0.8 — yet its generalization evidence is within-benchmark only: §7.4 calibrates T_d on 10% of each benchmark and tests on the remaining 90% of the same benchmark, same model, same action distribution. This is problematic because the five evaluated models use heterogeneous action parameterizations and normalizations (OpenVLA's quantile-normalized actions, RDT's diffusion-policy outputs, π0's flow-matching outputs, GR00T), so a threshold expressed in raw Cartesian action space has no a priori reason to transfer across models, or even across embodiments within ManiSkill. The 79.4% memory reduction is directly proportional to the transition-state fraction produced by this threshold, and the 2.5% success-rate drop depends on the classification being correct. The paper should either (a) demonstrate
- [§4 (State Prediction) vs. §7.1 (action chunking)] The state predictor uses "the output of the last VLA inference" to compute D, but §7.1 states that OpenVLA-OFT, RDT, π0, and GR00T use action chunks of lengths 8, 8, 5, and 16. The paper never specifies which chunk element (or aggregation) produces (Ax, Ay, Az) for the magnitude computation, nor when during chunk execution the state is re-evaluated. This is not a presentational gap: chunk-internal motion magnitude routinely straddles T_d (e.g., a fast approach followed by a slow grasp within one 8-step chunk), so the state label is ill-defined exactly at the phase boundary the method relies on. Please define the per-chunk state-assignment rule and report how sensitive the memory-savings and success-rate results are to that choice.
- [§7.2 / Fig. 14 (success-rate statistics)] The central accuracy claim ('negligible accuracy degradation', ~2.5% average success-rate reduction) is reported without rollout counts, number of seeds, or any variance estimate. In LIBERO/ManiSkill evaluation protocols, success-rate differences of 2–3% are frequently within run-to-run noise at typical rollout counts. Per-task bars in Fig. 14 appear to show drops larger than the 2.5% average for some model–task pairs, and these are not discussed. Please report the number of evaluation episodes per task, confidence intervals or standard deviations, and identify the worst-case per-task degradation, since the MotionVQ tradeoff is only acceptable if the degradation is statistically real and bounded.
- [§7.3 (baseline methodology)] Three fairness issues need clarification. (1) GPU baselines run at each model's default precision — FP16 for OpenVLA, FP32 for OpenVLA-OFT, BF16 for RDT/π0/GR00T — so the 6.5× speedup over A100 is an aggregate over incommensurate baselines; a like-for-like breakdown (e.g., VQVLA vs. FP16 A100 for all models, or per-model speedups) should be given. (2) Dadu-Corki and LUT-DLA are re-implemented in the authors' own cycle-level simulator; the fidelity of those re-implementations (validated against the originals' published numbers) is not established, and both the 2.8× and 1.9× claims rest on them. (3) Area/power are synthesized at 28nm and scaled to 7nm via [60]; given the 51.15 mm² total area and 19.28 W at 28nm, the scaled edge-deployment story deserves explicit numbers rather than a citation. Additionally, the per-action-latency comparison against Dadu-Corki (Fig. 18) normalizes by action
minor comments (7)
- [§3.2 / Fig. 5 (noise-injection motivation)] Fig. 5 validates tolerance to additive noise injected into actions per state, but the perturbation MotionVQ actually introduces is weight-VQ error, whose effect on actions is neither additive nor state-independent. Since Fig. 14 measures true success rates end-to-end this is not load-bearing, but the motivating experiment should either use the actual quantization error or be explicitly framed as a proxy.
- [§3.2 / Fig. 4 (single-task anecdote)] Fig. 4 presents the motion-magnitude vs. distance-to-object correlation for a single task with undefined axis units; a quantitative correlation over all benchmarks (with units and normalization stated) would better support the core insight.
- [§7.2] The sentence 'Such success rate reduction is acceptable, as verified in research papers [26]' cites the OpenVLA paper; [26] does not, to my knowledge, establish a 2.5% acceptability criterion. Please support or soften this claim.
- [§7.2 (VQ configuration)] Offline VQ training cost (k-means per weight group for two configurations per model) and the storage cost of carrying both codebook/index sets are not reported; both matter for the deployment story.
- [§7.3 (GPU-VQVLA result)] The 45.2% slowdown of GPU-VQVLA vs. GPU-A100 (Fig. 15) is an honest and useful result; it would benefit from a short breakdown of where the overhead goes (index traversal vs. kernel launch).
- [Table 1 (area breakdown)] Combined on-chip SRAM (7MB IPU + 1.5MB result cache + 5.6MB buffer) is large relative to the stated edge-robotics motivation; the spill-to-off-chip fallback mentioned in §7.3 deserves quantification.
- [Throughout (typos)] Several typos and spacing issues: 'sinceexisting VQ approaches' (§2.2), 'insight:the sensitivity' (§1), '6.5×, 2.8×,1.9×,3.3×' (Abstract), 'distribution law of multiplication' should be 'distributive law' (§5.1).
Circularity Check
Empirical systems co-design paper; speedups and success rates are measured, not derived identities.
full rationale
VQVLA’s load-bearing claims are hardware/algorithm measurements against external baselines (A100, Dadu-Corki, LUT-DLA, CodeGEMM, ShiftAddLLM) and task success rates on LIBERO/ManiSkill. MotionVQ selects high- vs low-precision codebooks from D=√(A_x²+A_y²+A_z²) vs T_d; merged-centroid GEMM rewrites multiplies via spatial aggregation and temporal centroid reuse. Neither step equates a claimed prediction to its fitted input by construction: T_d, cache size, adder-tree depth, and PE ratio are ordinary design knobs explored in §7.4 and then held fixed while reporting measured latency, energy, PE utilization, and success rate. There is no self-definitional loop, no uniqueness theorem imported from overlapping authors, and no renaming of a known closed-form result as a derivation. Calibration of T_d on a 10% within-benchmark split is standard engineering practice, not circularity under the stated patterns. Central results remain externally falsifiable measurements.
Axiom & Free-Parameter Ledger
free parameters (7)
- motion threshold T_d =
0.8
- high-precision VQ config =
VQ[256, 2, 256]
- low-precision VQ config =
VQ[128, 2, 64]
- result cache capacity =
1.5 MB
- adder-tree stage count =
3
- spatial-to-temporal PE ratio =
1:1
- Top-k hot centroids for temporal reuse =
not numerically fixed in text
axioms (6)
- domain assumption Robot execution phases near targets have small motion magnitude and higher sensitivity to action noise/quantization than large-motion transition phases.
- domain assumption After VQ, weight indices exhibit enough repeated centroids within and across columns that spatial input aggregation and temporal product reuse reduce multiplies without changing GEMM semantics.
- standard math ∑(I_i × Centroid) = (∑ I_i) × Centroid for identical centroids (distributivity), so spatial merging is exact.
- domain assumption Transformer backbone dominates VLA latency (~80.6% on A100), so accelerating VQ-GEMM there dominates end-to-end gains.
- ad hoc to paper Cycle-level simulator + Ramulator + 28nm→7nm scaling faithfully ranks VQVLA against A100 and re-implemented accelerators.
- domain assumption ~2.5% average success-rate drop is an acceptable accuracy cost for VLA deployment.
invented entities (3)
-
MotionVQ dual-precision state policy
no independent evidence
-
Merged-centroid vectorized GEMM (spatial merging + temporal reusing)
no independent evidence
-
Spatial/temporal centroid location tables and dual PE arrays with result cache
no independent evidence
read the original abstract
Vision-Language-Action (VLA) models have demonstrated strong potential for embodied AI, yet their high inference latency on GPUs limits real-time deployment. Existing accelerators, such as Dadu-Corki, improve efficiency but treat VLA models as full-precision workloads, leaving substantial redundancy in both memory and computation underexploited. In this paper, we propose VQVLA, an algorithm-hardware co-design framework that accelerates VLA inference by exploiting weight similarity and execution dynamics. We first introduce MotionVQ, a motion-aware vector quantization scheme that dynamically adjusts quantization precision based on the robot's execution state, reducing memory access while preserving task success rate. We then propose a merged-centroid vectorized GEMM paradigm that operates on the codebook-index representation, eliminating redundant multiplications through spatial aggregation and temporal reuse of centroids. To realize these optimizations, we design an accelerator that efficiently supports dynamic precision selection and centroid-reuse computation. Experimental results show that VQVLA achieves 6.5x, 2.8x, 1.9x, 3.3x, and 4.3x speedup over the A100 GPU, Dadu-Corki, LUT-DLA, CodeGEMM, and ShiftAddLLM, respectively, with negligible accuracy degradation.
Figures
Reference graph
Works this paper leans on
-
[1]
Binarybert: Pushing the limit of bert quantization.arXiv preprint arXiv:2012.15701, 2020
Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jing Jin, Xin Jiang, Qun Liu, Michael Lyu, and Irwin King. Binarybert: Pushing the limit of bert quantization.arXiv preprint arXiv:2012.15701, 2020
Pith/arXiv arXiv 2012
-
[2]
Racod: algorithm/hardware co-design for mobile robot path planning
Mohammad Bakhshalipour, Seyed Borna Ehsani, Mohamad Qadri, Dominic Guri, Maxim Likhachev, and Phillip B Gibbons. Racod: algorithm/hardware co-design for mobile robot path planning. InProceedings of the 49th Annual International Symposium on Computer Architecture, pages 597–609, 2022
2022
-
[3]
Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025
Pith/arXiv arXiv 2025
-
[4]
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. 𝜋0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024
Pith/arXiv arXiv 2024
-
[5]
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control.arXiv preprint arXiv:2307.15818, 2023
Pith/arXiv arXiv 2023
-
[6]
Yuzong Chen, Chao Fang, Xilai Dai, Yuheng Wu, Thierry Tambe, Marian Verhelst, and Mohamed S Abdelfattah. P3-llm: An integrated npu-pim accelerator for llm inference using hybrid numerical formats.arXiv preprint arXiv:2511.06838, 2025
Pith/arXiv arXiv 2025
-
[7]
Xnor neural engine: A hardware accelerator ip for 21.6-fj/op binary neural network inference
Francesco Conti, Pasquale Davide Schiavone, and Luca Benini. Xnor neural engine: A hardware accelerator ip for 21.6-fj/op binary neural network inference. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 37(11):2940–2951, 2018
2018
-
[8]
Vage Egiazarian, Andrei Panferov, Denis Kuznedelev, Elias Frantar, Artem Babenko, and Dan Alistarh. Extreme compression of large language models via additive quantization.arXiv preprint arXiv:2401.06118, 2024
Pith/arXiv arXiv 2024
-
[9]
Hardware implemen- tation of slam algorithms: a survey on implementation approaches and platforms
Reza Eyvazpour, Maryam Shoaran, and Ghader Karimian. Hardware implemen- tation of slam algorithms: a survey on implementation approaches and platforms. Artificial Intelligence Review, 56(7):6187–6239, 2023
2023
-
[10]
Foundation models in robotics: Applications, challenges, and the future.The International Journal of Robotics Research, page 02783649241281508, 2023
Roya Firoozi, Johnathan Tucker, Stephen Tian, Anirudha Majumdar, Jiankai Sun, Weiyu Liu, Yuke Zhu, Shuran Song, Ashish Kapoor, Karol Hausman, et al. Foundation models in robotics: Applications, challenges, and the future.The International Journal of Robotics Research, page 02783649241281508, 2023
2023
-
[11]
Fpga based hardware accelerator for calculations of the parallel robot inverse kinematics
Konrad Gac, Grzegorz Karpiel, and Maciej Petko. Fpga based hardware accelerator for calculations of the parallel robot inverse kinematics. InProceedings of 2012 IEEE 17th International Conference on Emerging Technologies & Factory Automation (ETFA 2012), pages 1–4. IEEE, 2012
2012
-
[12]
Eudoxus: Characterizing and accelerating localization in autonomous machines industry track paper
Yiming Gan, Yu Bo, Boyuan Tian, Leimeng Xu, Wei Hu, Shaoshan Liu, Qiang Liu, Yanjun Zhang, Jie Tang, and Yuhao Zhu. Eudoxus: Characterizing and accelerating localization in autonomous machines industry track paper. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pages 827–840. IEEE, 2021
2021
-
[13]
Olive: Accelerating large language models via hardware-friendly outlier-victim pair quantization
Cong Guo, Jiaming Tang, Weiming Hu, Jingwen Leng, Chen Zhang, Fan Yang, Yunxin Liu, Minyi Guo, and Yuhao Zhu. Olive: Accelerating large language models via hardware-friendly outlier-victim pair quantization. InProceedings of the 50th Annual International Symposium on Computer Architecture, pages 1–15, 2023
2023
-
[14]
Fast matrix multiplications for lookup table-quantized llms
Han Guo, William Brandon, Radostin Cholakov, Jonathan Ragan-Kelley, Eric Xing, and Yoon Kim. Fast matrix multiplications for lookup table-quantized llms. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 12419–12433, 2024
2024
-
[15]
An algorithm-hardware co-design based on revised microscaling format quantization for accelerating large language models
Yingbo Hao, Huangxu Chen, Yi Zou, and Yanfeng Yang. An algorithm-hardware co-design based on revised microscaling format quantization for accelerating large language models. In2025 62nd ACM/IEEE Design Automation Conference (DAC), pages 1–7. IEEE, 2025
2025
-
[16]
Orianna: An accelerator generation framework for optimization-based robotic applications
Yuhui Hao, Yiming Gan, Bo Yu, Qiang Liu, Yinhe Han, Zishen Wan, and Shaoshan Liu. Orianna: An accelerator generation framework for optimization-based robotic applications. InProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, pages 813–829, 2024
2024
-
[17]
Blitzcrank: Factor graph accelerator for motion planning
Yuhui Hao, Yiming Gan, Bo Yu, Qiang Liu, Shao-Shan Liu, and Yuhao Zhu. Blitzcrank: Factor graph accelerator for motion planning. In2023 60th ACM/IEEE Design Automation Conference (DAC), pages 1–6. IEEE, 2023
2023
-
[18]
Vapr: Variable-precision tensors to accelerate robot motion planning
Yu-Shun Hsiao, Siva Kumar Sastry Hari, Balakumar Sundaralingam, Jason Yik, Thierry Tambe, Charbel Sakr, Stephen W Keckler, and Vijay Janapa Reddi. Vapr: Variable-precision tensors to accelerate robot motion planning. In2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6304–6309. IEEE, 2023
2023
-
[19]
M2xfp: A metadata- augmented microscaling data format for efficient low-bit quantization
Weiming Hu, Zihan Zhang, Haoyan Zhang, Chen Zhang, Cong Guo, Yu Feng, Tianchi Hu, Guanglin Li, Guipeng Hu, Junsong Wang, et al. M2xfp: A metadata- augmented microscaling data format for efficient low-bit quantization. InPro- ceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2...
2026
-
[20]
Yafei Hu, Quanting Xie, Vidhi Jain, Jonathan Francis, Jay Patrikar, Nikhil Keetha, Seungchan Kim, Yaqi Xie, Tianyi Zhang, Hao-Shu Fang, et al. Toward general- purpose robots via foundation models: A survey and meta-analysis.arXiv preprint arXiv:2312.08782, 2023
Pith/arXiv arXiv 2023
-
[21]
Moped: Efficient mo- tion planning engine with flexible dimension support
Lingyi Huang, Yu Gong, Yang Sui, Xiao Zang, and Bo Yuan. Moped: Efficient mo- tion planning engine with flexible dimension support. In2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pages 483–497. IEEE, 2024
2024
-
[22]
Dadu-corki: Algorithm-architecture co-design for embodied ai-powered robotic manipulation
Yiyang Huang, Yuhui Hao, Bo Yu, Feng Yan, Yuxin Yang, Feng Min, Yinhe Han, Lin Ma, Shaoshan Liu, Qiang Liu, et al. Dadu-corki: Algorithm-architecture co-design for embodied ai-powered robotic manipulation. InProceedings of the 52nd Annual International Symposium on Computer Architecture, pages 327–343, 2025
2025
-
[23]
Biqgemm: matrix multiplication with lookup table for binary-coding-based quantized dnns
Yongkweon Jeon, Baeseong Park, Se Jung Kwon, Byeongwook Kim, Jeongin Yun, and Dongsoo Lee. Biqgemm: matrix multiplication with lookup table for binary-coding-based quantized dnns. InSC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–14. IEEE, 2020
2020
-
[24]
Beta: Binarized energy-efficient transformer accelerator at the edge
Yuhao Ji, Chao Fang, and Zhongfeng Wang. Beta: Binarized energy-efficient transformer accelerator at the edge. In2024 IEEE International Symposium on Circuits and Systems (ISCAS), pages 1–5. IEEE, 2024
2024
-
[25]
Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success.arXiv preprint arXiv:2502.19645, 2025
Pith/arXiv arXiv 2025
-
[26]
Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakr- ishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024
Pith/arXiv arXiv 2024
-
[27]
Ramulator: A fast and extensible dram simulator.IEEE Computer architecture letters, 15(1):45–49, 2015
Yoongu Kim, Weikun Yang, and Onur Mutlu. Ramulator: A fast and extensible dram simulator.IEEE Computer architecture letters, 15(1):45–49, 2015
2015
-
[28]
Automatic domain-specific soc design for autonomous unmanned aerial vehicles
Srivatsan Krishnan, Zishen Wan, Kshitij Bhardwaj, Paul Whatmough, Aleksandra Faust, Sabrina Neuman, Gu-Yeon Wei, David Brooks, and Vijay Janapa Reddi. Automatic domain-specific soc design for autonomous unmanned aerial vehicles. In2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 300–317. IEEE, 2022
2022
-
[29]
Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models
Changhun Lee, Jungyu Jin, Taesu Kim, Hyungjun Kim, and Eunhyeok Park. Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 13355–13364, 2024
2024
-
[30]
Mx+: Pushing the limits of microscaling formats for efficient large language model serving
Jungi Lee, Junyong Park, Soohyun Cha, Jaehoon Cho, and Jaewoong Sim. Mx+: Pushing the limits of microscaling formats for efficient large language model serving. InProceedings of the 58th IEEE/ACM International Symposium on Mi- croarchitecture, pages 869–883, 2025
2025
-
[31]
Spade: Sparse pillar-based 3d object detection accelerator for autonomous driving
Minjae Lee, Seongmin Park, Hyungmin Kim, Minyong Yoon, Janghwan Lee, Jun Won Choi, Nam Sung Kim, Mingu Kang, and Jungwook Choi. Spade: Sparse pillar-based 3d object detection accelerator for autonomous driving. In2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pages 454–467. IEEE, 2024
2024
-
[32]
Lut-dla: Lookup table as efficient extreme low-bit deep learning accelerator
Guoyu Li, Shengyu Ye, Chunyun Chen, Yang Wang, Fan Yang, Ting Cao, Cheng Liu, Mohamed M Sabry Aly, and Mao Yang. Lut-dla: Lookup table as efficient extreme low-bit deep learning accelerator. In2025 IEEE International Symposium on High Performance Computer Architecture (HPCA), pages 671–684. IEEE, 2025
2025
-
[33]
Vision-language foundation models as effective robot imitators.arXiv preprint arXiv:2311.01378, 2023
Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, et al. Vision-language foundation models as effective robot imitators.arXiv preprint arXiv:2311.01378, 2023
Pith/arXiv arXiv 2023
-
[34]
Dadu-p: A scalable accelerator for robot motion planning in a dynamic environment
Shiqi Lian, Yinhe Han, Xiaoming Chen, Ying Wang, and Hang Xiao. Dadu-p: A scalable accelerator for robot motion planning in a dynamic environment. In Proceedings of the 55th Annual Design Automation Conference, pages 1–6, 2018
2018
-
[35]
Dadu: Accelerating inverse kinematics for high-dof robots
Shiqi Lian, Yinhe Han, Ying Wang, Yungang Bao, Hang Xiao, Xiaowei Li, and Ninghui Sun. Dadu: Accelerating inverse kinematics for high-dof robots. In Proceedings of the 54th Annual Design Automation Conference 2017, pages 1–6, 2017
2017
-
[36]
Libero: Benchmarking knowledge transfer for lifelong robot learning
Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36:44776–44791, 2023. 13 MICRO 2026, October 31–November 04, 2026, Athens, Greece Zhuoran Song 1, Haozhe Jiang1, Chunyu Qi1, Minnan Pei2, Gang Li2, Xiaoy...
2023
-
[37]
eslam: An energy- efficient accelerator for real-time orb-slam on fpga platform
Runze Liu, Jianlei Yang, Yiran Chen, and Weisheng Zhao. eslam: An energy- efficient accelerator for real-time orb-slam on fpga platform. InProceedings of the 56th Annual Design Automation Conference 2019, pages 1–6, 2019
2019
-
[38]
Rdt-1b: a diffusion foundation model for bimanual manipulation.arXiv preprint arXiv:2410.07864, 2024
Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: a diffusion foundation model for bimanual manipulation.arXiv preprint arXiv:2410.07864, 2024
Pith/arXiv arXiv 2024
-
[39]
Archytas: A framework for synthesizing and dynamically optimiz- ing accelerators for robotic localization
Weizhuang Liu, Bo Yu, Yiming Gan, Qiang Liu, Jie Tang, Shaoshan Liu, and Yuhao Zhu. Archytas: A framework for synthesizing and dynamically optimiz- ing accelerators for robotic localization. InMICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture, pages 479–493, 2021
2021
-
[40]
Mobilesp: An fpga-based real-time keypoint extraction hardware accelerator for mobile vslam.IEEE transactions on circuits and systems I: regular papers, 69(12):4919–4929, 2022
Ye Liu, Jingyuan Li, Kun Huang, Xiangting Li, Xiuyuan Qi, Liang Chang, Yu Long, and Jun Zhou. Mobilesp: An fpga-based real-time keypoint extraction hardware accelerator for mobile vslam.IEEE transactions on circuits and systems I: regular papers, 69(12):4919–4929, 2022
2022
-
[41]
Vq-llm: High-performance code generation for vector quantization augmented llm inference
Zihan Liu, Xinhao Luo, Junxian Guo, Wentao Ni, Yangjie Zhou, Yue Guan, Cong Guo, Weihao Cui, Yu Feng, Minyi Guo, et al. Vq-llm: High-performance code generation for vector quantization augmented llm inference. In2025 IEEE Inter- national Symposium on High Performance Computer Architecture (HPCA), pages 1496–1509. IEEE, 2025
2025
-
[42]
Energy-efficient machine learning accelerator for binary neural networks
Wei Mao, Zhihua Xiao, Peng Xu, Hongwei Ren, Dingbang Liu, Shirui Zhao, Fengwei An, and Hao Yu. Energy-efficient machine learning accelerator for binary neural networks. InProceedings of the 2020 on Great Lakes Symposium on VLSI, pages 77–82, 2020
2020
-
[43]
A pro- grammable architecture for robot motion planning acceleration
Sean Murray, Will Floyd-Jones, George Konidaris, and Daniel J Sorin. A pro- grammable architecture for robot motion planning acceleration. In2019 IEEE 30th International Conference on Application-specific Systems, Architectures and Processors (ASAP), volume 2160, pages 185–188. IEEE, 2019
2019
-
[44]
The microarchitecture of a real-time robot motion planning accelerator
Sean Murray, William Floyd-Jones, Ying Qi, George Konidaris, and Daniel J Sorin. The microarchitecture of a real-time robot motion planning accelerator. In2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 1–12. IEEE, 2016
2016
-
[45]
Roboshape: Using topology patterns to scalably and flexibly deploy accelerators across robots
Sabrina M Neuman, Radhika Ghosal, Thomas Bourgeat, Brian Plancher, and Vijay Janapa Reddi. Roboshape: Using topology patterns to scalably and flexibly deploy accelerators across robots. InProceedings of the 50th Annual International Symposium on Computer Architecture, pages 1–13, 2023
2023
-
[46]
Fine-grained dram: Energy- efficient dram for extreme bandwidth systems
Mike O’Connor, Niladrish Chatterjee, Donghyuk Lee, John Wilson, Aditya Agrawal, Stephen W Keckler, and William J Dally. Fine-grained dram: Energy- efficient dram for extreme bandwidth systems. InProceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture, pages 41–54, 2017
2017
-
[47]
Codegemm: A codebook-centric approach to efficient gemm in quantized llms.Advances in Neural Information Processing Systems, 38:34603–34623, 2026
Gunho Park, Jeongin Bae, Byeongwook Kim, Baeseong Park, Jiwon Ryu, Hoseung Kim, Se Jung Kwon, and Dongsoo Lee. Codegemm: A codebook-centric approach to efficient gemm in quantized llms.Advances in Neural Information Processing Systems, 38:34603–34623, 2026
2026
-
[48]
Codegemm: A codebook-centric approach to efficient gemm in quantized llms
Gunho Park, Jeongin Bae, Byeongwook Kim, Jiwon Ryu, Hoseung Kim, Se Jung Kwon, Dongsoo Lee, et al. Codegemm: A codebook-centric approach to efficient gemm in quantized llms. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, volume 38, pages 34603–34623, 2026
2026
-
[49]
Pytorch: An imperative style, high-performance deep learning library.Advances in neural information processing systems, 32, 2019
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library.Advances in neural information processing systems, 32, 2019
2019
-
[50]
Microscopiq: Accel- erating foundational models through outlier-aware microscaling quantization
Akshat Ramachandran, Souvik Kundu, and Tushar Krishna. Microscopiq: Accel- erating foundational models through outlier-aware microscaling quantization. In Proceedings of the 52nd Annual International Symposium on Computer Architecture, pages 1193–1209, 2025
2025
-
[51]
Robox: an end-to-end solution to accelerate autonomous control in robotics
Jacob Sacks, Divya Mahajan, Richard C Lawson, Behnam Khaleghi, and Hadi Esmaeilzadeh. Robox: an end-to-end solution to accelerate autonomous control in robotics. In2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA), pages 479–490. IEEE, 2018
2018
-
[52]
Scale-sim: Systolic cnn accelerator simulator.arXiv preprint arXiv:1811.02883, 2018
Ananda Samajdar, Yuhao Zhu, Paul Whatmough, Matthew Mattina, and Tushar Krishna. Scale-sim: Systolic cnn accelerator simulator.arXiv preprint arXiv:1811.02883, 2018
Pith/arXiv arXiv 2018
-
[53]
Towards hardware accelerated reinforcement learning for application-specific robotic control
Shengjia Shao, Jason Tsai, Michal Mysior, Wayne Luk, Thomas Chau, Alexander Warren, and Ben Jeppesen. Towards hardware accelerated reinforcement learning for application-specific robotic control. In2018 IEEE 29th International Conference on Application-specific Systems, Architectures and Processors (ASAP), pages 1–8. IEEE, 2018
2018
-
[54]
A unified accelerator design for lidar slam algorithms for low-end fpgas
Keisuke Sugiura and Hiroki Matsutani. A unified accelerator design for lidar slam algorithms for low-end fpgas. In2021 International Conference on Field- Programmable Technology (ICFPT), pages 1–9. IEEE, 2021
2021
-
[55]
A universal lidar slam accelerator system on low-cost fpga.IEEE Access, 10:26931–26947, 2022
Keisuke Sugiura and Hiroki Matsutani. A universal lidar slam accelerator system on low-cost fpga.IEEE Access, 10:26931–26947, 2022
2022
-
[56]
Amr Suleiman, Zhengdong Zhang, Luca Carlone, Sertac Karaman, and Vivienne Sze. Navion: A 2-mw fully integrated real-time visual-inertial odometry accel- erator for autonomous navigation of nano drones.IEEE Journal of Solid-State Circuits, 54(4):1106–1119, 2019
2019
-
[57]
Stone Tao, Fanbo Xiang, Arth Shukla, Yuzhe Qin, Xander Hinrichsen, Xiaodi Yuan, Chen Bao, Xinsong Lin, Yulin Liu, Tse kai Chan, Yuan Gao, Xuanlin Li, Tongzhou Mu, Nan Xiao, Arnav Gurha, Zhiao Huang, Roberto Calandra, Rui Chen, Shan Luo, and Hao Su. Maniskill3: Gpu parallelized robotics simulation and rendering for generalizable embodied ai.arXiv preprint ...
Pith/arXiv arXiv 2024
-
[58]
Albert Tseng, Jerry Chee, Qingyao Sun, Volodymyr Kuleshov, and Christopher De Sa. Quip#: Even better llm quantization with hadamard incoherence and lattice codebooks.arXiv preprint arXiv:2402.04396, 2024
Pith/arXiv arXiv 2024
-
[59]
Gptvq: The blessing of dimensionality for llm quantization.arXiv preprint arXiv:2402.15319, 2024
Mart Van Baalen, Andrey Kuzmin, Ivan Koryakovskiy, Markus Nagel, Peter Couperus, Cedric Bastoul, Eric Mahurin, Tijmen Blankevoort, and Paul What- mough. Gptvq: The blessing of dimensionality for llm quantization.arXiv preprint arXiv:2402.15319, 2024
Pith/arXiv arXiv 2024
-
[60]
Scaling the power wall: a path to exascale
Oreste Villa, Daniel R Johnson, Mike Oconnor, Evgeny Bolotin, David Nellans, Justin Luitjens, Nikolai Sakharnykh, Peng Wang, Paulius Micikevicius, Anthony Scudiero, et al. Scaling the power wall: a path to exascale. InSC’14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 830–841. IEEE, 2014
2014
-
[61]
Gpt-4v (ision) for robotics: Multimodal task planning from human demonstration.IEEE Robotics and Automation Letters, 2024
Naoki Wake, Atsushi Kanehira, Kazuhiro Sasabuchi, Jun Takamatsu, and Katsushi Ikeuchi. Gpt-4v (ision) for robotics: Multimodal task planning from human demonstration.IEEE Robotics and Automation Letters, 2024
2024
-
[62]
Vlatest: Testing and evaluating vision-language-action models for robotic manipulation.Proceedings of the ACM on Software Engineering, 2(FSE):1615–1638, 2025
Zhijie Wang, Zhehua Zhou, Jiayang Song, Yuheng Huang, Zhan Shu, and Lei Ma. Vlatest: Testing and evaluating vision-language-action models for robotic manipulation.Proceedings of the ACM on Software Engineering, 2(FSE):1615–1638, 2025
2025
-
[63]
Oltron: Algorithm-hardware co-design for outlier-aware quantization of llms with inter-/intra-layer adaptation
Chenhao Xue, Chen Zhang, Xun Jiang, ZhuTianYa Gao, Yibo Lin, and Guangyu Sun. Oltron: Algorithm-hardware co-design for outlier-aware quantization of llms with inter-/intra-layer adaptation. InProceedings of the 61st ACM/IEEE Design Automation Conference, pages 1–6, 2024
2024
-
[64]
Dadu-rbd: Robot rigid body dynam- ics accelerator with multifunctional pipelines
Yuxin Yang, Xiaoming Chen, and Yinhe Han. Dadu-rbd: Robot rigid body dynam- ics accelerator with multifunctional pipelines. InProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture, pages 297–309, 2023
2023
-
[65]
Shiftaddllm: Accel- erating pretrained llms via post-training multiplication-less reparameterization
Haoran You, Yipin Guo, Yichao Fu, Wei Zhou, Huihong Shi, Xiaofan Zhang, Souvik Kundu, Amir Yazdanbakhsh, and Yingyan Celine Lin. Shiftaddllm: Accel- erating pretrained llms via post-training multiplication-less reparameterization. Advances in Neural Information Processing Systems, 37:24822–24848, 2024
2024
-
[66]
Building the computing system for autonomous micromobility vehicles: Design constraints and architectural optimizations
Bo Yu, Wei Hu, Leimeng Xu, Jie Tang, Shaoshan Liu, and Yuhao Zhu. Building the computing system for autonomous micromobility vehicles: Design constraints and architectural optimizations. In2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 1067–1081. IEEE, 2020
2020
-
[67]
Pqcache: Product quantization-based kvcache for long context llm inference.Proceedings of the ACM on Management of Data, 3(3):1–30, 2025
Hailin Zhang, Xiaodong Ji, Yilin Chen, Fangcheng Fu, Xupeng Miao, Xiaonan Nie, Weipeng Chen, and Bin Cui. Pqcache: Product quantization-based kvcache for long context llm inference.Proceedings of the ACM on Management of Data, 3(3):1–30, 2025
2025
-
[68]
Loam: Lidar odometry and mapping in real-time
Ji Zhang, Sanjiv Singh, et al. Loam: Lidar odometry and mapping in real-time. In Robotics: Science and systems, volume 2, pages 1–9. Berkeley, CA, 2014
2014
-
[69]
Exploiting intra-sm parallelism in gpus via persistent and elastic blocks
Han Zhao, Weihao Cui, Quan Chen, Jieru Zhao, Jingwen Leng, and Minyi Guo. Exploiting intra-sm parallelism in gpus via persistent and elastic blocks. In2021 IEEE 39th International Conference on Computer Design (ICCD), pages 290–298. IEEE, 2021
2021
-
[70]
Binary weight multi-bit activation quantization for compute-in-memory cnn accelerators.IEEE Transac- tions on Computer-Aided Design of Integrated Circuits and Systems, 2025
Wenyong Zhou, Zhengwu Liu, Yuan Ren, and Ngai Wong. Binary weight multi-bit activation quantization for compute-in-memory cnn accelerators.IEEE Transac- tions on Computer-Aided Design of Integrated Circuits and Systems, 2025. 14
2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.