Pith. sign in

REVIEW 3 major objections 6 minor 66 references

Saliency-Aware Quantized Imitation Learning for Efficient Robotic Control

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SQIL recovers full-precision performance in 4-bit quantized imitation policies by weighting training loss toward states where the full-precision policy is most sensitive to perturbation.

desk verdict Worth refereeing, but the paper over-sells its own numbers: the 'within 1%' claim fails on LIBERO-Long and NoCrash, and Table 9 never says what happens to the LoRA adapters at deployment. read the letter →

arxiv 2505.15304 v2 pith:GDPAH73Q submitted 2025-05-21 cs.RO

Seongmin Park , Hyungmin Kim , Sangwoo Kim , Wonseok Jeon , Juyoung Yang , Byeongwook Jeon , Yoonseon Oh , Jungwook Choi This is my paper · ORCID
classification cs.RO
keywords quantization-awaretrainingimitationlearningvision-language-actionmodelsroboticmanipulationmodelcompressionsaliency-basedstateimportanceautonomousdrivingedgedeployment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most quantization methods treat every timestep of an imitation-learned policy equally, but this paper observes that quantization error in robot policies is concentrated at a handful of mission-critical moments, such as the instant a gripper grasps an object. The authors propose SQIL, which uses a saliency-based state importance score (SIS) to identify those moments from the full-precision policy, then upweights a distillation loss at those states during 4-bit quantization-aware training. The reported result is that quantized vision-language-action models, driving policies, and physics-control policies all recover full-precision success rates within about one percentage point, while running two to four times faster with large energy savings on edge hardware. A reader should care because large imitation-learned policies are otherwise too slow and memory-heavy to deploy on battery-powered robots.

What carries the argument

The machinery has two parts. The first is the saliency-based state importance score (SIS), defined as $SIS(\pi^{FP}, s_t) = \mathbb{E}_k \left[ \tfrac{1}{2} \|\pi^{FP}(s_t) - \pi^{FP}(\phi(s_t,k))\|_2^2 \right]$, where $\phi(s_t,k)$ perturbates the state locally (a Gaussian blur over image patch $k$); it identifies states where the full-precision policy's action is highly sensitive to local visual changes. The second is quantization-robust action distillation (QRD), $L_{QRD} = \alpha_t \, \mathbb{E}_\tau \left[ \frac{1}{|T_i|} \sum_{s_t \in \tau_i} D(\pi^Q(s_t), \pi^{FP}(s_t)) \right]$, with $\alpha_t = \beta$ if $SIS > T$ and $\alpha_t = 1$ otherwise, $D$ an L2 discrepancy, $\beta=2$, and $T$ the top-20% SIS threshold. The two parts work together: QAT keeps the quantized policy faithful to expert data, QRD shapes the quantized policy's action distribution toward the FP teacher, and the SIS weighting concentrates the shaping at the states where divergence would be fatal.

What would settle it

Run the SQIL pipeline but replace the SIS-weighted QRD loss with a uniform QRD loss (all α_t = 1) at the same total loss weight; if the uniform version matches SQIL's success rate on LIBERO (within 0.5 percentage points), the saliency selection is not doing the claimed work. Alternatively, measure Spearman correlation between SIS values assigned by the FP policy and the per-state action discrepancy $\|\pi^Q(s_t) - \pi^{FP}(s_t)\|_2$ of an INT4 model; a correlation near zero would directly contradict the paper's mechanism.

Watch

Extended reading notes

Core claim

The paper's central claim is that quantization error in imitation learning is sparse in time rather than uniform: across most timesteps a 4-bit policy deviates only slightly from its full-precision teacher, but at a few mission-critical states—typically fine-grained physical interactions like grasping, releasing, or inserting—the action discrepancy spikes and causes the whole episode to fail. SQIL detects these states with a saliency-based state importance score (SIS), the average over image patches of the squared action change of the full-precision policy under local Gaussian perturbation, and then applies a quantization-robust action distillation (QRD) term that weights the teacher-student action discrepancy by an extra factor (β=2) at the top 20% of salient states, added to the standard QAT cross-entropy loss. With only this added weighting, the paper reports that INT4 weight-quantized OpenVLA matches the full-precision 73.8% success rate on LIBERO at 73.2%, INT4 W4A4 CILRS recovers FP driving success on NoCrash-dense (80% vs 82% in the easiest condition), and D4RL MuJoCo returns come within a few points of FP while LPPD-style 4-bit policies lose substantially. The authors further show the same fixed hyperparameters work across robot manipulation, autonomous driving, physics simulation, and real-world UR5 deployments, and that the method also applies to the π0 flow-matching policy.

Load-bearing premise

The approach assumes that the full-precision policy's action sensitivity to small visual perturbations marks the same states where the quantized policy's errors actually cause failures; if the correlation between SIS and quantization-induced action error is weak or reversed, the selective weighting would emphasize the wrong timesteps.

Editorial extensions

If this is right

  • INT4 weight quantization of OpenVLA-class VLA policies becomes practical on edge GPUs: 2.5× speedup, 2.5× energy savings, 4× memory reduction at success rates within 1% of full precision.
  • The same recipe—SIS-weighted distillation on top of QAT—restores full-precision behavior for weight+activation 4-bit driving policies (CILRS), yielding 3.7× speedup and 3.1× energy savings on a low-end GPU.
  • Because the hyperparameters (β=2, top-20% threshold, L2 discrepancy) transfer across manipulation, driving, and physics control, SQIL can serve as a drop-in modification to existing QAT fine-tuning pipelines for IL policies.
  • SQIL recovers the full-precision policy's visual attention at critical states (lower saliency divergence in Table 2), implying the quantized model reasons about the same task-relevant regions as the teacher.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same SIS weighting could be applied to other robustness interventions beyond quantization, such as pruning masks, data augmentation, or regularization of the FP policy itself; the paper's key mechanistic claim is that IL losses should be state-importance-weighted, not that quantization is the only error source.
  • One testable extension is an online SIS variant that recomputes saliency from the quantized policy during training; the quantized policy's own fragility points might differ from the FP teacher's, and updating the weights could improve recovery at very low bitwidths (INT2/INT3).
  • If the sparse-error observation generalizes, then episode success rate is a more appropriate metric for quantized IL than mean squared action error over all timesteps; a model with low average error can still fail catastrophically at a single critical state.
  • The finding that quantizing the LLM backbone (not the vision encoder) drives most of the performance loss suggests that future VLA quantization should focus saliency computation on language-conditioned reasoning rather than purely visual patches.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes SQIL (Saliency-Aware Quantized Imitation Learning), a fine-tuning scheme for 4-bit quantized imitation-learning policies. Given a full-precision (FP) policy and the expert dataset, SQIL computes a per-state saliency score SIS (Eq. 5) by averaging the L2 change in the policy's action under local patch perturbation (Eq. 4), then fine-tunes a quantized policy with a combined loss (Eq. 7): a behavior-cloning QAT term (Eq. 3) plus a quantization-robust action distillation term (Eq. 6) that up-weights high-SIS states by a factor beta. The method is evaluated on OpenVLA over the four LIBERO suites and two UR5 real-world setups, on CILRS over the NoCrash benchmark, on D4RL/MuJoCo control tasks, and on Jetson AGX Orin / RTX 2080 Ti hardware (Tables 8-9). The central claims are that SQIL recovers FP success rates within about one point on LIBERO (73.2 vs 73.8 on average) and enables 2-4x speedups and energy savings, with the mechanism attributed to suppressing quantization-induced action errors at mission-critical states.

Significance. If the result holds, SQIL is a practically attractive addition to IL pipelines: it reuses the existing expert dataset and FP policy, adds only a one-time SIS computation, and consistently improves over PTQ and plain QAT across manipulation, driving, and control domains. The paper's empirical breadth is a genuine strength: four LIBERO suites, NoCrash town/weather generalization, D4RL control, and real-world UR5 trials are all reported with repeated evaluation rounds, and the supplementary material discloses several ablations (Supp. Tables 10-14) including the unfavorable QRD-alone result and hyperparameter sensitivity. The main caveats, discussed below, concern the unspecified deployment recipe behind the hardware tables and an over-broad wording of the 'consistent recovery' claim; both are addressable without changing the method. The paper would also be strengthened by a code or data release, which is currently absent.

major comments (3)
  1. [Section 5.4, Table 9; Supplementary Section 1.2] The hardware claims in Table 9 are not tied to the model whose accuracy is reported. All OpenVLA accuracy numbers come from models fine-tuned with QLoRA (Supp. 1.2), where the 4-bit base is frozen and only 110M LoRA adapter parameters are trained in BF16. Table 9 reports an 'INT4' OpenVLA with 4.0 GB memory and 2.5x speedup, but the paper never states whether the adapters at deployment are (a) dropped, (b) kept as separate BF16 weights, or (c) merged into the base and re-quantized to INT4. In case (a) the deployed model is the AWQ-initialized base, whose LIBERO-Spatial success rate is 80.1% (Table 3), well below the reported SQIL values; in case (b) the deployed model is mixed-precision (7.6B x 0.5 bytes is about 3.8 GB plus roughly 0.22 GB of BF16 adapters, which is arithmetically consistent with the reported 4.0 GB), so the '4-bit weight-quantized' label is inaccurate; in case (c) the final merged-and-quantized weights were not the ones trained under QAT, so the success rates in Tables 3-5 may not transfer to the deployed artifact. The authors should specify the deployment recipe and, if adapters are merged and re-quantized, re-evaluate success rate on the exact artifact used in Table 9.
  2. [Abstract and Section 1 (Contributions), vs. Tables 3, 6, 7 and Figs. 7-8] The claims that SQIL 'consistently recovers full-precision performance' and 'maintains a success rate within 1% of the FP baseline' are stronger than several reported cells justify. On LIBERO, the per-suite SQIL-versus-FP gaps are 0.1, 0.4, 0.3, and 1.5 points (Table 3), so the 'within 1%' phrasing fails for LIBERO-Long (49.2 vs 50.7) even though it holds for the suite average (73.2 vs 73.8). The real-world trials also show gaps of 2-3 points (Figs. 7-8: 52.5% vs 55% and 76.7% vs 78.9%). More notably, Table 6 shows SQIL at 72% versus FP at 80% success on the new-town/dense (nt) condition of NoCrash, and the average across the four condition rows is 73 vs 76; this is better described as 'comparable' than as 'recovering'. Table 7 additionally shows a 44-point gap on Finger Spin (640 vs 684). I recommend restricting the 'within 1%' and 'recovering' wording to the LIBERO suite averages, or discussing the per-condition gaps and their statistical significance given the stated three-round protocol.
  3. [Sections 4.1-4.2 and 4.4; Supplementary Tables 10, 13] The mechanistic claims are only partially supported. First, the paper's premise that high-SIS states computed from the FP policy (Eq. 5) coincide with the states where quantization errors cause failures is never tested directly; a concrete check would be to report the per-state overlap or rank correlation between SIS and the action discrepancy between the FP policy and the PTQ/QAT policies along successful episodes, using data already available for Fig. 1. Second, Supp. Table 10 shows that QRD alone degrades success from its own AWQ initialization (69.5% vs 80.1%), so the two loss terms are strongly non-additive; the main text's Section 4.4 'synergy' discussion would be more informative if it presented the QRD-alone cell and explained why selective distillation helps only in combination with L_QAT. Third, there is no ablation with uniform weighting (beta = 1, i.e., alpha_t = 1) to isolate the effect of the SIS selection itself; Table 1 compares SIS against keyframes but not against unweighted distillation with the same total loss.
minor comments (6)
  1. [Throughout] Typos and wording: 'LIBIERO' in the Table 3 header, 'IN4' for INT4 in Section 5.2.2, 'the the LIBERO' in Section 5.1, 'Autonomus Driving' in Supp. Section 1.1, 'wih' in Supp. Section 1.2, and 'adaptor' should be 'adapter' throughout.
  2. [Algorithm 1 and Section 5.1] Algorithm 1 says 'Update pi_Q with partial LSQIL / partial theta' as if all weights are updated, but Supp. Section 1.2 states that QLoRA freezes the quantized base and updates only the 110M LoRA adapters; the main text should make this explicit so the reader does not misunderstand the QAT procedure.
  3. [Section 5.2.2, Table 5] Table 5 is captioned 'INT4 quantization results for pi0 with OpenVLA', but pi0 is introduced as a distinct compact model; please clarify pi0's training and quantization protocol (initialization, fine-tuning data, number of seeds) so that its rows can be compared with the OpenVLA rows.
  4. [Tables 3 and 10] Please state explicitly how the reported means and standard deviations are computed (e.g., 3 rounds of 500 episodes) and whether adjacent values such as 83.9 vs 84.0 in Table 3 are within the metric's noise; this would strengthen the 'within 1%' quantitative claim.
  5. [Supplementary Table 10] The 'QRD' row should state whether the SIS weighting alpha_t is applied in that ablation; as defined by Eq. (6), QRD always includes alpha_t, but the row is presented as if QRD were the plain, unweighted distillation term.
  6. [Data availability] No code, data, or model-release statement is provided; given that the headline hardware numbers depend on toolchain details (TensorRT for the vision encoder and MLC-LLM for the LLM backbone), releasing the deployment script and the SIS evaluation code would materially improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SQIL's central success-rate claim is validated against external benchmarks, and its loss does not encode the evaluation metric.

full rationale

I walked the derivation chain. SQIL's loss is L_SQIL = L_QAT + L_QRD (Eq. 7), where L_QAT (Eq. 3) maximizes the quantized policy's likelihood of expert actions and L_QRD (Eq. 6) distills the FP policy's action distribution with SIS weights. SIS (Eqs. 4-5) is computed from the FP policy and the expert dataset, so it is true by construction that QRD directly penalizes action discrepancy from the FP policy at SIS-selected states. That, however, is a design choice rather than a circular prediction. The paper's headline claim is that SQIL recovers FP success rates, and the reported evidence is external: LIBERO average success rates (Table 3), NoCrash success rate and driving score (Table 6), D4RL returns (Table 7), and real-robot task successes (Figs. 7-8). None of these success metrics appears in the loss or in the SIS computation; they come from environment interaction. Moreover, Table 10 shows that QRD alone actually degrades performance (69.5% on LIBERO-Spatial), so the success-rate outcome is not forced by the objective. The ablation comparing SIS to keyframe detection (Table 1) and the hyperparameter robustness tables (Supp. Tables 12-14) are empirical tests, not definitions. The only self-citations are [6] and [43], which appear in background and related-work lists and are not load-bearing for any central claim. The deployment-model ambiguity about whether Table 9's INT4 OpenVLA includes the LoRA adapters is a measurement-consistency and correctness concern, not a circularity concern. Therefore no circular step is exhibited and the derivation is self-contained against external benchmarks.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the behavior-cloning objective, the localization of quantization errors to critical states, and the transferability of FP saliency to quantized failure modes. The hand-selected SIS hyperparameters are free parameters; no new physical entities are introduced.

free parameters (4)
  • Importance weight beta = beta = 2
    Multiplies the QRD loss for states with SIS above threshold; chosen by hand and ablated in Supp. Table 13. Robust from 1.5 to 3 but not derived.
  • SIS threshold p = top 20%
    States in the top 20 percent of SIS values receive extra loss weight; chosen via ablations in Supp. Table 13 and stable across 10 to 30 percent.
  • SIS patch grid N = N = 8
    Number of spatial perturbation patches used in Eq. 4; chosen empirically in Supp. Sec. 3 and affects the cost of SIS computation.
  • SIS frame stride k = k = 4
    SIS is recomputed every fourth frame; chosen based on Supp. Table 12 to trade computation against success rate.
assumptions (5)
  • domain assumption The expert dataset DE contains reliable demonstrations of the target behaviors, making the behavior-cloning objective in Eq. 1 valid.
    IL assumes demonstrations are expert-quality; SQIL fine-tuning inherits this assumption from the OpenVLA and CILRS pipelines in Sec. 3.1.
  • ad hoc to paper States whose FP policy action changes strongly under local input perturbation are the mission-critical states where quantization failures occur.
    This is the core premise of SIS in Sec. 4.1, Eqs. 4-5, and it is used to weight the QRD loss. If FP saliency does not align with quantized-policy error modes, the method loses its mechanism.
  • domain assumption Quantization errors are concentrated at a few mission-critical states rather than accumulating over the whole trajectory.
    Stated in Sec. 3.3 and Fig. 1; this motivates selective weighting instead of uniform trajectory-wide error correction.
  • standard math The uniform clipped quantization model in Eq. 2 captures deployment error, and QAT with straight-through gradients approximates the quantized objective.
    Standard quantization-aware training assumption used in Eq. 3 and Algorithm 1.
  • domain assumption The full-precision policy is a valid teacher for distillation at all states used in QRD.
    QRD minimizes divergence to the FP policy in Eq. 6, so any FP policy mistakes are copied into the quantized policy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Saliency-Aware Quantized Imitation Learning for Efficient Robotic Control." pith.science (2026). https://pith.science/paper/GDPAH73Q

@misc{pith2026250515304,
  author       = {Pith},
  title        = {Pith review of: Saliency-Aware Quantized Imitation Learning for Efficient Robotic Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GDPAH73Q}},
  note         = {Machine review of arXiv:2505.15304}
}
read the original abstract

Deep neural network (DNN)-based policy models, such as vision-language-action (VLA) models, excel at automating complex decision-making from multi-modal inputs. However, scaling these models greatly increases computational overhead, complicating deployment in resource-constrained settings like robot manipulation and autonomous driving. To address this, we propose Saliency-Aware Quantized Imitation Learning (SQIL), which combines quantization-aware training with a selective loss-weighting strategy for mission-critical states. By identifying these states via saliency scores and emphasizing them in the training loss, SQIL preserves decision fidelity under low-bit precision. We validate SQIL's generalization capability across extensive simulation benchmarks with environment variations, real-world tasks, and cross-domain tasks (self-driving, physics simulation), consistently recovering full-precision performance. Notably, a 4-bit weight-quantized VLA model for robotic manipulation achieves up to 2.5x speedup and 2.5x energy savings on an edge GPU with minimal accuracy loss. These results underline SQIL's potential for efficiently deploying large IL-based policy models on resource-limited devices.

Figures

Figures reproduced from arXiv: 2505.15304 by the authors.

Figure 1
Figure 1. Comparison of action discrepancy (L2-norm) between the quantized and full-precision (FP) policies in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison of the structure, number of parameters of DNN-based policy models. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of timestep-wise state importance in the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Comparison of SIS and keyframe (KF) on trajectory. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Comparison of attention visualization in [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Comparison of action distributions for various quantiza [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: OpenVLA real-world UR5 robot evaluation tasks and results with various INT4 quantization. Task #Trials #Successes FP PTQ QAT SQIL (a) 30 25 22 25 24 (b) 30 24 20 22 24 (c) 30 22 18 17 21 Suc.Rate(%) 79 67 71 77 (a) Cloth sweeping (b) Cup stacking (c) Pick and place [P…
Figure 8
Figure 8. Figure 8: Real-world evaluation with OpenVLA on UR5. quantized using SQIL, in responding to a variety of lan￾guage instructions aimed at achieving a single goal. The results show that the IN4 policy’s performance closely matches that of the FP policy. Additionally, we analyze ho…
Figure 9
Figure 9. Figure 9: LIBERO benchmark. In our experiments, we used datasets that were specif￾ically modified for compatibility with the OpenVLA [27] framework, which included enhancements such as high￾resolution image processing, image rotation, and the exclu￾sion of unsuccessful demonstra…
Figure 10
Figure 10. Figure 10: Comparison of attention visualization for tasks successfully completed on the LIBERO-Spatial benchmark. [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Performance comparison of various quantization meth [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Hardware execution timeline comparison of 16-bit, 8-bit, and 4-bit weight datatypes for [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Timeline comparison of OpenVLA for 16-bit, 8-bit, and 4-bit datatypes over a total duration of 1,000 ms. V represents the ViT+MLP projector, P denotes the prefill operation, and D refers to the decode operation of the backbone LLM (Llama-2-7b) [PITH_FULL_IMAGE:figure…
Figure 14
Figure 14. Figure 14: Latency breakdown of OpenVLA for 16-bit, 8-bit, and 4-bit weights within a single step, including ViT+MLP, Llama-2-7b (Prefill), and Llama-2-7b (Decode) stages. 2.5× speedup in overall execution time. These experimental results confirm that the decode op￾eration in th…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 41 canonical work pages

  1. [1]

    Post train- ing 4-bit quantization of convolutional networks for rapid- deployment

    Ron Banner, Yury Nahshan, and Daniel Soudry. Post train- ing 4-bit quantization of convolutional networks for rapid- deployment. Advances in Neural Information Processing Systems, 32, 2019. 3

    work page 2019

  2. [2]

    π0: A vision-language- action flow model for general robot control

    Kevin Black and Noah Brown et al. π0: A vision-language- action flow model for general robot control. 8, 3

    work page

  3. [3]

    Rt-1: Robotics transformer for real-world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakr- ishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022. 1

    arXiv 2022

  4. [4]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818, 2023. 1, 3

    arXiv 2023

  5. [5]

    {TVM}: An automated {End- to-End} optimizing compiler for deep learning

    Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al. {TVM}: An automated {End- to-End} optimizing compiler for deep learning. In 13th USENIX Symposium on Operating Systems Design and Im- plementation (OSDI 18), pages 578–594, 2018. 8

    work page 2018

  6. [6]

    Pact: Parameterized clipping activa- tion for quantized neural networks

    Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. Pact: Parameterized clipping activa- tion for quantized neural networks. arXiv preprint arXiv:1805.06085, 2018. 1, 3

    arXiv 2018

  7. [7]

    Xception: Deep learning with depthwise separable convolutions

    Franc ¸ois Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 1251–1258, 2017. 3

    work page 2017

  8. [8]

    Exploring the limitations of behavior cloning for autonomous driving

    Felipe Codevilla, Eder Santana, Antonio M L ´opez, and Adrien Gaidon. Exploring the limitations of behavior cloning for autonomous driving. In Proceedings of the IEEE/CVF international conference on computer vision , pages 9329–9338, 2019. 2, 7, 1

    work page 2019

Show all 66 references
  1. [9]

    Qlora: Efficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems , 36,

  2. [10]

    Carla: An open urban driv- ing simulator

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Anto- nio Lopez, and Vladlen Koltun. Carla: An open urban driv- ing simulator. In Conference on robot learning, pages 1–16. PMLR, 2017. 1

    2017

  3. [11]

    Palm- e: An embodied multimodal language model

    Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm- e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378, 2023. 1

    2023 arXiv

  4. [12]

    Learned step size quantization

    Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S Modha. Learned step size quantization. In International Conference on Learning Representations, 2020. 1, 3, 7, 2

    2020

  5. [13]

    Quarot: Outlier-free 4-bit inference in ro- tated llms

    Ashkboos et al. Quarot: Outlier-free 4-bit inference in ro- tated llms. arXiv preprint arXiv:2404.00456, 2024. 7

    2024 arXiv

  6. [14]

    Quarl: Quantization for fast and environ- mentally sustainable reinforcement learning

    Aleksandra Faust, Gabe Barth-Maron, Maximilian Lam, Sharad Chitlangia, Srivatsan Krishnan, Vijay Janapa Reddi, and Zishen Wan. Quarl: Quantization for fast and environ- mentally sustainable reinforcement learning. Transactions on Machine Learning Research (TMLR) 2022, 2022. 3 9

    2022

  7. [15]

    Gptq: Accurate post-training quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323, 2022. 1

    2022 arXiv

  8. [16]

    OPTQ: Accurate quantization for generative pre- trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. OPTQ: Accurate quantization for generative pre- trained transformers. In The Eleventh International Confer- ence on Learning Representations, 2023. 1, 3

    2023

  9. [17]

    D4rl: Datasets for deep data-driven rein- forcement learning

    Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4rl: Datasets for deep data-driven rein- forcement learning. arXiv preprint arXiv:2004.07219, 2020. 3, 7, 8

    2004 arXiv

  10. [18]

    Visualizing and understanding atari agents

    Samuel Greydanus, Anurag Koul, Jonathan Dodge, and Alan Fern. Visualizing and understanding atari agents. In Inter- national conference on machine learning, pages 1792–1801. PMLR, 2018. 5

    2018

  11. [19]

    Apple in- telligence foundation language models

    Tom Gunter, Zirui Wang, Chong Wang, Ruoming Pang, Andy Narayanan, Aonan Zhang, Bowen Zhang, Chen Chen, Chung-Cheng Chiu, David Qiu, et al. Apple in- telligence foundation language models. arXiv preprint arXiv:2407.21075, 2024. 3

    2024

  12. [20]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 3

    2016

  13. [21]

    1.1 computing’s energy problem (and what we can do about it)

    Mark Horowitz. 1.1 computing’s energy problem (and what we can do about it). In 2014 IEEE international solid-state circuits conference digest of technical papers (ISSCC), pages 10–14. IEEE, 2014. 3

    2014

  14. [22]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 3

    2021 arXiv

  15. [23]

    Fixed-point feedfor- ward deep neural network design using weights +1, 0, and -1

    Kyuyeon Hwang and Wonyong Sung. Fixed-point feedfor- ward deep neural network design using weights +1, 0, and -1. In Signal Processing Systems (SiPS), 2014 IEEE Workshop on, pages 1–6. IEEE, 2014. 3

    2014

  16. [24]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceed- ings of the IEEE conference on computer vision ...

    2018

  17. [25]

    A study of bfloat16 for deep learning training

    Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellem- pudi, Dipankar Das, Kunal Banerjee, Sasikanth Avan- cha, Dharma Teja V ooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, et al. A study of bfloat16 for deep learning training. arXiv preprint arXiv:1905.12322, 2019. 1

    1905 arXiv

  18. [26]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361,

    2001 arXiv

  19. [27]

    Openvla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Fos- ter, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kol- lar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Ope...

    2024 arXiv

  20. [28]

    Kisa: A unified keyframe identifier and skill annotator for long-horizon robotics demonstrations

    Longxin Kou et al. Kisa: A unified keyframe identifier and skill annotator for long-horizon robotics demonstrations. 2, 5

  21. [29]

    Quantizing deep convolu- tional networks for efficient inference: A whitepaper

    Raghuraman Krishnamoorthi. Quantizing deep convolu- tional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342, 2018. 3

    2018 arXiv

  22. [30]

    Stabilizing off-policy q-learning via boot- strapping error reduction

    Aviral Kumar, Justin Fu, Matthew Soh, George Tucker, and Sergey Levine. Stabilizing off-policy q-learning via boot- strapping error reduction. Advances in neural information processing systems, 32, 2019. 4

    2019

  23. [31]

    Brecq: Pushing the limit of post-training quantization by block reconstruc- tion

    Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. Brecq: Pushing the limit of post-training quantization by block reconstruc- tion. arXiv preprint arXiv:2102.05426, 2021. 1

    2021 arXiv

  24. [32]

    Loftq: Lora- fine-tuning-aware quantization for large language models

    Yixiao Li, Yifan Yu, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, and Tuo Zhao. Loftq: Lora- fine-tuning-aware quantization for large language models. arXiv preprint arXiv:2310.08659, 2023. 3

    2023 arXiv

  25. [33]

    Cirl: Controllable imitative reinforcement learning for vision-based self-driving

    Xiaodan Liang, Tairui Wang, Luona Yang, and Eric Xing. Cirl: Controllable imitative reinforcement learning for vision-based self-driving. In Proceedings of the European conference on computer vision (ECCV) , pages 584–599,

  26. [34]

    Awq: Activation-aware weight quan- tization for llm compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, and Song Han. Awq: Activation-aware weight quan- tization for llm compression and acceleration. arXiv, 2023. 1, 3, 7, 2

    2023

  27. [35]

    Libero: Benchmarking knowl- edge transfer for lifelong robot learning

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowl- edge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36, 2024. 2, 3, 7, 1

    2024

  28. [36]

    Nvidia tensor core pro- grammability, performance & precision

    Stefano Markidis, Steven Wei Der Chien, Erwin Laure, Ivy Bo Peng, and Jeffrey S Vetter. Nvidia tensor core pro- grammability, performance & precision. In 2018 IEEE in- ternational parallel and distributed processing symposium workshops (IPDPSW), pages 522–531. IEEE, 2018. 3

    2018

  29. [37]

    Low precision policy distillation with application to low-power, real-time sensation-cognition-action loop with neuromorphic comput- ing

    Jeffrey L Mckinstry, Davis R Barch, Deepika Bablani, Michael V Debole, Steven K Esser, Jeffrey A Kusnitz, John V Arthur, and Dharmendra S Modha. Low precision policy distillation with application to low-power, real-time sensation-cognition-action loop with neuromorphic comput-...

    2018 arXiv

  30. [38]

    Up or down? adap- tive rounding for post-training quantization

    Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Chris- tos Louizos, and Tijmen Blankevoort. Up or down? adap- tive rounding for post-training quantization. In International conference on machine learning, pages 7197–7206. PMLR,

  31. [39]

    Ni and The ncnn contributors

    H. Ni and The ncnn contributors. ncnn: high-performance neural network inference computing framework optimized for mobile platforms. https : / / github . com / Tencent/ncnn, 2017. 8

    2017

  32. [40]

    Nvidia tegrastats toolkit

    NVIDIA Corporation. Nvidia tegrastats toolkit. https: //docs.nvidia.com/drive/drive_os_5.1.6. 1L/nvvib_docs/index.html#page/DRIVE_OS_ 10 Linux _ SDK _ Development _ Guide / Utilities / util_tegrastats.html. 8, 4

  33. [41]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 3

    2023 arXiv

  34. [42]

    An algorithmic perspective on imitation learning

    Takayuki Osa, Joni Pajarinen, Gerhard Neumann, J Andrew Bagnell, Pieter Abbeel, Jan Peters, et al. An algorithmic perspective on imitation learning. Foundations and Trends® in Robotics, 7(1-2):1–179, 2018. 4

    2018

  35. [43]

    Pruning with scaled policy constraints for light- weight reinforcement learning

    Seongmin Park, Hyungmin Kim, Hyunhak Kim, and Jung- wook Choi. Pruning with scaled policy constraints for light- weight reinforcement learning. IEEE Access , 12:36055– 36065, 2024. 3

    2024

  36. [44]

    Very deep convo- lutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. 3

    2014 arXiv

  37. [45]

    Algorithm-hardware co-design of adaptive floating-point encodings for resilient deep learning infer- ence

    Thierry Tambe, En-Yu Yang, Zishen Wan, Yuntian Deng, Vijay Janapa Reddi, Alexander Rush, David Brooks, and Gu-Yeon Wei. Algorithm-hardware co-design of adaptive floating-point encodings for resilient deep learning infer- ence. In 2020 57th ACM/IEEE Design Automation Confer- en...

    2020

  38. [46]

    Deepmind control suite

    Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, Yazhe Li, Diego de Las Casas, David Budden, Abbas Ab- dolmaleki, Josh Merel, Andrew Lefrancq, et al. Deepmind control suite. arXiv preprint arXiv:1801.00690, 2018. 1

    2018 arXiv

  39. [47]

    Carla autonomous driving leaderboard

    CARLA team. Carla autonomous driving leaderboard. https://leaderboard.carla.org/ , 2020. Ac- cessed: 2021-02-11. 1

    2020

  40. [48]

    MLC-LLM, 2023

    MLC team. MLC-LLM, 2023. 9

    2023

  41. [49]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems , pages 5026–5033. IEEE, 2012. 3

    2012

  42. [50]

    Llama 2: Open foundation and fine- tuned chat models, 2023

    Hugo Touvron et al. Llama 2: Open foundation and fine- tuned chat models, 2023. 3

    2023

  43. [51]

    Bridgedata v2: A dataset for robot learning at scale

    Homer Rich Walke, Kevin Black, Tony Z Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen-Estruch, An- dre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. Bridgedata v2: A dataset for robot learning at scale. In Con- ference on Robot Learning, pages 1723–1736. PMLR, 2023. 7

    2023

  44. [52]

    Tinyvla: Towards fast, data-efficient vision- language-action models for robotic manipulation

    Junjie Wen, Yichen Zhu, Jinming Li, Minjie Zhu, Kun Wu, Zhiyuan Xu, Ran Cheng, Chaomin Shen, Yaxin Peng, Feifei Feng, et al. Tinyvla: Towards fast, data-efficient vision- language-action models for robotic manipulation. arXiv preprint arXiv:2409.12514, 2024. 1

    2024 arXiv

  45. [53]

    Smoothquant: Accurate and effi- cient post-training quantization for large language models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and effi- cient post-training quantization for large language models. In International Conference on Machine Learning , pages 38087–38099. PMLR, 2023. 1

    2023

  46. [54]

    Ptq4vit: Post-training quantization for vision transformers with twin uniform quantization

    Zhihang Yuan, Chenhao Xue, Yiqi Chen, Qiang Wu, and Guangyu Sun. Ptq4vit: Post-training quantization for vision transformers with twin uniform quantization. In European conference on computer vision , pages 191–207. Springer,

  47. [55]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11975–11986, 2023. 3

    2023

  48. [56]

    Fastact: A lightweight actor compression framework for fast policy learning

    Hongjie Zhang, Haoming Ma, and Zhenyu Chen. Fastact: A lightweight actor compression framework for fast policy learning. In 2023 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2023. 3

    2023

  49. [57]

    End-to-end urban driving by imitat- ing a reinforcement learning coach

    Zhejun Zhang, Alexander Liniger, Dengxin Dai, Fisher Yu, and Luc Van Gool. End-to-end urban driving by imitat- ing a reinforcement learning coach. In Proceedings of the IEEE/CVF international conference on computer vision, pages 15222–15232, 2021. 1, 2, 3, 7

    2021

  50. [58]

    Poster: Self-supervised quantization-aware knowledge distillation

    Kaiqi Zhao and Ming Zhao. Poster: Self-supervised quantization-aware knowledge distillation. In 2023 IEEE/ACM Symposium on Edge Computing (SEC). IEEE. 3

    2023

  51. [59]

    Dorefa-net: Training low bitwidth convo- lutional neural networks with low bitwidth gradients

    Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. Dorefa-net: Training low bitwidth convo- lutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160, 2016. 3

    2016 arXiv

  52. [60]

    Lidar-ptq: Post-training quantization for point cloud 3d ob- ject detection

    Sifan Zhou, Liang Li, Xinyu Zhang, Bo Zhang, Shipeng Bai, Miao Sun, Ziyu Zhao, Xiaobo Lu, and Xiangxiang Chu. Lidar-ptq: Post-training quantization for point cloud 3d ob- ject detection. arXiv preprint arXiv:2401.15865, 2024. 1

    2024 arXiv

  53. [61]

    Towards effective low-bitwidth convolutional neural networks

    Bohan Zhuang, Chunhua Shen, Mingkui Tan, Lingqiao Liu, and Ian Reid. Towards effective low-bitwidth convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7920–7928,

  54. [63]

    Pick up the black bowl next to the ramekin and place it on the plate

    Experiments Details 1.1. Benchmark Details Robot Manipulation: We employed the LIBERO bench- mark [35] to assess the efficacy of SQIL within the realm of robot manipulation. The LIBERO benchmark includes four distinct task suites, each designed to facilitate life-long learn- i...

  55. [64]

    10 for Sec

    Attention Map Analysis Additional visualizations are provided in Fig. 10 for Sec. 4.4, illustrating a broad spectrum of tasks. The full-precision (FP) policy consistently demonstrates precise focus on rel- evant task objects and their specific locations, particularly where int...

  56. [65]

    To assess the distinct im- pacts of different quantization methods, we separately eval- uate each approach, as detailed in Table 10

    Ablation Study Quantization Impact Analysis. To assess the distinct im- pacts of different quantization methods, we separately eval- uate each approach, as detailed in Table 10. Applying RTN quantization without any calibration for reducing quantiza- tion errors results in a s...

  57. [66]

    Detailed settings Device Settings: For our experimental setup, we utilized NVIDIA Jetson AGX Orin 64GB and RTX 2080Ti GPU

    Implementation Details 4.1. Detailed settings Device Settings: For our experimental setup, we utilized NVIDIA Jetson AGX Orin 64GB and RTX 2080Ti GPU. NVIDIA Jetson AGX Orin 64GB is equipped with a 12-core Arm Cortex-A78AE CPU, an NVIDIA Ampere architecture GPU. The device run...

  58. [2018]

    3 11 Saliency-Aware Quantized Imitation Learning for Efficient Robotic Control Supplementary Material

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.