{"id":"4cd951a8-4d0e-44d1-b281-24c05db9df84","arxiv_id":"2412.01034","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Quantization-aware imitation learning with teacher-distillation (QAIL+QBC) recovers full-precision policy accuracy under 4-bit weight and activation quantization across robot manipulation, driving, and control benchmarks.","lead":"This paper proposes fine-tuning quantized robot-control policies so they imitate both expert demonstrations and a full-precision teacher model, reducing accuracy loss from 4-bit compression. On standard robot manipulation and driving benchmarks the method nearly matches full-precision success while cutting inference latency and energy by 2.5 to 3.7 times on edge hardware.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The QBC loss in Eq. (3) samples states from the quantized policy, but Algorithm 1 trains on FP/expert states; the resulting covariate shift is unaddressed and untested.","rationale":"The reader's weakest assumption identifies the same load-bearing gap: Eq. (3) is an on-policy expectation, while Algorithm 1 is an offline distillation over full-precision and expert rollouts. In imitation learning with closed-loop deployment, covariate shift is the classic failure mode, and this paper's method is explicitly designed to prevent compounding action errors. Because the paper provides no quantization-policy rollouts, no state-marginal divergence measurement, and no ablation that varies the data collection policy, the mechanism behind QBC is not demonstrated. The concern is concrete and testable rather than a matter of taste. It does not, however, by itself collapse the empirical claims: the reported results are directionally plausible, and the mismatch could be benign if the quantized and full-precision policies visit nearly the same states. That is exactly why the appropriate verdict remains CONDITIONAL rather than REJECT: the authors should either fix the algorithm to match Eq. (3) or provide evidence that the off-policy approximation is adequate. Reproducibility concerns such as missing code and error bars are real but secondary; the covariate-shift issue bears directly on whether the proposed method, as stated, implements its own loss.","tokens_in":18118,"tokens_out":3644,"duration_ms":36474,"concrete_test":"Modify Algorithm 1 to include a third dataset D_Q collected by periodically rolling out the current quantized policy in the LIBERO and CARLA environments, and train on D_QAIL U D_Q with the same hyperparameters. Re-run the LIBERO-Spatial row of Table 4 and the NoCrash-dense row of Table 5, reporting means and variances over multiple seeds. If the on-policy variant does not improve success rate or reduce the divergence between the deployed quantized state distribution and the training-state distribution relative to the original algorithm, the covariate-shift concern is settled; if it improves substantially, Algorithm 1 as written is not implementing Eq. (3).","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 defines L_QBC = E_{s_t~pi^q_theta} D(pi^q_theta(a_t|s_t), pi_fp(a_t|s_t)), so the distillation signal is supposed to be evaluated on the quantized policy's own state distribution. Algorithm 1 instead populates D_QAIL = D_FP U DE, where D_FP is collected by rolling out the full-precision policy, and both loss terms are evaluated on those states. The paper never collects or re-weights states from the quantized policy during training. In a closed-loop sequential policy, a small action error at time t shifts the state distribution at time t+1, so QBC trained on teacher/expert states can be optimizing the discrepancy on states the quantized policy never visits, leaving compounding errors unregularized. This is not merely a notational gap: the claimed 'robustness throughout the sequence' in Sec. 4.2 depends on the training-state distribution matching deployment, and no ablation in Tables 3-5 or the appendix measures or corrects this mismatch. The central accuracy claim (73.1% vs. 74.0% on OpenVLA LIBERO; near-FP NoCrash scores for CILRS) could still survive, but it is currently conditional on an unvalidated on-policy assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes quantization-aware imitation learning (QAIL) combined with quantization-robust behavior cloning (QBC) to fine-tune quantized policy models for robot manipulation, autonomous driving, and continuous control. The core idea is to fine-tune a quantized policy with an imitation-learning loss on expert plus full-precision rollouts while also minimizing a distributional discrepancy against the full-precision policy. Experiments with OpenVLA on LIBERO, CILRS on NoCrash-dense, and D4RL/DeepMind Control Suite report near-full-precision success rates with 4-bit weights or weights-and-activations, together with 2.5–3.7× speedups and comparable energy savings on edge hardware such as the Jetson AGX Orin and RTX 2080 Ti. The paper claims this is the first approach to recover and deploy the performance of quantized IL-based policies successfully.","tokens_in":18318,"tokens_out":4429,"duration_ms":42316,"significance":"If the empirical claims hold, the paper demonstrates a practically important result: aggressive 4-bit quantization can be applied to large imitation-learning policies (including a 7.6B-parameter VLA model) with only a small accuracy penalty and substantial deployment gains on resource-limited hardware. The strengths of the paper include real hardware measurements with detailed latency/energy breakdowns, evaluation across three distinct policy classes, and comparison to prior quantization-aware RL methods (LPPD, QuaRL). The attention and action-distribution analyses provide useful qualitative insight even if they are not fully independent evidence. However, the central accuracy claim currently rests on single-seed comparisons with no error bars, and the QBC loss is defined on a state distribution that the algorithm does not actually sample from; both issues need to be resolved before the headline conclusions can be accepted.","major_comments":[{"comment":"The QBC loss is defined as L_QBC = E_{s_t ~ π^q_θ} D(π^q_θ(a_t|s_t), π_FP(a_t|s_t)), i.e., an expectation over states visited by the quantized policy. Algorithm 1 instead populates D_QAIL = D_FP ∪ D_E, where D_FP is collected by rolling out the full-precision policy π_FP, and both L_QAIL and λL_QBC are evaluated on those states. The training state distribution therefore differs from the deployment distribution of the quantized policy. This is load-bearing because the paper's claim of 'robustness throughout the sequence' (Sec. 4.2) relies on QBC regularizing the quantized policy's own covariate-shifted state distribution; with the implemented objective, the distillation signal is only provided on states the quantized policy may never visit. Please clarify which distribution is actually used in the experiments. If QBC is indeed computed off-policy, provide an ablation that measures the mismatch, or a theoretical argument that the off-policy objective still controls compounding errors.","section":"§4.2, Eq. (3) vs. Algorithm 1"},{"comment":"No error bars, confidence intervals, or numbers of seeds are reported anywhere. The headline result of Table 3 is a 73.1% versus 74.0% average success rate (a 0.9-point gap) on LIBERO, where each suite is evaluated over 500 episodes; with binomial sampling noise, this gap is within one standard error, and the task-level swings in Table 4 (e.g., task8: 70 vs. 82, task6: 88 vs. 94) are also consistent with noise. Similarly, Table 5 shows per-condition differences of up to 10 percentage points between QAIL+QBC and the FP baseline. Without repeated seeds or confidence intervals, the central claim that QAIL+QBC 'preserves' full-precision accuracy cannot be statistically assessed. Please report variance over multiple fine-tuning runs or at least binomial confidence intervals for the success rates.","section":"§5.2, Tables 3–5"},{"comment":"The attention-divergence analysis is partly self-confirming: QBC, as defined in Eq. (3), explicitly minimizes a discrepancy between the quantized and full-precision action distributions, so reporting that QAIL+QBC yields lower AttDiv than AWQ is expected from the construction and does not provide independent evidence that the method improves task-relevant reasoning. Please either reframe Table 2 as a sanity check of the optimization objective, or use a metric that is not directly optimized (e.g., task success conditioned on attention fixation, or a behavioral measure such as object-touch counts).","section":"§4.3, Table 2"}],"minor_comments":[{"comment":"The phrase 'first to recover and deploy the performance of quantized IL-based policies successfully' is stronger than what the evidence supports, given the absence of statistical error bars and the existence of prior quantization-aware policy methods (LPPD, QuaRL) that the paper itself compares against; please temper the claim.","section":"§1 and §6"},{"comment":"The pseudocode states 'for each state-action pair (s_t, a_t) in D_QAIL' and computes L_total = L_IL + λL_QBC, but Eq. (3) requires full action distributions rather than a single action; the pseudocode should specify how D(·,·) is computed in practice (e.g., average L2 over sampled actions, or a closed-form distance for Gaussian policies).","section":"Algorithm 1"},{"comment":"The wQBC formulation introduces three hyperparameters (β, threshold T, and the 20% / 10% selection rule) but Table 1 reports only a single LIBERO-Long comparison with no sensitivity analysis. A short ablation over β or T would help establish that the reported 2.6-point improvement is not an artifact of a single threshold choice.","section":"§4.2, Eq. (7)"},{"comment":"The main text cites the NVIDIA tegrastats tool [34] for energy measurement, while the appendix says the jetson-stats library is used; please reconcile which tool actually produced the reported energy numbers, since this affects reproducibility.","section":"§A.1.2 and §A.4.1"},{"comment":"There are several typos and minor formatting issues, including 'Autonomus Driving' in Section A.1.1, 'training proceeds wih a learning rate' in Section A.1.2, and the rendering of Eq. (6) where '1/I × J' appears garbled; a careful proofread is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and practically relevant problem, and the hardware measurements are a useful contribution. However, the statistical foundation is thinner than typical for a robotics/ML venue: the headline accuracy claims rest on single runs, and the mismatch between Eq. (3) and Algorithm 1 raises a real correctness question about the proposed training objective. I would be willing to reconsider after the authors clarify the actual QBC implementation, add confidence intervals or repeated-seed results, and address the circularity of the attention analysis. The paper's fit to the journal is acceptable; the main risk is that the central empirical claim may not survive more careful evaluation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful empirical paper, not a methodological breakthrough. The authors show that fine-tuning a quantized IL policy while distilling from a full-precision teacher can get OpenVLA and CILRS close to their original accuracy at 4-bit, with real speedups and energy savings on an edge GPU. That is worth having, and the hardware measurements are the strongest part: they break down latency per stage and measure energy on actual devices.\n\nThe new pieces are modest: QAT for imitation learning is a natural extension of standard QAT, and teacher-student distillation is an old trick. The wQBC saliency weighting is a small twist, and the paper is honest about showing QAIL and QBC separately. The D4RL results and the RL extension in the appendix add breadth.\n\nThe soft spots are real but not fatal. First, there are no error bars or seed counts anywhere. The headline LIBERO gap is 0.9 points (73.1 vs 74.0), and Table 4 shows swings like task8 dropping from 82 to 70 while task2 jumps from 66 to 94. That is entirely consistent with sampling noise, especially with 500 episodes per task. Same for NoCrash, where the differences between QAIL and QAIL+QBC are often a few points.\n\nSecond, the covariate-shift concern is legitimate. Eq. (3) defines the QBC loss over states sampled from the quantized policy, but Algorithm 1 trains on FP rollouts and expert demos. If the quantized policy drifts off the training distribution during closed-loop deployment, the distillation signal is mis-targeted. The authors do not address this or ablate an on-policy variant. I would not call it a load-bearing flaw because the empirical results still largely hold, but it undercuts the 'robustness throughout the sequence' claim in Sec. 4.2.\n\nThird, the 'first to recover and deploy' claim in Sec. 1 is too strong given LPPD already did low-precision policy distillation for RL, and the combination of QAT and distillation is standard in perception. Finally, no code or checkpoints are released, which limits verification.\n\nWho is this for? Practitioners who want to deploy VLA or driving policies on edge hardware will get value from the latency and energy numbers, and quantization researchers will find the recipe useful as a baseline. It deserves a serious referee: the scope is appropriate, the experiments are broad, and the idea is sensible. But the authors should be pushed to add seeds/error bars, test an on-policy distillation variant, and moderate the novelty claims. I would not accept it as is, but I would not desk-reject it either.","headline":"A practical quantized-IL recipe with credible hardware numbers, but the accuracy claims need error bars and the distillation setup has an unaddressed covariate-shift gap.","tokens_in":18947,"tokens_out":2677,"would_cite":false,"duration_ms":24953,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a two-part quantization-aware imitation learning loss recovers the accuracy of full-precision robotic policies at 4-bit precision, preserving near-baseline success while cutting memory several-fold and speeding up…","keywords":["quantization-aware training","imitation learning","behavior cloning","vision-language-action models","autonomous driving","robot manipulation","low-bit quantization","edge deployment"],"falsifier":"Run the QAIL+QBC trained quantized policy in closed loop, record the states it actually visits, and compare them against the states in the training set $\\mathcal{D}_{\\text{QAIL}}$; if the distribution gap is large, or if removing the full-precision rollouts from $\\mathcal{D}_{\\text{QAIL}}$ collapses success to QAIL-only levels, then the QBC benefit depends on a training-distribution assumption that may not hold at deployment.","tokens_in":17856,"feed_emoji":"🤖","tokens_out":4021,"duration_ms":36730,"temperature":0.7,"pith_summary":"Large imitation-learning policies such as vision-language-action models are too slow and memory-hungry for onboard robots, but naive low-bit quantization degrades action quality. The paper proposes quantization-aware imitation learning (QAIL) combined with quantization-robust behavior cloning (QBC): fine-tune the quantized policy on expert demonstrations while also matching the action distribution of the full-precision teacher. On OpenVLA with INT4 weights, the method reaches a 73.1 percent average success rate on LIBERO versus 74.0 percent for the Bfloat16 baseline, and on the CILRS driving model at W4A4 it approaches full-precision success on NoCrash-dense while cutting infractions. The paper reports up to 2.5x speedup and 2.5x energy savings on an edge GPU for manipulation, and up to 3.7x speedup and 3.1x energy savings on a low-end GPU for driving. If the claim holds, aggressive 4-bit quantization becomes a practical path for deploying large robotic policies on resource-limited hardware without sacrificing decision quality.","feed_headline":"4-bit policies match full-precision control accuracy","feed_subtitle":"QAIL+QBC keeps LIBERO success near baseline while cutting memory 4x and speeding up edge inference 2.5–3.7x.","key_machinery":"The machinery is a sum of two losses: the QAIL loss $\\mathcal{L}_{\\text{IL}} = -\\mathbb{E}_{(s_t,a_t)\\sim\\mathcal{D}_{\\text{QAIL}}} \\log \\pi^q_\\theta(a_t|s_t)$, trained on expert demonstrations merged with state-action pairs collected from the full-precision policy, and the QBC loss $\\mathcal{L}_{\\text{QBC}} = \\mathbb{E}_{s_t\\sim\\pi^q_\\theta} D(\\pi^q_\\theta(a_t|s_t), \\pi_{\\text{FP}}(a_t|s_t))$, which pushes the quantized policy's action distribution toward the full-precision teacher's. A weighted version increases the QBC term on states deemed important by a perturbation-based saliency score over the image. The two losses do complementary work: QAIL anchors the policy to demonstrator actions, while QBC prevents the distribution from being distorted by quantization errors, and their combination is what the paper credits for recovering baseline-level performance.","core_discovery":"The central claim is that a policy's robustness to low-bit quantization errors can be trained into it through imitation learning, and that the first successful recovery of full-precision performance for quantized IL-based policies is achieved by combining two losses: QAIL, which maximizes the likelihood of actions from expert demonstrations plus full-precision policy rollouts, and QBC, which minimizes the divergence between the quantized policy's action distribution and the full-precision policy's distribution over states. Together these losses keep the quantized policy faithful to the teacher's overall behavior while retaining the high-quality actions present in expert data. Concretely, OpenVLA with INT4 weights reaches 73.1 percent average success on LIBERO compared to 74.0 percent for the Bfloat16 baseline, and CILRS at W4A4 reaches near full-precision success rates on NoCrash-dense with reduced pedestrian collisions and red-light infractions. A saliency-weighted variant of QBC improves long-horizon manipulation tasks, and the appendix extends the same QBC idea to quantized reinforcement learning policies.","pith_inferences":["The QBC term is effectively a policy-distillation objective from the full-precision teacher, so the same idea could likely transfer to other sequential decision-making settings where a trusted full-precision policy exists and a quantized student must be deployed under latency or memory constraints.","The training data for QAIL is collected from full-precision rollouts rather than from the quantized policy's own state distribution, leaving a covariate-shift gap; collecting rollouts from the evolving quantized policy, in the style of DAgger, would be a natural test of whether closing that gap pushes the INT4 results even closer to baseline.","The reported speedups and energy savings are tied to the specific inference stacks and kernels used on the Jetson and 2080Ti platforms, and may not transfer to other hardware without re-tuning.","The attention-divergence analysis suggests QAIL+QBC restores the quantized policy's focus to task-relevant regions; a follow-up could test whether the same alignment holds under distribution shift, such as novel object layouts or weather conditions."],"forward_implications":["A 7.6B-parameter vision-language-action model can run at INT4 weights with a roughly one-percentage-point drop in LIBERO success versus Bfloat16, while using about one quarter of the memory.","A W4A4 self-driving model can reach near full-precision success and driving scores on NoCrash-dense, with pedestrian collisions reduced to near full-precision levels.","On the measured hardware, deploying these quantized policies yields up to 2.5x speedup and 2.5x energy savings for the manipulation model, and up to 3.7x speedup and 3.1x energy savings for the driving model, with 1.7x speedup and 1.3x energy savings for an 8-bit CPU deployment.","Applying saliency-weighted QBC improves long-horizon task success, raising LIBERO-Long from 47.8 percent with plain QBC to 50.4 percent."],"supporting_citations":[{"why":"Supplies the OpenVLA vision-language-action model used for all robot manipulation experiments.","marker":"[24]"},{"why":"Supplies the AWQ weight-quantization method used as the post-training-quantization baseline and as the initialization point for QAIL fine-tuning.","marker":"[28]"},{"why":"Defines the LIBERO benchmark and its four task suites used to evaluate manipulation success.","marker":"[29]"},{"why":"Supplies the CILRS image-based driving model that is quantized and evaluated on NoCrash.","marker":"[51]"},{"why":"Defines the NoCrash benchmark and the dense traffic scenario used to evaluate driving success, driving score, and infractions.","marker":"[10]"},{"why":"Supplies the LSQ learned-step-size quantization method used for weight and activation quantization in the driving and physics-simulation experiments.","marker":"[14]"},{"why":"Supplies QLoRA, the low-rank adaptation method used to fine-tune the OpenVLA weights (110M trainable parameters) during QAIL.","marker":"[11]"},{"why":"Defines the D4RL benchmark and DeepMind Control Suite tasks used to evaluate generalization to classical physics simulation.","marker":"[17]"}],"fun_headline_variants":["Quantization-aware imitation learning keeps 4-bit policies accurate","4-bit policies achieve near-lossless imitation learning","Training quantized policies with QAIL+QBC preserves accuracy","QAIL and QBC losses make 4-bit robot policies nearly lossless","4-bit policies: up to 3.7x faster, accuracy maintained"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The QBC loss assumes that the states the quantized policy will meet during deployment are well-covered by the full-precision policy rollouts and expert demonstrations used during training, and the paper does not directly validate that coverage.","fun_headline_variants_meta":{"raw":{"variants":["Quantization-aware imitation learning keeps 4-bit policies accurate","4-bit policies achieve near-lossless imitation learning","Training quantized policies with QAIL+QBC preserves accuracy","QAIL and QBC losses make 4-bit robot policies nearly lossless","4-bit policies: up to 3.7x faster, accuracy maintained"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001685,"raw_usage":{"total_tokens":6681,"prompt_tokens":951,"completion_tokens":5730,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":5642}},"tokens_in":567,"tokens_out":5730,"duration_ms":36586,"temperature":1.0,"reasoning_tokens":5642,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:44:46.564846+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the QAIL+QBC trained quantized policy in closed loop, record the states it actually visits, and compare them against the states in the training set $\\mathcal{D}_{\\text{QAIL}}$; if the distribution gap is large, or if removing the full-precision rollouts from $\\mathcal{D}_{\\text{QAIL}}$ collapses success to QAIL-only levels, then the QBC benefit depends on a training-distribution assumption that may not hold at deployment.","supporting_citations":[{"cited_title":"Awq: Activation-aware weight quan- tization for llm compression and acceleration","cited_arxiv_id":null,"evidence_quote":"Supplies the AWQ weight-quantization method used as the post-training-quantization baseline and as the initialization point for QAIL fine-tuning."},{"cited_title":"Libero: Benchmarking knowl- edge transfer for lifelong robot learning","cited_arxiv_id":null,"evidence_quote":"Defines the LIBERO benchmark and its four task suites used to evaluate manipulation success."},{"cited_title":"End-to-end urban driving by imitat- ing a reinforcement learning coach","cited_arxiv_id":null,"evidence_quote":"Supplies the CILRS image-based driving model that is quantized and evaluated on NoCrash."},{"cited_title":"Exploring the limitations of behavior cloning for autonomous driving","cited_arxiv_id":null,"evidence_quote":"Defines the NoCrash benchmark and the dense traffic scenario used to evaluate driving success, driving score, and infractions."},{"cited_title":"Learned step size quantization","cited_arxiv_id":null,"evidence_quote":"Supplies the LSQ learned-step-size quantization method used for weight and activation quantization in the driving and physics-simulation experiments."}],"review_version":1}