{"id":"d5e34acd-6421-49f6-81cd-8ccd46d0069d","arxiv_id":"2607.20175","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"PerceptDrive reaches 90.4 PDMS on NAVSIM v1 and 90.2 EPDMS on NAVSIM v2 with a single camera by retaining per-branch perception priors and using scene-conditioned, metric-distilled expert routing.","lead":"A self-driving planner that keeps geometry, semantics, and motion knowledge separate and re-weights them per scene reaches new top scores on the NAVSIM driving benchmark. It outputs one trajectory with no test-time candidate search, and part of the gain comes from training the router to match the benchmark's own scoring metric.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"NAVSIM SOTA scores are credible, but the broader 'direct planning' claim depends on NAVSIM's rule-based scorer transferring to closed-loop objectives, which the paper leaves untested.","rationale":"The reader's weakest assumption — that NAVSIM's non-reactive rule-based sub-metrics are a valid proxy for driving quality — is exactly the point on which the paper's broader conclusion rests. The reported benchmark numbers themselves are not seriously in doubt given the internal evidence: standard errors are small, the full model separates from every ablation via paired bootstrap, and the official evaluator is used for all headline scores. The soft spot is generalization: because Laux and Lroute are supervised from the same sub-metric family that defines EPDMS, the headline gain could in principle be mostly an artifact of fitting that specific scorer. The authors acknowledge this explicitly in the Discussion, so the concern is not hidden or misleading; it is a stated limitation. That makes the appropriate verdict conditional on external validation, not rejection. I would not change the reader's CONDITIONAL verdict, because my assessment aligns with the reader's weakest assumption rather than uncovering a new flaw. A closed-loop or re-weighted-objective evaluation is the one check that would move the verdict to ACCEPT or REJECT depending on outcome.","tokens_in":20672,"tokens_out":10211,"duration_ms":115670,"concrete_test":"Once checkpoints are released, run the PerceptDrive policy in an interactive closed-loop simulator (e.g., reactive NAVSIM v2 environment or nuPlan closed-loop benchmark) under the same sensor setup, and compare against DriveFuture and DriveWAM baselines. If PerceptDrive's relative ranking inverts or its margin over baselines collapses, the SOTA claim is specific to NAVSIM's non-reactive rule-based scorer. A cheaper complementary check: re-train with Lroute/Laux targets restricted to a subset of sub-metrics (e.g., drop LK/EC) and see whether the reported 90.2 EPDMS advantage persists; if it flips under re-weighting, the gain is largely scorer alignment.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The benchmark claim (90.4 PDMS, 90.2 EPDMS) is internally well supported: official evaluator scores, three-seed stability, paired bootstrap separation, and surrogate validation. The load-bearing weakness is the step from 'best on NAVSIM metrics' to 'improves direct planning.' PerceptDrive's metric-aware branch and router are explicitly trained to maximize the same privileged rule-based sub-metrics that define EPDMS: Laux (Eq. A4) and Lroute (Eq. A7) distill NC/DAC/DDC/TLC/EP/TTC/LK/EC estimates from navtrain offline pools. The Discussion concedes that 'part of the gain reflects evaluator alignment' and that 'transfer to differently weighted objectives is untested'; closed-loop evaluation is 'left open.' All internal validations (Table A6, A7, masking profiles) are likewise within the same scorer family, so they cannot show transfer. If NAVSIM's non-reactive rule-based scorer is not a valid proxy for closed-loop driving quality, the system may be a better NAVSIM-scorer fitter rather than a better driver. This is a generalization risk, not an internal inconsistency.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"PerceptDrive proposes a perception-prior world-action model for end-to-end driving. A frozen, driving-adapted VLM provides geometric/semantic/dynamic expert tokens; a frozen V-JEPA encoder provides dense observation latents. A trainable world-action model compresses these streams with query banks, anchors each expert branch to its prior via a retention loss, predicts an action-free future latent, and fuses the expert conditions through a scene-conditioned router. Training additionally uses privileged NAVSIM sub-metrics to supervise a quality regressor and to distill soft-gate targets from one-step branch drafts. At inference the system produces a single trajectory from one front camera without candidate scoring or test-time search. The paper reports 90.4 PDMS on NAVSIM v1 navtest, 90.2 EPDMS on NAVSIM v2 navtest, and 34.5 EPDMS on navhard, with ablations across provider design, routing/fusion, and retention choices.","tokens_in":21056,"tokens_out":8150,"duration_ms":86750,"significance":"If the empirical results hold, this is a strong benchmark contribution: state-of-the-art NAVSIM v1/v2 numbers with single-trajectory, single-camera inference. The paper's support is unusually careful: official evaluator scores, three training seeds, paired bootstrap separation from ablations, surrogate validation of the routing supervision (Spearman 0.91 for k-NN interpolation, 86.2% draft-rollout agreement), hyperparameter sensitivity analysis, and a transparent Discussion of limitations. The conceptual mechanism — prior-retention-anchored expert branches with metric-distilled scene-conditioned routing — is also well motivated. The main caveat is that the routing and quality supervision are trained on exactly the rule-based sub-metrics that define the evaluation score, so part of the gain is evaluator alignment; the paper acknowledges this but does not quantify it, and transfer to closed-loop or differently weighted objectives is untested.","major_comments":[{"comment":"Data-provenance check is missing and potentially serious. The provider is full fine-tuned on 1,398,858 samples including NuScenes-QA and DriveQA, which are built on the nuScenes dataset, the same dataset from which NAVSIM navtest/navhard scenes are drawn. The paper does not state that navtest/navhard scenes or their frames were excluded from Stage 1a QA fine-tuning or from the multi-teacher distillation corpus. Since the VLM stream contributes a large share of the final EPDMS (81.9 without the stream vs. 90.2 full, Table A1), even partial overlap could materially inflate the reported SOTA. Please provide a scene-level overlap analysis and, if any test scenes are present in the provider training corpus, rerun with exclusion filtering.","section":"§3.2, Appendix B"},{"comment":"The broader claim that PerceptDrive 'improves direct planning' is not cleanly separable from 'optimizes the NAVSIM scorer.' The routing targets are α* = softmax(q/T_r), where q is the mean of privileged sub-metric scores (NC, DAC, DDC, TLC, EP, TTC, LK, EC) evaluated by the same rule-based planner family that defines EPDMS; the auxiliary metric branch uses the same pool. The ablations show that metric-distilled routing gives a real score gain (Table A2), but every internal validation is inside the same scorer family, so the gain could reflect alignment to the benchmark's particular weighting rather than a general improvement in driving quality. The Discussion concedes that transfer to differently weighted objectives and closed-loop evaluation are untested. This is acceptable for a benchmark-specific claim, but the abstract and conclusion should say 'under NAVSIM metrics' rather than impl","section":"§4.3 Discussion, Appendix C Eq. (A7)"},{"comment":"Reproduction is incomplete because the four provider-loss weights λ_geo, λ_sem, λ_dyn, λ_rep are never given. Appendix C reports routing/retention weights (λ_f, λ_ret, λ_r) and temperature, but the provider construction is a central component and its weighting is unspecified. Please report the values or the search range used.","section":"§3.2, Eq. (5)"}],"minor_comments":[{"comment":"The phrase 'improves direct planning' should be qualified as 'improves NAVSIM PDMS/EPDMS' unless transfer evidence is added. This is partly addressed in the Discussion, but the abstract/conclusion currently overstate the scope.","section":"Abstract and §5"},{"comment":"The axis label 'uniform 7-point range below Full' is unclear. Please state explicitly what the rings correspond to and how the bootstrap bands were computed.","section":"Figure 4(b)"},{"comment":"The navhard table formatting splits PerceptDrive sub-scores across two stage rows without a per-stage EPDMS column, making the final 34.5 EPDMS attribution ambiguous. Add per-stage EPDMS and mark which stage produces the final score.","section":"Table A3"},{"comment":"The provider is described as 'frozen' after adaptation, but it was first full fine-tuned and LoRA-adapted. Please clarify in the main text that 'frozen' refers to the deployed inference-time state, to avoid confusion about what is being frozen.","section":"§3.2"},{"comment":"The paper notes that sensor coverage, pretraining, proposal count, and inference budget differ across baselines, but this is only mentioned in passing. A small table or footnote listing these properties for each compared method would improve fairness assessment.","section":"§4.1, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The benchmark work is technically strong and transparent, and I am sympathetic to it. The key blocker is the data-provenance question: because the provider is fine-tuned on nuScenes-derived QA corpora and NAVSIM is also nuScenes-based, the paper must verify scene-level exclusion of navtest/navhard before the SOTA claim can be accepted. If the authors resolve this and add the metric-transfer qualification, I would support acceptance after minor revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the headline result is credible: 90.4 PDMS on NAVSIM v1 and 90.2 EPDMS on v2, supported by the official evaluator, three training seeds with tiny variance, paired bootstraps, and a real attempt to validate their routing surrogates. Second, the two genuinely new mechanisms—per-branch prior retention and metric-distilled expert routing—are coherent and not present in the cited baselines, but the routing targets are softmaxes of privileged rule-based sub-metric scores from the same NAVSIM PDM family that defines EPDMS. The authors admit this is evaluator alignment, and they leave closed-loop evaluation and transfer to differently weighted objectives untested.\n\nWhat the paper does well: the architecture is a sensible new combination of known machinery—frozen VLM with geometry/semantic/dynamics expert slots distilled from VGGT, V-JEPA 2, and Wan, a trainable world-action model with query banks, a scene-conditioned router, and a flow-matching actor. The ablations are better than what we usually see in this subfield: nested construction, design-choice controls, hyperparameter sensitivity that is flat rather than tuned, retention diagnostics (CKA, cross-reconstruction selectivity), and the surrogate validation for the k-NN score interpolation (Spearman 0.91) and for one-step branch drafts (86.2% agreement with full rollouts). The paper also reports compute and parameters honestly.\n\nThe soft spots are real but proportionate. The biggest one is external validity: the system is trained to maximize NAVSIM's non-reactive scorer, and all internal validation lives inside that same scorer family, so nothing in the paper tests whether this transfers to closed-loop driving or even to a differently weighted objective. That is a generalization risk, not an internal inconsistency. The absolute margin over the previous SOTA is small (0.3 in both metrics), and no code or data are released. I'd also note that the \"direct planning\" framing is accurate only relative to NAVSIM's non-reactive protocol; the authors are appropriately careful about this in the discussion.\n\nThis is a paper worth reading for anyone working on end-to-end driving or on fusing frozen perception priors into planners. The internal evidence is strong enough that I'd send it to serious peer review, and I would cite it for the architecture and the empirical result, while remaining cautious about the closed-loop claim. The right outcome is a conditional accept after the authors either release code or add an independent evaluation under a different objective or closed-loop setup.","headline":"PerceptDrive is a believable new NAVSIM SOTA with unusually careful internal validation, but part of its gain is explicitly alignment to the NAVSIM rule-based scorer, and the closed-loop transfer question is left open.","tokens_in":21584,"tokens_out":1670,"would_cite":true,"duration_ms":18345,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single-camera planner that routes frozen perception priors per scene reaches the top open-loop driving scores without test-time search.","keywords":["end-to-end autonomous driving","perception priors","expert routing","world-action model","flow-matching actor","knowledge distillation","query compression","open-loop driving benchmark"],"falsifier":"Run the planner in interactive closed-loop simulation, or re-score it with a differently weighted objective that is not used during training (e.g., a learned safety critic or naturalistic driving costs); if the routing and retention gains shrink or reverse, the improvements are largely an artifact of aligning to the benchmark's specific non-reactive scorer.","tokens_in":20585,"feed_emoji":"🚗","tokens_out":6170,"duration_ms":53836,"temperature":0.7,"pith_summary":"This paper tries to solve what it calls the prior-to-plan transfer problem: how to pass rich geometric, semantic, and dynamic knowledge from frozen perception models into a planner without losing it in compression or mixing it with fixed weights. The proposed system, PerceptDrive, feeds teacher-distilled priors and dense video latents into a trainable world-action model with three expert branches. A per-branch retention objective anchors each branch to its own prior, and a scene-conditioned router softly weights the branches before a flow-matching actor generates one trajectory. With one front-facing camera and no candidate scoring or test-time search, the paper reports the highest scores on the v1 and v2 versions of the standard open-loop driving benchmark — 90.4 and 90.2 respectively — and its ablations attribute the gain to retention and routed fusion. A sympathetic reader would care because the results suggest that frozen foundation knowledge can be preserved and adaptively recruited for direct planning at deployment.","feed_headline":"Top open-loop driving scores from one camera, no reranking","feed_subtitle":"Adaptive routing of geometry, semantics, and dynamics priors beats multi-camera and search-based planners on both metrics.","key_machinery":"The carrying mechanism is an expert-routed world-action model built around three expert conditioning branches — geometric, semantic, and dynamic — each with its own query bank. Two trainable pieces do the work: a per-branch retention probe that reconstructs the frozen prior's token states from each branch's compressed readout (with stop-gradient targets), and a lightweight router that maps a shared scene vector to soft gates on the simplex, mixing the three expert conditions before trajectory generation. A flow-matching actor is conditioned on the gated mixture plus a self-predicted action-free future latent from a frozen video encoder. During training, privileged rule-based sub-metric score","core_discovery":"The central claim is that the bottleneck between frozen perception priors and continuous trajectory generation is not the priors themselves but the interface: query compression tends to discard source-specific cues, and static fusion cannot adapt expert contributions per scene. PerceptDrive addresses both with two coupled mechanisms. Per-branch prior retention makes each compressed readout reconstruct its own frozen prior, preventing branch collapse and preserving geometry, semantics, and dynamics through the query bottleneck. Metric-distilled routing uses privileged rule-based sub-metric scores, available only during training, to supervise soft gates that recombine expert conditions at the","pith_inferences":["Editorial extension: Because the router is supervised by the same sub-metrics that define the benchmark, part of the reported margin likely reflects alignment to that specific scorer; retraining the gate on a differently weighted objective is a direct way to test how much of the gain is transferable.","Editorial extension: The gate values could be read as an interpretable per-scene signal of which prior matters most (curves toward geometry, straight roads toward dynamics), suggesting they might be useful for downstream monitoring or explanation, something the paper does not claim.","Editorial extension: The same retention-plus-routing recipe could apply to other embodied tasks that condition on frozen foundation models — for example manipulation or navigation — but the paper only demonstrates it for driving."],"forward_implications":["A direct single-trajectory planner can match or exceed systems that rely on candidate scoring, reranking, or multiple cameras on open-loop benchmarks.","Evaluator knowledge can be amortized: distilling privileged sub-metrics into the router during training keeps inference a single feed-forward pass.","Frozen, heterogeneous perception priors can be combined at the conditioning level rather than through static fusion, with per-branch retention preventing the compression bottleneck from erasing source-specific information.","Per-branch prior retention induces specialized reads — branch readouts become more distinguishable and reconstruct their own priors better — so additional frozen priors can in principle be added with dedicated branches.","The ablations show the gains are not solely from learnable weights or trajectory averaging, so routing before generation is the component that carries the improvement."],"fun_headline_variants":["One camera, no reranking: adaptive priors top driving benchmarks","Adaptive routing of perception priors hits SOTA driving scores","Scene-adaptive expert routing beats multi-camera planners","Single camera, no search: adaptive priors set new driving records","Retain and route perception priors for top open-loop driving"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the benchmark's rule-based sub-metrics for safety, comfort, compliance, and progress are a valid proxy for real driving quality, because the router and quality regressor are trained to maximize exactly those metrics; if those metrics do not reflect closed-loop or differently weighted objectives, the reported advantage could shrink or disappear.","fun_headline_variants_meta":{"raw":{"variants":["One camera, no reranking: adaptive priors top driving benchmarks","Adaptive routing of perception priors hits SOTA driving scores","Scene-adaptive expert routing beats multi-camera planners","Single camera, no search: adaptive priors set new driving records","Retain and route perception priors for top open-loop driving"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000815,"raw_usage":{"total_tokens":3432,"prompt_tokens":789,"completion_tokens":2643,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":2557}},"tokens_in":533,"tokens_out":2643,"duration_ms":18132,"temperature":1.0,"reasoning_tokens":2557,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T10:33:44.536677+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the planner in interactive closed-loop simulation, or re-score it with a differently weighted objective that is not used during training (e.g., a learned safety critic or naturalistic driving costs); if the routing and retention gains shrink or reverse, the improvements are largely an artifact of aligning to the benchmark's specific non-reactive scorer.","supporting_citations":[],"review_version":1}