{"id":"bd379d05-0749-4e42-8bb7-efd6977769ff","arxiv_id":"2607.23511","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Block-wise Modal Joint Attention over image, LiDAR, and diffusion action tokens yields 88.9 PDMS / 88.4 EPDMS on NAVSIM without anchors or auxiliary supervision.","lead":"MOJITO replaces cascaded perception-then-planning stacks in end-to-end driving with three parallel Transformer branches that jointly update image, LiDAR, and action tokens every block. On NAVSIM it reports strong closed-loop scores without trajectory anchors or heavy auxiliary losses, suggesting denser sensor–planner coupling can simplify the stack.","discovery_kind":"new_method","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"The causal claim that joint attention removes the perception-to-planning bottleneck is never isolated: all gains are confounded with the DINOv3/Uni3D backbone upgrade, and camera-only MOJITO (86.8) actually trails the cascaded DiffusionDrive (88.1).","rationale":"The reader's weakest_assumption already named exactly this confound — \"that metric gains versus DiffusionDrive/WoTE are caused by removing the information bottleneck rather than by backbone choice (DINOv3+Uni3D), diffusion formulation, or hyper-parameters\" — and set CONDITIONAL/MODERATE accordingly. My pass confirms that identification and adds two concrete pieces of internal evidence the reader did not cite: (1) camera-only MOJITO underperforming cascaded baselines (Table 5 ID 1 vs Table 1), which is in tension with the claim that removing the bottleneck is the decisive factor; and (2) the observation that Table 5 ID 3 is not a true compressed-context control, so the bottleneck hypothesis has no direct test anywhere in the paper. I also note mitigating strengths consistent with the reader's posture: the code/models are promised public, the tokenization and attention-direction ablations are genuine attempts at isolation, the navhard Stage-II degradation is honestly reported, and the abstract's unqualified SOTA next to ReCogDrive-RL's 90.8 PDMS is a framing overreach the reader already caught. Because the reader's verdict already prices in this attribution risk and nothing here rises to internal contradiction or likely-wrong numbers, the verdict should remain CONDITIONAL. The proposed cascaded-same-backbone control is the single experiment that would convert this from an open attribution question into a settled one, in either direction.","tokens_in":16206,"tokens_out":1909,"duration_ms":23748,"concrete_test":"Train one controlled variant: keep DINOv3-S+ and Uni3D-S/PillarGroup frozen-identical, but replace joint self-attention with a cascaded interface — compress perception tokens into a compact latent (e.g., 64–256 tokens via a perceiver-style cross-attention pooler) and condition the same DiT planner on it via cross-attention only, with identical data, recipe, and compute. If this cascaded-same-backbone control reaches ≥88.4 PDMS, the bottleneck-removal claim is refuted and the gain is backbone-driven; if it drops to ≤87, the architectural thesis is supported. As a cheaper second check, report DiffusionDrive's planner retrained on DINOv3 features.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central thesis (§1, §3.2 Challenge 1) is that the cascaded \"compressed context\" interface discards fine-grained detail, and that block-wise Modal Joint Attention fixes this. But the experimental evidence never holds the perception backbone constant while varying only the interface. DiffusionDrive/WoTE/Transfuser use ResNet-class backbones; MOJITO uses pretrained DINOv3-S+ and Uni3D-S (§5.2), initialized from released weights and fine-tuned. Any PDMS gap is therefore jointly attributable to (a) joint attention, (b) far stronger pretrained backbones, (c) the anchor-free diffusion formulation, and (d) training recipe (batch 512, 8×H200). Two internal signals sharpen the doubt. First, Table 5 ID 1: camera-only MOJITO scores 86.8 — *below* DiffusionDrive (88.1) and WoTE (88.3), which are cascaded. If removing the bottleneck were the decisive advantage the introduction claims, the unified camera-only model should at least match cascaded camera+LiDAR baselines; it does not. Second, the nearest ablation control (Table 5 ID 3, cross-attention, 85.7) still lets action tokens query dense sensor tokens every block — it is not a compressed-context baseline, so the actual bottleneck hypothesis is tested nowhere. The one ablation that does vary tokenization (ID 2, FPS+KNN, 86.1) shows adding LiDAR can *hurt* depending on tokenizer choice, further suggesting representation quality, not architecture topology, drives results. The 88.9 number may well be real and reproducible (code is released), but the paper's narrative attributes it to the wrong, or at least unproven, cause. This is an attribution gap, not an internal inconsistency, so it bears on the framing rather than the benchmark numbers themselves.","agreement_with_reader":"agree"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The paper proposes MOJITO, a unified end-to-end driving architecture in which image tokens (DINOv3-S+), LiDAR tokens (Uni3D-S with a new \"PillarGroup\" BEV grid tokenizer), and diffusion action tokens are concatenated and processed by shared block-wise self-attention (\"Modal Joint Attention\"), eliminating the cascaded perception→planning interface, predefined anchors, and auxiliary supervision. The authors argue this removes an information bottleneck that discards fine-grained detail. Empirically, MOJITO reports 88.9 PDMS on NAVSIM-v1 navtest, 88.4 EPDMS on NAVSIM-v2 navtest, and 29.0 EPDMS on the navhard two-stage split (camera-only), each at or near the top of the respective tables, plus ablations over modality, tokenizer, and attention type, a depth-scaling study, attention visualizations, and latency numbers. Code and models are promised at a public repository.","tokens_in":16670,"tokens_out":3230,"duration_ms":60876,"significance":"If the results hold, the paper is a useful data point for the field: a compact (127M) anchor-free imitation-only model reaching 88.9/88.4 on NAVSIM-v1/v2 without RL, VLM backbones, or auxiliary tasks is a genuinely competitive and parameter-efficient result, and the anchor-free diffusion design with only 2 denoising steps at 187.65 ms latency is practically relevant. Strengths worth naming: head-to-head tables on both NAVSIM versions including the harder navhard two-stage protocol; an internal ablation (Table 5) that isolates attention type (cross- vs. self-attention) with all other factors fixed, showing a 3.2 PDMS gap in favor of joint attention; a scaling study with from-scratch-planner controls; a public code/model release; and a falsifiable architectural claim. The PillarGroup tokenizer with absolute metric scale is a simple, transferable contribution independent of the main thesis.","major_comments":[{"comment":"Abstract and §5.3, Tables 1-2: the claim of 'setting a new state-of-the-art' is not supported by the paper's own tables. Table 2 lists ReCogDrive-Base-RL at 90.8, ReCogDrive-Large-RL at 90.4, AdaThinkDrive-RL at 90.3, and AutoVLA-RL at 89.1 — all above MOJITO's 88.9. The text in §5.3 is careful ('competitive score compared to VLM-RL-based methods'), but the abstract and §1 are not. The accurate claim — best among imitation-only, anchor-free, non-RL methods — is still strong and should be stated precisely in the abstract, introduction, and conclusion.","section":"Abstract; §5.3, Tables 1-2"},{"comment":"The central causal framing (§1, §3.2 Challenge 1) — that the PDMS gains come from removing the compressed-context bottleneck via joint attention — is confounded in all cross-method comparisons by the simultaneous backbone upgrade to pretrained DINOv3-S+/Uni3D-S, the anchor-free diffusion formulation, and the training recipe. Table 5 ID 1 sharpens this: camera-only MOJITO scores 86.8, below the cascaded DiffusionDrive (88.1) and WoTE (88.3), so the unified interface alone does not dominate cascaded designs when the modality set is held to camera. Table 5 ID 3 does isolate attention topology (85.7 cross-attn vs 88.9 self-attn, same backbones) and is the paper's best evidence for the mechanism — but cross-attention still lets action tokens query dense sensor tokens every block, so the 'compressed context' hypothesis itself is tested nowhere. A controlled comparison (e.g., a cascaded variant","section":"§1, §3.2, §5.6 Table 5"},{"comment":"No variance estimates anywhere. The headline margin over WoTE is 0.6 PDMS (88.9 vs 88.3) and over DiffusionDrive 0.8, with single-run numbers and no seeds reported. NAVSIM PDMS is known to have non-trivial run-to-run variation under imitation training, and the ablation deltas in Table 5 (e.g., ID 2 at 86.1 vs ID 1 at 86.8) are interpreted directionally without any notion of noise. At minimum: report the main result and Table 5 over ≥3 seeds with std, or state explicitly the observed seed variance. This is a standard expectation for margins of this size.","section":"§5.3-5.6, Tables 1, 3, 5"},{"comment":"PillarGroup is one of the paper's load-bearing components (Table 5 ID 2 vs ID 4 attributes a 2.8 PDMS gap to tokenization), yet it is specified only as 'a fixed grid (e.g., 32×32 pillars), covering a specific metric range', top-K=512, N=64. The metric crop range, the ranking criterion in step C of Fig. 2, padding behavior, and z/height handling are unspecified, and none of (grid size, K, N) is ablated. Since the FPS+KNN degradation is attributed entirely to tokenization while backbone, point budget, and density also differ between ID 2 and ID 4, the paper should (i) fully specify PillarGroup and (ii) add at least a small sensitivity sweep over grid resolution and K, or soften the attribution.","section":"§4.2, §5.6 Table 5"}],"minor_comments":[{"comment":"Eq. (2): the network is denoted εθ but the regression target is x(0) — this is x0-prediction, not ε-prediction; the notation is misleading. Also k is called a 'diffusion step' yet sampled as k∼U(0,1); please clarify continuous vs discrete time and state the sampler and the number of inference steps (2 steps appears only in §7, with no sensitivity to step count).","section":"§4.3, Eq. (2)"},{"comment":"Fig. 3 caption says 'the red line represents the trajectory generated by DiffusionDrive, ReCogDrive, and MOJITO, respectively' — three methods cannot share one color; the figure needs distinct colors/labels per method. Fig. 5's attention maps are averaged over 8 waypoints and 3 camera views; specifying the averaging and normalization would aid interpretation.","section":"§5.5 Fig. 3; §6 Fig. 5"},{"comment":"§5.8: 'instruction following' is implemented by mapping language to a fixed NAVSIM command vocabulary via Qwen3-1.7B, i.e., the model itself is conditioned only on discrete commands. The framing in §5.8 and the contributions list should be tempered accordingly; a quantitative instruction-compliance metric would strengthen the claim.","section":"§5.8, Fig. 4"},{"comment":"§7 latency: measured on a single H200 with batch size unspecified; please report batch size, whether backbone precompute is amortized, and the latency breakdown across branches, since the joint-attention sequence length scales with NI+NL+NA.","section":"§7"},{"comment":"Reference and text glitches: [24] arXiv id '2512.107192' appears malformed; [55] author list contains 'Zhao, Seth Z.and Zhang' (missing space/formatting); §5.2 'follow Transfuser for image pre-processing' should specify exactly which augmentations/crops; Table 4 Stage I/II rows share one EPDMS column — clarify that EPDMS aggregates both stages.","section":"References; §5.2; Table 4"},{"comment":"§6 is titled 'Cross Modal Learning' but contains only an attention-map visualization analysis; either rename (e.g., 'Attention Analysis') or add the promised analysis of cross-modal learning dynamics. Also consider noting that attention weights are not, by themselves, evidence of information flow — a sentence of caveat would suffice.","section":"§6"}],"recommendation":"major_revision","confidential_remarks":"Solid empirical NAVSIM entry with released code, but the headline framing currently outruns the evidence in two places: the SOTA phrasing versus the RL-VLA rows in its own Table 2, and the bottleneck-removal causal story that is confounded with the DINOv3/Uni3D backbone upgrade (the skeptic's concern lands for cross-method attribution, though Table 5 ID 3 partially rescues the internal mechanism claim). All requested fixes are within scope — wording corrections, seed variance, and one controlled cascade-interface ablation — so I recommend major rather than minor revision only because the margins (0.6 PDMS) are small relative to unreported run variance. Industry-affiliated large-compute training (8×H200, batch 512) may limit exact reproducibility for academic readers; worth encouraging release of training configs and seeds."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: this is a clean systems paper that wires DINOv3, a metric-scale LiDAR tokenizer, and an anchor-free DiT planner through block-wise joint self-attention, and the NAVSIM numbers are strong. The causal claim that joint attention alone kills the perception–planning bottleneck is weaker than the abstract suggests.\n\nWhat is actually new is the full stack, not any single gadget. Modal Joint Attention (concat image∥LiDAR∥action tokens into shared MHSA every block), PillarGroup instead of FPS+KNN for scene-scale LiDAR, and dropping anchors/aux losses while staying ViT/DiT-compatible is a coherent design relative to UniAD/Transfuser/DiffusionDrive and the compressed-token VLAs. They earn credit for focused ablations (camera-only, FPS+KNN vs PillarGroup, cross- vs self-attn), a depth scaling curve, latency, instruction-following demos, and code. 88.9 PDMS / 88.4 EPDMS with a 127M model and no RL is a real data point against both cascaded E2E and heavy VLA pipelines.\n\nSoft spots, in proportion. The stress-test lands on framing, not on fabricated scores: baselines mostly use weaker backbones, so backbone + recipe + joint attention are entangled. Camera-only MOJITO at 86.8 sits under cascaded DiffusionDrive/WoTE, which undercuts “removing the bottleneck is decisive” if you read the intro literally. The cross-attn control still lets actions query dense tokens; it is not a compressed-context baseline. No seeds/error bars. SOTA language should sit next to the higher RL-VLA PDMS numbers they already table. None of that breaks the method result; it means the narrative over-attributes.\n\nWho it is for: people building multi-modal E2E planners who care about foundation-model compatibility and anchor-free diffusion. Worth a serious referee. I would engage—read the architecture, try the code if you are on NAVSIM—not treat the bottleneck story as settled causal science.","headline":"Solid unified E2E driving system with real NAVSIM gains and released code; the “bottleneck removed” story is only partly isolated from the backbone upgrade.","tokens_in":17686,"tokens_out":532,"would_cite":true,"duration_ms":20195,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Letting action, image, and LiDAR tokens update each other in every block removes the cascaded perception bottleneck and plans without anchors.","keywords":["Autonomous Driving","End-to-End Planning","Multi-modal Joint Learning","Modal Joint Attention","Diffusion Transformer","LiDAR Tokenization","NAVSIM"],"falsifier":"Train an otherwise identical MOJITO that freezes perception tokens from attending to action tokens (or replaces joint self-attention with one-way cross-attention from action to sensors only) and check whether PDMS/EPDMS and the qualitative failure modes collapse back toward strong cascaded baselines such as DiffusionDrive on the same navtest split.","tokens_in":17311,"feed_emoji":"🚗","tokens_out":985,"duration_ms":22874,"temperature":0.7,"pith_summary":"Most end-to-end driving stacks still squeeze cameras and LiDAR into a compact context, then hand that summary to a separate planner. The authors argue that squeeze discards the fine geometric detail trajectory planning needs and also keeps modern vision transformers from helping the planner directly. MOJITO instead runs three parallel transformer branches—image, LiDAR, and a diffusion action planner—and, in every block, concatenates their tokens so a shared self-attention layer updates all three at once. Because the planner can query dense sensor tokens while it denoises waypoints, the system needs neither hand-crafted trajectory anchors nor auxiliary 3D detection losses. On the NAVSIM benchmarks this yields 88.9 PDMS and 88.4 EPDMS, ahead of recent cascaded and vision-language baselines, while remaining a compact 127M-parameter model that also follows high-level commands and produces diverse paths.","feed_headline":"Joint attention kills the driving perception bottleneck","feed_subtitle":"Action, camera, and LiDAR tokens update together; no anchors, new NAVSIM highs at 127M params.","key_machinery":"Modal Joint Attention: inside each transformer block the image, LiDAR, and action token sequences are concatenated, given shared positional embeddings, and passed through one multi-head self-attention layer so action tokens query dense sensor features while perception features adapt to the current planning state; the joint sequence is then split and fed to each branch’s own FFN.","core_discovery":"A fully unified sensor-to-action architecture, built on block-wise Modal Joint Attention among image, LiDAR, and diffusion action tokens, can generate stable anchor-free trajectories without auxiliary supervised tasks and reaches state-of-the-art closed-loop scores on NAVSIM v1 and v2 by letting the planner read fine-grained multi-modal features at every layer instead of a compressed perception context.","pith_inferences":["If bidirectional perception–action attention is the real driver, the same joint-token pattern should transfer to other continuous control domains (manipulation, drone flight) that today still cascade a frozen encoder into a policy head.","The camera-only drop on navhard Stage II suggests joint attention still depends on complementary geometry; pure vision variants may need explicit depth or temporal cues to close that gap.","Because the action branch is trained from scratch while perception starts from foundation weights, further gains may come from co-pretraining the diffusion planner on large unlabeled trajectory corpora before NAVSIM fine-tuning.","Latency near 188 ms with two diffusion steps implies real-time use will hinge on fewer steps or distilled samplers rather than on shrinking the joint attention itself."],"forward_implications":["End-to-end planners can drop predefined trajectory anchors and auxiliary 3D box or map losses if action tokens continually attend to dense multi-modal features.","Standard ViT and DiT stacks with pretrained vision and 3D weights can be wired straight into planning without a separate compressed context interface.","Scaling aligned depth across image, LiDAR, and action branches should continue to lift closed-loop scores when more data are available.","High-level command conditioning plus diffusion sampling yields instruction-following and continuous trajectory diversity without a large vision-language backbone.","A metric-preserving pillar grid for LiDAR is required for joint attention to help; relative FPS+KNN tokenization does not."],"fun_headline_variants":["MOJITO lets planner read raw image and LiDAR at every layer","Block-wise joint attention drops the perception-planning bottleneck","Anchor-free trajectories via shared image-LiDAR-action tokens","Unified sensor-to-action hits 88.9 PDMS on NAVSIM v1","Modal joint learning sets NAVSIM v2 SOTA without cascaded context"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That higher NAVSIM non-reactive scores prove the joint attention truly preserves the fine sensor detail the authors say cascaded pipelines lose, rather than gains coming mainly from backbone choice or diffusion setup.","fun_headline_variants_meta":{"raw":{"variants":["MOJITO lets planner read raw image and LiDAR at every layer","Block-wise joint attention drops the perception-planning bottleneck","Anchor-free trajectories via shared image-LiDAR-action tokens","Unified sensor-to-action hits 88.9 PDMS on NAVSIM v1","Modal joint learning sets NAVSIM v2 SOTA without cascaded context"]},"model":"grok-4.5","effort":"low","cost_usd":0.00418,"raw_usage":{"total_tokens":1259,"prompt_tokens":789,"num_sources_used":0,"completion_tokens":79,"cost_in_usd_ticks":41804000,"prompt_tokens_details":{"text_tokens":789,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":391,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":789,"tokens_out":79,"duration_ms":6715,"temperature":1.0,"reasoning_tokens":391,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-30T20:29:55.871542+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train an otherwise identical MOJITO that freezes perception tokens from attending to action tokens (or replaces joint self-attention with one-way cross-attention from action to sensors only) and check whether PDMS/EPDMS and the qualitative failure modes collapse back toward strong cascaded baselines such as DiffusionDrive on the same navtest split.","supporting_citations":[],"review_version":1}