{"id":"9b11400a-cac2-4b1b-9b25-5085a6740708","arxiv_id":"2504.19614","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"DiVE generates multi-view driving videos conditioned on text, boxes, road sketches, and camera poses, reporting SOTA FID/FVD/KPM on nuScenes plus a 2.62x faster accelerated variant.","lead":"This paper introduces DiVE, a video-diffusion-transformer system that generates multi-view driving videos from text, 3D object boxes, road sketches, and camera poses, reporting state-of-the-art quality scores on the nuScenes benchmark. The system combines a new classifier-free-guidance distillation scheme with progressive-resolution sampling to speed up inference by 2.62x, making synthetic driving data more practical for training perception models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA and augmentation-superiority claims rest on cross-paper metrics and on Table 2 baselines that differ in resolution; no shared evaluation protocol is reported.","rationale":"The reader's conditional verdict is appropriate. The central claim of state-of-the-art generation quality and superior downstream augmentation is exactly what would be falsified if the comparisons are not controlled, and the paper gives several indications that they are not controlled: mixed resolutions in Table 1, no FVD backbone or KPM configuration, an internally inconsistent characterization of the accelerated variant's quality drop, and different StreamPETR baselines in Table 2. These are not aesthetic objections; they change what the numbers mean. The paper deserves credit for extensive ablations, re-implementing PnP/DICE, and reporting a downstream perception experiment, but those efforts do not remove the need for a shared evaluation protocol. A conditional acceptance asking for re-measurement of at least the closest prior methods (Panacea, Drive-WM) under the same pipeline, plus a controlled Table 2 rerun, is the right call. No claim of dishonesty or sloppiness is intended; the issue is that the evidence as reported cannot distinguish generator quality from evaluation-setup effects.","tokens_in":19146,"tokens_out":8942,"duration_ms":88852,"concrete_test":"Use the released checkpoints of Panacea and Drive-WM and re-evaluate them under DiVE's exact pipeline: same nuScenes validation annotations, same 16-frame clips, same FVD implementation and feature extractor, same KPM matching settings, and evaluate at each method's native resolution and at 480x854. If DiVE's FID/FVD/KPM margins do not persist within bootstrap confidence intervals, the SOTA claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline numbers are compared against values taken from prior papers, not re-measured under a single protocol. Table 1 mixes resolutions from 192x384 to 480x854, video lengths, guidance settings, and does not specify the FVD feature extractor or KPM matching settings. FVD is sensitive to frame count and backbone, and KPM depends on the matching model and thresholds, so DiVE's FVD 68.4 and KPM 73.2 margin over Panacea (139.0 / 59.2) and Drive-WM (122.7 / 45.8) cannot be taken at face value. The paper's own accelerated variant shows the scale of this sensitivity: DiVE+ has FVD 93.2 and KPM 63.0, a 36% FVD increase, yet is described as 'minimal quality degradation.' Separately, Table 2 compares augmentation gains against different real-data StreamPETR baselines: the Panacea block starts at 34.6 mAP / 47.0 NDS, while the DiVE block starts at 38.0 / 49.0. Appendix B.4 states the StreamPETR training resolution for DiVE is 480p, different from the 256x704 baseline, so the +2.9% vs +1.7% mAP comparison mixes the effect of the generator with the effect of training resolution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents DiVE, a DiT-based framework for generating multi-view driving videos under multimodal control (text, 3D object instances, road sketches, camera parameters). The architecture combines unified cross-attention, a SketchFormer for road guidance, a parameter-free view-inflated attention for cross-view consistency, and a multi-scale training schedule. To reduce inference cost, the authors introduce Multi-Control Auxiliary Branch Distillation (MAD) and Resolution Progressive Sampling (RPS), reporting a combined 2.62x speedup. Experiments on nuScenes report state-of-the-art FID 7.14, FVD 68.4, and KPM 73.2, and show that training StreamPETR with DiVE-generated data plus real data raises mAP from 38.0 to 40.9 and NDS from 49.0 to 52.0, claiming larger gains than Panacea and Glad.","tokens_in":19424,"tokens_out":6370,"duration_ms":56284,"significance":"If validated, DiVE would be a useful step toward controllable, high-resolution synthetic video for autonomous driving perception. The view-inflated attention trick is simple and parameter-free, the MAD/RPS acceleration ideas are practically motivated, and the paper includes extensive ablations and a downstream augmentation study. However, the headline SOTA and augmentation-superiority claims currently rest on cross-paper metric comparisons and on augmentation baselines that differ in resolution and starting performance. The architectural contributions are solid, but the evidence for the strongest claims needs to be re-established under a shared evaluation protocol.","major_comments":[{"comment":"The SOTA claim is based on metric values taken from prior papers without a shared evaluation protocol. Table 1 lists resolutions from 192x384 to 480x854 but does not report the number of frames, the FVD feature extractor, or the KPM matching model and thresholds for each method. FVD is sensitive to frame count and backbone, and KPM depends on the matcher and thresholds, so the reported margins (DiVE FVD 68.4 vs Panacea 139.0) cannot be taken at face value. Please either re-run all prior methods under the exact protocol used for DiVE, or provide a detailed protocol table and a sensitivity analysis. Without this, the 'state-of-the-art' claim in Section 4.3 is not fully supported.","section":"Section 4.3, Table 1"},{"comment":"There is an unexplained discrepancy in DiVE's FVD. Table 1 reports DiVE FVD 68.4 and DiVE+ FVD 93.2, while Figure 7 and Tables 4 and 5 report DiVE FVD 86.13 and DiVE+ FVD 100.18 under a reduced protocol (first 16 frames per scene across four runs). The paper does not state which protocol underlies Table 1 or whether the reduced protocol matches the protocols used for prior methods. Because the headline SOTA margin depends on the 68.4 FVD value, the exact protocol (frames per clip, number of clips, resolution, FVD implementation) must be specified for both the main table and the ablations, and the discrepancy must be explained.","section":"Section 4.4 and Table 1"},{"comment":"The augmentation comparison is confounded. The DiVE block starts from a higher real-data StreamPETR baseline (38.0 mAP / 49.0 NDS vs 34.6 / 47.0 for Panacea) and uses a different training resolution: Appendix B.4 states that StreamPETR training for DiVE is at 480p, differing from the 256x704 baseline. The +2.9 mAP improvement for DiVE over the +1.7 mAP for Panacea could therefore be due to higher perception training resolution rather than generator quality. To support the claim that DiVE-generated data is superior for augmentation, compare all generators at the same training resolution and from the same real-data baseline, e.g., by upsampling Panacea/Glad data to 480p or downsampling DiVE data to 256x512.","section":"Table 2 and Appendix B.4"},{"comment":"The claim of 'minimal quality degradation' for DiVE+ is not supported by the reported FVD. The FVD increases from 86.13 (DiVE) to 100.18 (DiVE+) in Figure 7, and from 68.4 to 93.2 in Table 1, the latter being a 36% relative increase. Since FVD is the paper's primary temporal-quality metric, 'minimal' should be replaced with a quantitative statement, or task-relevant evidence should be provided showing that the FVD change does not affect downstream perception utility.","section":"Figure 7 and Section 4.5"}],"minor_comments":[{"comment":"There are typos: 'coniguration' should be 'configuration', and 'researches' should be 'research'.","section":"Abstract"},{"comment":"The sentence 'a guidance scale value is uniformly sampled fron the continous interval' contains two typos: 'fron' and 'continous'.","section":"Section 4.2"},{"comment":"'In constrast' should be 'In contrast'.","section":"Related Work"},{"comment":"The '+1.7%' and '+2.9%' entries are absolute percentage-point changes in mAP, not relative percentages; please use 'pp' or write 'mAP +1.7 points' to avoid ambiguity.","section":"Table 2"},{"comment":"'CFG denots' should be 'CFG denotes'.","section":"Figure 1 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper builds on the authors' workshop paper (ref. [22]); the manuscript should clarify the incremental contribution relative to that earlier version. The evaluation-protocol issues in the main comparison are substantial but fixable, and I believe the architectural ideas are worth pursuing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely engineered system, but the headline SOTA and data-augmentation claims are built on cross-paper metric comparisons that aren't apples-to-apples. I'd trust the architecture and the ablations more than the absolute numbers.\n\nWhat's actually new: a DiT-based framework for multi-view driving video generation that accepts four conditioning modalities through a unified cross-attention, plus SketchFormer for BEV layouts and a parameter-free view-inflated attention that helps cross-view consistency. The two inference accelerators—MAD, which distills multi-condition CFG into cheap auxiliary branches, and RPS, which progressively raises resolution during sampling—are practical, and the ablations suggest they work. This is one of the first DiT takes on this task, and the qualitative results look a step up from MagicDrive and Panacea.\n\nNow the soft spots. The SOTA numbers in Table 1 are compared against prior papers' reported values without re-running them under a single protocol. Resolutions range from 192x384 to 480x854, and the paper never specifies the FVD feature extractor or the number of frames used for FVD/KPM. Both metrics are sensitive to those choices, so the margins over Panacea (FVD 139 down to 68, KPM 59 up to 73) can't be taken at face value. The paper's own accelerated variant underscores this: DiVE+ jumps from FVD 68.4 to 93.2, a 36% increase, yet is described as having 'minimal quality degradation.' That is an overstatement.\n\nThe augmentation comparison in Table 2 has a similar confound: DiVE's StreamPETR baseline is trained at 480p while the Panacea block uses a 256x704 baseline, but the gains are reported as percentage-point improvements without isolating resolution from generator quality. So the '+2.9% vs +1.7% mAP' claim mixes two variables. This is a testable fix: train both at the same resolution or report both configurations.\n\nNone of this kills the paper. The architecture is sensible, the ablations are mostly well-designed, and the downstream perception evaluation is an external, reasonable benchmark. But the evidence doesn't yet support 'state-of-the-art' or 'superior augmentation' claims. A serious referee should send it to major revision and ask for a unified evaluation protocol, code, and a more honest description of the quality/acceleration tradeoff.\n\nI'd bring this to our reading group to discuss the protocol issues—it's a good example of how SOTA claims in generative driving data get slippery. I wouldn't cite the headline numbers until independently verified, but I'd cite the method for its architectural ideas.","headline":"A real engineering contribution, but the headline SOTA and augmentation claims rest on uncontrolled cross-paper comparisons, so the numbers are promising, not proven.","tokens_in":19968,"tokens_out":3697,"would_cite":true,"duration_ms":36542,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Multi-view driving model lifts 3D detection mAP by 2.9 points","keywords":["controllable video generation","multi-view generation","diffusion transformer","driving scene generation","bird's-eye-view control","multi-modal conditioning","classifier-free guidance distillation","progressive resolution sampling"],"falsifier":"Run every compared generator through a single evaluation pipeline—same resolution, frame count, guidance scale, sampling steps, and identical FID/FVD/KPM implementations—then train the same 3D detector on each synthetic set at matched resolution; if DiVE no longer leads on the generation metrics or loses its downstream mAP/NDS gain, the paper's main claims are not supported.","tokens_in":18945,"feed_emoji":"🚗","tokens_out":12388,"duration_ms":109917,"temperature":0.7,"pith_summary":"DiVE is a video-generation system for autonomous-driving scenes that sets out to establish that a diffusion-transformer generator, conditioned on text, bird's-eye-view road sketches, 3D object boxes, and camera geometry, can produce multi-camera driving videos that are photorealistic, temporally coherent, and consistent across views. The paper argues that such synthetic videos are genuinely useful as training data: adding DiVE-generated clips to real nuScenes data lifts the 3D detection model's mAP from 38.0 to 40.9 and NDS from 49.0 to 52.0. To make high-resolution generation affordable, it introduces two mechanisms—a distillation scheme that replaces per-condition classifier-free guidance selection, and a training-free progressive-resolution sampler—and reports a combined 2.62x speedup with modest quality loss. The stakes are practical: if the central claim holds, controllable generation can lower the cost of collecting and annotating diverse driving video.","feed_headline":"Multi-view driving model lifts 3D detection mAP by 2.9 points","feed_subtitle":"DiVE also reports the best FID/FVD/KPM on nuScenes and a 2.62x inference speedup.","key_machinery":"The load-bearing mechanism is a diffusion transformer that interleaves spatial, temporal, cross-attention, and MLP layers, extended by three components. View-inflated attention reshapes multi-view tokens so all cameras attend together without added parameters, enforcing cross-view consistency. A unified cross-attention fuses text tokens, Fourier-encoded 3D object boxes fused with instance captions, and camera-pose embeddings; SketchFormer, a geometry-aware sketch encoder, injects bird's-eye-view road layouts through mirrored zero-initialized fusion cells. Efficiency comes from Multi-Control Auxiliary Branch Distillation (MAD), which trains lightweight condition-specific auxiliary branches to imitate multi-condition classifier-free guidance in one forward pass, and Resolution Progressively Sampling (RPS), which runs early sampling stages at lower resolutions with a resolution-aware timestep shift before refining at full resolution.","core_discovery":"The paper's central claim is that DiVE achieves state-of-the-art quality for multi-view driving video generation on nuScenes, with FID 7.14, FVD 68.4, and a cross-view key-point matching score of 73.2, improving over the prior generators it compares against. It further claims that the generated videos are controllable enough to match real-data geometry: BEV road and vehicle segmentation scores and 3D object-detection mAP/NDS approach the real-data reference, and augmenting real training data with DiVE output raises downstream perception performance by a larger margin than prior synthetic generators. The authors state these results as evidence that a diffusion transformer with multi-modal conditioning, cross-view attention, and efficient guidance handling can serve as a practical data simulator for autonomous driving perception.","pith_inferences":["Editorial inference: view-inflated attention is a generic token-grouping operation; it should transfer to any multi-view or panoramic video generator, and a direct test is measuring KPM gains when the same reshape is added to a UNet-based generator.","Editorial inference: MAD's mixed-control training—randomly nullifying subsets of conditions during distillation—is a portable recipe for multi-condition diffusion models; a layout-to-image model with text and box conditions could validate it outside driving.","Editorial inference: the RPS timestep-shift formula is stated for arbitrary resolution ratios and should port to other rectified-flow models as a training-free upsampling sampler; the paper only demonstrates the 240p-to-480p ladder.","Editorial inference: if the downstream gains reproduce, the practical endgame is targeted generation of rare driving conditions (night, rain, crowded scenes) rather than collecting them; the paper demonstrates editing ability but does not quantify a corner-case benchmark."],"forward_implications":["DiVE-generated video becomes a viable augmentation source: real-plus-synthetic training raises the video-based 3D detector's mAP by 2.9 points and NDS by 3.0 points over real data alone.","Multi-condition classifier-free guidance can be compressed to a single function evaluation, so combinatorial guidance tuning is replaced by a fixed auxiliary-branch recipe.","Parameter-free view-inflated attention gives cross-view consistency at zero added parameters, making it a drop-in change for existing DiT video architectures.","Progressive-resolution sampling with a timestep shift yields 2.62x faster high-resolution generation with only small FVD degradation.","The generated scenes remain controllable across long horizons, with stable object appearance and layout over 240-frame videos."],"supporting_citations":[{"why":"Supplies the baseline video-diffusion-transformer architecture that DiVE extends for multi-condition input.","marker":"[63]"},{"why":"Prior street-view generator with 3D geometry control whose conditioning design and scores are part of the comparison.","marker":"[11]"},{"why":"Prior panoramic video generator whose evaluation protocol, FVD/KPM scores, and downstream augmentation setup DiVE follows and compares against.","marker":"[53]"},{"why":"The video-based 3D detector used to measure whether generated data improves perception.","marker":"[47]"},{"why":"The dataset that provides driving sequences, annotations, and interpolated high-frame-rate labels for training and evaluation.","marker":"[3]"},{"why":"Classifier-free guidance, the technique MAD distills and whose multi-condition selection problem motivates the auxiliary branches.","marker":"[16]"},{"why":"Rectified flow, the sampling framework used for inference and for the stage transitions in progressive-resolution sampling.","marker":"[31]"},{"why":"BEVFormer, used in ablations as the object-detection backbone for scoring controllability of generated data.","marker":"[27]"},{"why":"Cross-view transformer used for BEV segmentation controllability scoring.","marker":"[64]"}],"fun_headline_variants":["DiVE: SOTA multi-view driving videos, 2.62x faster","Multi-view video diffusion hits SOTA, lifts 3D mAP","2.62x speedup for SOTA multi-view driving video generation","SOTA driving videos with consistent multi-view coherence and speed","DiVE: efficient multi-view driving scene generation, SOTA on nuScenes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison against prior methods assumes their published FID, FVD, and KPM scores were measured under the same evaluation protocol; because these metrics depend on resolution, frame count, and the perceptual-metric backbone, a fair re-run could narrow or change the reported margin.","fun_headline_variants_meta":{"raw":{"variants":["DiVE: SOTA multi-view driving videos, 2.62x faster","Multi-view video diffusion hits SOTA, lifts 3D mAP","2.62x speedup for SOTA multi-view driving video generation","SOTA driving videos with consistent multi-view coherence and speed","DiVE: efficient multi-view driving scene generation, SOTA on nuScenes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001086,"raw_usage":{"total_tokens":4570,"prompt_tokens":1003,"completion_tokens":3567,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":619,"completion_tokens_details":{"reasoning_tokens":3470}},"tokens_in":619,"tokens_out":3567,"duration_ms":24622,"temperature":1.0,"reasoning_tokens":3470,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:46:48.960197+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run every compared generator through a single evaluation pipeline—same resolution, frame count, guidance scale, sampling steps, and identical FID/FVD/KPM implementations—then train the same 3D detector on each synthetic set at matched resolution; if DiVE no longer leads on the generation metrics or loses its downstream mAP/NDS gain, the paper's main claims are not supported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior street-view generator with 3D geometry control whose conditioning design and scores are part of the comparison."},{"cited_title":"In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","cited_arxiv_id":null,"evidence_quote":"Prior panoramic video generator whose evaluation protocol, FVD/KPM scores, and downstream augmentation setup DiVE follows and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The video-based 3D detector used to measure whether generated data improves perception."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Rectified flow, the sampling framework used for inference and for the stage transitions in progressive-resolution sampling."}],"review_version":1}