{"id":"f49a3487-980c-460b-b218-92bc43aface6","arxiv_id":"2412.01522","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A driving world model that generates over 1500 frames of 576x1024 driving video, claiming state-of-the-art long-term consistency.","lead":"InfinityDrive generates driving videos up to two minutes long at 576x1024 resolution. It combines memory injection, an adaptive loss, and curriculum training to reduce the drift that limits existing world models.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 1500-frame headline claim rests on an unmeasured extrapolation: all quantitative metrics stop at 100–120 frames, while Eq. (6) deliberately down-weights distant frames, so minute-scale drift control is not established.","rationale":"The paper's stated contribution is precisely the temporal extension: spatio-temporal co-modeling, extended temporal training, and memory mechanisms are introduced to 'break time limits.' The quantitative results at 100–120 frames (Table 2, Fig. 3) are competitive and the ablations in Table 3 support the individual components, which is real evidence for the method's short-to-medium-horizon behavior. However, the headline claim is the 1500-frame, minute-scale capability, and that is exactly the regime for which no quantitative measurement is provided. The discrepancy between Fig. 4's 'up to 1200 frames' and the abstract/Table 1's '1500 frames' reinforces that the minute-scale evidence is not systematic. Moreover, the mechanism is not obviously length-agnostic: Eq. (6) deliberately down-weights the loss for distant frames, so the objective explicitly relaxes fidelity to the condition as distance grows. Whether this relaxation is harmless at 120 frames and still harmless at 1500 frames is an empirical question that the paper does not answer. The reader's conditional verdict is therefore appropriate: the request for metrics at 1500 frames, error bars, the full experimental protocol, and released artifacts is directly tied to the central claim and does not overreach. My stress-test identifies no separate fatal flaw; the concern is the same load-bearing extrapolation, sharpened by the design of the loss weighting.","tokens_in":13965,"tokens_out":4712,"duration_ms":44226,"concrete_test":"Run the released inference pipeline under the same evaluation protocol as Table 2 for at least 10 seeds, and compute FID, FVD, and a temporal-consistency metric (e.g., LPIPS between adjacent frames or a scene-retention metric) on fixed windows anchored at frames 120, 500, 1000, and 1500. If FVD/FID at 1500 exceeds the 120-frame value by more than the variance observed at 100 frames, or if the model cannot complete the full rollout, the headline claim is unsupported. Additionally, perform a λ sweep (e.g., 0.1×, 1×, and 10× the reported value) at 1500 frames; since Eq. (6) uses a single decay rate, the claim requires stable performance across the plausible λ range. Report the full experimental protocol, error bars, and released artifacts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is 'consistent video generation lasting over 1500 frames (more than 2 minutes)' at 576×1024. The only quantitative support is Table 2, which reports FID/FVD at 25 and 100 frames, and Fig. 3, which extends to 120 frames. There is no metric, error bar, or statistical comparison at 500, 1000, or 1500 frames; the 1500-frame evidence is a set of stills (Figs. 1 and 5), and Fig. 4's caption actually says 'up to 1200 frames,' not 1500. The load-bearing assumption is that the mechanisms that control drift over 100–120 frames—memory injection/retention and the memory curve adaptive loss with a single exponential decay rate λ (Eqs. 3 and 6)—continue to control drift over a 1500-frame rollout. That assumption is structurally risky because Eq. (6) assigns exponentially smaller loss weight to frames farther from the memory block; the model is trained to be least constrained precisely in the regime where the headline claim is made. With a single λ and no reported sweep or sensitivity analysis, there is no evidence that the trained trade-off between retention and generative freedom is correct at minute scale. The missing 'Experimental Setup' section (deferred to 'Appendix' in Sec. 4.1, with no appendix present in this version), absence of error bars, and absence of released code or data make these points impossible to check. This does not disprove the claim, but it is the exact point on which the claim's validity rests.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"InfinityDrive proposes a driving world model that generates 576×1024 videos at 10 Hz and claims consistent generation for over 1500 frames (more than 2 minutes). To achieve this it introduces: (i) a spatio-temporal co-modeling (STCM) module that dynamically varies resolution and temporal density and uses skip-ROPE positional embeddings; (ii) a curriculum that extends the training window to 128 frames; (iii) memory injection and retention mechanisms that feed clean historical frames with timestep t=0; and (iv) a memory curve adaptive loss with exponentially decaying weights w(T)=e^−λT. Experiments report FID/FVD at 25 and 100 frames against prior driving world models, together with ablations, qualitative long-rollout examples, and diversity demonstrations.","tokens_in":14305,"tokens_out":5462,"duration_ms":43598,"significance":"If the 1500-frame claim were quantitatively substantiated, this would be a major step for driving world models: the paper's own comparisons show baselines degrading within 80–120 frames while InfinityDrive preserves visible structure at 1200–1500 frames. The proposed components are plausible, and the ablation study in Table 3 isolates the contributions of ETST, STCM, and MCAL. However, the manuscript provides no quantitative evidence at the claimed horizon, omits its entire experimental setup by deferring to an absent Appendix, and reports no uncertainty or significance measures. The qualitative examples are impressive but do not by themselves support the 'state-of-the-art' and 'first' claims. The work is promising but not yet substantiated.","major_comments":[{"comment":"The central claim of 'consistent video generation lasting over 1500 frames' is not evaluated quantitatively. Table 2 reports FID/FVD only at 25 and 100 frames, and Fig. 3 plots metrics only up to 120 frames. No metric is reported at 500, 1000, or 1500 frames. Moreover, the text is internally inconsistent: Sec. 4.2 states 'sustaining quality up to 1,200 frames, approximately 2 minutes,' while Fig. 1 shows 1500 frames and Fig. 4's caption says 'up to 1200 frames.' At the stated 10 Hz, 1200 frames is 2 minutes and 1500 frames is 2.5 minutes. Please add quantitative long-horizon evaluations at the claimed frame counts or revise the claims to match the evaluated horizon.","section":"§4.2, Table 2, Fig. 3"},{"comment":"The 'Experimental Setup' section consists of a single sentence referring to an Appendix, but no appendix is present in this version of the manuscript. This omits information needed to reproduce or interpret the reported numbers: dataset splits (including the 'opendv2k' dataset of Fig. 5), the FID/FVD evaluation protocol (number of generated videos, resolution, frame sampling, and reference-video set), the text-captioning pipeline, and all training hyperparameters. Without this section, Tables 2–3 and the comparison claims cannot be verified. Please include the full appendix and state the essential setup in the main text.","section":"§4.1 / Appendix"},{"comment":"The memory curve adaptive loss uses a single exponential decay rate λ in w(T)=e^−λT, and the memory segment length M (Sec. 3.2.1) is chosen by hand. Neither λ nor M is reported, and no sensitivity analysis or ablation over these hyperparameters is given. Because the loss assigns exponentially smaller weight to frames far from the memory block, the training signal is weakest precisely in the regime covered by the 1500-frame claim. This is a load-bearing design choice: please report λ and M, explain how they were selected, and provide a sensitivity study with long-horizon metrics.","section":"§3.2.2, Eqs. (3) and (6)"},{"comment":"No error bars, confidence intervals, sample counts, or statistical significance tests are reported for any FID/FVD value. Several comparisons are close (for example, adding STCM in Table 3 changes FID by only 0.11, and the short-term FVD gap between Ours and Vista is about 19), so without uncertainty estimates it is impossible to know whether these differences are meaningful. Please report means and standard deviations over multiple evaluation runs or seeds, together with the number of evaluation videos.","section":"Tables 2–3, Fig. 3"}],"minor_comments":[{"comment":"The phrases 'the first driving world model with exceptional generalization capabilities' and 'state-of-the-art performance' are stronger than the current evidence supports; please qualify these claims to match the quantitative evaluation.","section":"Abstract and §1"},{"comment":"The section heading contains a typo: 'Autonoumous' should be 'Autonomous.'","section":"§2.1"},{"comment":"The resolution scaling factor α is said to be 'randomly select[ed]' from [1,4], but the surrounding text implies discrete buckets and α=1 would mean no scaling; please clarify whether α is continuous or discrete and how the buckets are defined.","section":"§3.1.1"},{"comment":"The symbol t is used both for the diffusion timestep in Eq. (1) and for the sampled timestep U(0,1000) in Eq. (2), which is confusing; please use distinct notation for the frame index and the diffusion timestep.","section":"§3.2.1, Eqs. (1)–(2)"},{"comment":"The figure captions disagree with the abstract: Fig. 4 states 'up to 1200 frames' while Fig. 1 and the abstract claim 1500 frames; please reconcile these numbers.","section":"Fig. 4 and Fig. 1"},{"comment":"Reference [1], listed as 'Anonymous. Imagen video: ... Submitted to TMLR, 2022. Rejected,' is not a standard citation and should be replaced with the proper published or preprint reference.","section":"References"},{"comment":"The diversity evaluation is entirely qualitative; a quantitative diversity metric (for example, LPIPS variance or FVD over multiple rollouts from the same condition) would substantially strengthen the claim.","section":"§4.5, Diversity"},{"comment":"The limitations paragraph does not mention the absence of quantitative long-horizon evaluation, which is the most important limitation of the current version.","section":"Limitations and future work"}],"recommendation":"major_revision","confidential_remarks":"The core technical idea is plausible and the qualitative results are intriguing, but the manuscript in its current form does not substantiate its headline claim: the 1500-frame assertion rests on 25–120 frame quantitative evaluations, the Experimental Setup is missing because the Appendix is absent, and no uncertainty information is provided. These gaps are fixable with additional experiments and a completed manuscript, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The headline claim—a 576x1024 driving world model that rolls out 1500+ frames with consistency—is exactly what the autonomous driving community wants. If true, it would make large-scale synthetic OOD data generation practical. The architecture is a sensible combination of known pieces: dynamic resolution training (STCM with skip-ROPE), noise-free memory injection, curriculum expansion to 128-frame training windows, and an exponential memory-curve loss. Each component is motivated, the ablations show each helps at 100 frames, and the short-term numbers are competitive with Vista on FVD. The paper is clearly written and the qualitative stills are suggestive.\n\nThe soft spot is the gap between the claim and the evidence. The abstract promises 1500 frames, but the quantitative evaluation stops at 100 frames in Table 2 and 120 frames in Fig. 3. Fig. 4's caption actually says 1200 frames. There are no error bars, no sample counts, no significance tests, and no released code or data. Stills cannot demonstrate the absence of drift at minute scale. The stress-test worry about Eq. (6) lands: with a single decay rate lambda, the model receives the least training signal exactly in the far-future regime where the claim lives. That doesn't falsify the claim—autoregressive conditioning can still hold things together—but it makes the mechanism most responsible for long-horizon behavior unmeasured where it matters.\n\nThe missing pieces are also mundane: the Experimental Setup section is deferred to an appendix that is absent from this version, so the evaluation protocol, datasets, and hyperparameters are unverifiable. If I'm refereeing this, I would require metrics at 500, 1000, and 1500 frames (or a justification for why 100 frames suffices), error bars over multiple seeds, and ideally code or a demo that can be probed.\n\nBottom line: this paper is for researchers working on driving world models and long-video generation. It's a serious paper attacking a real problem, and the method deserves referee time rather than a desk reject. The 1500-frame claim is not yet supported, but the path to supporting it is clear. I would recommend a conditional accept after the authors supply full evaluation at the claimed duration, error bars, and released artifacts.","headline":"A plausible architecture for minute-scale driving world models, but the core 1500-frame claim is supported by only 100-120 frames of quantitative evaluation; worth referee time, not belief yet.","tokens_in":14818,"tokens_out":3710,"would_cite":false,"duration_ms":32658,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"InfinityDrive is presented as the first driving world model to generate over 1,500 frames of consistent 576×1024 video.","keywords":["driving world model","long video generation","diffusion transformer","spatio-temporal co-modeling","memory injection","error accumulation","autonomous driving simulation","text-to-video generation"],"falsifier":"Run the Table 2 protocol on InfinityDrive's actual 1,500-frame generations: compute FID and FVD on frames 1,300–1,500 with the same reference distribution used for the 100-frame scores, and compare with those scores. If the numbers climb steeply, or if a specific vehicle or building tracked in the released videos changes identity between frames 1,000 and 1,500, the extrapolated minute-scale claim fails; the model would still be a strong 100- to 120-frame world model, but not the 1,500-frame one advertised.","tokens_in":13791,"feed_emoji":"🚗","tokens_out":9209,"duration_ms":73526,"temperature":0.7,"pith_summary":"InfinityDrive is presented as the first driving world model that can synthesize more than 1,500 frames of 576×1024 driving video, lasting over two minutes, while keeping the scene spatially and temporally coherent. The paper argues that existing driving world models fall short either by generating only a few seconds of video, by dropping resolution to reach longer horizons, or by drifting into blur when rolled out autoregressively. Its proposed remedy has three parts: an efficient spatio-temporal co-modeling module that shifts processing effort between spatial detail and temporal density depending on resolution, a curriculum that extends training windows to 128 frames, and memory injection/retention paired with a memory-curve loss that down-weights distant frames so errors do not compound. If the claim holds, world models become a practical source of minutes-long, diverse, out-of-distribution driving footage for training and testing autonomous systems.","feed_headline":"World model drives 1,500 frames at full resolution","feed_subtitle":"Clean-memory injection and an Ebbinghaus-style loss keep two-minute driving rollouts from drifting into blur.","key_machinery":"The load-bearing object is the memory block inside the autoregressive rollout. The input is split into a memory segment of the first $M$ frames and a future segment; during training the memory frames receive diffusion timestep $t=0$ (no noise), so the model conditions on clean history while the future frames undergo full denoising. The memory curve adaptive loss then weights each future frame by $w(T)=e^{-\\lambda T}$, where $T$ is the distance from the memory segment: near frames are strongly held to the history, far frames are allowed generative freedom, and the single decay rate $\\lambda$ controls how quickly that transition happens. Around this, the Efficient Spatio-Temporal Co-Modeling module dynamically changes temporal density with resolution, and skip-ROPE positional embeddings preserve the original frame-rate indices when training frames are subsampled, letting 128-frame windows fit in GPU memory at 576×1024. These mechanisms are what the paper credits with preventing cumulative error and with keeping FID/FVD flat over time.","core_discovery":"On its own terms, the paper establishes that a diffusion-transformer driving world model can roll out high-resolution video over minute-scale horizons without the usual collapse into blur or scene drift. The central quantitative evidence is the long-term comparison in Table 2: at 100 frames, InfinityDrive reports FID 14.92 and FVD 113.91, lower than Vista (FID 34.61, FVD 234.60), SVD-XT (FID 42.99, FVD 275.08), and StreamingT2V-SVDXT (FID 181.22, FVD 1255.30), and its FID/FVD curves stay roughly flat across 40, 80, and 120 frames while those baselines climb. Qualitatively, the authors show rollouts of 1,200 and 1,500 frames in which buildings and vehicles persist and the scene continues to evolve rather than repeating the initial frames. The discovery, as the authors state it, is that the combination of an extended training schedule, clean-memory injection, and exponential loss weighting removes the error-accumulation barrier that previously limited driving world models to short clips.","pith_inferences":["The paper's own limitation note says the focus is single-view, long-duration, high-fidelity generation; the 1,500-frame result should therefore be read as a single-camera claim, not a multi-view or controllable-simulation claim.","The missing experiment is a quantitative FID/FVD measurement at 1,500 frames; the paper reports numbers only through 100 frames in Table 2 and 120 frames in Fig. 3, so running the same protocol over the full rollout would directly test the headline claim.","The single decay rate $\\lambda$ in $w(T)=e^{-\\lambda T}$ is a free parameter; a natural extension is a learned or scenario-dependent rate, since optimal retention may differ between highway cruising and dense intersections.","The $t=0$ clean-memory block plus exponential frame-distance weighting is not driving-specific; it could be transferred to other long-horizon video domains such as egocentric navigation or surveillance, with the same goal of keeping memory separate from generation."],"forward_implications":["Minutes-long, high-resolution rollouts would let a single conditioned clip generate an entire simulated trip, making world models a practical data engine for perception training rather than a source of short clips.","The roughly flat FID/FVD curves over 40–120 frames imply that the drift-control machinery should keep quality approximately constant over much longer rollouts, so simulation episodes can match the length of real drives.","Joint text-to-video and image-to-video training on re-captioned driving data means one model can sample diverse weather, layout, and traffic-rule scenarios from text while staying anchored to a given initial image.","Training windows of 128 frames let the model learn behaviors that unfold over tens of seconds, such as deceleration at intersections and yielding to turning vehicles, which short-window models cannot capture."],"supporting_citations":[{"why":"Vista is the main high-resolution driving world model baseline; its 15-second limit and quality degradation in complex scenarios define the gap InfinityDrive targets.","marker":"[12]"},{"why":"Stable Video Diffusion (SVD-XT) is the video-diffusion baseline used in short- and long-term comparisons and in the StreamingT2V extension.","marker":"[2]"},{"why":"StreamingT2V is the autoregressive long-video baseline with anchor-frame and long-short memory that InfinityDrive is compared against for consistency and drift.","marker":"[18]"},{"why":"The Ebbinghaus forgetting curve motivates the exponential memory-curve loss $w(T)=e^{-\\lambda T}$.","marker":"[10]"},{"why":"DiT is the diffusion-transformer backbone whose variational loss term $L_{VB}$ is included in the total training loss.","marker":"[34]"},{"why":"RoFormer's rotary position embedding is the basis for skip-ROPE, which keeps original temporal indices when training frames are subsampled.","marker":"[40]"},{"why":"The vision-language model used to re-caption driving datasets, enabling joint text-to-video and image-to-video training.","marker":"[60]"},{"why":"GAIA-1 demonstrates minute-scale driving generation at low resolution, setting the resolution-and-duration bar that InfinityDrive claims to meet together.","marker":"[21]"}],"fun_headline_variants":["Driving world model generates stable 2-minute videos","World model rolls out 1,500 frames of driving scenes","InfinityDrive: minute-scale video from a world model","Long-horizon driving world model beats drift for minutes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's two-minute claim rests on extrapolating from quantitative evaluation on sequences of at most 120 frames; the load-bearing assumption is that the drift-control mechanisms that keep FID and FVD flat over those frames will continue to keep a 1,500-frame rollout coherent, since the longer generation is shown only in qualitative stills.","fun_headline_variants_meta":{"raw":{"variants":["Driving world model generates stable 2-minute videos","World model rolls out 1,500 frames of driving scenes","InfinityDrive: minute-scale video from a world model","Long-horizon driving world model beats drift for minutes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000458,"raw_usage":{"total_tokens":2317,"prompt_tokens":983,"completion_tokens":1334,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":599,"completion_tokens_details":{"reasoning_tokens":1267}},"tokens_in":599,"tokens_out":1334,"duration_ms":9049,"temperature":1.0,"reasoning_tokens":1267,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:17:47.363571+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the Table 2 protocol on InfinityDrive's actual 1,500-frame generations: compute FID and FVD on frames 1,300–1,500 with the same reference distribution used for the 100-frame scores, and compare with those scores. If the numbers climb steeply, or if a specific vehicle or building tracked in the released videos changes identity between frames 1,000 and 1,500, the extrapolated minute-scale claim fails; the model would still be a strong 100- to 120-frame world model, but not the 1,500-frame one advertised.","supporting_citations":[{"cited_title":"Internlm-xcomposer-2.5: A versatile large vision language model supporting long-contextual input and output, 2024","cited_arxiv_id":null,"evidence_quote":"The vision-language model used to re-caption driving datasets, enabling joint text-to-video and image-to-video training."},{"cited_title":"Vista: A generalizable driving world model with high fidelity and versatile controllability, 2024","cited_arxiv_id":null,"evidence_quote":"Vista is the main high-resolution driving world model baseline; its 15-second limit and quality degradation in complex scenarios define the gap InfinityDrive targets."},{"cited_title":"Stable video diffusion: Scaling latent video diffusion models to large datasets, 2023","cited_arxiv_id":null,"evidence_quote":"Stable Video Diffusion (SVD-XT) is the video-diffusion baseline used in short- and long-term comparisons and in the StreamingT2V extension."},{"cited_title":"Streamingt2v: Con- sistent, dynamic, and extendable long video generation from text, 2024","cited_arxiv_id":null,"evidence_quote":"StreamingT2V is the autoregressive long-video baseline with anchor-frame and long-short memory that InfinityDrive is compared against for consistency and drift."},{"cited_title":"Ebbinghaus","cited_arxiv_id":null,"evidence_quote":"The Ebbinghaus forgetting curve motivates the exponential memory-curve loss $w(T)=e^{-\\lambda T}$."},{"cited_title":"Gaia-1: A generative world model for au- tonomous driving, 2023","cited_arxiv_id":null,"evidence_quote":"GAIA-1 demonstrates minute-scale driving generation at low resolution, setting the resolution-and-duration bar that InfinityDrive claims to meet together."}],"review_version":1}