{"id":"211a9fd1-e544-4bd6-80ac-f40ce69e82b2","arxiv_id":"2506.17213","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"InfGen is a single next-token-prediction transformer that interleaves motion simulation with scene generation, keeping traffic realistic over 30-second rollouts better than motion-only simulators.","lead":"This paper introduces InfGen, a traffic simulator that predicts how visible cars move and, at the same time, decides when new cars should appear or disappear. It is designed for long 30-second driving simulations, where older models leave the scene unrealistically empty.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Long-term superiority claim lacks a scene-generation-capable baseline, so 'significantly outperforms all other methods' is not established; the reported composite margin over CatK is also within noise absent error bars.","rationale":"The reader's weakest assumption identifies the unvalidated long-term NLL metrics and the validation-set-tuned R as the load-bearing risk. I agree that these are serious, but the more concrete and more easily falsifiable gap is the absence of any scene-generation-capable baseline. The paper's own framing in Sec. 5.3 says baselines are 'derived by improving upon' SMART and CatK, which are motion-only; the resulting comparison demonstrates that inserting agents helps under the proposed metrics, not that InfGen is better than other ways of generating long-term scenes. The metric concerns are not independent: ACE and placement NLL are the metrics on which InfGen gains most, and those metrics are built around the same insertion/removal behavior that only InfGen possesses. A rule-based inserter is a cheap and decisive control. The short-term SOTA claim is more modest ('performs at the state-of-the-art'), and Table 1 supports competitive rather than superior performance, so my concern does not overturn the method's engineering contribution. The paper is honest about limitations, but the headline long-term claim needs either a scene-generation-capable comparison or explicit uncertainty quantification before it can be taken at face value. Since the reader already recommends CONDITIONAL, my stress-test reinforces that verdict rather than changing it.","tokens_in":20643,"tokens_out":7704,"duration_ms":86219,"concrete_test":"Run the same long-term evaluation with a non-InfGen scene-generation-capable baseline: for example, take SMART or CatK rollouts and populate empty regions using LCTGen or a simple rule-based/statistical inserter that samples agent placements from the WOMD validation distribution, then recompute ACE, placement NLL, and the adapted composite. If such a baseline closes the gap to InfGen or exceeds it, the claimed long-term superiority is an artifact of comparing against motion-only methods. As a second check, bootstrap Table 2 over validation scenarios (e.g., 100 resamples) and report 95% confidence intervals; if the InfGen-vs-CatK composite interval overlaps zero, the 'significantly outperforms' wording should be withdrawn or softened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims InfGen 'significantly outperforms all other methods in long-term (30s) simulation.' The evidence for this is Table 2, but the only baselines are SMART and CatK, both closed-loop motion simulators with no agent insertion mechanism. Their long-term rollouts necessarily empty out, so any method that adds agents will improve ACE and placement NLL almost by construction. The paper's own ablation (Table 5) substitutes a 'heuristic approach to remove agents' when the REMOVE token is disabled, showing that the authors are aware of how much the placement mechanism drives the result. No scene-generation-capable baseline is evaluated, even though LCTGen, SceneDiffuser++, ScenarioDreamer, and GPD-1 are cited in Related Work. Additionally, the long-term metrics are distributional NLLs against empirical validation-set statistics rather than paired rollouts, so a model producing generic traffic marginals can score well without modeling the specific scenario. The baseline entry/exit radius R is tuned on the validation set to maximize placement scores, while InfGen uses its fixed tokenizer radius R=75 m, making the placement comparison asymmetric. The headline composite difference over CatK is only 0.0022 (0.6606 vs 0.6584), InfGen is worse on map-based realism (0.8087 vs 0.8186), and no uncertainty estimates are provided, so 'significantly outperforms' is not supported by the reported numbers.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"InfGen proposes a unified autoregressive transformer that interleaves closed-loop motion simulation with spatial scene generation for long-term traffic simulation. Motion and pose/control tokens are arranged into a single next-token-prediction sequence, trained on WOMD with a standard cross-entropy objective. The model is evaluated on the short-term WOSAC Sim Agent benchmark and on a newly introduced long-term (30 s) protocol with adapted sliding-window metrics, placement-based NLL statistics, and a new Agent Count Error (ACE) metric. The paper claims state-of-the-art short-term performance and significant superiority over SMART and CatK in long-term simulation.","tokens_in":20966,"tokens_out":2067,"duration_ms":22863,"significance":"The central idea--unifying agent insertion, removal, and motion within one autoregressive model--is timely and addresses a real limitation of existing closed-loop simulators, which empty out over long rollouts. The method is trained on real data with a standard objective rather than a bespoke adversarial procedure, and the code release is promised. If the long-term evaluation were supported by proper baselines and uncertainty quantification, the contribution could be useful for trip-level simulation research. The short-term results are competitive, though not state-of-the-art as claimed.","major_comments":[{"comment":"The headline claim that InfGen 'significantly outperforms all other methods in long-term (30s) simulation' is not established, because the only baselines (SMART and CatK) are closed-loop motion simulators with no agent-insertion mechanism. Their long-term rollouts necessarily become sparse, so a model that adds agents will improve ACE and placement-based NLL almost by construction. The paper's own ablation in Table 5, which substitutes a heuristic agent-removal procedure when the REMOVE token is disabled, shows that the placement mechanism drives much of the reported gain. A scene-generation-capable baseline (e.g., LCTGen, SceneDiffuser++, ScenarioDreamer, or GPD-1, all cited in Related Work) or a clear explanation of why such a comparison is infeasible is needed before the superiority claim can be accepted.","section":"§5.3, Table 2"},{"comment":"The long-term evaluation rests on three newly introduced components--sliding-window adaptation of WOSAC, placement-based NLL against empirical validation-set distributions, and ACE--whose validity is not demonstrated. In particular, computing NLL against global empirical marginals can reward a model that produces generic traffic statistics rather than scenario-conditioned rollouts. The placement-based baseline radius R is tuned on the validation set to maximize baseline scores, whereas InfGen uses its fixed tokenizer radius R=75 m, making the placement comparison asymmetric. No error bars, confidence intervals, or significance tests are reported; the composite margin over CatK is 0.0022 (0.6606 vs 0.6584), and InfGen is worse on map-based realism (0.8087 vs 0.8186). The phrase 'significantly outperforms' is therefore not supported by the numbers as reported.","section":"§5.2, metrics"},{"comment":"The abstract states that InfGen 'performs at the state-of-the-art in short-term (9s) traffic simulation,' but Table 1 shows InfGen with composite 0.7514, below SMART-7M (0.7521) and CatK (0.7603). The claim should be softened to 'competitive' unless additional evidence (e.g., variance across seeds) is provided.","section":"§5.1, Table 1"},{"comment":"The motion-only analysis disables insertion and removal for all methods and reports that all methods perform similarly. This is presented as evidence that long-horizon motion simulation alone does not reveal long-term issues, but it also underscores that InfGen's advantage in Table 2 comes from the scene-generation component, which is exactly the component with no proper scene-generation baseline. The comparison should be framed accordingly.","section":"§5.3, Table 3"}],"minor_comments":[{"comment":"The text contains a placeholder 'Sec. ??' when referring to additional experiments in the Supplementary Material; this should be fixed to the actual section number.","section":"§5.3"},{"comment":"The caption contains a duplicated word: 'determines determines which task to execute next.'","section":"Figure 3 caption"},{"comment":"The phrase 'In contract, InfGen conducts dynamic scenario layout generation' should read 'In contrast.'","section":"§2, Related Work"},{"comment":"The loss notation is slightly confusing because L_pose is defined in the text as λ2 Lpos + λ3 Lhead, but the equation lists λ3 implicitly through the sum; please make the coefficient indices fully explicit.","section":"Equation (8) and text"},{"comment":"The legend in Figure 5 (and the supplementary figures) uses colored markers for initially placed agents, new agents, and the ego agent, but the caption does not define all colors; please add a legend for each marker type.","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is sound and the short-term results are credible, but the long-term evaluation is not yet convincing: the lack of a scene-generation-capable baseline, the asymmetric evaluation radius, and the absence of uncertainty measures all bear directly on the paper's primary claim. These issues are fixable with additional experiments and a more measured presentation, so I do not recommend rejection, but the current submission is not ready. I would also ask the editor to ensure that the reviewers of any revision scrutinize the implementation details of the new metrics, since the NLL-based comparisons are nonstandard."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, what to know: the interleaving of motion simulation and scene generation through mode-control tokens is a genuinely new idea, and it does produce visibly denser, more realistic 30-second rollouts than SMART or CatK. But the abstract's claim that it significantly outperforms all methods in long-term simulation is not supported by the reported evaluation. The short-term SOTA claim is also wrong on the paper's own numbers.\n\nWhat is good: the dynamic agent matrix framing is clean, and training a single next-token model on interleaved motion, insertion, and removal tokens is a natural extension of SMART's line of work. The paper documents design decisions well, the qualitative results are striking, and the limitations section and failure cases are honest. Code release is promised.\n\nThe soft spots are in the long-term evaluation. Only SMART and CatK are compared, both without any agent-insertion mechanism. Their rollouts necessarily empty out, so a model that adds agents improves placement-based NLL and ACE almost by construction. No scene-generation-capable baseline (LCTGen, SceneDiffuser++, ScenarioDreamer, GPD-1) is included, even though all are cited. The long-term metrics are distributional NLLs against empirical validation-set statistics, not paired rollouts, and they come without error bars. The composite margin over CatK is 0.0022, and InfGen is worse on map-based realism (0.8087 vs 0.8186). Calling that 'significantly outperforms' is a stretch. The evaluation radius for baselines is tuned on the validation set while InfGen uses its fixed 75 m tokenizer radius, making the placement comparison asymmetric. The ablation in Table 5 substitutes a heuristic removal when the REMOVE token is disabled, which shows the authors know how much the placement mechanism drives the result.\n\nNone of this breaks the central method. The motion-only analysis in Table 3 is reassuring: all methods are similar when insertion and removal are disabled, so InfGen's gains do come from scene generation. The method deserves peer review, but the evaluation needs to be rebuilt around paired baselines with scene generation and error bars before the headline claims can stand.\n\nWho is this for: people building trip-level traffic simulators and anyone working on tokenized agent simulation. I would bring it to our reading group and would send it to referees; expect major revisions on the evaluation, not a desk rejection.","headline":"InfGen's interleaved motion-and-scene generation is a real step forward; the long-term SOTA claim outruns the evidence.","tokens_in":21451,"tokens_out":3546,"would_cite":true,"duration_ms":31503,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"InfGen claims that a single autoregressive transformer can interleave closed-loop motion simulation with scene generation, keeping 30-second traffic rollouts realistically populated while staying competitive on the standard 9-second…","keywords":["traffic simulation","closed-loop simulation","next-token prediction","scene generation","agent insertion and removal","long-term rollout","autoregressive transformer","WOMD"],"falsifier":"Re-seed the SMART and CatK baselines with a simple non-learned rule that spawns agents from the logged entry distribution whenever the scene becomes sparse, and compare their Mean ACE and placement-based likelihood scores to InfGen on the same validation split; if they match or beat InfGen, the measured advantage comes from density maintenance rather than realistic scene generation. A second check is a human audit of the paper's own failure cases--new agents standing on road boundaries or motionless in lanes--to see whether the placement-based metric actually penalizes those errors.","tokens_in":20421,"feed_emoji":"🚗","tokens_out":10730,"duration_ms":102986,"temperature":0.7,"pith_summary":"Long-horizon traffic simulators have a known failure mode: as the ego vehicle moves, logged agents leave the observable area and no replacements appear, so the scene slowly drains empty. InfGen treats simulation as one ordered token sequence in which a single autoregressive transformer alternates between predicting the motions of active agents and generating positions and headings for newly entering agents. Given a 1.1-second log, it rolls out 30 seconds of traffic while keeping agent counts close to the logged distribution. The paper reports that InfGen stays competitive with the best short-term models on the standard 9-second benchmark and outperforms the SMART and CatK baselines on its long-term realism metrics.","feed_headline":"Traffic model spawns new agents to survive 30-second rollouts","feed_subtitle":"InfGen alternates motion prediction with scene generation, keeping long scenes populated instead of empty","key_machinery":"The dynamic agent matrix is the central object: rows are created when a new agent is inserted and deleted when an agent exits, while columns carry motion tokens over time. Motion simulation extends the matrix horizontally; scene generation extends it vertically with pose tokens sampled from a position grid and heading intervals, and removes rows marked <REMOVE AGENT>. The four mode-control tokens are the mechanism that makes both extensions trainable as a single ordered token sequence, so one next-token prediction loss supervises motion, placement, heading, and the timing of task switches. The agent query, which attends to an occupancy grid plus surrounding agents and map tokens, carries the scene-generation branch.","core_discovery":"InfGen's central claim is that fixed-agent motion simulation is the wrong unit for long horizons; the right unit is an interleaved token stream over a dynamic agent matrix whose rows are agent lifecycles and whose columns are timesteps. Four mode-control tokens--<BEGIN MOTION>, <ADD AGENT>, <KEEP AGENT>, <REMOVE AGENT>--let one decoder choose whether the next prediction is a motion token for an existing agent or a pose token for a new one, and when to switch. A motion step extends the matrix forward in time; a scene-generation step uses an agent query with grid attention over an occupancy grid to propose new agents autoregressively until <BEGIN MOTION> hands control back. Training is end-to-end next-token prediction on logged data, including the switching decision. With this, the paper reports a 30-second composite score of 0.6606 against 0.6584 for CatK and 0.6519 for SMART, a Mean Agent Count Error of 8.1 against 12.2 and 12.0, and an ACE slope of 0.15 against 0.32 and 0.31, while remaining competitive on the 9-second benchmark.","pith_inferences":["Inference: the control-token recipe generalizes: any simulator with entering and exiting entities--crowds, mixed traffic, multi-agent fleets--could be recast as interleaved next-token prediction over a dynamic entity matrix.","Inference: because the ACE slope gap is so large, a cheap baseline of wrapping existing simulators with a density-maintenance re-seeding rule would be a useful test of whether the advantage is learned scene generation or just agent-count maintenance.","Inference: the paper names map size as the rollout ceiling, so the natural follow-up is to generate map tokens in the same autoregressive stream, turning the 30-second horizon into a trip-length one."],"forward_implications":["Sustained rollouts no longer need a fixed agent set: trained on short logs, the model emits stable 30-second rollouts in which agents enter and exit as the ego moves.","The same model works on the standard short-term benchmark without task-specific changes, scoring a WOSAC composite of 0.7514, close to CatK's 0.7603.","Long-horizon evaluation can be done without one-to-one agent correspondence, using sliding-window likelihood scores and placement statistics against reference distributions instead of the original WOSAC matching assumption.","Disabling insertion and removal makes all methods score about the same, which shows that scene depletion, not motion error, is the distinguishing long-horizon problem InfGen targets."],"supporting_citations":[{"why":"Supplies the dataset used for training and validation, and the empirical distributions against which long-term placement statistics are computed.","marker":"[9]"},{"why":"Defines the WOSAC Sim Agents challenge and the 8-second evaluation setup that the paper adapts to 30-second rollouts.","marker":"[14]"},{"why":"Provides the map and motion tokenizer plus the decoder-only base model that InfGen extends, and serves as the main long-term baseline.","marker":"[31]"},{"why":"CatK is the closed-loop fine-tuned tokenized traffic model that serves as the strongest short-term comparison and a long-term baseline.","marker":"[41]"},{"why":"Represents the fixed-agent closed-loop simulation approach that the paper identifies as the cause of empty long-term scenes.","marker":"[25]"}],"fun_headline_variants":["Interleaved motion and scene generation stabilizes 30-second traffic rollouts","Autoregressive traffic model alternates motion and scene generation for long rollouts","Spawning agents on the fly: new traffic model extends simulation horizon to 30s","InfGen's mode-switching token prediction enables stable long-term traffic simulation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that scoring long rollouts against distributions estimated from the validation set, with the comparison radius for baselines tuned on that set, is a valid and sufficient measure of realism; if that scoring does not track realism, the claimed long-term superiority is not established.","fun_headline_variants_meta":{"raw":{"variants":["Interleaved motion and scene generation stabilizes 30-second traffic rollouts","Autoregressive traffic model alternates motion and scene generation for long rollouts","Spawning agents on the fly: new traffic model extends simulation horizon to 30s","InfGen's mode-switching token prediction enables stable long-term traffic simulation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000577,"raw_usage":{"total_tokens":2714,"prompt_tokens":931,"completion_tokens":1783,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":547,"completion_tokens_details":{"reasoning_tokens":1699}},"tokens_in":547,"tokens_out":1783,"duration_ms":13519,"temperature":1.0,"reasoning_tokens":1699,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:09:39.811326+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-seed the SMART and CatK baselines with a simple non-learned rule that spawns agents from the logged entry distribution whenever the scene becomes sparse, and compare their Mean ACE and placement-based likelihood scores to InfGen on the same validation split; if they match or beat InfGen, the measured advantage comes from density maintenance rather than realistic scene generation. A second check is a human audit of the paper's own failure cases--new agents standing on road boundaries or motionless in lanes--to see whether the placement-based metric actually penalizes those errors.","supporting_citations":[{"cited_title":"Large scale interactive motion forecasting for autonomous driving: The waymo open mo- tion dataset","cited_arxiv_id":null,"evidence_quote":"Supplies the dataset used for training and validation, and the empirical distributions against which long-term placement statistics are computed."},{"cited_title":"The waymo open sim agents challenge","cited_arxiv_id":null,"evidence_quote":"Defines the WOSAC Sim Agents challenge and the 8-second evaluation setup that the paper adapts to 30-second rollouts."},{"cited_title":"Smart: Scalable multi-agent real-time motion generation via next-token prediction","cited_arxiv_id":null,"evidence_quote":"Provides the map and motion tokenizer plus the decoder-only base model that InfGen extends, and serves as the main long-term baseline."},{"cited_title":"Closed- loop supervised fine-tuning of tokenized traffic models","cited_arxiv_id":null,"evidence_quote":"CatK is the closed-loop fine-tuned tokenized traffic model that serves as the strongest short-term comparison and a long-term baseline."},{"cited_title":"Promptable closed-loop traffic simulation","cited_arxiv_id":null,"evidence_quote":"Represents the fixed-agent closed-loop simulation approach that the paper identifies as the cause of empty long-term scenes."}],"review_version":2}