{"id":"8f5596e4-d874-4f49-9e01-3de39fe0ed20","arxiv_id":"2606.05480","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"CAPE benchmark shows the proposed EPM learned dynamics model yields faster laps and lower errors than DPM and DDM across five controllers in both nominal and disturbance-aware simulations.","lead":"The paper introduces the CAPE framework to benchmark five racing controllers using a new learned vehicle dynamics model called EPM, which it claims outperforms two prior models in lap time and tracking accuracy. A smart generalist might read it to see how learned models affect real controller performance in simulated uncertain conditions rather than just prediction accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Central performance claims rest on unvalidated fidelity of the learned EPM and injected disturbance model to real vehicle behavior.","rationale":"The reader’s weakest_assumption directly identifies the simulation-to-reality gap as the load-bearing assumption; the abstract-level numbers supply no independent evidence (real-vehicle runs, cross-validation metrics, or sensitivity analysis) that would falsify it. No other internal inconsistency in the reported experimental design is visible from the given material.","tokens_in":1828,"tokens_out":371,"duration_ms":29195,"concrete_test":"Acquire a fresh set of vehicle trajectories from the physical platform under operating conditions (speed range, surface, or external disturbance) deliberately excluded from the original training data; re-identify or re-evaluate all three models on this hold-out set, then re-execute the five closed-loop controllers inside the same disturbance-aware simulator; if the lap-time ordering or the reported percentage improvements reverse or shrink below 10 %, the original claims are sensitive to the particular data and disturbance distribution.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline result (Adaptive NMPC + EPM at 5.82 s vs. 12.99 s DPM / 8.80 s DDM, plus 17–29 % error reductions under η = 1 disturbances) requires that both the EPM and the composite disturbance model (measurement noise + process disturbances + actuator delay + parametric uncertainty) reproduce the dynamics and uncertainty structure that actually limit closed-loop racing performance. Because all reported numbers are generated inside this single simulation environment, any mismatch between the learned models’ training distribution and the true plant (or between the chosen disturbance statistics and physical actuator/sensor behavior) would make the relative ordering an artifact of the testbed rather than a general property of the models.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes the CAPE framework for systematic benchmarking of racing controllers using learned vehicle dynamics models. It introduces an Enhanced Physics Model (EPM) and compares it to the Deep Pacejka Model (DPM) and Deep-learning Dynamics Model (DDM) in closed-loop simulations involving five control architectures. The key results indicate that EPM achieves the best average lap times across models and controllers, with Adaptive NMPC + EPM at 5.82 s versus 12.99 s for DPM and 8.80 s for DDM, and lower tracking errors. Under disturbance-aware simulations with η = 1, EPM shows reductions in longitudinal and lateral errors and increased velocity compared to the baselines.","tokens_in":1978,"tokens_out":548,"duration_ms":32892,"significance":"If the learned EPM and the disturbance model faithfully represent the vehicle dynamics and uncertainties, this work offers a valuable methodology for evaluating learned dynamics in closed-loop control settings and demonstrates potential advantages of physics-enhanced models for improving robustness in autonomous racing applications. The use of multiple controllers and disturbance scaling provides a more comprehensive evaluation than open-loop model comparisons alone.","major_comments":[{"comment":"Abstract and §4 (Experiments): The reported numerical results (e.g., lap times of 5.82 s, 12.99 s, 8.80 s and percentage error reductions of 29.0%, 17.2%, etc.) are presented without accompanying details on the training dataset size and source, the neural network architectures for DPM, DDM, and EPM, the validation split or cross-validation procedure, or any statistical tests for significance of the differences. This information is necessary to evaluate whether the performance gains are robust or could be due to overfitting or specific hyperparameter choices.","section":"Abstract and §4"},{"comment":"Disturbance-aware simulation framework (likely §3 or §5): The parameters for measurement noise, process disturbances, actuator delay, and parametric uncertainty are not specified with reference to physical measurements or literature on real vehicle sensor/actuator characteristics. Without this grounding, the claim that the framework incorporates 'realistic uncertainties' and that EPM improves performance under them rests on unverified modeling choices that could bias the comparison.","section":"Disturbance-aware simulation framework (likely §3 or §5)"}],"minor_comments":[{"comment":"The definition of the global disturbance scaling factor η should be clarified with an equation or explicit formula in the methods section.","section":"Notation"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback on our manuscript. The comments highlight important aspects of experimental rigor and grounding that will strengthen the presentation. We address each major comment below and will revise the manuscript accordingly.","responses":[{"response":"We agree that these details are essential for assessing reproducibility and robustness. The full manuscript contains some of this information in §4, but we acknowledge it is not presented with sufficient clarity or completeness. In the revised version we will expand §4 to explicitly report: (i) the training dataset size and source (number of trajectories collected from the simulator and any real-vehicle data if used), (ii) the precise neural-network architectures (layer counts, neuron sizes, activation functions) for DPM, DDM and EPM, (iii) the validation split and any cross-validation procedure employed, and (iv) statistical tests (e.g., paired t-tests or Wilcoxon tests) on the lap-time and error differences across the five controllers, including p-values and confidence intervals. These additions will directly address concerns about overfitting or hyperparameter sensitivity.","revision_made":"yes","referee_comment":"[Abstract and §4] Abstract and §4 (Experiments): The reported numerical results (e.g., lap times of 5.82 s, 12.99 s, 8.80 s and percentage error reductions of 29.0%, 17.2%, etc.) are presented without accompanying details on the training dataset size and source, the neural network architectures for DPM, DDM, and EPM, the validation split or cross-validation procedure, or any statistical tests for significance of the differences. This information is necessary to evaluate whether the performance gains are robust or could be due to overfitting or specific hyperparameter choices."},{"response":"We accept that the disturbance parameters require explicit grounding. The current manuscript states the functional forms but does not cite supporting literature or measurements. In the revision we will add, in the disturbance-aware simulation section, specific numerical values together with references to established sources (e.g., sensor noise characteristics from automotive IMU datasheets, actuator delay measurements reported in autonomous racing literature, and tire-parameter uncertainty ranges from Pacejka-model identification studies). We will also clarify how the global scaling factor η is applied and why the chosen magnitudes are representative of real-world conditions. This will substantiate the claim of realistic uncertainties and allow readers to judge potential bias.","revision_made":"yes","referee_comment":"[Disturbance-aware simulation framework (likely §3 or §5)] Disturbance-aware simulation framework (likely §3 or §5): The parameters for measurement noise, process disturbances, actuator delay, and parametric uncertainty are not specified with reference to physical measurements or literature on real vehicle sensor/actuator characteristics. Without this grounding, the claim that the framework incorporates 'realistic uncertainties' and that EPM improves performance under them rests on unverified modeling choices that could bias the comparison."}],"tokens_in":1567,"tokens_out":624,"duration_ms":22395,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point here is that the authors built CAPE as a way to run the same set of controllers against three learned vehicle models and added a disturbance layer on top. Their EPM comes out on top for lap time and tracking error in those closed-loop runs.\n\nWhat the paper actually delivers is a consistent comparison setup. They test five controllers, swap in DPM, DDM, and EPM, then inject measurement noise, process disturbances, actuator delay, and parametric uncertainty at different scales. The headline numbers (Adaptive NMPC at 5.82 s with EPM versus 12.99 s and 8.80 s for the others, plus 12–29 % error drops at η=1) come from that single environment. That kind of controlled head-to-head is useful when you want to see how model choice affects closed-loop behavior under the same disturbances.\n\nThe limitation is exactly what the stress-test note flags: everything is generated inside their simulation. The relative ordering depends on EPM and the chosen disturbance statistics matching the dynamics that actually limit performance on hardware. The abstract gives no training data details, no real-vehicle validation, and no external check on whether the disturbance model reflects physical sensor or actuator behavior. If those assumptions are off, the ranking becomes specific to this sim rather than a general property of the models.\n\nThis is for groups working on learned dynamics for vehicle control or autonomous racing who need a ready benchmark. It has enough concrete runs and a clear methodology to go to a serious referee, though any review will probably press for hardware results or more transparent training and validation steps.","headline":"CAPE and EPM give a clean simulation benchmark for learned dynamics in racing control, but the reported gains stay inside one testbed with no real-vehicle check.","tokens_in":2489,"tokens_out":404,"would_cite":false,"duration_ms":24400,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"The CAPE framework shows that an enhanced physics model yields faster lap times and lower tracking errors than two other learned vehicle dynamics models when tested in closed-loop racing controllers under disturbances.","keywords":["vehicle dynamics modeling","learned dynamics","control benchmarking","racing controllers","nonlinear model predictive control","disturbance simulation","autonomous racing","closed-loop evaluation"],"falsifier":"Running the same five controllers on a physical vehicle and measuring whether EPM still produces lower lap times and tracking errors than DPM and DDM would confirm or refute the performance advantage.","tokens_in":2735,"feed_emoji":"🏎","tokens_out":767,"duration_ms":26817,"temperature":0.7,"pith_summary":"The paper presents the CAPE framework as a way to benchmark racing controllers using learned dynamics models in simulation. It introduces an enhanced physics model (EPM) and compares it to Deep Pacejka Model and Deep-learning Dynamics Model across five controller types. Experiments show EPM produces the lowest lap times and tracking errors, with further gains in robustness when noise, delays, and uncertainty are added to the simulation. A reader would care because accurate dynamics models directly affect whether simulated controller performance translates to real vehicles.","feed_headline":"EPM model cuts racing lap times to 5.82 s versus 12.99 s","feed_subtitle":"CAPE benchmark finds enhanced physics model lowers tracking errors by up to 29 percent under noise and delays compared with prior learned mo","key_machinery":"The enhanced physics model (EPM), a learned vehicle dynamics representation that combines physics structure with data-driven terms to improve closed-loop prediction accuracy.","core_discovery":"The CAPE framework evaluates five closed-loop control architectures under three learned vehicle dynamics models and finds that the proposed enhanced physics model (EPM) achieves the best average lap times and lowest longitudinal and lateral tracking errors across all tested controllers. Specifically, Adaptive NMPC with EPM records 5.82 s lap times versus 12.99 s for DPM and 8.80 s for DDM. When evaluated in a disturbance-aware simulator that includes measurement noise, process disturbances, actuator delay, and parametric uncertainty at moderate scaling, EPM reduces longitudinal tracking error by 29.0% and 17.2%, lateral tracking error by 24.6% and 12.3%, and increases average velocity by 39.","pith_inferences":["If EPM generalizes beyond the tested track and vehicle, the same benchmarking approach could rank dynamics models for other autonomous tasks such as highway driving or aerial navigation.","The framework could be extended by replacing the simulator with logged real-vehicle data to test how well each model predicts actual closed-loop behavior.","Designers of racing controllers might prioritize EPM-style models when simulation-to-reality transfer is the main performance bottleneck."],"forward_implications":["Controllers using EPM maintain higher average velocities while reducing both longitudinal and lateral errors under identical configurations.","The CAPE benchmark allows direct comparison of model quality through closed-loop metrics rather than open-loop prediction error alone.","EPM improves robustness when measurement noise, process disturbances, actuator delay, and parametric uncertainty are present at moderate levels.","Average results across controllers show consistent gains in velocity magnitude and error reduction relative to the two comparison models."],"fun_headline_variants":["EPM lap time 5.82s in CAPE vs 12.99s DPM","CAPE shows EPM reduces errors 29% versus DPM","CAPE tests EPM with Adaptive NMPC at 5.82s laps","EPM achieves lower tracking errors in disturbance aware CAPE"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The learned EPM and the disturbance-aware simulation framework accurately represent the vehicle dynamics and uncertainties that matter for closed-loop controller performance.","fun_headline_variants_meta":{"raw":{"variants":["EPM lap time 5.82s in CAPE vs 12.99s DPM","CAPE shows EPM reduces errors 29% versus DPM","CAPE tests EPM with Adaptive NMPC at 5.82s laps","EPM achieves lower tracking errors in disturbance aware CAPE"]},"model":"grok-4.3","cost_usd":0.009845,"raw_usage":{"total_tokens":4375,"prompt_tokens":820,"num_sources_used":0,"completion_tokens":81,"cost_in_usd_ticks":98453000,"prompt_tokens_details":{"text_tokens":820,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3474,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":820,"tokens_out":81,"duration_ms":34284,"temperature":1.0,"reasoning_tokens":3474,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T04:40:11.143423+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the same five controllers on a physical vehicle and measuring whether EPM still produces lower lap times and tracking errors than DPM and DDM would confirm or refute the performance advantage.","supporting_citations":[],"review_version":1}