{"id":"8c99e2ab-a7b8-4d9d-9c3c-2c502758a790","arxiv_id":"2412.18408","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A prototype pipeline reconstructs road scenes in the Godot game engine from images, with an unvalidated STL-based method to constrain road modifications.","lead":"This paper presents a prototype pipeline that extracts road shapes from images and rebuilds them as drivable roads inside the Godot game engine. The authors position this as a flexible alternative to formal-language scenario generation for testing autonomous driving systems.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central image-to-road claim is untested: the only demonstration uses a synthetic pixel drawing as input, and the paper's own Discussions state that contour extraction on real images requires manual threshold tuning and fails under occlusions.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the pipeline depends on reliable contour extraction from real images, but the only test uses a synthetic image that avoids this failure mode. My stress-test confirms this is the central weakness. The paper's contribution is a prototype pipeline, and the strongest claim is about reconstructing roads from images. However, the experimental section evaluates only the downstream tile-generation step on a clean binary mask, while the upstream image-understanding step is acknowledged to be fragile and manually tuned. No quantitative accuracy metrics, baselines, code, or data are provided. The STL-based refinement component is only proposed, not implemented or tested, so it cannot rescue the reconstruction claim. The paper is honest about its limitation in the Discussions section, and the idea has practical potential, but the evidence is too thin to support the central claim as a validated result. Therefore the reader's REJECT verdict stands, and I see no additional load-bearing concern beyond this one.","tokens_in":5880,"tokens_out":2200,"duration_ms":22262,"concrete_test":"Run the pipeline with fixed preprocessing and a single fixed threshold on a held-out set of at least 20 real road images with known ground-truth road geometry (e.g., KITTI road segmentation labels or manually drawn masks). Report quantitative agreement between the generated road tiles and ground truth using mask IoU or centerline Hausdorff distance, and report the fraction of images for which contour extraction fails catastrophically. A substantial failure fraction or poor metrics would show that the Section 1 claim about image-based reconstruction is unsupported; success would resolve the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 1's strongest claim is that the pipeline can generate road models from images and 'roughly replicate original road geometries.' The only end-to-end demonstration is Fig. 3, whose input is described as 'a basic pixel drawing' — a synthetic binary mask, not a real image. The reconstruction stage consumes this mask via per-pixel white detection, so it never exercises the image-processing stage (brightness/contrast/sharpness adjustment, Gaussian blur, contour extraction, thresholding) that is supposed to handle real images. The first stage is precisely where the paper locates its primary limitation: the Discussions section admits that contour detection accuracy is the main limitation, that each image may require a different threshold, and that Fig. 1a shows tree shadows causing occlusions. Since output fidelity is entirely downstream of this untested stage, the evidence does not support the claim that the pipeline works 'from images.' This is an empirical-evidence gap rather than an internal inconsistency; the pipeline may work on clean inputs, but the paper's own stated limitation prevents the general claim from being accepted as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a scenario-generation pipeline for the Godot simulator that extracts road contours from images using OpenCV and reconstructs them as road tiles inside the simulator. It also proposes Signal Temporal Logic (STL) specifications to constrain road perturbations and filter generated variations. The experimental section demonstrates the pipeline on a synthetic pixel drawing, showing that a binary mask can be converted into a Godot scene, but it reports no quantitative error metrics and does not exercise the image-processing stage on a real image. The STL component is described but not implemented or evaluated. The paper's Discussions section identifies contour-detection accuracy and manual threshold tuning as the main limitations.","tokens_in":6200,"tokens_out":4895,"duration_ms":45086,"significance":"If validated, the idea of reconstructing editable road scenes in a game engine from images could be a useful complement to formal-language-based scenario generation for CPS testing, particularly because the sequential tile construction allows human-in-the-loop adjustment. The paper is honest about its main limitation, and the synthetic demonstration does provide some evidence that tile-based road reconstruction from a binary mask works. However, the central claim that the pipeline works 'from images' is not supported by the reported experiments, and the formal-methods contribution remains only a proposal. The conceptual contribution is modest but potentially useful if the missing evidence is supplied.","major_comments":[{"comment":"The only end-to-end demonstration uses a 'basic pixel drawing' as input (Figure 3a), and the text states that this image is generated from handwritten data. The central claim in Section 1 that the pipeline generates road models 'from images' is therefore unsupported: the image-processing stage (brightness/contrast/sharpness adjustment, Gaussian blur, contour extraction, thresholding) is not exercised on a realistic input. The Discussions section itself identifies contour-detection accuracy as the primary limitation and notes that each image may require a different manually tuned threshold, so the reported experiment does not address the stage where the paper locates its main risk.","section":"Section 4, 'Scenario Reconstruction Using Synthetic Roads'"},{"comment":"The reconstruction step consumes the contour only as a binary white/non-white pixel mask ('If a pixel is white—indicating it is part of the road—'), so the simulator-side tile placement is validated, but the computer-vision part of the pipeline is bypassed in the reported experiment. A demonstration on at least one real road image, with the full contour-extraction and thresholding stages applied before the mask is passed to the generator, is needed to support the abstract's claim of reconstructing scenes 'directly from provided images.'","section":"Section 4, 'Scenario Reconstruction Using Image'"},{"comment":"The STL specifications are not operationalized. The signals e1 and d1 are described only verbally, the thresholds (10, t1, t2) are never assigned values or units, and no experiment applies an STL monitor to filter generated perturbations. As a result, the contribution listed in Section 1 as 'Propose integrating formal method specifications' remains a proposal, and the paper provides no evidence that the formal-method component of the pipeline works.","section":"Section 3.2, specifications φ1–φ3"},{"comment":"No quantitative error metric is reported for the reconstruction. Section 2.2 defines distance metrics d_p on splines, but the paper never uses these metrics to compare the original road geometry with the reconstructed one, so the claim in Section 1 that the approach 'can roughly replicate original road geometries' is supported only by visual inspection of Figure 3.","section":"Section 4, Figure 3"}],"minor_comments":[{"comment":"The sentence 'Figure 1a shows how shadows from trees can affect the contour, causing occlusions along the edge of the road' does not match the caption of Figure 1, where subfigure (a) is labeled 'Fit splines'; please correct the cross-reference or the figure caption.","section":"Section 4, Discussions"},{"comment":"The text says 'the third figure illustrates how it is reconstructed in the Godot simulator,' but Figure 3 contains four subfigures and subfigure (c) is labeled 'Add new pixel colors'; the intended reference is unclear.","section":"Figure 3 caption and surrounding text"},{"comment":"The thresholds in φ1–φ3 are all written as '10'; if this is intentional, please state the units, and if not, please distinguish the three thresholds. The parameters t1 and t2 are introduced but never defined.","section":"Section 3.2, φ1–φ3"},{"comment":"It is unclear how the spline distance metric d_p relates to the pixel contour data used in Section 4; please clarify the mapping from contour points to spline representations and whether this metric is used anywhere in the experiments.","section":"Section 2.2"},{"comment":"There are several typographical errors, including 'Peraltai' in the author list and 'prop/hastieerties' in reference [18]; these should be corrected.","section":"Title page and references"}],"recommendation":"major_revision","confidential_remarks":"The gap between the paper's central claim and the reported experiments is substantial but empirical rather than logical. If the authors can add a real-image case study and quantitative evaluation, the paper could become a solid short contribution; if not, a reject would be justified. I chose major_revision because the missing evidence is addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The idea is real and the execution is honest, but the evidence is too thin to accept. The paper builds a pipeline that takes a road image, extracts a contour, fits a spline, and instantiates a drivable road in Godot, plus a notional STL layer for constraining perturbations. That combination is new in the cited literature, and it is a sensible way to get flexible, scriptable maps without the manual construction overhead of CARLA or AirSim.\n\nThe writing is straightforward and the paper does not oversell itself. The Discussions section explicitly admits that contour detection is the main limitation, that thresholds need manual tuning per image, and that tree shadows cause occlusions. The related work is competent, and the one self-citation (Qin et al.) is contextually appropriate, not self-promotion. The circularity burden is low — nothing is fitted to a held-out label.\n\nThe central claim that the pipeline reconstructs road geometries \"from images\" is untested. The only end-to-end demo, Figure 3, uses a basic pixel drawing — a binary white-road mask — so the image-processing stage (brightness, blur, threshold, contour extraction) is bypassed. You cannot tell whether the pipeline works on real photos from this paper. The STL-based refinement is proposed but never implemented; we see formulas, not monitoring results. There are no quantitative error metrics, no baselines, no code or data release. The free parameters (contour threshold, STL thresholds t1/t2, perturbation parameters) are unspecified. The paper's own limitation statement confirms that the risk is real: if contour extraction fails on a real image, the whole pipeline fails.\n\nThis reads as a workshop-scale prototype description, not a validated research result. I agree with the reviewer's rejection. The idea could mature into a useful engineering contribution, but it needs at least (1) evaluation on real street-view or dashcam images with and without shadows, (2) quantitative road-shape error metrics, and (3) an implemented STL monitoring step that actually filters perturbations.\n\nFor a serious venue, a desk reject is appropriate; sending this to referees would mostly produce the same critique. If the authors resubmit with real-image evaluations and a working STL component, the idea would be worth a look.","headline":"A genuinely new pipeline idea, but the evidence is a synthetic-mask demo and the STL refinement is only proposed, so this is a workshop prototype, not a validated result.","tokens_in":6606,"tokens_out":2797,"would_cite":false,"duration_ms":26365,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A pipeline reconstructs road geometries in the Godot simulator from images, using splines and Signal Temporal Logic to constrain generated variations.","keywords":["scenario generation","cyber-physical systems","simulation testing","Godot","road reconstruction from images","Signal Temporal Logic","contour extraction","spline fitting"],"falsifier":"Take a real photograph of a road with tree shadows crossing the pavement edge and run the pipeline without hand-tuning the detection threshold for that image; if the extracted contour misses or splits the road boundary and the reconstructed road visibly departs from the source geometry, the central claim that images can roughly reproduce road geometries is falsified. The synthetic test in the paper does not exercise this shadow-occlusion failure mode.","tokens_in":5661,"feed_emoji":"🛣️","tokens_out":11680,"duration_ms":93602,"temperature":0.7,"pith_summary":"The paper introduces a pipeline that turns images of roads into drivable road models inside the Godot game engine, offering an alternative to formal-language scenario generation for testing cyber-physical systems. The central claim is that computer-vision extraction of a road contour, followed by spline fitting and procedural tile placement, can roughly reproduce the original road geometry in simulation. The paper also argues that representing roads as splines makes it possible to generate constrained variations, with Signal Temporal Logic formulas filtering out perturbations that violate design thresholds. A demonstration on a hand-drawn synthetic road shows the reconstructed road appearing in the simulator, while the authors identify contour-detection accuracy and manual threshold tuning as the main limitations.","feed_headline":"Turn road photos into testable simulator roads","feed_subtitle":"A pipeline turns road photos into drivable Godot road maps; temporal-logic formulas filter the variations.","key_machinery":"The load-bearing object is the road-contour spline: a smooth curve fitted to the contour extracted from a source image. This spline carries the road geometry from the image into the simulator, where it is pixelated and converted into per-pixel tile values; Godot then instantiates the corresponding road tiles using a tile-selection rule. The same spline representation is what makes modification possible, since perturbations are applied to the spline and then filtered by Signal Temporal Logic (STL), a formalism for expressing constraints on real-valued signals over time. The specifications $\\phi_1 := G(e_1 < 10)$, $\\phi_2 := G(d_1 < 10)$, and $\\phi_3 := G((e_1 > 10) \\Rightarrow F_{[t_1,t_2]} G(d_1 < 10 \\wedge e_1 < 10))$ constrain the perturbation error $e_1$ and the distance $d_1$ between generated splines.","core_discovery":"The paper claims that road reconstruction from images can be broken into a repeatable pipeline: preprocess the image, detect the road contour, fit splines to that contour, pixelate the result, and transmit per-pixel tile choices to Godot, which instantiates road tiles programmatically. Because the road is represented as splines rather than as a single 3D object, the geometry can be perturbed, for example with sinusoidal variations, and the perturbations can be constrained by STL specifications that keep the added error and the distance between splines below specified thresholds. The synthetic-road experiment demonstrates that this pipeline yields a road in the simulator whose shape matches the drawn input; the paper presents this as evidence that image-based techniques can roughly replicate original road geometries.","pith_inferences":["The decisive test for the pipeline is on real photographs with shadows and lighting variation; the synthetic hand-drawn evaluation in the paper does not exercise the contour-detection failure mode the authors themselves flag, so a measurable next step is to run the pipeline on such images and compare reconstructed contours against ground truth.","The STL constraints could be inverted into a search objective: maximizing or minimizing the robustness degree of the formulas would let an automated sampler generate road variants at a chosen distance from the original, turning scenario refinement into an optimization problem rather than a manual threshold choice.","The contour-to-spline representation is not tied to Godot specifically; the same coordinate and tile-selection logic could be ported to any simulator that exposes procedural scene generation, so the core claim is about the image-to-geometry mapping rather than the engine.","The current reconstruction covers only road geometry; extending it to lane markings, curbs, and buildings would require additional segmentation stages, so the result should be read as road-shape reconstruction, not full scene reconstruction."],"forward_implications":["Road maps for simulation testing could be generated from photographs instead of being hand-constructed, reducing a major bottleneck in current autonomous-driving simulators.","Because the road is stored as splines, the reconstruction supports real-time human interaction and on-the-fly adjustments inside the simulator.","STL-filtered perturbations let one source road yield many variants whose deviations from the original stay within quantifiable thresholds, increasing scenario diversity for CPS testing.","The pipeline can be combined with existing scenario-generation methods to vary both vehicle arrangements and map geometry at the same time, broadening the set of testable scenarios.","The design anticipates connecting multiple reconstructed scenarios seamlessly and replacing manual threshold tuning with learning-based road detection."],"supporting_citations":[{"why":"supplies the Godot engine's code-driven node and tile system that the pipeline uses to instantiate roads from transmitted data.","marker":"[10]"},{"why":"provides the tile-selection instructions the pipeline follows to choose which road segment to instantiate from neighboring pixels.","marker":"[17]"},{"why":"defines Signal Temporal Logic and its monitoring algorithm, which the paper uses to specify and filter admissible road perturbations.","marker":"[18]"}],"fun_headline_variants":["Photo-to-drivable road in Godot","Road photos become Godot test scenes","Image-based road reconstruction for CPS testing","Spline-road pipeline from single images","Godot road tiles from photo contours"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline assumes that a road's contour can be reliably extracted from a given image using current OpenCV preprocessing and a manually tuned threshold; the paper's own discussion identifies contour-detection accuracy as the primary limitation.","fun_headline_variants_meta":{"raw":{"variants":["Photo-to-drivable road in Godot","Road photos become Godot test scenes","Image-based road reconstruction for CPS testing","Spline-road pipeline from single images","Godot road tiles from photo contours"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000113,"raw_usage":{"total_tokens":986,"prompt_tokens":792,"completion_tokens":194,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":408,"completion_tokens_details":{"reasoning_tokens":132}},"tokens_in":408,"tokens_out":194,"duration_ms":2941,"temperature":1.0,"reasoning_tokens":132,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:40:43.641603+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a real photograph of a road with tree shadows crossing the pavement edge and run the pipeline without hand-tuning the detection threshold for that image; if the extracted contour misses or splits the road boundary and the reconstructed road visibly departs from the source geometry, the central claim that images can roughly reproduce road geometries is falsified. The synthetic test in the paper does not exercise this shadow-occlusion failure mode.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the Godot engine's code-driven node and tile system that the pipeline uses to instantiate roads from transmitted data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the tile-selection instructions the pipeline follows to choose which road segment to instantiate from neighboring pixels."},{"cited_title":"In: FORMATS, pp","cited_arxiv_id":null,"evidence_quote":"defines Signal Temporal Logic and its monitoring algorithm, which the paper uses to specify and filter admissible road perturbations."}],"review_version":1}