{"id":"120d89fa-dafb-4945-9072-977afb238f10","arxiv_id":"2606.13368","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"IterCAD is a multimodal agent framework using progressive SFT and geometry-aware RL for CAD tasks, with a new data pipeline, IterCAD-Bench, and CD-TR metric showing outperformance in executability and precision.","lead":"This paper introduces IterCAD, a multimodal AI agent for iterative, closed-loop CAD generation and editing via drawing-to-code, text-to-code, and interactive tasks in a sandbox. A smart generalist might read it to see how AI can better match real-world design workflows that involve repeated refinement rather than single attempts.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Data synthesis pipeline's match to real-world CAD task distributions is the least secure link in the performance claims","rationale":"The reader's weakest_assumption exactly matches the load-bearing point; the provisional UNVERDICTED status is appropriate given the abstract-only input. No other internal inconsistency is detectable without the full text, and the absence of formal verification or shipped code noted by the reader reinforces that the data-representativeness issue remains the primary untested assumption.","tokens_in":1721,"tokens_out":366,"duration_ms":18239,"concrete_test":"Sample 50 real industrial CAD files (e.g., from GrabCAD STEP/IGES collections), extract their multi-view drawing statistics (feature count, tolerance histograms, view complexity) and editing trajectory length distributions; regenerate equivalent data with the paper's synthesis pipeline and run a two-sample Kolmogorov-Smirnov test on the key statistics; divergence p<0.01 on two or more metrics would indicate the generalization risk is realized.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim (outperformance in executability, geometric precision, and closed-loop refinement) depends on the synthesized data being representative. The abstract describes a pipeline that adds industrial features to produce drawings, editing tasks, and trajectories, then trains via SFT + geometry-aware RL and evaluates on IterCAD-Bench with CD-TR/AUC-TR. No mention is made of quantitative validation (distributional statistics, expert review, or comparison to external CAD corpora) that the generated multi-view drawings and interaction sequences reproduce the feature complexity, tolerance variability, or editing patterns of actual manufacturing workflows. If the synthetic distribution diverges, the reported gains and the new metric's claimed survivor-bias-free status become benchmark artifacts rather than evidence of general capability.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper presents IterCAD, a multimodal agent framework for closed-loop, interactive CAD generation and editing formulated as multi-turn interactions in an executable sandbox across Drawing-to-Code, Text-to-Code, and Interactive Editing tasks. It introduces a data synthesis pipeline that adds industrial manufacturing features to produce multi-view drawings, editing tasks, and trajectories; trains the agent via progressive SFT followed by geometry-aware RL with viable-prefix masking; and proposes IterCAD-Bench with the CD-TR curve and AUC-TR metric as a survivor-bias-free evaluation standard. The central claim is that IterCAD achieves highly competitive performance, significantly outperforming prior methods in code executability and geometric precision while demonstrating superior closed-loop iterative refinement.","tokens_in":1860,"tokens_out":503,"duration_ms":29320,"significance":"If the empirical claims hold, the work addresses a clear mismatch between one-shot CAD generation and iterative real-world practice, with the new benchmark and metric potentially serving as a useful standard for the community. The geometry-aware RL component with viable-prefix masking represents a concrete technical contribution for improving executability. The data synthesis approach incorporating industrial features is a positive step toward more realistic training data.","major_comments":[{"comment":"Data synthesis pipeline (described in the methods section following the task formulation): the central performance claims (outperformance in executability, geometric precision, and iterative refinement on IterCAD-Bench) rest on the assumption that the generated multi-view drawings and interaction trajectories match the feature complexity, tolerance variability, and editing patterns of real manufacturing workflows, yet no quantitative validation (distributional statistics, expert review, or comparison against external CAD corpora) is reported to support this match.","section":"Data synthesis pipeline"},{"comment":"Experimental evaluation section: the abstract and results claim significant outperformance across benchmarks with the new CD-TR/AUC-TR metric, but the provided description supplies no details on baseline implementations, number of runs, error bars, or full experimental protocol, preventing assessment of whether the reported gains are robust or benchmark-specific.","section":"Experimental evaluation"}],"minor_comments":[{"comment":"Notation for the CD-TR curve and AUC-TR metric should be defined with an explicit equation or pseudocode in the evaluation section to clarify how tolerance thresholds are applied and how survivor bias is avoided.","section":"Evaluation metric definition"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback and positive assessment of the work's significance. We address each major comment below and will revise the manuscript to incorporate the suggested improvements.","responses":[{"response":"We agree that explicit quantitative validation of the data synthesis pipeline against real manufacturing data would strengthen the central claims. In the revised manuscript we will add a dedicated subsection (or appendix) reporting distributional statistics (feature-type histograms, tolerance ranges, and editing-operation frequencies) comparing the synthesized data against external CAD corpora such as ABC and DeepCAD, together with a brief expert-review summary from two domain practitioners. This addition will be placed immediately after the data-pipeline description.","revision_made":"yes","referee_comment":"[Data synthesis pipeline] Data synthesis pipeline (described in the methods section following the task formulation): the central performance claims (outperformance in executability, geometric precision, and iterative refinement on IterCAD-Bench) rest on the assumption that the generated multi-view drawings and interaction trajectories match the feature complexity, tolerance variability, and editing patterns of real manufacturing workflows, yet no quantitative validation (distributional statistics, expert review, or comparison against external CAD corpora) is reported to support this match."},{"response":"We acknowledge that the current experimental section lacks sufficient implementation and reproducibility details. In the revision we will expand the section to include: (i) complete descriptions and hyper-parameter settings for all baselines, (ii) results averaged over five independent runs with standard deviations shown as error bars in all tables and figures, and (iii) a full experimental protocol (including random seeds, hardware, and evaluation scripts) placed in the appendix. These changes will allow readers to assess robustness directly.","revision_made":"yes","referee_comment":"[Experimental evaluation] Experimental evaluation section: the abstract and results claim significant outperformance across benchmarks with the new CD-TR/AUC-TR metric, but the provided description supplies no details on baseline implementations, number of runs, error bars, or full experimental protocol, preventing assessment of whether the reported gains are robust or benchmark-specific."}],"tokens_in":1445,"tokens_out":414,"duration_ms":9710,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that this paper builds IterCAD as a multimodal agent for closed-loop CAD work. It sets up three tasks—drawing-to-code, text-to-code, and interactive editing—inside an executable sandbox, trains with progressive SFT then geometry-aware RL that masks invalid prefixes, and evaluates on a new IterCAD-Bench using CD-TR curves and AUC-TR to combine validity and precision without survivor bias.\n\nThe work does address a real gap: most prior CAD generation is one-shot, while engineers iterate. The data synthesis pipeline that adds industrial features to create multi-view drawings and editing trajectories is a concrete effort to move closer to practice. The RL masking step and the new metric show some care about geometric constraints and evaluation.\n\nThe soft spots are in the evidence. The abstract states clear outperformance in executability and precision plus better iterative refinement, yet supplies no numbers, baselines, protocols, or error bars. That makes the strength of the gains impossible to judge from what is here. The data pipeline is the bigger concern. No distributional checks, expert review, or comparison to external CAD corpora are described, so it is unclear whether the synthetic tasks capture real manufacturing variability in features, tolerances, or editing patterns. If they do not, the reported advantages and the metric's claimed generality become benchmark-specific rather than general.\n\nThis is for researchers working on agents for design code or geometric generation tasks. A reader who wants ideas for closed-loop setups or new evaluation curves in constrained code domains would get something usable. The task framing and metric are novel enough that the paper deserves a serious referee to press on the data validation and experimental details.","headline":"IterCAD adds an iterative agent loop, synthetic data pipeline, and CD-TR metric to CAD generation, but the outperformance claims rest on unshown experiments and unvalidated data realism.","tokens_in":2421,"tokens_out":413,"would_cite":false,"duration_ms":22073,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"IterCAD frames CAD generation and editing as closed-loop multi-turn interactions between a multimodal agent and an executable code sandbox.","keywords":["CAD generation","multimodal agent","iterative refinement","code generation","reinforcement learning","geometric precision","data synthesis pipeline"],"falsifier":"Run the trained IterCAD agent on a collection of real industrial CAD drawings and models never produced by the synthesis pipeline and measure whether the reported improvements in code validity and Chamfer-distance precision remain.","tokens_in":2616,"feed_emoji":"🤖","tokens_out":621,"duration_ms":21425,"temperature":0.7,"pith_summary":"The paper establishes that open-loop one-shot methods mismatch real iterative CAD practice and proposes a unified agent that handles drawing-to-code, text-to-code, and interactive editing through repeated observation and code revision. It supplies a data pipeline that produces standard-compliant multi-view drawings and editing trajectories, then trains the agent first with progressive supervised fine-tuning and then with geometry-aware reinforcement learning that uses viable-prefix masking. The resulting system is evaluated on a new benchmark suite whose CD-TR curve and AUC-TR metric jointly measure code executability and geometric fidelity without survivor bias. Experiments indicate that this closed-loop procedure yields higher rates of valid, precise CAD code than prior approaches across the three tasks.","feed_headline":"Iterative agent raises CAD code executability and precision","feed_subtitle":"Multi-turn interaction with an executable sandbox outperforms one-shot generation on validity and geometric metrics","key_machinery":"The multimodal agent that performs closed-loop turns with an executable CAD sandbox, refined by progressive supervised fine-tuning and geometry-aware reinforcement learning using viable-prefix masking.","core_discovery":"IterCAD formulates CAD tasks as multi-turn agent-sandbox interactions, generates training trajectories via an industrial-feature data pipeline, and optimizes the agent through progressive SFT followed by geometry-aware RL with viable-prefix masking, producing measurable gains in code executability and geometric precision on the introduced IterCAD-Bench and CD-TR metric.","pith_inferences":["If the sandbox accurately reflects downstream manufacturing constraints, the agent could reduce the number of human revision cycles needed in production CAD workflows.","The same closed-loop pattern might transfer to other domains where code must satisfy geometric or physical constraints, such as procedural modeling or robotic task planning."],"forward_implications":["The same agent architecture unifies three previously separate CAD tasks under one interactive loop.","Viable-prefix masking during RL directly raises the fraction of executable code outputs.","The CD-TR curve supplies a single scalar that trades off validity against geometric tolerance without discarding failed samples.","Iterative refinement produces larger gains on complex editing tasks than on single-pass generation."],"fun_headline_variants":["IterCAD casts CAD as multi-turn agent-sandbox interactions","Pipeline generates multi-view CAD drawing and editing tasks","SFT followed by RL optimizes agent for CAD code validity","New bench uses CD-TR curve for CAD geometric metrics","RL masking improves CAD executability and precision"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The data synthesis pipeline that adds advanced industrial manufacturing features creates training and test distributions close enough to real CAD work for the measured gains to carry over.","fun_headline_variants_meta":{"raw":{"variants":["IterCAD casts CAD as multi-turn agent-sandbox interactions","Pipeline generates multi-view CAD drawing and editing tasks","SFT followed by RL optimizes agent for CAD code validity","New bench uses CD-TR curve for CAD geometric metrics","RL masking improves CAD executability and precision"]},"model":"grok-4.3","cost_usd":0.007197,"raw_usage":{"total_tokens":3312,"prompt_tokens":651,"num_sources_used":0,"completion_tokens":73,"cost_in_usd_ticks":71974500,"prompt_tokens_details":{"text_tokens":651,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2588,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":651,"tokens_out":73,"duration_ms":20497,"temperature":1.0,"reasoning_tokens":2588,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-01T07:31:03.974026+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Run the trained IterCAD agent on a collection of real industrial CAD drawings and models never produced by the synthesis pipeline and measure whether the reported improvements in code validity and Chamfer-distance precision remain.","supporting_citations":[],"review_version":2}