{"id":"28cd7301-8f11-405a-a165-81dbbffdffb4","arxiv_id":"2606.31252","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"Embodied CAD deploys solver-grounded LLM agents with stratified L0-L4 skills, action grammar, and solver rewards to achieve high executable rates on multi-step mechanical and mold assembly tasks.","lead":"The paper presents Embodied CAD, where LLM agents iteratively pick actions from a CAD skill library, execute them via a geometric solver, and use feedback to plan and repair parametric B-Rep assemblies. This could improve reliability of AI-generated industrial CAD models over one-shot script generation.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"Reader's weakest_assumption correctly flags the solver-dependence as the key precondition. With only the abstract available and no contradictory detail supplied, the argument is internally consistent on its own terms; the UNVERDICTED status is therefore appropriate rather than a flaw in the claim itself.","tokens_in":1703,"tokens_out":265,"duration_ms":15813,"concrete_test":"Re-run the reported benchmark tasks while logging every solver response for completeness and latency; if any workflow requires external constraints or produces ambiguous feedback that the agent cannot resolve within the described L0-L4 grammar, the execution claim weakens.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on solver-grounded iterative planning and learning via reliable CAD backend feedback. The abstract describes the mechanism (action grammar, deterministic resolution, solver-derived rewards) and reports that solver-grounded planning executes all benchmark workflows while learned controllers expose a policy gap. No internal inconsistency or unsupported step is visible from the provided description; the framework is explicitly designed around the solver loop. Because the full manuscript was referenced but not supplied in the query, no concrete technical flaw (e.g., hidden assumption in a specific equation or untested regime) can be isolated.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces Embodied CAD, a framework for solver-grounded LLM agents performing parametric B-Rep assembly modeling. Agents iteratively select actions from a stratified L0-L4 CAD skill library, resolve them via deterministic geometric operations, execute in a CAD backend, and leverage solver feedback for planning, repair, and learning via action grammar constraints and solver-derived rewards (including supervised warm-up and GRPO-style refinement). On multi-step mechanical, industrial, and mold-oriented assembly tasks, solver-grounded planning is reported to execute all strong-planner workflows in the benchmark, while learned controllers achieve high executable rates but expose a gap between valid tool calls and exact long-horizon policy prediction. Evaluation uses solver-aligned metrics: executable rate, skill accuracy, operation-family accuracy, exact policy accuracy, and task completion success.","tokens_in":1790,"tokens_out":423,"duration_ms":19478,"significance":"If the empirical claims hold with detailed supporting data, the work offers a concrete mechanism for ensuring geometric and parametric validity in LLM-generated CAD, which is load-bearing for industrial use cases where syntactic validity alone is insufficient. The explicit separation of planning success from learned policy gaps, combined with solver-derived rewards, provides a falsifiable testbed for embodied design agents and could inform follow-on work on long-horizon geometric reasoning.","major_comments":[{"comment":"Abstract and Evaluation section: the central claims that 'solver-grounded planning executes all strong-planner workflows' and that learned controllers 'reach high executable rates' are stated without any numerical results, benchmark size, task breakdown, baselines, error bars, or data exclusion criteria. This absence prevents verification of whether the solver feedback loop actually scales without compounding errors, which is load-bearing for the framework's contribution.","section":"Abstract / Evaluation"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":"Full manuscript text referenced but not supplied in the review package; all comments are therefore limited to the abstract. The citation pattern and novelty relative to prior CAD-LLM work cannot be assessed without the body."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback and the recommendation for major revision. The concern about missing numerical support for the central claims is valid for the abstract as written and we address it directly below.","responses":[{"response":"We agree that the abstract as currently written does not include the requested numerical results, benchmark size, task breakdown, baselines, error bars, or data exclusion criteria, and that this limits immediate verification. In the revised manuscript we will expand the abstract to report the key quantitative outcomes (executable rates for planning and learned controllers, benchmark size and task composition, and main baselines). We will also ensure the Evaluation section explicitly states benchmark size, per-task breakdown, data exclusion criteria, and includes error bars or variance measures where multiple runs were performed. These additions will make the scalability of the solver feedback loop directly verifiable from the text.","revision_made":"yes","referee_comment":"[Abstract / Evaluation] Abstract and Evaluation section: the central claims that 'solver-grounded planning executes all strong-planner workflows' and that learned controllers 'reach high executable rates' are stated without any numerical results, benchmark size, task breakdown, baselines, error bars, or data exclusion criteria. This absence prevents verification of whether the solver feedback loop actually scales without compounding errors, which is load-bearing for the framework's contribution."}],"tokens_in":1379,"tokens_out":292,"duration_ms":13830,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's main move is to replace one-pass CAD script generation with an agent that picks actions from an L0-L4 skill library, resolves parameters deterministically, runs them in a real geometric solver, and feeds the outcome back for planning and learning. The GRPO refinement step uses solver-derived rewards on top of supervised warm-up. That setup directly targets the gap between syntactically valid output and kernel-accepted parametric B-Rep geometry.\n\nWhat the work does cleanly is tie evaluation to solver-aligned metrics: executable rate, skill accuracy, operation-family accuracy, exact policy accuracy, and task completion. The claim that the planner covers every strong-planner workflow on the benchmark while learned controllers expose the remaining long-horizon policy gap is a useful distinction. The framework is built around the solver loop rather than hoping external constraints will save it.\n\nThe soft spot is the absence of concrete numbers, baselines, task counts, or error bars in the abstract. Without those it is hard to tell whether the reported executable rates represent a real advance or whether the assemblies stay short enough that compounding errors do not yet appear. The central assumption that solver feedback stays reliable and timely across multi-step sequences also needs explicit checking in the full experiments.\n\nThis is for groups working on grounded agents for engineering design or manufacturing pipelines. A reader who cares about LLM use in CAD would get value from the skill stratification and reward design. It deserves a serious referee so the implementation details and quantitative results can be examined.","headline":"The iterative solver-grounded loop for CAD agents is a practical framing, but the abstract gives too little on benchmarks and numbers to judge the actual gains.","tokens_in":2265,"tokens_out":375,"would_cite":false,"duration_ms":26302,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Solver-grounded LLM agents perform parametric B-Rep assembly modeling by iteratively executing actions and using CAD solver feedback.","keywords":["LLM agents","CAD modeling","B-Rep","parametric modeling","solver feedback","assembly modeling","geometric kernel","action planning"],"falsifier":"A test where solver feedback is noisy, incomplete, or delayed on a long assembly task, causing the agent to fail task completion despite valid individual actions.","tokens_in":2603,"feed_emoji":"🔧","tokens_out":650,"duration_ms":29936,"temperature":0.7,"pith_summary":"The paper presents Embodied CAD, where LLM agents build parametric CAD assemblies not by writing full scripts at once but by choosing actions from a skill library, running them in a CAD backend, and adjusting based on solver feedback. This approach aims to ensure every feature and relation is geometrically valid and editable. A sympathetic reader would care because it addresses the gap between plausible code from LLMs and the exact requirements of industrial CAD kernels for mechanical and mold tasks. The evaluation uses metrics like executable rate and task completion to show that planning with solver grounding succeeds on all benchmark workflows while learned models highlight remaining challenges in long-horizon decisions.","feed_headline":"LLM agents build CAD assemblies using solver feedback","feed_subtitle":"Iterative skill selection and backend execution succeeds on all benchmark mechanical tasks where one-pass scripts fail.","key_machinery":"The solver-grounded iterative loop that selects actions from the L0-L4 skill library, resolves them into geometric operations, executes in the CAD backend, and uses feedback for planning and learning.","core_discovery":"Solver-grounded planning executes all strong-planner workflows in the current benchmark, while learned controllers reach high executable rates and expose the remaining gap between valid tool calls and exact long-horizon policy prediction. The framework uses a stratified L0-L4 CAD skill library, action grammar constraints, deterministic parameter resolution, and solver-derived rewards for supervised warm-up and GRPO-style refinement on multi-step assembly tasks.","pith_inferences":["Similar solver-grounded approaches could apply to other domains requiring precise geometric or physical validation, like robotics or architecture.","Improving the learned controllers to close the policy prediction gap might require better long-horizon reasoning techniques.","Extending the skill library to more complex assemblies could test the scalability of the framework.","The method suggests that hybrid planning and learning is key for reliable CAD automation."],"forward_implications":["Solver-grounded planning completes all workflows in the benchmark.","Learned controllers achieve high rates of executable actions.","Metrics such as skill accuracy and operation-family accuracy can be measured against solver outcomes.","The gap between valid tool calls and exact policy prediction remains for learned models.","Task completion success depends on the reliability of solver feedback in multi-step tasks."],"fun_headline_variants":["Solver-grounded agents model parametric B-Rep assemblies","LLM agents use solver feedback for editable CAD models","Iterative skill selection builds assembly models with solver input","CAD agents achieve full task completion via solver grounding","Parametric assembly modeling via stratified CAD skill agents"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The CAD backend and solver provide reliable, complete, and timely feedback that guides multi-step planning and learning without compounding errors.","fun_headline_variants_meta":{"raw":{"variants":["Solver-grounded agents model parametric B-Rep assemblies","LLM agents use solver feedback for editable CAD models","Iterative skill selection builds assembly models with solver input","CAD agents achieve full task completion via solver grounding","Parametric assembly modeling via stratified CAD skill agents"]},"model":"grok-4.3","cost_usd":0.003421,"raw_usage":{"total_tokens":1803,"prompt_tokens":654,"num_sources_used":0,"completion_tokens":72,"cost_in_usd_ticks":34212000,"prompt_tokens_details":{"text_tokens":654,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1077,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":654,"tokens_out":72,"duration_ms":8473,"temperature":1.0,"reasoning_tokens":1077,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-01T05:42:35.412007+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A test where solver feedback is noisy, incomplete, or delayed on a long assembly task, causing the agent to fail task completion despite valid individual actions.","supporting_citations":[],"review_version":1}