{"id":"f7a41723-0f4d-4d37-82f2-e42e422fb2a3","arxiv_id":"2411.19886","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A tool that fuses two PDDL domains with random action mutations to produce new, guaranteed-solvable planning problems.","lead":"PDDLFuse fuses two existing planning domains and randomly mutates their actions to generate new benchmark problems. It aims to give AI planners more diverse test domains, but the paper does not yet prove the domains are diverse or release the tool.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 omits parameter/type handling for added predicates, so the published method may not produce valid PDDL domains at all.","rationale":"The reader's weakest assumption concerned semantic meaningfulness and diversity of mutated unions; I sharpen this to a more concrete, load-bearing gap: Algorithm 1 does not specify how randomly added predicates are attached to action parameters or how object types from two source domains are reconciled. Without this, the method may not even produce syntactically valid PDDL, let alone diverse domains. The paper's own validator is tested only on existing domains (Table 2), not on PDDLFuse-generated outputs, so the core generation loop is unvalidated. The solvability-by-construction idea (Algorithm 3) is sound in principle and the planner experiments are suggestive, but they presuppose that valid domain files exist. Since the paper provides no code or data, the most direct resolution is to release the implementation or specify the omitted parameter/type handling. This does not change the reader's conditional verdict; it reinforces the condition that artifacts and a validity check on generated domains be supplied. I therefore keep the verdict unchanged.","tokens_in":37765,"tokens_out":3939,"duration_ms":39828,"concrete_test":"Implement Algorithm 1 exactly as written (without inventing parameter/type handling) on a pair of IPC domains with disjoint type signatures, e.g., Blocksworld and Satellite, with prob_add_pre = prob_add_eff = 0.5 and 10 random seeds. Feed each generated domain file to a strict PDDL 3.1 type checker such as VAL. If a substantial fraction are rejected for unbound variables, undeclared types, or ill-typed action schemas, the published algorithm is not self-contained and the paper must release code or specify how parameters and types are handled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central generation step (Algorithm 1, Methods) randomly 'adds new predicates' to action preconditions and effects, but PDDL actions are lifted schemas with typed parameters. Adding a predicate such as (at ?x ?y) to an action requires either reusing existing action parameters of matching types or introducing and declaring new parameters. The paper provides no rule for this, and the formal domain tuple (O,P,A) in the Background ignores types entirely. When fusing two IPC domains with different type signatures (e.g., Blocksworld and Gripper), the union of objects and predicates without a type-unification strategy makes it undefined how added predicates are grounded. The validator is only demonstrated on existing domains (Table 2), not on generated ones, so no evidence shows that Algorithm 1 actually produces valid PDDL. If the implementation silently performs parameter instantiation and type reconciliation, that is a core omitted mechanism; if it does not, most generated 'domains' will be rejected by a PDDL parser or be semantically ill-formed. This is more fundamental than the unmeasured diversity concern: the tool's ability to generate any valid novel domain is unestablished.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"PDDLFuse is presented as a tool for generating novel, diverse planning domains by fusing two existing PDDL domains. The method first renames overlapping predicates and actions, then unions the objects, predicates, and actions of the two domains, and randomly mutates action preconditions and effects according to tunable probabilities (adding/removing predicates, negation, predicate reversibility). Problem files are generated by executing a random action sequence from an initial state and using a subset of the reached predicates as the goal, which is intended to guarantee solvability by construction. The paper evaluates the resulting domains by running Fast Downward (FF and lmcut heuristics) and LPG on generated instances, reporting success counts across parameter settings and generation depths. The central claim is that PDDLFuse can create complex and varied planning domains that challenge domain-independent planners and support training and evaluation of planning systems.","tokens_in":37993,"tokens_out":3950,"duration_ms":36043,"significance":"If fully established, the contribution would address a genuine bottleneck in planning research: the limited pool of diverse benchmark domains. The solvability-by-construction mechanism is simple and, if implemented correctly, gives a strong guarantee for problem generation. The paper also makes a concrete, falsifiable claim that parameter settings can modulate planner difficulty, which is a useful capability for benchmarking. However, the current evaluation does not demonstrate that the generated domains are either novel or diverse: no diversity metric is defined, no baseline is compared, all experiments use a single random seed, and sample sizes are small and unequal. Moreover, a fundamental technical detail—how predicates added to actions are grounded with typed parameters—is omitted from the specification of the generation algorithm. The significance is therefore conditional on resolving these load-bearing gaps; the tool could be valuable, but the paper as written does not yet substantiate the headline claims.","major_comments":[{"comment":"Algorithm 1 and the formal domain definition omit parameter and type handling, which is load-bearing for PDDL validity. In PDDL, actions are lifted schemas with typed parameters; adding a predicate such as (at ?x ?y) to a precondition or effect requires either reusing parameters of matching types in the action or introducing and declaring new parameters. The paper never specifies this, and the domain tuple (O,P,A) in the Background section contains no type information at all. When fusing domains with different type signatures (e.g., Blocksworld and Gripper), the union operations on lines 1-2 of Algorithm 1 are undefined with respect to how predicates are grounded. The validator is only demonstrated on existing, hand-written domains (Table 2), not on generated domains, so there is no evidence that Algorithm 1 actually produces parseable PDDL. If the implementation silently performs parameter instantiation and type reconciliation, that core mechanism must be described and validated; if it does not, many generated 'domains' would be rejected by a PDDL parser or be semantically ill-formed.","section":"Methods, Algorithm 1 and Background"},{"comment":"The paper's central claim of generating 'intricate and varied domains' is not supported by any diversity measurement. No diversity metric is defined or computed, no baseline or comparison to existing domain generation methods is provided, all experiments use a single random seed (seed 42 in System Configuration), and sample sizes are very small (typically 5 problems per configuration, with some rows containing only 1-4 problems due to 'memory constraints' in Solvability Across Parameter Variations and Depth). The heat maps in Figures 1-3 report per-cell counts of solved instances out of at most 5, which is too coarse to establish diversity or to support the parameter-sensitivity conclusions drawn from them. The authors should report quantitative diversity measures over generated domains (e.g., structural or behavioral distances between generated domains), compare against baselines such as mutating a single domain or sampling from existing IPC domains, and use multiple seeds with error bars.","section":"Results and Conclusion"},{"comment":"The solvability-by-construction guarantee is not tied to the actual generation procedure in the main text, and the paper's own results show a large fraction of unsolved instances. Algorithm 1 line 22 says only 'Execute actions to derive the goal state from the initial state' without specifying whether the action sequence is generated under the mutated domain (as in Algorithm 3) or under the original base domains. If the witness sequence is executed before mutation, the mutations can invalidate it, which would explain why planners often fail on purportedly solvable instances (e.g., Tables 5 and 8 show FD(ff) solving 3/5 and 1/5 at some depths, and LPG frequently solving 0/5). If the witness sequence is generated after mutation, then every problem should be solvable by that witness and the low solver success rates are due only to planner incompleteness; the paper must clarify this point because the interpretation of all experimental tables depends on it. The authors should also state explicitly that Algorithm 3 is the mechanism referenced by Algorithm 1, or integrate the two.","section":"Problem File Generation and Algorithm 3"},{"comment":"The manuscript contains a serious internal inconsistency: the Background and Related Works sections repeatedly refer to a system called 'DomGenX' as the proposed contribution (e.g., 'DomGenX addresses this gap' and 'aligning with DomGenX's goal'), while the title, abstract, and Methods describe 'PDDLFuse'. Moreover, the 'Background and Related Works' section appears twice with overlapping but non-identical text, and the second occurrence introduces DomGenX without any definition. This is not a purely cosmetic issue: the reader cannot determine which tool is being described, and the repeated text suggests a leftover from a previous paper version. The authors must reconcile the terminology and remove the duplicated section before the paper can be evaluated coherently.","section":"Background and Related Works (entire repeated section)"}],"minor_comments":[{"comment":"The heading 'System Configuraiton' is a typo for 'System Configuration'.","section":"System Configuration"},{"comment":"In the parameter list, 'revf lag' should be spelled as 'rev_flag' or 'reversibility flag' for consistency with the algorithm notation.","section":"Domain Generation"},{"comment":"The captions claim the heat maps show 'solvability' but the cells are described in the text as 'the number of solvable instances'; given the maximum of 5 instances per cell, the figures would be much more informative with per-cell success rates and confidence intervals rather than raw counts.","section":"Figures 1-3"},{"comment":"The parameter names are inconsistent between the text ('Prob add precond', 'Prob add effect') and Algorithm 1 ('prob_add_pre', 'prob_add_eff'), which makes it difficult to map the reported configurations to the formal notation; a single notation should be used throughout.","section":"Experimental Setup and Parameters"},{"comment":"The paper repeatedly refers to 'Supplementary Material' for details of the algorithms, but no supplementary material is provided; this should be included or the descriptions should be made self-contained.","section":"Methods (multiple)"}],"recommendation":"major_revision","confidential_remarks":"The paper shows signs of a very early draft: a duplicated background section, leftover references to a different tool name (DomGenX vs. PDDLFuse), missing supplementary material, and a small single-seed evaluation. The underlying idea is reasonable and could mature into a useful contribution, but the current manuscript is not yet ready for a full archival venue. For the editor: if the authors can address the type-handling question, validate generated domains directly, and provide a proper diversity evaluation with baselines and multiple seeds, the paper would be within reach; otherwise it may be better suited to a workshop."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things: the core generation idea is simple and solvability-by-construction is sound, but as written the method likely cannot produce valid PDDL because it never says how newly added predicates get their arguments or types.\n\nWhat's new: fusing two PDDL domains by renaming symbols, taking the union, and randomly mutating preconditions and effects. That is not in the cited LLM-translation work. The problem-generation trick—execute a random action sequence and take a subset of the reached state as the goal—does guarantee a solution exists. The parameter controls for add/remove probabilities and negation are a reasonable way to vary difficulty.\n\nWhat's soft: the typed-parameter omission is load-bearing. PDDL actions are lifted schemas. Adding a predicate like (at ?x ?y) to an action requires either reusing existing parameters of the right types or declaring new ones. The paper's formal tuple (O,P,A) has no types, and Algorithm 1 just says \"add new predicates.\" If the implementation does some type reconciliation, that is a core mechanism missing from the paper. If it does not, most generated domains will be rejected by a PDDL parser. The validator is only run on eight existing domains, never on generated ones, so we have no evidence the pipeline actually emits valid domain files. The diversity claim is also unmeasured: no diversity metric, no baselines, one seed, and sample sizes from one to five problems per configuration.\n\nThere are presentation problems worth noting: the Background section appears twice, and the second copy repeatedly refers to a different tool called DomGenX. That looks like leftover text from another paper and should be cleaned up.\n\nWho is this for? People who want benchmark generators for planning research or training data for learned planners. The idea is worth pursuing, and if the authors release code and fix the type-handling description, it could be a useful tool. In current form, though, the central construction is under-specified and the evaluation does not support the diversity claims.\n\nI would send it to peer review, but with the expectation of major revision. The concept is new enough and the problem is real. A good referee should ask for the missing parameter/type mechanism, validation of generated domains, a diversity measure, and artifact release.","headline":"Simple, sound solvability trick, but the published algorithm may never produce valid PDDL because it omits type/parameter handling for added predicates.","tokens_in":38500,"tokens_out":2145,"would_cite":false,"duration_ms":21275,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PDDLFuse fuses two PDDL domains into new, solvable planning benchmarks with tunable difficulty.","keywords":["PDDL","planning domain generation","domain randomization","automated planning","domain fusion","planner evaluation","solvable problem generation","Fast Downward"],"falsifier":"One concrete check: generate a large batch of fused domains and inspect the plans that planners return; if the random mutations are never used in any successful plan and nearly every goal is reachable in one or two steps, the claimed difficulty tuning would be refuted.","tokens_in":37555,"feed_emoji":"🧩","tokens_out":6243,"duration_ms":43255,"temperature":0.7,"pith_summary":"PDDLFuse proposes that the scarcity of diverse planning domains is a bottleneck in planning research, and that existing LLM-based methods only reconstruct known domains from natural language rather than invent new ones. The tool starts from two existing PDDL domains, renames overlapping predicates and actions, and randomly adds or removes preconditions and effects to produce a fused domain with tunable mutation probabilities. Problems are generated by rolling out random applicable actions and setting the goal to a subset of the reached state, which makes every problem solvable by construction. The paper reports that Fast Downward and LPG solve many of these generated instances, with success rates dropping as mutation probabilities, negation, object counts, and fusion depth increase, which the authors interpret as evidence of controllable difficulty. If correct, this gives researchers a parameterized source of novel planning domains for benchmarking planners and training planning models.","feed_headline":"Fuse two PDDL domains to mint new planning benchmarks","feed_subtitle":"Random mutations plus guaranteed-solvable rollouts give planners a tunable source of novel test domains.","key_machinery":"The central object is the domain-fusion mutation procedure (Algorithm 1): take the union of objects, predicates, and actions of two domains after renaming overlaps; for each action, with probability prob_add_pre add predicates to preconditions, with prob_add_eff add effects, with prob_rem_pre and prob_rem_eff remove them, and apply negation with prob_neg. Problems are generated by a random action rollout (Algorithm 3): execute random applicable actions from an initial state for N steps and choose a subset of true predicates as the goal, guaranteeing that a plan exists. This mutation-rollout pair is what carries the claim of novel, solvable, difficulty-tunable domains.","core_discovery":"The central claim is that novel planning domains can be produced by fusing two existing PDDL domains through randomized edits to action preconditions and effects, rather than by translating natural language descriptions of existing domains. The paper argues that this is the planning analogue of domain randomization in reinforcement learning, and that the resulting domains, together with problems generated from random action rollouts, are solvable and diverse enough to stress-test domain-independent planners. The reported experiments show that planner solvability degrades as mutation probabilities, negation probabilities, object counts, and iterative domain depth grow, which the authors take as evidence that PDDLFuse generates complex, adjustable-difficulty domains.","pith_inferences":["The manuscript refers to the tool as DomGenX in the Background section; this appears to be a naming inconsistency rather than a different method.","Diversity is asserted but not measured; a natural extension is to quantify structural distance between generated and seed domains, which the paper does not do.","Because problems are solvable by construction, planner success on them may reflect incremental reachability rather than genuine planning difficulty; filtering for non-trivial plans would test this.","The depth-level procedure suggests a curriculum-like use of recursively fused domains, but the reported data show declining solvability rather than evidence that deeper domains are meaningfully more complex."],"forward_implications":["Researchers can generate new solvable planning benchmarks without manual domain authoring, using pairs of existing domains as seeds.","Tuning the mutation, negation, and object-count parameters shifts planner success rates, giving a way to produce easier or harder test sets.","The tool brings the domain-randomization idea from reinforcement learning into PDDL planning, potentially supporting generalization experiments.","Generated domains can be used to validate new planners and to probe the limits of domain-independent planners like Fast Downward and LPG.","The built-in validator checks PDDL 3.1 syntax and executes plans to confirm goals are reachable."],"supporting_citations":[{"why":"Baseline that reconstructs PDDL from natural language and requires a reference domain; PDDLFuse positions itself against this limitation.","marker":"Oswald et al. 2024"},{"why":"Iterative LLM refinement of domains from environment feedback; represents the refinement-based approach PDDLFuse contrasts with.","marker":"Mahdavi et al. 2024"},{"why":"Introduces Active Domain Randomization in RL, the inspiration for generating diverse training domains.","marker":"Mehta et al. 2020"},{"why":"Shows domain randomization improves RL generalization by varying physical parameters, supporting the motivation.","marker":"Ajani, Hur, and Mallipeddi 2023"},{"why":"Supplies the formal definition of planning domains and problems used throughout the paper.","marker":"Ghallab, Nau, and Traverso 2004"},{"why":"Defines the Fast Downward planner used in the solvability experiments.","marker":"Helmert 2006"},{"why":"Defines the LPG planner used as the second domain-independent planner in experiments.","marker":"Gerevini and Serina 2002"},{"why":"Source of the eight domains and optimal plans used to evaluate the PDDLFuse validator.","marker":"Chen, Thiébaux, and Trevizan 2023"}],"fun_headline_variants":["Fuse PDDL domains to mint new planning benchmarks","Randomized domain fusion makes tunable test suites","PDDLFuse: random edits spawn diverse planning domains","Two PDDL domains in, many harder ones out"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Randomly mutating the union of two PDDL domains yields domains that are semantically meaningful and diverse, not merely syntactically valid and solvable by construction.","fun_headline_variants_meta":{"raw":{"variants":["Fuse PDDL domains to mint new planning benchmarks","Randomized domain fusion makes tunable test suites","PDDLFuse: random edits spawn diverse planning domains","Two PDDL domains in, many harder ones out"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000325,"raw_usage":{"total_tokens":1788,"prompt_tokens":879,"completion_tokens":909,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":844}},"tokens_in":495,"tokens_out":909,"duration_ms":8398,"temperature":1.0,"reasoning_tokens":844,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:41:58.705122+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete check: generate a large batch of fused domains and inspect the plans that planners return; if the random mutations are never used in any successful plan and nearly every goal is reachable in one or two steps, the claimed difficulty tuning would be refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Baseline that reconstructs PDDL from natural language and requires a reference domain; PDDLFuse positions itself against this limitation."},{"cited_title":"J.; and Paull, L","cited_arxiv_id":null,"evidence_quote":"Introduces Active Domain Randomization in RL, the inspiration for generating diverse training domains."},{"cited_title":"S.; Hur, S.-h.; and Mallipeddi, R","cited_arxiv_id":null,"evidence_quote":"Shows domain randomization improves RL generalization by varying physical parameters, supporting the motivation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the formal definition of planning domains and problems used throughout the paper."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the LPG planner used as the second domain-independent planner in experiments."},{"cited_title":"Z.; Thi \\'e baux, S.; and Trevizan, F","cited_arxiv_id":null,"evidence_quote":"Source of the eight domains and optimal plans used to evaluate the PDDLFuse validator."}],"review_version":1}