{"id":"1c250f34-3c23-40e4-af8d-5fecc2e0c53f","arxiv_id":"2505.02576","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A recursive divide-and-conquer prompting method with LLM-generated sub-problem dependencies outperforms chain-of-thought baselines on two synthetic list tasks at higher difficulty, with lower token and time cost.","lead":"This paper introduces RDD, a prompting strategy that recursively splits reasoning problems into sub-problems with dependencies, lets an LLM solve the pieces, and merges the answers. On two word-list tasks, RDD beats chain-of-thought with self-consistency at higher difficulty while using fewer tokens and less wall-clock time.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The compute-matched comparison is not actually matched: under the paper's own formula, baseline budgets exceed RDD's by 2-3x, so Hypothesis 1 is unsupported.","rationale":"The paper's central claim is that RDD outperforms baselines in a compute-matched setting while being more efficient. The reader's weakest assumption was that the token formula n_context_tokens + 3*n_output_tokens creates a fair inference budget. My stress-test confirms this is not merely a missing detail: applying the formula to the paper's own resource tables shows that baseline budgets exceed RDD's by roughly 2-3x. For example, in Table 1 at n=90, CoT+SC has total cost about 7.66M under the formula while RDD has about 3.11M; LtM+SC is about 10.05M. This is an internal inconsistency with the stated compute-matching procedure. The nonstandard self-consistency voting further weakens the baseline comparison, but the compute-budget mismatch is the load-bearing issue because it directly undermines Hypothesis 1. Hypotheses 2-4 about generic applicability, time, and context savings are less affected and may still hold. Since the reader already issued a CONDITIONAL verdict centered on this exact assumption, my finding reinforces rather than changes that verdict; the condition should be a corrected compute-matched rerun with standard SC aggregation and full methodological transparency.","tokens_in":26847,"tokens_out":8111,"duration_ms":109335,"concrete_test":"Reproduce the SC sample-count selection described in App. E for each difficulty. Then rerun the n=70 and n=90 conditions of Tables 1-3 with k chosen so that for each baseline, k*(C_per_sample + 3*O_per_sample) exactly equals RDD's total (C + 3*O) for that condition, using standard majority-vote SC as the aggregation method. If RDD's accuracy lead over CoT+SC/LtM+SC disappears or falls within noise, Hypothesis 1 is refuted; if the lead persists, the compute-matched claim is restored.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix E states that self-consistency sample counts were chosen to match resource usage via n_context_tokens + 3*n_output_tokens. Applying that formula to the reported totals in Table 1 gives CoT+SC 1.50M at n=5 vs. RDD 0.51M; at n=90, CoT+SC 7.66M and LtM+SC 10.05M vs. RDD 3.11M. The same pattern appears in Tables 2 and 3. Thus, under the paper's own cost metric, the baselines receive substantially more compute than RDD, so the comparison is not compute-matched. The SC sample-count selection procedure is not otherwise specified, and the SC aggregation is a nonstandard LLM/binary-search procedure (Sec. 3) rather than the majority vote that defines standard self-consistency. Because the central claim is that RDD outperforms baselines in a compute-matched setting, the one condition that would make that claim true—equal budgets—is not met. The accuracy advantage at high difficulty may therefore reflect a compute advantage in disguise. Time-efficiency claims (Hypotheses 3-4) are less affected, but the headline empirical claim is unsupported as reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Recursive Decomposition with Dependencies (RDD), a divide-and-conquer prompting framework for LLMs that recursively decomposes a reasoning problem into sub-problems, optionally models dependencies between them, and merges solutions. The method is intended to be task-agnostic, requiring no task-specific demonstrations beyond a fixed set of generic meta-task examples. The authors evaluate RDD on letter concatenation (with and without task-specific examples) and length reversal (with dependencies), comparing against chain-of-thought (CoT) and least-to-most (LtM) prompting with self-consistency (SC). They claim that RDD outperforms these baselines in a compute-matched setting as task complexity increases, while also reducing time and token usage. The theoretical contribution (Appendix A) consists of two theorems stating conditions for recursive decomposition to improve accuracy.","tokens_in":27124,"tokens_out":4638,"duration_ms":55432,"significance":"If the empirical claims hold, RDD would be a practical and scalable method for improving LLM accuracy on complex reasoning tasks while reducing computational cost, and its support for dependencies and error recovery is a genuine extension over prior decomposition methods such as least-to-most and Tree-of-Thoughts. The paper provides complete prompt templates and a detailed error analysis, which are useful for replication. However, the strength of the evidence is substantially weakened by the lack of a properly compute-matched comparison, the absence of error bars or significance tests, and the use of a non-standard self-consistency aggregation procedure. The theoretical results in Appendix A are mathematically trivial and do not contribute predictive content. The core idea is plausible and worth pursuing, but the current empirical support for the headline claim (Hypothesis 1) is not convincing as reported.","major_comments":[{"comment":"The compute-matched claim is not supported by the reported resource-usage data. The paper states in Appendix E that SC sample counts were chosen using the formula n_context_tokens + 3 * n_output_tokens. Applying this formula to the row totals in Table 1 for the task-specific letter-concatenation experiment yields CoT+SC budgets of 1.50M at n=5 and 7.66M at n=90, LtM+SC budgets of 1.14M at n=5 and 10.05M at n=90, versus RDD+LtM budgets of 0.51M at n=5 and 3.11M at n=90. The same discrepancy appears in Tables 2 and 3 (e.g., Table 2, n=90: CoT+SC 6.93M vs. RDD+CoT 3.75M). Thus, under the paper's own cost metric, the baselines are allocated roughly 2–3 times more compute than RDD, so the comparison is not compute-matched and Hypothesis 1 is not validated. The authors should either re-run the comparison with matched budgets or report accuracy as a function of compute (e.g., budget–accuracy curves) to substantiate the compute-matched claim.","section":"Appendix E, Tables 1–3; Section 3, Hypothesis 1"},{"comment":"The empirical evaluation reports only point estimates of accuracy, averaged over 100 instances per difficulty level, with no error bars, confidence intervals, or significance tests. Without uncertainty quantification, the claimed performance advantages—especially around the observed 'transition points' (e.g., 20 < n* < 50 in Fig. 3, 10 < n* < 20 in Fig. 4a)—may be within sampling noise. The authors should provide bootstrap confidence intervals or pairwise significance tests (e.g., McNemar's test) for the comparisons at each difficulty level.","section":"Section 3, Figures 3–4"},{"comment":"The self-consistency baseline deviates from the standard SC procedure (Wang et al., 2022b) in two ways: the paper states that 'we employ the LLM itself to decide the most consistent answer given the set of sampled solutions' and that a binary-search procedure is used to find the most consistent answer instead of a majority vote over sampled chains. This non-standard aggregation may behave differently from standard SC, and no evidence is provided that it preserves SC's accuracy or is comparable to the original method. Because SC is the mechanism used to give the baselines their compute budget, this deviation should be validated (e.g., by comparing the modified SC against majority-vote SC on a subset of problems).","section":"Section 3, Self-Consistency implementation"},{"comment":"The two theorems restate immediate consequences of the recursive accuracy formula in Eq. (1). Theorem 1 follows directly from the bound ϕ_RDD(X0) ≤ ϕ_d(X0)ϕ_m(X0) in Eq. (5), and Theorem 2 follows from the analogous bound ϕ_RDD(X0) ≤ ϕ_u(Xi). These observations are correct but tautological; they add no predictive or prescriptive content and are not used elsewhere in the paper. The authors should either remove these theorems or re-frame them as simple observations rather than formal results.","section":"Appendix A, Theorems 1–2"}],"minor_comments":[{"comment":"There is a typo: 'ϕRDD to be the overall accuracy of RRD applied' should read 'RDD' instead of 'RRD'.","section":"Section 2.1"},{"comment":"The sentence 'Moreover, RDD is allows for ordered execution of subtasks' contains a grammatical error; it should be 'RDD allows for ordered execution of subtasks'.","section":"Section 5"},{"comment":"The time value for LtM+SC at n=70 is recorded as '12,18h' with a comma as decimal separator, while all other entries use a period; this should be made consistent.","section":"Appendix E, Table 1"},{"comment":"The indentation in Algorithm 1 suggests that 'SCHEDULE DFS(problem, [ ])' is inside the 'while unsolved is not empty' loop, which is likely incorrect; the indentation should be clarified.","section":"Section 2.1, Algorithm 1"},{"comment":"The error analysis averages ϕ_d, ϕ_m, and ϕ_u over all sub-problems that appear recursively; the paper notes this, but it is worth stating explicitly in the figure caption that ϕ_u is not the accuracy at the root difficulty n0, to avoid misinterpretation.","section":"Section 3.4, Figure 5"},{"comment":"The width parameter w (max branching factor) is treated as a fixed constant throughout, but no value is reported and no sensitivity analysis is provided. Since w directly controls the decomposition, its value should be given for each experiment, or at least a default should be stated.","section":"Section 2.1"}],"recommendation":"major_revision","confidential_remarks":"The paper appears to be from a NeurIPS workshop submission and is currently formatted as an extended abstract. The primary concern is the compute-matched comparison, which is a central empirical claim: the reported token budgets do not match under the paper's own formula, and the direction of the imbalance (baselines receiving more compute) means the current data cannot support the claimed compute-matched advantage. This is fixable with additional experiments or re-analysis, so I recommend major revision rather than rejection. The non-standard SC variant and lack of error bars are secondary but also need to be addressed before the paper can be considered for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know up front. First, the method is a real incremental advance: recursive decomposition where the LLM proposes dependencies between subproblems, plus error recovery at merge time, with generic prompts that transfer to new tasks without task-specific examples. Second, the headline empirical claim—that RDD wins in a compute-matched setting—is not supported by the paper's own data.\n\nWhat's new and good. Prior divide-and-conquer prompting (Zhang et al.) builds trees without dependencies; DecomP and Graph-of-Thoughts need user-provided decomposition or graph structures. RDD lets the model emit dependency identifiers during decomposition and handles them with a BFS/DFS scheduler. The paper evaluates on letter concatenation and length reversal at six difficulty levels, in both task-specific and generic demonstration settings. The generic setting is the more interesting result: a fixed set of cross-task decomposition and merge examples appears to suffice. The appendix includes full prompts, resource usage tables, and an error analysis that breaks down decomposition, unit, and merge accuracies. That is useful, honest work.\n\nThe main soft spot is the compute-matched claim. Appendix E says baseline sample counts were chosen using n_context + 3*n_output. Applying that formula to the reported totals gives CoT+SC and LtM+SC budgets 2-3x larger than RDD's at every difficulty level (e.g., n=90: CoT+SC ~7.7M vs RDD ~3.1M). So the comparison is not compute-matched as claimed. The direction of the imbalance is actually favorable to RDD—it wins despite having less compute—but that is a different claim, and with the nonstandard self-consistency voting (binary search with LLM adjudication rather than majority vote) it is hard to know what a fair comparison would show. I also found no error bars or significance tests; the curves look clean, but we are trusting single runs on 100 instances per condition. Theorems 1-2 are tautological restatements of the product formula, adding no predictive content. And there is no code or data released, which makes the resource tables hard to verify.\n\nBottom line: the method deserves serious attention, and the generic-supervision angle is the most novel part. The paper overstates what the experiments establish. I would send it to peer review—there is a solid kernel here—but with a clear request to either redo the compute-matching or drop that phrase and report accuracy-versus-compute tradeoffs directly, add confidence intervals, release code and data, and remove or heavily reframe the theorems.","headline":"A genuinely useful decomposition method, but the compute-matched headline claim is not supported by the paper's own resource tables; the evaluation needs another pass before publication.","tokens_in":27621,"tokens_out":4654,"would_cite":false,"duration_ms":56444,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A task-agnostic recursive splitting method lets LLMs beat step-by-step reasoning on hard problems while doing less work.","keywords":["recursive decomposition","divide-and-conquer reasoning","large language models","chain-of-thought prompting","least-to-most prompting","self-consistency","sub-task dependencies","directed acyclic graph reasoning"],"falsifier":"Run RDD and CoT+SC on the letter-concatenation benchmark at list size 90 on identical hardware and compare total wall-clock time per solved instance, counting every decomposition, unit-solving, merging, and self-consistency call; if CoT+SC matches or beats RDD's time per correct answer under this direct cost measure, the paper's efficiency and compute-matched claims are falsified.","tokens_in":26686,"feed_emoji":"🧩","tokens_out":9675,"duration_ms":95406,"temperature":0.7,"pith_summary":"Recursive Decomposition with Dependencies (RDD) is a prompting method that repeatedly asks a large language model to split a reasoning problem into smaller sub-problems, solve each one (or split again), and merge the answers. The paper's central claim is that this generic split–solve–merge loop improves accuracy on complex problems relative to chain-of-thought and least-to-most prompting with self-consistency, once the problem passes a certain difficulty threshold, and that it does so while generating fewer tokens and running faster. RDD needs no task-specific examples: the same fixed set of generic demonstration decompositions works on unseen problem classes. The method also lets the model mark dependencies between sub-problems with identifiers, so the decomposition graph becomes a directed acyclic graph rather than a simple tree, and the merge step is told it may fix mistakes found in sub-solutions.","feed_headline":"Recursive splitting beats chain-of-thought on hard reasoning","feed_subtitle":"A task-agnostic split-solve-merge loop matches or beats baselines on complex tasks while using fewer tokens and less time.","key_machinery":"The load-bearing mechanism is the recursive decomposition pipeline with three fixed meta-tasks—decompose, unit-solve, and merge—plus a scheduler that executes them (breadth-first decomposition until an unsatisfied dependency is found, then depth-first unit-solving and merging). A prompt-level convention gives each sub-problem an identifier such as P-1 and lets other sub-problems reference its solution through placeholders like {P-1}, turning the decomposition tree into a directed acyclic graph. The accuracy recursion $\\phi_{RDD}(X_0) = \\phi_d(X_0)\\phi_m(X_0)\\prod_i [1[C(X_i)]\\phi_u(X_i) + 1[\\neg C(X_i)]\\phi_{RDD}(X_i)]$ carries the argument: it shows that recursion beats direct solving when decomposing and merging are jointly more reliable than solving the root directly, and when each sub-problem is easier than the root.","core_discovery":"On the paper's own terms, the core discovery is a performance transition point. For a problem class with within-class difficulty $n_0$, direct unit solving has accuracy $\\phi_u$, while recursively decomposing has overall accuracy $\\phi_{RDD}$; the authors prove that decomposition helps only when $\\phi_d(X_0)\\phi_m(X_0) > \\phi_u(X_0)$ and each sub-problem's unit accuracy exceeds the root's. Empirically, on the letter-concatenation and length-reversal benchmarks with six difficulty levels each, they observe such a transition: below $n^*$ the baselines CoT+SC or LtM+SC win, and above it RDD wins, with the margin growing as difficulty increases. In the generic setting with no task-specific examples, RDD+CoT overtakes CoT+SC after a transition point, and the length-reversal task, which benefits from dependency modeling, also shows RDD ahead after a transition point. The authors further report that RDD uses fewer output tokens and less wall-clock time, and that its merge step can recover from errors made in sub-solutions.","pith_inferences":["If the transition-point theorem is right, an adaptive router could estimate $\\phi_d$, $\\phi_m$, and $\\phi_u$ for each problem instance and apply recursion only when $\\phi_d\\phi_m > \\phi_u$ holds, instead of choosing by difficulty alone.","The compute-matched comparison uses the formula $n_{\\text{context}} + 3\\,n_{\\text{output}}$ to pick self-consistency sample counts; since RDD makes many small calls, its wall-clock advantage may shrink or grow depending on per-call overhead, which the paper does not quantify.","A stress test that injects known errors into sub-solutions and measures how often the merge step repairs them would isolate the error-recovery mechanism from the decomposition's baseline accuracy.","The DAG mechanism suggests testable applications to multi-hop question answering or program synthesis with shared intermediate results, though the paper itself only evaluates synthetic list tasks."],"forward_implications":["On problems harder than the transition point, RDD gives higher exact-match accuracy than CoT+SC and LtM+SC under the paper's compute-matched budget; on easier problems the baselines remain preferable.","RDD can be applied to a new problem class with no task-specific demonstrations, so its decomposition prompts do not have to be rewritten per domain.","Because it outputs fewer tokens per problem, RDD reduces context-window pressure and wall-clock time, and its independent sub-problems are parallelizable by design.","The dependency mechanism extends decomposition from trees to DAGs, so tasks where one intermediate result feeds several later steps can be handled, with sub-problems executed in dependency order.","The merge step can correct errors from earlier steps, making the system more tolerant of imperfect sub-solutions."],"supporting_citations":[{"why":"Defines chain-of-thought prompting, used as the main baseline and as RDD's unit-solving strategy.","marker":"Wei et al., 2022"},{"why":"Defines least-to-most prompting, a baseline that requires task-specific decomposition examples and is the unit-solving method in the task-specific RDD experiments.","marker":"Zhou et al., 2022"},{"why":"Defines self-consistency, used to match compute between the baselines and RDD and to aggregate baseline reasoning chains.","marker":"Wang et al., 2022b"},{"why":"Earlier recursive divide-and-conquer prompting that does not model sub-task dependencies, the gap RDD aims to fill.","marker":"Zhang et al., 2024"},{"why":"Decomposed prompting, whose decomposition structure must be demonstrated for each problem class, in contrast to RDD's task-agnostic decomposition.","marker":"Khot et al., 2022"},{"why":"Graph of Thoughts, an explicit DAG method whose structure and node meanings are user-supplied, in contrast to RDD's model-generated dependency graph.","marker":"Besta et al., 2024"},{"why":"Tree of Thoughts, a tree-structured sampling process rather than a recursive decomposition, providing related-work context for RDD's graph structure.","marker":"Yao et al., 2023"}],"fun_headline_variants":["Recursive decomposition wins on hard reasoning after a tipping point","Split-solve-merge beats chain-of-thought on complex tasks","Task-agnostic recursive reasoning: fewer tokens, better on tough problems","Dependency-aware recursion outperforms CoT past a difficulty threshold","Recursive reasoning with dependencies: efficiency and accuracy at scale"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the token-count formula $n_{\\text{context}} + 3\\,n_{\\text{output}}$ creates a fair compute budget between RDD's many small calls and the baselines' fewer large calls; if that formula undercounts the real cost of RDD's extra calls, the compute-matched performance conclusion collapses.","fun_headline_variants_meta":{"raw":{"variants":["Recursive decomposition wins on hard reasoning after a tipping point","Split-solve-merge beats chain-of-thought on complex tasks","Task-agnostic recursive reasoning: fewer tokens, better on tough problems","Dependency-aware recursion outperforms CoT past a difficulty threshold","Recursive reasoning with dependencies: efficiency and accuracy at scale"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000205,"raw_usage":{"total_tokens":1411,"prompt_tokens":980,"completion_tokens":431,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":358}},"tokens_in":596,"tokens_out":431,"duration_ms":4832,"temperature":1.0,"reasoning_tokens":358,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:47:28.230688+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run RDD and CoT+SC on the letter-concatenation benchmark at list size 90 on identical hardware and compare total wall-clock time per solved instance, counting every decomposition, unit-solving, merging, and self-consistency call; if CoT+SC matches or beats RDD's time per correct answer under this direct cost measure, the paper's efficiency and compute-matched claims are falsified.","supporting_citations":[{"cited_title":"Le, and Ed H","cited_arxiv_id":null,"evidence_quote":"Defines least-to-most prompting, a baseline that requires task-specific decomposition examples and is the unit-solving method in the task-specific RDD experiments."},{"cited_title":"Decomposed Prompting : A Modular Approach for Solving Complex Tasks","cited_arxiv_id":null,"evidence_quote":"Decomposed prompting, whose decomposition structure must be demonstrated for each problem class, in contrast to RDD's task-agnostic decomposition."},{"cited_title":"Graph of Thoughts : Solving Elaborate Problems with Large Language Models","cited_arxiv_id":null,"evidence_quote":"Graph of Thoughts, an explicit DAG method whose structure and node meanings are user-supplied, in contrast to RDD's model-generated dependency graph."},{"cited_title":"Griffiths, Yuan Cao, and Karthik R","cited_arxiv_id":null,"evidence_quote":"Tree of Thoughts, a tree-structured sampling process rather than a recursive decomposition, providing related-work context for RDD's graph structure."}],"review_version":1}