REVIEW 5 major objections 8 minor 2 cited by
PLAN-TUNING: Post-Training Language Models to Learn Step-by-Step Planning for Complex Problem Solving
T0 review · 5 major / 8 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Teaching smaller language models to write a natural-language plan before solving a math problem lifts in-domain accuracy by about 7% and out-of-domain accuracy by 10-12%.
desk verdict Plan-tuning is a plausible idea, but the headline gains are carried by M2, which hands answer generation to an external executor, so the central claim is not yet supported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the synthetic planning trajectory, defined in the paper as a sequence of operators $\tau_k$ that map a partial reasoning state to the next state, so that the policy factorizes as $$\pi_{\$\theta$}(\tau,y\mid x)=\pi_{\$\theta$}(y\mid \tau,x)\,\pi_{\$\theta$}(\tau\mid x).$$ In practice, a trajectory is a natural-language list of subgoals followed by their execution. The pipeline generates five candidate plans per problem, scores each with a constraint-aware plan verifier, keeps plans above a threshold, executes them, and keeps only plans whose executed answer equals the gold answer. Training then uses two objectives: supervised fine-tuning on the full <plan, execution, answer> sequence or on the plan alone, and a modified GRPO objective whose total reward is a plan-similarity score plus a binary answer-correctness reward. This reward design is what pushes the policy toward plans that look like the distilled gold plan and still solve the problem.
What would settle it
Train the same base model on the same answer-verified filtered problems used for plan-tuning, but format each solution as an ordinary chain of thought with no plan tags, using identical hyperparameters, and evaluate on GSM8k, MATH, OlympiadBench, and AIME. If the filtered chain-of-thought model matches or beats the plan-tuned model, the planning structure is not what carries the gains; if plan-tuning still wins by roughly the reported margins, the paper's central claim is supported.
Extended reading notes
Core claim
The central claim is that structured natural-language planning, distilled from a large model and stored in a smaller model's weights, transfers across math benchmarks and helps on problems outside the training distribution. The main evidence is the consistent gap over a conventional supervised fine-tuning baseline: on MATH the plan-tuned variants gain between roughly 9 and 26 accuracy points, and on AIME the gains are proportionally even larger. The paper also reports that the plan-only variant, in which the model learns only to produce a plan and a fixed external model executes it, generally outperforms the joint plan-and-execution variant, and that two-stage filtering of the training corpus (plan score plus gold-answer check) is worth about 4 points on GSM8k and 9 points on AIME. The authors take these results to show that the plan structure itself, not extra tokens or larger models, carries the improvement.
Load-bearing premise
The load-bearing premise is that the gains come from the planning structure itself and not from the stricter answer-verified filtering of the training corpus or from the external executor used in the plan-only variant; the paper does not train a chain-of-thought baseline on the same filtered set, nor a chain-of-thought baseline using the same external executor, and its own limitations section concedes that errors in the upstream generation and verification model can propagate into the training data.
Editorial extensions
If this is right
- Plan-tuned smaller models can get a substantial share of planning ability without scaling up parameters or paying inference-time search costs.
- Plan-only training with a fixed external executor is competitive with or better than joint plan-and-answer training, suggesting that a model able to decompose a problem is the key ingredient.
- Two-stage filtering (plan quality plus gold-answer verification) is needed for the benefit; using only the plan verifier costs roughly 4 points on GSM8k and 9 points on AIME.
- Training on a single dataset beats mixing GSM8k and MATH for the same method, so planning policies learned this way are to some degree dataset-specific.
- Out-of-domain gains on OlympiadBench and AIME indicate the learned planning behavior transfers to harder, unseen math competitions.
Reading between the lines
- A direct test the paper leaves open: add a filtered chain-of-thought baseline (same answer-verified problems, same hyperparameters, no plan tags) and a chain-of-thought baseline that hands its final step to the same external executor; without those, the exact contribution of the plan format is not isolated.
- Because the plan-only variant delegates execution to a fixed external model, the method's ceiling may be set by that executor; trying different executors, or fine-tuning the executor too, is a natural next step.
- The planning reward relies on an LLM-based similarity scorer comparing generated plans to gold plans; replacing it with a programmatic plan-validity measure would show whether the RL gains depend on that scorer's judgment.
- The finding that mixing datasets hurts suggests a single global planner may not emerge from this recipe; a curriculum over problem types or per-domain rewards might recover unified planning without the degradation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes PLAN-TUNING, a post-training framework that teaches smaller open-weights LLMs to solve math problems by first producing a natural-language 'planning trajectory' and then executing it. Planning data are distilled from Gemini-2.0-Flash using a PlanGEN-style Best-of-N procedure (N=5 per problem on GSM8k and MATH); candidate plans are filtered by a verifier score threshold (alpha=80) and by checking that the executed plan yields the exact gold answer. Three training variants are evaluated: M1 (SFT over plan, execution, and answer jointly), M2 (SFT over the plan only, with the final answer produced by an untrained 'off-the-shelf' executor), and Plan-GRPO (GRPO with a Gemini-based plan-similarity reward Rplan plus a binary answer-correctness reward Rans). Experiments on GSM8k, MATH, OlympiadBench, and AIME 2024 report consistent gains for M1 and M2 over CoT SFT baselines, and mixed gains for Plan-GRPO over vanilla GRPO, alongside ablations on dataset mixing and two-stage filtering. The central claim is that post-training with planning structure improves both in-domain accuracy and out-of-domain generalization of smaller LLMs on math reasoning.
Significance. The problem addressed (moving planning structure from inference-time prompting into post-training for small models) is timely and relevant, and the framework is simple enough to be reproducible: the appendices contain the data-synthesis and reward prompts, and the ablations (single- vs. two-stage filtering in Table 3; single- vs. mixed-corpus training in Figure 4) are informative and honestly reported. The authors also state their limitations clearly (upstream-model dependence, cost, math-only scope, manual thresholds, LLM-based plan reward). If the evaluation confounds below are resolved, the framework would be a reasonable empirical contribution to reasoning post-training. At present, however, the load-bearing comparisons have two confounds that the manuscript does not resolve: M2's final answers are produced by an external executor rather than by the fine-tuned model, and the plan-tuning corpus is filtered by verifier score plus answer correctness while the baseline SFT's data status is unspecified. The planning reward in the RL variant is self-referential (teacher-generated references scored by the same teacher model).
major comments (5)
- [§3.3.1 (Method 2); Table 2] The M2 evaluation is mismatched with the baseline. Method 2 states that after fine-tuning the model to generate a plan, 'we use the same off-the-shelf base LLM to execute that plan and produce the final answer,' yet the manuscript never identifies which model this executor is at inference. If it is Gemini-2.0-Flash (the data-synthesis LLM of §4.1), then the M2 results in Table 2, including the largest gains (Gemma MATH 83.8 and AIME 29.37 vs. 65.4 and 14.04 for baseline SFT), measure a Gemini-assisted pipeline rather than the smaller model's competence. If it is the untrained base checkpoint of the same model, the comparison is still unequal: M2 receives an additional model and an extra inference pass that the baseline SFT does not have, and no matched control (e.g., an executor applied to the baseline's CoT outputs) is reported. Either way, the M2 numbers cannot support the abstract's claim that PLAN-TUNING improves the task-specific performance of smaller LLMs as evaluated.
- [§3.2 (Training Data Quality); §4.2 (Baselines)] The plan-tuning corpus is filtered to problems where a Gemini plan passes the verifier threshold and executes to the exact gold answer, while the baseline SFT is described only as 'trained using reasoning chains and answers'; the paper does not state that the baseline uses the same filtered problem set, nor does it report the baseline's effective training-set size. The gains in Table 2 therefore conflate the planning format with the effect of selecting problems and trajectories that a strong teacher can solve. Table 3 shows that the answer-correctness filter alone changes results by roughly 4% on GSM8k and 9% on AIME, demonstrating that filtering is a substantial factor in the reported improvements. The central claim requires a baseline SFT trained on the same filtered problem subset with conventional CoT supervision, and ideally M1/M2 comparisons on identical subsets.
- [§3.3.2 (Details on Planning Reward)] The planning reward Rplan is a Gemini-2.0-Flash similarity score between the generated plan and a reference plan that was also generated by Gemini-2.0-Flash. 'High-quality plan' is therefore operationalized as 'resembles the teacher plan,' so the planning component of Plan-GRPO is a similarity/distillation signal rather than an externally grounded quality measure. Because the abstract presents reinforcement learning as part of the framework's contribution, the paper should either ablate Rplan against Rans alone with matched rollout counts, or use a plan-quality metric that is not defined by the same model that produced the references. The Limitations section acknowledges the use of LLM-based verification, but the main results in Table 2 are presented without this caveat.
- [Abstract/Introduction vs. Table 2] The headline numbers are not transparently derivable from Table 2. The abstract's 'average ~7%' (and the Introduction's '~7% on GSM8k and ~20% on MATH') do not match the mean absolute gains over the four in-domain SFT cells (~11.9% overall; ~5.3% on GSM8k, ~18.5% on MATH), nor do the ~10% and ~12% OOD claims match the mean OOD gains (~6.4% on OlympiadBench and ~9.5% on AIME). The aggregation rule (which cells, absolute vs. relative, which baseline) should be stated explicitly, and the large dispersion across configurations (e.g., Qwen MATH-trained M2 AIME +25.9 vs. Qwen GSM8k-trained M2 OlympiadBench +1.3) should be visible in the reporting.
- [§4.2 (Improvements with Plan-GRPO); Conclusions] The paper claims that plan-tuned models show better OOD generalization, but the GRPO results in Table 2 are mixed: Qwen3-4B Plan-GRPO underperforms vanilla GRPO on AIME (15.22 vs. 17.68) and essentially ties on OlympiadBench (25.07 vs. 24.63), while the Gemma-3-1B results are very low in absolute terms (28.35 on GSM8k, 3.00 on AIME). Since the GRPO experiments are confined to GSM8k-trained 1B/4B models with one epoch and four rollouts, the Conclusions' unqualified statement that plan-tuned models 'significantly improved' OOD performance should be scoped to the SFT variants, or the RL claim should be presented as preliminary.
minor comments (8)
- [§5 (Conclusions)] The Conclusions state that plan-tuning 'substantially mitigates the token-repetition issue,' but no experiment in Section 4 measures token repetition or arithmetic drift directly; either add the supporting analysis or remove the claim.
- [Table 2] Table 2 has presentation problems: the header 'Datasets Evaluated usingM(GSM8k)' is missing a space; the Gemma GRPO rows report '02.52' and '00.00' with leading zeros and leave the MATH-trained columns empty; and the table should state explicitly that the GRPO rows come from GSM8k-only training.
- [§4.1 (Datasets)] The 'AIME 2024' benchmark is described as having 933 problems and is cited to Sun et al. (2025), but standard AIME 2024 contains 30 problems; the composition of this benchmark (years covered, answer format, extraction and matching protocol) should be specified, since the reported absolute accuracies (e.g., 28.62 for a 12B baseline) are much higher than typical AIME results for models of this scale. The OlympiadBench micro-average procedure should also be described.
- [§4.2 (additional case study)] The Gemma-3-1B-It SFT run achieving 5.16% on GSM8k is surprisingly low for a 1B instruct model and is not a controlled comparison for the model-size argument (different objective, different data format, and no 1B SFT baseline in Table 2); please report the training details and sanity-check this number, since the GRPO discussion is anchored to it.
- [§4.1–4.2 (Experimental Setup)] No random seeds, inference decoding settings (temperature, sampling), or uncertainty estimates are reported; with only 500 held-out MATH problems, differences of 5 to 20 points would benefit from confidence intervals or significance testing.
- [§2 and §4.2] The paper motivates post-training against inference-time planning methods, but no prompting control (e.g., 'first make a plan, then solve' on the base model, or PlanGEN-style inference on the base model) is included; such a control is needed to support the claim that the gains come from post-training rather than from the planning format at inference.
- [§4.3 (Analysis)] The claim that 'enforcing intermediate correctness reduces arithmetic drift' is asserted without step-level error analysis; please soften or support it with evidence.
- [§4.2 (reproducibility)] The phrase 'during the limited rebuttal period' reads as a response-to-reviewers artifact and should be removed from a public manuscript; also, the repository link currently points only to qualitative examples, and releasing the filtered corpora, checkpoints, and evaluation scripts would materially improve reproducibility.
Circularity Check
No significant circularity: headline gains are measured against external gold-answer benchmarks, and the self-referential plan reward is a training objective, not the reported metric.
full rationale
The paper's central claim is that plan-tuned models achieve higher accuracy on GSM8k, MATH, OlympiadBench, and AIME. These accuracies are exact-match or micro-average scores computed against held-out gold answers, so the claimed improvements do not reduce to the training inputs by construction. The planning reward Rplan is defined as Gemini-based similarity between a generated plan and a Gemini-synthesized reference plan, which is self-referential as a definition of 'plan quality'; however, the paper does not report Rplan as evidence of success, and the conclusion that planning helps is supported by external benchmark accuracy, not by the reward itself. The self-citations to Parmar et al. (2025) (PlanGEN) supply a data-synthesis mechanism, a verification agent, and a score threshold (alpha = 80), but the paper also applies an independent answer-correctness filter using gold labels, and its own ablation (single-stage vs. two-stage filtering) provides internal evidence for the filtering design. These citations are tools, not uniqueness theorems, and they are not the sole support for the empirical claims. The M2 variant evaluates a plan generator plus an off-the-shelf executor, which is a potential confound and a correctness risk, but it is not a circular reduction: M2's reported accuracy is a genuine measurement of the combined pipeline against external answers. Overall, no step in the derivation is equivalent to its own input by definition, no fitted parameter is renamed as a prediction in the headline results, and the central empirical claim is anchored to independent benchmarks.
Assumptions & free parameters
free parameters (3)
- Plan verifier score threshold alpha =
80
- Number of candidate plans N and sampling temperature =
N=5, T=0.7
- Reward weights for plan and answer rewards =
implicit weights 1.0 for both
assumptions (4)
- domain assumption Gemini-2.0-Flash produces high-quality, transferable planning trajectories for math problems.
- domain assumption LLM-based plan verifier scores reliably rank plan quality above threshold 80.
- domain assumption Gemini similarity score is a valid reward signal for planning in GRPO.
- domain assumption The external execution LLM used in M2 is competent enough to execute generated plans faithfully.
Cite this review
Pith. "Pith review of PLAN-TUNING: Post-Training Language Models to Learn Step-by-Step Planning for Complex Problem Solving." pith.science (2026). https://pith.science/paper/LY2EVX43
@misc{pith2026250707495,
author = {Pith},
title = {Pith review of: PLAN-TUNING: Post-Training Language Models to Learn Step-by-Step Planning for Complex Problem Solving},
year = {2026},
howpublished = {\url{https://pith.science/paper/LY2EVX43}},
note = {Machine review of arXiv:2507.07495}
}
abstract
Recently, decomposing complex problems into simple subtasks--a crucial part of human-like natural planning--to solve the given problem has significantly boosted the performance of large language models (LLMs). However, leveraging such planning structures during post-training to boost the performance of smaller open-source LLMs remains underexplored. Motivated by this, we introduce PLAN-TUNING, a unified post-training framework that (i) distills synthetic task decompositions (termed "planning trajectories") from large-scale LLMs and (ii) fine-tunes smaller models via supervised and reinforcement-learning objectives designed to mimic these planning processes to improve complex reasoning. On GSM8k and the MATH benchmarks, plan-tuned models outperform strong baselines by an average $\sim7\%$. Furthermore, plan-tuned models show better generalization capabilities on out-of-domain datasets, with average $\sim10\%$ and $\sim12\%$ performance improvements on OlympiadBench and AIME 2024, respectively. Our detailed analysis demonstrates how planning trajectories improves complex reasoning capabilities, showing that PLAN-TUNING is an effective strategy for improving task-specific performance of smaller LLMs.
Figures
Figures from the paper (2 more)
Forward citations
Cited by 2 Pith papers
-
ThinkTuning: Instilling Cognitive Reflections without Distillation
ThinkTuning uses teacher feedback grafted onto GRPO rollouts to teach small language models reflective reasoning, improving MATH-500, AIME, and GPQA-Diamond accuracy over vanilla GRPO.
-
CoreThink: A Symbolic Reasoning Layer to reason over Long Horizon Tasks with LLMs
The paper claims a symbolic orchestration layer, CoreThink, achieves state-of-the-art results on seven coding and reasoning benchmarks with no training, but provides no verifiable implementation or method details.
Reference graph
Works this paper leans on
-
[1]
Decide on Travel Dates – Choose the season (e.g., cherry blossom in spring or autumn foliage) and trip duration
-
[2]
Set a Budget – Estimate total costs including flights, transport, lodging, food, and attractions
-
[3]
Check Entry Requirements – Look into visa policies, passport validity, and any travel restric- tions
-
[4]
Choose Destinations – Select cities and regions (e.g., Tokyo, Kyoto, Osaka, Nara, Hokkaido, Oki- nawa) based on your interests
-
[5]
Book Flights and Accommodation – Use plat- forms like Google Flights, Booking.com, or Airbnb; book early in peak seasons
-
[6]
Plan Transportation Within Japan – Consider a Japan Rail Pass and IC cards (Suica, Pasmo) for local travel
-
[7]
Create an Itinerary– Map out a day-by-day plan including travel time, attractions, and rest periods
-
[8]
Arrange Travel Essentials – Pocket Wi-Fi/SIM card, power adapter (Type A/B), Japanese yen (cash is still widely used)
Show all 43 references
-
[9]
Learn Local Etiquette – Study basic customs (e.g., removing shoes indoors, quiet on trains, no tipping)
-
[10]
How to schedule a meeting with your PhD colleague Response
Pack Appropriately– Bring weather-appropriate clothing, walking shoes, and a small daypack for city exploration. How to schedule a meeting with your PhD colleague Response
-
[14]
Define the Purpose – Clarify the topic (e.g., pa- per discussion, experiment planning)
-
[15]
Propose a Timeframe – Suggest a few potential date/time slots that work for you
-
[16]
Use Scheduling Tools – Use Doodle, Google Calendar, or Outlook to find mutual availability
-
[17]
Check Their Availability – Send a quick mes- sage (email/Slack/WhatsApp) asking for their availability
-
[18]
Suggest a Duration – Be clear about how long you expect the meeting to take (30 mins, 1 hour, etc.)
-
[19]
Decide the Format – In-person or virtual (Zoom, Teams, Google Meet)
-
[20]
Send Calendar Invite – Include topic, agenda (if needed), and location or meeting link
-
[21]
Follow Up Politely – If they don’t respond in a day or two, send a short, polite reminder
-
[22]
Prepare Materials – If needed, send any relevant documents or reading material beforehand
-
[23]
How to organize a party for the birthday Response
Confirm on the Day– Send a quick message con- firming the meeting a few hours before, especially for virtual calls. How to organize a party for the birthday Response
-
[24]
Set a Budget – Decide how much you want to spend overall
-
[25]
Choose a Date & Time – Pick a convenient day for the birthday person and guests
-
[26]
Pick a Venue – At home, a restaurant, park, or rented space—based on budget and guest size
-
[27]
Decide a Theme (optional) – Themes like retro, beach, movie night, or favorite colors add fun
-
[28]
Make a Guest List – Know how many people you’re inviting to plan food and space
-
[29]
Send Invitations– Use digital invites (WhatsApp, email, Evite) or printed cards
-
[30]
Plan Food & Drinks – Homemade, catered, or ordered—match it to the guest profile and theme
-
[31]
Arrange Decorations & Cake – Buy or DIY decorations and order a cake in advance
-
[32]
Plan Activities or Entertainment – Music, games, speeches, or photo booth to keep guests engaged
-
[33]
How to solve a given mathematical reasoning problem Response
Prepare a Backup Plan – Especially for outdoor events, have a Plan B in case of weather or other issues. How to solve a given mathematical reasoning problem Response
-
[34]
Understand the Problem – Read carefully and identify what is being asked
-
[35]
Identify Key Information – Extract numbers, relationships, and conditions from the problem
-
[36]
Break Down the Problem – Divide the complex problem into smaller, manageable parts
-
[37]
Visualize or Represent – Use diagrams, equa- tions, or tables if helpful
-
[38]
Recall Relevant Concepts – Think of formulas, theorems, or reasoning strategies that apply
-
[39]
Formulate a Plan – Decide on the method (e.g., algebra, logic, geometry) to approach the prob- lem
-
[40]
Execute Step-by-Step – Carry out your plan me- thodically, solving subparts if needed
-
[41]
Check Units and Logic – Make sure the answer is in the correct form and logically consistent
-
[42]
Verify Your Answer– Plug it back into the origi- nal problem to confirm it satisfies all conditions
-
[43]
B Data Synthesis Prompts These prompts form a structured framework for evaluating and improving mathematical reason- ing in large language models
Reflect on the Method – Consider if there’s a more efficient or alternative solution strategy. B Data Synthesis Prompts These prompts form a structured framework for evaluating and improving mathematical reason- ing in large language models. The Plan Gener- ation Prompt encour...
-
[2023]
In Thirty-seventh Conference on Neural Information Processing Sys- tems
Direct preference optimization: Your language model is secretly a reward model. In Thirty-seventh Conference on Neural Information Processing Sys- tems. David Rein, Betty Li Hou, Asa Cooper Stickland, Jack- son Petty, Richard Yuanzhe Pang, Julien Dirani, Ju- lian Michael, and ...
2024 arXiv
-
[2024]
arXiv preprint arXiv:2412.19437
Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing...
2023 arXiv
-
[2025]
arXiv preprint arXiv:2502.16111
Plangen: A multi-agent framework for gener- ating planning and reasoning trajectories for complex problem solving. arXiv preprint arXiv:2502.16111. Pruthvi Patel, Swaroop Mishra, Mihir Parmar, and Chitta Baral. 2022. Is a question decomposition unit all we need? In Proceedings...
2022 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.