CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks

Ilia Kulikov; Jack Lanchantin; Jason Weston; Jing Xu; Olga Golovneva; Ping Yu; Sainbayar Sukhbaatar; Tianlu Wang; Weizhe Yuan

REVIEW 4 major objections 5 minor 4 cited by

CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that making an LLM reason and plan before writing new training prompts produces synthetic data that beats human-curated and existing synthetic reasoning datasets.

desk verdict A practical synthetic-data recipe with strong ablations, but the s1k seeds make benchmark contamination a real open question. read the letter →

arxiv 2507.23751 v2 pith:JKGA6TUV submitted 2025-07-31 cs.AI cs.CL

Ping Yu , Jack Lanchantin , Tianlu Wang , Weizhe Yuan , Olga Golovneva , Ilia Kulikov , Sainbayar Sukhbaatar , Jason Weston

show 1 more author

Jing Xu

This is my paper · ORCID

classification cs.AIcs.CL

keywords syntheticdatachain-of-thoughtself-instructcurationmathematicalreasoninginstructionfollowingreinforcementlearninganswerconsistency

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CoT-Self-Instruct claims that the quality of synthetic training data improves substantially when the generating model is made to reason and plan before it writes a new example, and then a filter removes the examples the model cannot reliably solve. The paper extends Self-Instruct, which prompts an LLM with seed tasks to produce new ones, by inserting a Chain-of-Thought stage that analyzes the seeds, brainstorms candidates, and self-evaluates them, generating both the question and its target answer in one reasoned pass. A curation step then rejects low-quality examples: for verifiable reasoning tasks it uses Answer-Consistency, where the target answer must match the majority vote of fresh samples, and for open-ended instruction following it uses a reward-model-based RIP filter. Trained with GRPO, Qwen3-4B-Base reaches 57.2% average accuracy on MATH500, AMC23, AIME24, and GPQA-Diamond, ahead of s1k's 44.6% and OpenMathReasoning's 47.5%, while Llama 3.1-8B-Instruct trained with DPO on the instruction-following data reaches 54.7 on AlpacaEval 2 and Arena-Hard, above human WildChat data at 46.8. The paper aims to show that reasoning-in-the-loop generation plus automatic filtering can replace ever-larger human-curated datasets.

What carries the argument

The load-bearing object is the CoT-Self-Instruct prompt template: given two seed instructions, the model must list their common elements, write a plan for a new self-contained instruction of comparable quality and complexity, and produce the instruction in a strict bracketed format; for reasoning tasks it must also solve the new problem and emit a boxed final answer in the same pass. The second component is the curation filter. Answer-Consistency, the paper's new filter for verifiable tasks, samples K fresh solutions to each generated question, takes the majority vote, and deletes any example whose CoT-time target answer does not match that vote; the paper argues the CoT-time answer is more informative than a vote alone because it was produced together with the problem's construction. For non-verifiable tasks the paper adapts RIP, which judges instructions by the distribution of reward-model scores over sampled responses, modifying it to sample several instructions per few-shot prompt and keep the highest-scoring one, which avoids the topic shift a single global threshold causes.

What would settle it

An overlap audit would settle it: compute ROUGE-L or embedding similarity between every training example used in the experiments and every question in MATH500, AMC23, AIME24, GPQA-Diamond, AlpacaEval 2, and Arena-Hard, and if a substantial fraction of the solved training questions nearly duplicates benchmark items, the central claim fails. Re-running the same GRPO and DPO pipelines after deleting all examples above a similarity threshold would directly measure how much of the reported margin is contamination.

Watch

Extended reading notes

Core claim

The central claim is that CoT-Self-Instruct produces training data good enough to beat both the seed instructions it starts from and the largest public synthetic reasoning datasets, in verifiable and non-verifiable settings alike. For reasoning, generating an instruction jointly with its answer through explicit step-by-step reasoning, then discarding any example whose target answer disagrees with the majority-vote answer from additional samples, yields 57.2% average accuracy on MATH500, AMC23, AIME24, and GPQA-Diamond when Qwen3-4B-Base is trained with GRPO, versus 44.6% for s1k and 47.5% for OpenMathReasoning; scaling the filtered set to 10,000 examples raises the average to 58.7%. For instruction following, generating prompts from domain-clustered seeds with a planning step, then keeping the single highest-reward-model-scored prompt per few-shot batch, gives a DPO-trained Llama 3.1-8B-Instruct 54.7 average on AlpacaEval 2 and Arena-Hard, beating human WildChat data at 46.8 and standard Self-Instruct at 47.4, and online-DPO training lifts the same data to 67.1 versus 63.1 for human data. The paper's stated conclusion is that reasoning-based creation plus curation creates high-quality synthetic instructions for RL training in both domains.

Load-bearing premise

The load-bearing premise is that the generated training questions do not overlap with the evaluation questions, since the paper never filters or checks for contamination against MATH500, AMC23, AIME24, GPQA-Diamond, AlpacaEval, or Arena-Hard; if the seed-derived synthetic pool contains benchmark questions, the reported gains are inflated.

Editorial extensions

If this is right

GRPO-trained models built on CoT-Self-Instruct data outperform models trained on the s1k seed set itself, meaning the pipeline extracts more training signal from 1,000 seed questions than the seeds provide directly.
Less filtered data beats more unfiltered data: filtering shrinks the training pool yet consistently improves accuracy, so data quality, not volume, is the dominant lever in this regime.
Answer-Consistency filtering outperforms Self-Consistency filtering, implying that the answer produced during question construction carries information beyond a fresh majority vote.
Adding more filtered synthetic data, from 5,000 to 10,000 examples, keeps improving results, suggesting the method scales where fixed public sets cannot.
Reasoning-based generation helps even non-reasoning tasks: longer CoT planning beats short CoT and no CoT for instruction-following, so the benefit is not specific to mathematics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test the paper does not run is a generation-evaluation overlap audit: measuring n-gram or embedding similarity between the synthetic training pool and the evaluation benchmarks would show how much of the reported margin is contamination-free.
A natural extension is to vary seed-pool composition and domain balance to test whether CoT-Self-Instruct amplifies seed quality or merely copies it, since the paper reports only the single s1k and WildChat seed configurations.
The Answer-Consistency filter treats the generator's majority vote as a correctness oracle; probing filter accuracy on seeds with known answers would reveal whether the filter can admit confidently wrong labels.
Because the same model family generates and trains, the gap over Self-Instruct may depend on the generator being only modestly stronger than the trainee; re-running the pipeline with a much stronger generator would test whether the advantage scales in a fully self-improving loop.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a useful, well-run empirical study, and the headline claim is plausible but not fully earned until they measure overlap between their s1k-seeded synthetic data and the four benchmarks. That measurement is missing, and it is load-bearing.

What's actually new: taking Self-Instruct and adding a CoT step that generates both the instruction and its target answer, then an Answer-Consistency filter that keeps samples where the majority-vote solution matches the generated target. The recipe is simple, and the paper does the right work to support it: ablations that isolate CoT versus no CoT, instruction-only versus instruction-plus-target, and three filtering methods; size-matched training sets; multiple seed models; and both math and instruction-following benchmarks. The gains over s1k and OpenMathReasoning (57.2 vs 44.6, 47.5) are large, and the component-wise ablations are consistent.

The soft spots. The missing contamination analysis is the one that matters. The synthetic instructions are seeded from s1k, which is itself a curated set of public benchmark-style problems, and the evaluation is on MATH500, AMC23, AIME24, GPQA-Diamond. There is no deduplication, no n-gram overlap, no embedding similarity check, no manual audit, and no release of data or code so a reviewer can do it themselves. This is not evidence of wrongdoing, but it is evidence of an unmeasured confound. The instruction-following head has the same gap, though the risk looks lower there because AlpacaEval and Arena-Hard prompts are not the kind of thing a few-shot generator seeds from WildChat would reproduce verbatim.

Smaller weaknesses: no variance or significance across the reported numbers; the AlpacaEval/Arena-Hard runs use GPT-4-Turbo and GPT-4o judges rather than the standard judge versions, which makes cross-paper comparison approximate; and an odd raw chain-of-thought trace appears inside the manuscript text, which should be cleaned up.

Overall, I believe the central claim holds up conditionally. The method is a straightforward extension of known components, the ablations are internally consistent, and the gains are big enough that even an inflated-but-positive result would still point to something useful. The paper is for people building synthetic post-training data for RLVR and instruction following. It deserves a serious referee. I'd send it out, and ask for the contamination measurement and for data/code release as revision conditions, not reject-and-resubmit conditions.

Referee Report

4 major / 5 minor

Summary. The paper proposes CoT-Self-Instruct, a two-stage synthetic data pipeline. In the first stage, an LLM is prompted with seed instructions and asked to reason about their common elements and complexity before generating a new instruction (and, for verifiable reasoning tasks, a target answer). In the second stage, generated data are filtered: Answer-Consistency for verifiable tasks, and RIP (Rejecting Instruction Preferences) for non-verifiable instruction-following tasks. The authors train Qwen3-4B-Base with GRPO on the generated reasoning data and Llama 3.1-8B-Instruct with DPO/online DPO on the instruction-following data. Their headline results are that CoT-Self-Instruct with Answer-Consistency filtering reaches 57.2% average accuracy on MATH500, AMC23, AIME24, and GPQA-Diamond, outperforming s1k (44.6%) and OpenMathReasoning (47.5%); and that CoT-Self-Instruct with RIP filtering reaches 54.7 on AlpacaEval 2 / Arena-Hard, outperforming human WildChat data (46.8).

Significance. If the central claims hold, the paper would be a useful empirical contribution: it provides extensive ablations, including size-matched comparisons (Tables 3 and 5), multiple filter variants, and different target-generation schemes (Tables 4-10), and it documents its prompts in the appendix. The use of external benchmarks and the same model family for generation and training is a standard self-training setup and is not circular by itself. However, two issues currently weaken the significance: the absence of any contamination analysis between the seed-driven synthetic data and the evaluation benchmarks, and the absence of variance or significance testing for the headline comparisons. The paper also contains no code or data release, which limits reproducibility and makes independent verification of the contamination question impossible.

major comments (4)

[Section 4.1 / Section 5.1, Table 1] The paper reports no contamination check between the generated synthetic training data and the evaluation benchmarks. The seed pool is s1k (Section 4.1), and s1k is curated from public reasoning benchmarks; evaluation is then run on MATH500, AMC23, AIME24, and GPQA-Diamond. Since the same model family is prompted with s1k seeds and asked to produce novel questions of comparable difficulty, exact or near-duplicate reproduction of benchmark questions is plausible. The headline gains (57.2 vs 44.6 for s1k, and 57.2 vs 47.5 for OpenMathReasoning) are only meaningful if the synthetic training questions are not duplicated in the test sets. Please add a contamination analysis: normalized exact-match removal, n-gram overlap statistics, embedding-similarity checks, and a manual audit of the nearest neighbors, and report the main results after excluding any overlapping examples. The same issue, though with lower a priori risk, applies to the WildChat-seeded instruction-following track evaluated on AlpacaEval 2 and Arena-Hard.
[Section 5.1, Tables 1, 4, and 5] The headline comparisons are reported without any variance or significance testing. All numbers are point estimates of pass@1 averaged over 16 sampled rollouts, and no repeated GRPO training runs are reported. The abstract and Section 5.1 use the word "significantly" (e.g., "significantly outperform" and "still significantly higher"), but no standard errors, confidence intervals, or hypothesis tests support that wording. For example, the AIME difference between Answer-Consistency filtering (24.6) and Self-Consistency filtering (22.5) in Table 1 may be within sampling noise. Please provide standard errors or confidence intervals for the main comparisons, or per-problem matched-pair tests, so the reader can assess whether the ranking of methods is reliable.
[Section 5.1, paragraph after Table 1] The text states that 'in each case we see the same trend that CoT-Self-Instruct with Answer-Consistency is superior to Self-Instruct and other competing baselines' when targets are generated by other means (Appendix Tables 6, 7, and 8). This claim is not supported by those tables: Table 6 (majority-voted targets) and Table 7 (Best-of-K targets) contain no Answer-Consistency condition, and Table 8 (base-model targets) also contains no Answer-Consistency row. Either add the missing Answer-Consistency experiments for these target-generation settings or soften the claim to reflect only the conditions actually tested.
[Section 3.2 / Section 4.1] The Answer-Consistency filter is defined as generating K responses and comparing the majority vote to the target answer, but the value of K is never reported in the main text or appendix. Similarly, the Self-Consistency threshold (SC >= 0.5) and RIP score threshold (RSc >= 0.5) are fixed without a sensitivity analysis. Without reporting K and without showing how results vary with the chosen thresholds, the method is not fully reproducible and the filter comparisons may depend on arbitrary parameter choices. Please state K explicitly and include a small sensitivity sweep over the filter thresholds.

minor comments (5)

[Table 5 caption] The caption contains the typo 'AMIE24'; it should read 'AIME24'.
[Section 5.1, last paragraph] The word 'perfomance' should be 'performance'.
[Section 4.2, first paragraph] The model name is written as 'LLama 3.3-70B-Instruct' and the training model as 'LLama 3.1-8B-Instruct'; please use consistent capitalization (Llama).
[Table 1, Answer-Consistency (more data) row] The 10,000-example Answer-Consistency condition is listed in Table 1 but is not described in the experimental setup; please explain how this larger set was generated and filtered, and whether it is the same pipeline as the 5,000-example set.
[General] No code or data release is mentioned. Releasing the synthetic training sets and the filtering scripts would substantially help readers verify the contamination analysis and reproduce the results.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation chain found: the synthetic-data pipeline is evaluated against external benchmarks, and the only self-citation (RIP) is used as a non-load-bearing filtering tool.

full rationale

The paper's central claims are empirical results from a synthetic-data self-training pipeline, not analytic derivations that reduce to their own inputs. In the reasoning branch, generation is seeded from s1k instructions and evaluated on MATH500, AMC23, AIME24, and GPQA-Diamond; the generated questions and target answers are produced by Qwen3-4B, filtered by internal consistency, and then used for GRPO. The evaluation is external to the synthetic-data construction, so the reported gains (57.2% vs 44.6% for s1k and 47.5% for OpenMathReasoning) are not forced by definition or by a fitted parameter. The Answer-Consistency filter compares a self-generated target answer to a majority-vote solution from the same model; this is an internal quality heuristic, not a quantity later 'predicted' on the benchmarks. The instruction-following branch uses RIP, cited to Yu et al. (2025), which shares authors with this paper; however, RIP is applied as a filtering tool, the unfiltered CoT-Self-Instruct already outperforms the human-data baseline (53.9 vs 46.8), and the evaluation judge is external (GPT-4-turbo and GPT-4o), so the self-citation is not load-bearing. The absence of a contamination check between the s1k-seeded synthetic data and the public evaluation benchmarks is a correctness and validity risk, but it is not a circular reduction: no equation or construction in the paper equates the synthetic training distribution with the evaluation set. Overall, the derivation chain is self-contained against external benchmarks and warrants a low circularity score, with only a minor self-citation that does not carry the central argument.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method relies on hand-chosen filtering thresholds and a length-reward coefficient, plus assumptions that reward models and consistency voting are valid quality signals. The most consequential unstated assumption is the absence of benchmark contamination. No new physical or conceptual entities are introduced.

free parameters (4)

Self-Consistency filter threshold = 0.5 (majority vote ratio)
Chosen by hand; controls how many synthetic examples are kept (e.g., 4034 of 5000).
RIP score threshold/quantile = 0.5
Chosen by hand; used to filter 5000 generated instructions down to 2419-2491.
Length normalization coefficient = 0.2
Chosen to mitigate length explosion during DPO; applied to reward when selecting preferred responses.
Number of synthetic candidates per prompt (RIP selection) = 32
For each few-shot prompt, 32 instructions are generated and the highest RIP score is kept.

assumptions (3)

domain assumption Reward model scores (INF-ORM-Llama3.1-70B, Athene-RM-8B) are reliable indicators of instruction and response quality for filtering.
Used in RIP filtering for both reasoning and instruction-following data (Sections 3.2, 4.2).
domain assumption Majority-vote consistency between the CoT-generated target answer and sampled responses identifies high-quality, answerable questions.
Basis of the Answer-Consistency filter (Section 3.2), following Self-Consistency filtering (Prasad et al., 2024).
domain assumption Generated synthetic instructions do not overlap with evaluation benchmarks (no contamination).
Unstated; no deduplication or overlap check against MATH500, AMC23, AIME24, GPQA-Diamond, AlpacaEval, or Arena-Hard is described (Sections 4 and 5).

how reviews work

0 comments

Cite this review

Pith. "Pith review of CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks." pith.science (2026). https://pith.science/paper/JKGA6TUV

@misc{pith2026250723751,
  author       = {Pith},
  title        = {Pith review of: CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JKGA6TUV}},
  note         = {Machine review of arXiv:2507.23751}
}

read the original abstract

We propose CoT-Self-Instruct, a synthetic data generation method that instructs LLMs to first reason and plan via Chain-of-Thought (CoT) based on given seed tasks, and then generate a new synthetic example of similar quality and complexity. This is followed by a filtering step to select high-quality data using automatic metrics, which are then used for LLM training. In verifiable reasoning, our synthetic data significantly outperforms existing training datasets, such as s1k and OpenMathReasoning, when evaluated on MATH500, AMC23, AIME24, and GPQA-Diamond. For non-verifiable instruction-following tasks, our method surpasses the performance of both human and standard Self-Instruct training data on the AlpacaEval 2.0 and Arena-Hard benchmarks.

Figures

Figures reproduced from arXiv: 2507.23751 by the authors.

**Figure 1.** CoT-Self-Instruct. Our method first prompts LLMs to reason and generate new instructions given seed instructions, followed by automatic curation of high-quality data using either Answer-Consistency for verifiable reasoning tasks, or RIP (Yu et al., 2025) for non-verifiable tasks. examples are either incorrectly labeled or too difficult. For non-verifiable data, we use the recent Rejecting Instruction Preferences (R… view at source ↗

**Figure 2.** CoT-Self-Instruct instruction generation template for verifiable reasoning tasks. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** CoT-Self-Instruct instruction generation template for general instruction following tasks. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Self-Instruct instruction generation template for general instruction following tasks. [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

**Figure 5.** Figure 5: Short CoT instruction generation template for general instruction following tasks. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

**Figure 6.** Figure 6: Self-Instruct (standard, without CoT) instruction generation template for verifiable rea [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

**Figure 7.** Figure 7: Self-Instruct instruction & target generation template (standard, without CoT) for verifi [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

**Figure 8.** Figure 8: CoT-Self-Instruct (No-Solve) instruction generation template for verifiable reasoning tasks [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

**Figure 9.** Figure 9: Self-Instruct-Then-Solve (i.e. No CoT) instruction generation template for verifiable rea [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Refine After Generation: Toward Correct and Concise Patches in LLM-based Program Repair
cs.SE 2026-08 conditional novelty 6.0 of 10

A post-generation refinement adapter, trained with SFT and DPO on constructed verbose-to-concise pairs, makes LLM repair patches much smaller across four SWE-bench systems while preserving or improving resolution.
Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills
cs.CL 2026-07 conditional novelty 6.0 of 10

Skill Self-Play uses an evolving skill library to guide LLM self-training, improving tool-call and reasoning accuracy beyond unguided self-play on five model backbones.
Autodata: An agentic data scientist to create high quality synthetic data
cs.AI 2026-06 unverdicted novelty 6.0 of 10

Autodata trains meta-optimized AI agents to generate superior synthetic datasets, yielding performance gains over classical methods on CS research, legal, and math reasoning tasks.
Less is Enough: Synthesizing Diverse Data in LLM Feature Space with Sparse Autoencoders
cs.CL 2026-02 conditional novelty 6.0 of 10

Coverage of sparse-autoencoder-identified task features predicts post-training performance and can guide synthesis of small, high-impact datasets (2,000 vs. 300,000 samples).

Reference graph

Works this paper leans on

10 extracted references · 10 canonical work pages · cited by 4 Pith papers

[2]

Below are sample tasks from user

<begin>{INSTRUCTION 2}</end> Come up with one new task, wrapped with <begin>and </end> Figure 5: Short CoT instruction generation template for general instruction following tasks. Below are sample tasks from user

work page
[3]

<begin>{INSTRUCTION 1}</end>

work page
[4]

reference

<begin>{INSTRUCTION 2}</end> Come up with one new task, wrapped with <begin>and </end>. Please provide your Chain- of-Thought first and then provide the new generated task. 7 A PPENDIX We report results when matching the training size to 893 the same as our seed tasks in Table 3. Table 3: CoT-Self-Instruct results on reasoning tasks with same size trainin...

work page
[7]

Do not include any solution, hint, or answer-—only the question statement itself. Please put your generated problem strictly in the format of [New Question Begin]{your generated question}[New Question End] Figure 7: Self-Instruct instruction & target generation template (standard, without CoT) for verifi- able reasoning tasks. This template prompts the LL...

work page
[9]

You are a reasoning question generator assistant

Then solve the new question and format your output as follows: [New Question Begin]{your generated question}[New Question End] [Final Answer to New Question Begin]\boxed{your final answer}[Final Answer to New Question End] 15 Figure 8: CoT-Self-Instruct (No-Solve) instruction generation template for verifiable reasoning tasks without answering (i.e., gene...

work page
[10]

The question draws inspiration from the seed question without copying it verbatim, remaining novel and of comparable difficulty

work page
[11]

The question’s final answer should be a single, unambiguous scalar value (e.g., an integer, reduced fraction, exact radical), or another answer type that can be verified in one step (e.g., ‘yes/no,’ a choice from A to D)

work page
[12]

Do not include any solution, hint, or answer-—only the question statement itself. Please reason step by step and put your generated problem strictly in the format of [New Question Begin]{your generated question}[New Question End] Figure 9: Self-Instruct-Then-Solve (i.e. No CoT) instruction generation template for verifiable rea- soning tasks. You are a re...

work page

Show all 10 references

[13]

Write a brand-new, self-contained reasoning question that meets the following requirements: (a) The question draws inspiration from the seed question without copying it verbatim, remaining novel and of comparable difficulty. (b) The question’s final answer should be a single, ...
[14]

We conduct GRPO-training using Qwen3-4B-Base model on synthetic instructions generated by different templates, with targets sampled from Qwen3-4B

Then solve the new question and format your output as follows: [New Question Begin]{your generated question}[New Question End] [Final Answer to New Question Begin]\boxed{your final answer}[Final Answer to New Question End] 16 Table 4: Results of CoT-Self-Instruct, comparing to...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [2]

Below are sample tasks from user

<begin>{INSTRUCTION 2}</end> Come up with one new task, wrapped with <begin>and </end> Figure 5: Short CoT instruction generation template for general instruction following tasks. Below are sample tasks from user

work page

[2] [3]

<begin>{INSTRUCTION 1}</end>

work page

[3] [4]

reference

<begin>{INSTRUCTION 2}</end> Come up with one new task, wrapped with <begin>and </end>. Please provide your Chain- of-Thought first and then provide the new generated task. 7 A PPENDIX We report results when matching the training size to 893 the same as our seed tasks in Table 3. Table 3: CoT-Self-Instruct results on reasoning tasks with same size trainin...

work page

[4] [7]

Do not include any solution, hint, or answer-—only the question statement itself. Please put your generated problem strictly in the format of [New Question Begin]{your generated question}[New Question End] Figure 7: Self-Instruct instruction & target generation template (standard, without CoT) for verifi- able reasoning tasks. This template prompts the LL...

work page

[5] [9]

You are a reasoning question generator assistant

Then solve the new question and format your output as follows: [New Question Begin]{your generated question}[New Question End] [Final Answer to New Question Begin]\boxed{your final answer}[Final Answer to New Question End] 15 Figure 8: CoT-Self-Instruct (No-Solve) instruction generation template for verifiable reasoning tasks without answering (i.e., gene...

work page

[6] [10]

The question draws inspiration from the seed question without copying it verbatim, remaining novel and of comparable difficulty

work page

[7] [11]

The question’s final answer should be a single, unambiguous scalar value (e.g., an integer, reduced fraction, exact radical), or another answer type that can be verified in one step (e.g., ‘yes/no,’ a choice from A to D)

work page

[8] [12]

Do not include any solution, hint, or answer-—only the question statement itself. Please reason step by step and put your generated problem strictly in the format of [New Question Begin]{your generated question}[New Question End] Figure 9: Self-Instruct-Then-Solve (i.e. No CoT) instruction generation template for verifiable rea- soning tasks. You are a re...

work page