REVIEW 11 cited by
1.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Training
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The AM-DeepSeek-R1-Distilled is a large-scale dataset with thinking traces for general reasoning tasks, composed of high-quality and challenging reasoning problems. These problems are collected from a multitude of open-source datasets, subjected to semantic deduplication and meticulous cleaning to eliminate test set contamination. All responses within the dataset are distilled from reasoning models (predominantly DeepSeek-R1) and have undergone rigorous verification procedures. Mathematical problems are validated by checking against reference answers, code problems are verified using test cases, and other tasks are evaluated with the aid of a reward model. The AM-Distill-Qwen-32B model, which was trained through only simple Supervised Fine-Tuning (SFT) using this batch of data, outperformed the DeepSeek-R1-Distill-Qwen-32B model on four benchmarks: AIME2024, MATH-500, GPQA-Diamond, and LiveCodeBench. Additionally, the AM-Distill-Qwen-72B model surpassed the DeepSeek-R1-Distill-Llama-70B model on all benchmarks as well. We are releasing these 1.4 million problems and their corresponding responses to the research community with the objective of fostering the development of powerful reasoning-oriented Large Language Models (LLMs). The dataset was published in \href{https://huggingface.co/datasets/a-m-team/AM-DeepSeek-R1-Distilled-1.4M}{https://huggingface.co/datasets/a-m-team/AM-DeepSeek-R1-Distilled-1.4M}.
Forward citations
Cited by 11 Pith papers
-
POINTS-Seeker: An Open Recipe for Multimodal Search Agents with Visual Memory Management
POINTS-Seeker-8B is an 8B multimodal model trained from scratch for agentic search that uses seeding and visual-space history folding to outperform prior models on six visual reasoning benchmarks.
-
The Challenge of Teaching Reasoning to LLMs Without RL or Distillation
Twenty high-quality chain-of-thought examples from a reasoning model are enough to activate strong math reasoning in a 32B base model with lightweight fine-tuning.
-
Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Prompting
Fine-tuning small models on difficulty-adapted, shortened reasoning traces (LiteCoT) yields equal or better benchmark accuracy than training on much longer traces, with far fewer tokens.
-
Not All Tokens Are What You Need In Thinking
A method that scores each chain-of-thought token by answer-conditioned perplexity and trains models on the compressed traces preserves or improves reasoning accuracy with significantly fewer tokens.
-
Not All Correct Answers Are Equal: Why Your Distillation Source Matters
A large-scale comparison of three distillation teachers shows that the choice of teacher model strongly affects student reasoning performance, with AM-Thinking-v1 distilled data leading on all tested benchmarks.
-
Reasoning Fine-Tuning Induces Persistent Latent Policy States
Reasoning fine-tuning reorganizes chain-of-thought into more differentiated latent switching states, and pruning with those states beats self-consistency in 11 of 12 settings.
-
SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning
SwS uses failures during RL training to synthesize targeted math problems, improving reasoning accuracy on eight benchmarks.
-
Weak-Driven Learning: How Weak Agents make Strong Agents Stronger
Mixing an LLM's logits with an earlier weak checkpoint during fine-tuning yields math and code accuracy gains beyond standard SFT saturation.
-
MiroMind-M1: An Open-Source Advancement in Mathematical Reasoning via Context-Aware Multi-Stage Policy Optimization
MiroMind-M1 open-sources a two-stage SFT plus RLVR recipe with a new context-aware multi-stage policy optimization (CAMPO) that claims competitive AIME24, AIME25, and MATH500 scores among Qwen-2.5-based models.
-
OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Models
The paper advocates protecting and leveraging OpenReview's peer review corpus as a community asset for LLM-based review assistance, benchmarks, and alignment.
-
Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.
Discussion (0). Continue with ORCID to comment.