{"id":"a646eee2-548f-4467-8109-422318c09fe5","arxiv_id":"2501.13126","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A Perplexity Difference based easy-to-hard curriculum improves LLM pretraining accuracy on several benchmarks, with an 8.1 point gain on MMLU and CMMLU at 3B scale.","lead":"The authors propose a pretraining curriculum that sorts text by Perplexity Difference, a measure of how much harder a sample is for a weak model than for a strong model, and feeds easy samples first and hard samples later. They report consistent accuracy gains over random ordering at 1.3B and 3B scale, including an 8.1 point average improvement on MMLU and CMMLU for a 3B model trained on 1 trillion tokens.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 8.1% 3B headline gain rests on an unvalidated extrapolation: reference-model PD is never checked against the actual 3B model's preferences on the benchmarks that carry the claim.","rationale":"The reader's weakest assumption is also the load-bearing point: reference-model PD must approximate the experimental model's true preferences. My stress-test narrows it further: the only validation (Figure 4) uses a different model size and different evaluation tasks than the headline 3B/MMLU-CMMLU claim, and the preference curve itself is not measured (Section 6, Appendix B not run). This does not make the paper internally inconsistent, and the ablations and honest limitations deserve credit, but the central empirical claim currently rests on an extrapolation from 1.3B to 3B and from easy tasks to MMLU/CMMLU. A conditional verdict remains appropriate; the condition should explicitly include demonstrating the proxy on the 3B model and reporting seed variance for the 8.1% gain.","tokens_in":20416,"tokens_out":6360,"duration_ms":70180,"concrete_test":"Run the 3B/1T experiment on the same bilingual corpus with three arms: (i) the paper's RM-PD ordering, (ii) PD computed from early and final checkpoints of the Random 3B run, and (iii) random order, all using the reported S-shape with a=10. If arm (ii) reproduces the MMLU/CMMLU gain and its PD ranking correlates with arm (i) at Spearman >= 0.8 on a held-out sample, the proxy is validated. If arm (ii) loses the gain or the rankings diverge, the headline 8.1% rests on an unvalidated approximation. The same run also provides multi-seed variance for the single-run result.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's 3B/1T result — +8.1% on MMLU/CMMLU — is a reordering effect, so everything depends on the signal used to order the data. That signal is PD computed from 100M and 1.3B reference models, under the claim that this approximates what the 3B experimental model prefers at early versus late training. This is the mechanism that separates PDPC from Random, and the only direct evidence is the ablation in Figure 4. That figure is at 1.3B scale, on ARC-E/ARC-C/SciQ/HellaSwag/PIQA only; it compares RM-based PD against early/end checkpoints of a randomly trained 1.3B model, and the aggregate excludes MMLU/CMMLU. It therefore does not establish that RM-PD ranks data the way the 3B model would, nor that the proxy works for the benchmarks that produce the headline gain. Figure 11's Spearman correlations only show consistency among different RM pairs, not agreement with the 3B model. Section 6 concedes the S-shape preference curve was chosen by functional exploration rather than measured, and Appendix B's annealing-based validation is not actually run. So the 'preference' in PDPC is an assumed form, and the 8.1% could come from the particular schedule/partition rather than from the model's true data preference. This is a load-bearing gap, not an internal inconsistency.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PDPC, a curriculum pretraining framework that orders data by Perplexity Difference (PD), defined as the relative perplexity gap between a weak and a strong reference model. The data are partitioned into low-PD and high-PD halves, and an S-shape preference function (Eq. 8) schedules the mixing ratio of low-PD data over training progress, gradually shifting from low-PD to high-PD data. The authors report consistent gains over random sampling at 1.3B scale on 100B tokens, and a headline 3B/1T result with +8.1% average accuracy on MMLU and CMMLU (Table 2). The data ordering is computed offline with small reference models, so pretraining runs without interruption.","tokens_in":20710,"tokens_out":6142,"duration_ms":62549,"significance":"If the 3B result is reliable, the paper offers a practical and surprisingly effective data-reordering curriculum that requires no extra data, no extra compute, and no training interruption. The 1.3B controlled comparisons in Tables 1 and 3 are a genuine strength: they span multiple metrics (PD, PPL, QuRating) and multiple schedule families, and the PD-based S-shape curve consistently wins. The reference-model approximation for PD is a sensible cost-reduction idea, and the distributional analyses in Figures 6-9 help build intuition. However, the central mechanism the paper advertises in its title and abstract---that the method 'always perceives and uses the data preferred by LLMs'---is not empirically grounded. The preference curve is fitted to downstream benchmark accuracy, and the reference-model PD is validated only at 1.3B on a subset of benchmarks that exclude MMLU and CMMLU. The authors honestly acknowledge the functional-exploration origin of the curve in Section 6, but that acknowledgment also marks the load-bearing gap: the title claim is currently a hypothesis, not a measured property.","major_comments":[{"comment":"The 3B/1T headline gain of +8.1% on MMLU/CMMLU rests on PD computed from 100M and 1.3B reference models, yet the only direct validation of this approximation is at 1.3B scale and on ARC-E/ARC-C/SciQ/HellaSwag/PIQA (Figure 4). The paper never checks whether RM-PD agrees with the 3B model's actual preferences---for example, by computing PD from early and late checkpoints of a randomly trained 3B model and comparing it with the RM-based PD, or by reporting any 3B-scale correlation. Moreover, the validation benchmarks exclude exactly the two benchmarks (MMLU, CMMLU) that produce the headline improvement. Because the entire method is a reordering based on this preference signal, the 3B result is currently unexplained by the proposed mechanism; it could be an artifact of the specific schedule rather than of the model's true data preference.","section":"§3.2, Table 2; §3.3, Figure 4"},{"comment":"The S-shape preference function f(p)=1/(1+exp(a(p-0.5))) with a=10 is selected by maximizing downstream accuracy on the evaluation benchmarks themselves (Table 3), and the same benchmarks are used to report the main results. The paper's own limitation section states that the curve was 'determined through functional exploration' rather than measured, and the annealing-based measurement described in Appendix B is not actually executed---no experimental results are reported there. Thus the 'preference function' is a fitted hyperparameterized schedule, not an empirical measurement of what the model prefers. The central claim that the method 'always perceives and uses the data preferred by LLMs' therefore overstates what is demonstrated. The authors should either run the annealing experiments and show that the fitted S-shape matches measured preferences, or reframe the contribution as a tuned curriculum schedule and discuss the risk of benchmark overfitting.","section":"§2.3, Eq. (8); Table 3; §6; Appendix B"},{"comment":"All reported results are single runs with no variance, confidence intervals, or significance tests. For a claim as large as +8.1% on MMLU/CMMLU, a single run is insufficient evidence of a real effect. Furthermore, the 3B comparison (Table 2) includes only Random as a baseline; the 1.3B table shows that Sequential-PD-Low2High and Preference CL-PPL are markedly worse than PDPC, but at 3B we do not know whether PDPC beats these alternatives or merely beats random sampling. The abstract's claim that PDPC 'significantly surpasses baselines' is not substantiated for the 3B scale without at least one strong baseline (e.g., Sequential-PD-Low2High or Preference CL-PPL) and preferably a small number of seeds to gauge run-to-run variation.","section":"§3.1-3.2, Tables 1-2"}],"minor_comments":[{"comment":"The benchmark name 'CEV AL' in the text and Table 2 header contains a spurious space; it should be 'CEVAL' for consistency with the Appendix figures.","section":"§3.1, Table 2"},{"comment":"The legend notation '(PDPC, PD, Sig.)' is ambiguous; the text elsewhere uses 'S.' for S-shape. Please use a consistent abbreviation such as '(PDPC, PD, S.)'.","section":"Figure 3 caption"},{"comment":"The semantic-properties analysis uses GPT-4o annotations but reports no sample size, no inter-rater agreement, and no statistical test. The absolute differences between high-PD and low-PD proportions are often small (e.g., 0.83 vs 0.82 for Rule 1), so the claim that 'PD is independent of other linguistic features' is not supported without additional quantification.","section":"§3.5, Figure 9"},{"comment":"The text of Appendix B is written in the present tense as if the annealing experiments were carried out ('We conduct experiments on checkpoints...'), but no results are reported. Since Section 6 lists this as a future direction, Appendix B should be clearly labeled as a proposed method or preliminary sketch, not as an executed experiment.","section":"Appendix B"},{"comment":"The caption says 'during model training of 1B model,' but the experiment described in the text is the 1.3B model; please correct the label.","section":"Figure 5 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper is borderline between major revision and reject. I lean toward major revision because the 1.3B-controlled experiments are encouraging and the cost-reduction idea (using small reference models for PD) is practically valuable. The fixes are within the manuscript's scope: validate the RM proxy at the scale and benchmarks of the headline claim, add a second 3B baseline, and report variance or acknowledge single-run results. The authors should also recalibrate the causal language in the title and abstract, since the preference function is fitted rather than measured. No concerns about citation patterns or novelty disclosure; the related work coverage appears adequate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Main thing to know: this paper proposes an offline data-curriculum method that reorders pretraining data by a perplexity-difference (PD) metric between a weak and a strong reference model, then feeds low-PD data first and high-PD later following an S-shaped schedule. Reported gains are real in shape — consistent across 1.3B/100B and 3B/1T runs, several ablations, and against Random and curriculum baselines. The 3B result claims +8.1 average on MMLU/CMMLU, and if that held at that precision it would be a strong no-extra-data, no-extra-compute improvement. What's actually new: PD as a difficulty signal and the offline preference-function scheduler; prior work either interrupts training (MATES) or scores quality rather than difficulty (QuRating). That is a fair contribution.\n\nWhat the paper does well: the design is sensible, the ablations on RM sizes and function shapes support the qualitative claim, and the limitations section openly admits n=2, single runs, and unexplored iteration. The case study on data source and quality distributions is a nice sanity check.\n\nWhere it is soft, in proportion: the headline magnitude is not pinned down. The S-shape steepness a=10 is chosen by searching on the evaluation benchmarks; all runs are single runs with no variance; the 3B comparison has only Random as baseline; and the mechanism story relies on reference-model PD standing in for the 3B model's preferences, which is never checked against the actual 3B model on the benchmarks that carry the gain. The Figure 4 ablation at 1.3B is relevant but does not cover MMLU/CMMLU, and Appendix B's annealing validation is described but not actually run. So the stress-test note is right: the 'preference' is in part a fitted schedule, and the 8.1% number should be read as an existence proof, not a measured property of the model's preference.\n\nThat said, the central claim — low-to-high PD reordering helps pretraining — holds up in shape. The soft spots are addressable, not fatal. A serious referee should see it: ask for multiple seeds, a fixed schedule chosen before evaluation, a direct comparison of RM-based PD against the actual model's early and late checkpoints, and code or data release. I would take it for review and expect heavy revision; I would not desk-reject.","headline":"A promising offline data-ordering method whose qualitative gains look real, but the headline 3B numbers rest on tuned, single-run estimates and a partly assumed preference mechanism.","tokens_in":21253,"tokens_out":2166,"would_cite":true,"duration_ms":22603,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that LLM pretraining should follow the model's evolving data preferences, and proposes a curriculum that reorders an existing corpus offline by a new Perplexity Difference (PD) metric.","keywords":["LLM pretraining","curriculum learning","perplexity difference","data ordering","preference function","data efficiency","MMLU","CMMLU"],"falsifier":"Train a 3B model on 1T tokens with the reverse schedule (high-PD data first and low-PD data last) and check whether it matches or exceeds PDPC; or compute PD from the actual early and late checkpoints of the experimental model and check whether the reference-model PD ordering agrees with it—if the reverse schedule wins or the orderings diverge, the curriculum's mechanism is not what drives the reported gains.","tokens_in":20207,"feed_emoji":"🧠","tokens_out":4150,"duration_ms":44123,"temperature":0.7,"pith_summary":"The paper claims that during pretraining, a language model's preferred data changes as the model grows stronger, so training should gradually shift from easy, low-PD samples to hard, high-PD samples. It introduces Perplexity Difference (PD), the normalized gap between a weak model's perplexity and a strong model's perplexity on a sample, and uses it to partition the corpus. An S-shaped preference function then decides, at each training step, what fraction of the batch should come from the low-PD versus high-PD partition, all arranged offline so pretraining never pauses. In experiments with 1.3B and 3B models, this reordering alone—with no extra data or compute—beats uniform sampling and several curriculum baselines; the 3B model trained on 1T tokens gains 8.1% average accuracy on MMLU and CMMLU. If correct, the result identifies a cheap, orthogonal axis for improving LLM pretraining: the order in which existing data is consumed.","feed_headline":"Reordering pretraining data lifts LLM accuracy 8.1%","feed_subtitle":"A perplexity-gap curriculum schedules easy data first, hard data later, with no extra tokens or compute.","key_machinery":"The load-bearing mechanism is the Perplexity Difference (PD) metric combined with an S-shaped preference function. PD is computed from two reference models of different sizes trained on a subset of the corpus, approximating an early and a late checkpoint of the experimental model. The S-shaped function $f(p) = 1/(1+\\exp(a(p-0.5)))$ prescribes the concentration of low-PD data at each training step, ensuring low-PD data dominates early and high-PD data dominates late, while the integral constraint keeps the total volumes of the two partitions equal.","core_discovery":"The central claim is that high-PD samples—texts a strong model fits far better than a weak model—are exactly the samples a training model should encounter late, while low-PD samples should appear early. The paper defines $\\mathrm{PD}(x) = (\\mathrm{PPL}_{M_w}(x) - \\mathrm{PPL}_{M_s}(x))/\\mathrm{PPL}_{M_w}(x)$ and partitions the corpus into two equal halves, low-PD and high-PD. A preference function $f(p)$ maps training progress $p = k/K$ to the proportion of low-PD data in the current batch, and the paper shows empirically that an S-shaped $f(p) = 1/(1+\\exp(a(p-0.5)))$ outperforms linear, Z-shaped, and uniform schedules. With this schedule, test loss drops faster in the later half of training, gradient norms stabilize, and downstream accuracy improves most on benchmarks like MMLU, CMMLU, and CEVAL; the reported 3B result is 4.1% average gain across all benchmarks and 8.1% across MMLU and CMMLU.","pith_inferences":["Editorial inference: The same 'learnability gap' logic could be applied to other domains—vision, speech, multimodal data—wherever a weak and strong reference model can be trained on a sample of the corpus.","Editorial inference: The 3B/1T result may not transfer to frontier-scale training, and the paper's own limitation section notes that only $n=2$ partitions were tested; larger $n$ might improve or change the picture.","Editorial inference: Iteratively refitting the preference function from new checkpoints, as sketched in the appendix, could compound gains but adds compute; the paper does not claim this is already validated.","Editorial inference: PD is a difficulty signal that appears orthogonal to quality ratings, so combining PDPC with data selection or quality filtering is a natural next test."],"forward_implications":["Data ordering alone can yield substantial downstream gains at a fixed token budget and fixed model size.","Because the entire arrangement is performed offline, PDPC can be added as a final data preprocessing step without interrupting pretraining.","PD is stable across reference-model sizes, so a cheap 100M/700M model pair suffices to compute the ordering.","The observed gains concentrate in the later half of training, suggesting that the high-PD portion of the corpus is what unlocks the largest accuracy increases.","The S-shape preference function, rather than full sample-level sorting, preserves batch diversity and avoids the homogeneity that hurts sequential curricula."],"supporting_citations":[{"why":"Supplies SlimPajama, the dataset used for the 1.3B runs and for the case-study analysis of PD distributions.","marker":"(Soboleva et al., 2023)"},{"why":"Defines the bilingual Matrix-style corpus composition used for the 3B, 1T-token run.","marker":"(Zhang et al., 2024a)"},{"why":"Provides the QuRating data-quality baseline and the in-context evaluation protocol the paper adopts.","marker":"(Wettig et al., 2024)"},{"why":"Introduces MATES, the dynamic data-influence method that PDPC contrasts with because MATES interrupts training to reselect data.","marker":"(Yu et al., 2024)"},{"why":"Supplies the Llama architecture used for all experimental and reference models.","marker":"(Touvron et al., 2023)"},{"why":"Provides the Megatron pretraining framework in which all models are trained.","marker":"(Shoeybi et al., 2019)"}],"fun_headline_variants":["Perplexity-gap curriculum reorders data, lifts MMLU/CMMLU by 8.1%","Easy data early, hard data late: perplexity-gap curriculum pays 8.1%","Preference-aware pretraining: use data the model likes, gain 8.1%","Schedule pretraining data by model preference for 8.1% accuracy boost"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the data preference of the actual 1.3B or 3B model during training is well approximated by perplexity differences between two smaller reference models trained on a subset of the same corpus.","fun_headline_variants_meta":{"raw":{"variants":["Perplexity-gap curriculum reorders data, lifts MMLU/CMMLU by 8.1%","Easy data early, hard data late: perplexity-gap curriculum pays 8.1%","Preference-aware pretraining: use data the model likes, gain 8.1%","Schedule pretraining data by model preference for 8.1% accuracy boost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000989,"raw_usage":{"total_tokens":4220,"prompt_tokens":996,"completion_tokens":3224,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":3126}},"tokens_in":612,"tokens_out":3224,"duration_ms":21862,"temperature":1.0,"reasoning_tokens":3126,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:29:37.881643+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a 3B model on 1T tokens with the reverse schedule (high-PD data first and low-PD data last) and check whether it matches or exceeds PDPC; or compute PD from the actual early and late checkpoints of the experimental model and check whether the reference-model PD ordering agrees with it—if the reverse schedule wins or the orderings diverge, the curriculum's mechanism is not what drives the reported gains.","supporting_citations":[],"review_version":1}