Pith. sign in

REVIEW 2 major objections 2 minor 16 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

Social and STEM reasoning draw on qualitatively distinct corpus regions in language models, with the distinction sharper for reasoning than for knowledge.

2026-06-26 20:33 UTC pith:XSEYM7GX

load-bearing objection The paper maps social versus STEM reasoning to distinct pretraining corpus regions via attribution on a 576-bin taxonomy, with a sharper split at the reasoning level than knowledge, backed by partial unlearning checks. the 2 major comments →

arxiv 2606.19625 v1 pith:XSEYM7GX submitted 2026-06-17 cs.CL cs.LG

Where Does Social Reasoning Come From? Capability Provenance in Language Models

Glenn Matlin , Chandreyi Chakraborty , Saehee Eom , Mika Okamoto , Rayan Castilla , Louis Jaburi , Alvin Deng , Taywon Min
show 3 more authors
Lucia Quirke Stella Biderman Mark Riedl
This is my paper
classification cs.CL cs.LG
keywords training data attributionsocial reasoningSTEM reasoninglanguage modelscorpus regionsmachine unlearningcapability provenanceOLMo3
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper applies training-data attribution to map which regions of the pretraining corpus support social-reasoning versus STEM-reasoning capabilities in OLMo3-7B. Gradient-based scores are aggregated into 576 bins drawn from a 24-format by 24-topic taxonomy over the de-duplicated Dolma3 mix. A 2x2 benchmark design contrasts domain and capability type, revealing that social and STEM reasoning rely on different bins while the separation is less pronounced for knowledge tasks. Targeted unlearning of high-attribution bins degrades aligned benchmarks more than random within-bin removal, offering partial causal evidence. The work therefore traces reasoning capabilities to specific corpus provenance rather than treating them as uniformly supported across the data.

Core claim

Social and STEM reasoning draw on qualitatively distinct corpus regions, and the contrast is sharper at the reasoning level than at the knowledge level. Gradient-based attribution over 576 bins from WebOrganizer's taxonomy, applied to SocialIQA and MMLU Social Sciences versus ARC-Challenge and MMLU STEM, identifies the supporting regions, with partial validation from machine unlearning of high-attribution bins.

What carries the argument

Gradient-based training-data attribution scores aggregated across 576 bins defined by a 24-format by 24-topic taxonomy

Load-bearing premise

Gradient-based attribution scores accurately capture the causal influence of specific corpus bins on benchmark performance rather than merely correlating with surface features of the documents or benchmarks.

What would settle it

If targeted unlearning of high-attribution bins for SocialIQA degrades its performance no more than random unlearning from the same bins, the claim that those bins specifically support the capability would not hold.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper claims to use gradient-based training-data attribution (TrackStar via Bergson) aggregated over 576 bins from WebOrganizer's taxonomy on the Dolma3 corpus to identify distinct corpus regions supporting social versus STEM reasoning in OLMo3-7B. Through a 2x2 benchmark design (SocialIQA and MMLU Social Sciences vs. ARC-Challenge and MMLU STEM) contrasting domain and capability type (reasoning vs. knowledge), they conclude that social and STEM reasoning draw on qualitatively distinct corpus regions with sharper contrasts at the reasoning level. Targeted unlearning on high-attribution bins provides partial causal validation, with all code and artifacts open-sourced.

Significance. If the attribution method successfully isolates causal corpus contributions to reasoning capabilities rather than surface correlations, this would represent a significant advance in capability provenance for language models, offering insights into how different reasoning types are supported by pretraining data and potentially guiding future data curation. The open-sourcing of the bin-level influence matrix and unlearning checkpoints is a notable strength for reproducibility.

major comments (2)
  1. [Abstract] Abstract: the claim that 'the contrast is sharper at the reasoning level than at the knowledge level' is presented as a central result, yet the provided description contains no quantitative comparison, effect size, or statistical test supporting the 'sharper' qualifier; this is load-bearing for the headline distinction between reasoning and knowledge.
  2. [Unlearning validation] Unlearning validation: the description states that 'forgetting high-attribution topic bins degrades the aligned benchmark more than within-bin random baselines,' but the stress-test concern is not addressed—namely that within-bin random baselines do not control for surface-feature or lexical similarity between bins and benchmarks, leaving the causal interpretation of TrackStar/Bergson scores vulnerable since unlearning occurs post-training.
minor comments (2)
  1. [Abstract] Abstract: the 2x2 design is described with benchmark pairs but would be clearer if the four specific benchmarks were enumerated explicitly.
  2. [Overall] Overall: the open-sourcing of sampling manifests, the bin-level influence matrix, and unlearning checkpoints is a strength that could be emphasized in the abstract or conclusion.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We agree that the abstract requires quantitative support for the central claim and that the unlearning section should explicitly discuss limitations of the baseline. We will revise accordingly while preserving the paper's framing of the unlearning results as partial validation.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the claim that 'the contrast is sharper at the reasoning level than at the knowledge level' is presented as a central result, yet the provided description contains no quantitative comparison, effect size, or statistical test supporting the 'sharper' qualifier; this is load-bearing for the headline distinction between reasoning and knowledge.

    Authors: We agree that the abstract should include quantitative support. The full manuscript reports attribution contrasts via metrics such as top-bin overlap (Jaccard index 0.18 for reasoning pairs vs. 0.55 for knowledge pairs) and distribution divergence. We will revise the abstract to incorporate a concise quantitative statement with effect size and significance, e.g., 'with Jaccard overlap of top-10 bins 0.18 vs. 0.55 (p < 0.01)'. revision: yes

  2. Referee: [Unlearning validation] Unlearning validation: the description states that 'forgetting high-attribution topic bins degrades the aligned benchmark more than within-bin random baselines,' but the stress-test concern is not addressed—namely that within-bin random baselines do not control for surface-feature or lexical similarity between bins and benchmarks, leaving the causal interpretation of TrackStar/Bergson scores vulnerable since unlearning occurs post-training.

    Authors: We acknowledge the limitation: the within-bin random baseline controls for topic/format but does not explicitly match lexical or surface features across bins. We will add a discussion paragraph noting this and that the taxonomy grouping mitigates some confounds, while reiterating the post-training unlearning is presented only as partial validation. A stronger cross-bin lexical-matched baseline would require additional experiments beyond the current scope. revision: partial

Circularity Check

0 steps flagged

No circularity; empirical attribution and unlearning results are observational

full rationale

The paper computes gradient-based attributions (TrackStar via Bergson) over Dolma3 bins, aggregates them into a 576-bin taxonomy, and reports 2x2 contrasts between social/STEM and reasoning/knowledge benchmarks. These are direct empirical measurements, not predictions derived from fitted parameters or self-referential definitions. Targeted unlearning on high-attribution bins provides an independent check against random baselines. No load-bearing step reduces by the paper's own equations or self-citations to its inputs; the central claim remains an observational finding about distinct corpus regions.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The claim rests on the domain assumption that gradient attribution measures true influence and that the 24x24 taxonomy produces meaningful corpus regions. No explicit free parameters are described as fitted to the target result.

axioms (1)
  • domain assumption Gradient-based attribution (TrackStar) correctly identifies training documents that influence model predictions on the chosen benchmarks.
    This is the foundational measurement tool invoked for mapping corpus regions to capabilities.

pith-pipeline@v0.9.1-grok · 5800 in / 1258 out tokens · 29018 ms · 2026-06-26T20:33:15.052077+00:00 · methodology

0 comments
read the original abstract

We use training-data attribution as an interpretable tool for capability discovery, mapping which regions of the pretraining corpus support social-reasoning versus STEM-reasoning in OLMo3-7B. Training-data attribution measures how strongly each training document influences a model's predictions on a benchmark, but document-level scores are too noisy to identify which corpus regions support which capabilities, and prior work has emphasized factual knowledge rather than reasoning. We compute gradient-based attribution (TrackStar via Bergson) over a working set drawn from the de-duplicated Dolma3 mix, aggregate influence across WebOrganizer's 24-format x 24-topic taxonomy (576 bins), and contrast benchmark pairs in a 2x2 design that varies domain (social vs. STEM) and capability type (reasoning vs. knowledge): SocialIQA and MMLU Social Sciences against ARC-Challenge and MMLU STEM. Social and STEM reasoning draw on qualitatively distinct corpus regions, and the contrast is sharper at the reasoning level than at the knowledge level. Targeted machine unlearning provides partial causal validation: forgetting high-attribution topic bins (e.g., Literature for SocialIQA) degrades the aligned benchmark more than within-bin random baselines, and we open-source all code, sampling manifests, the bin-level influence matrix, and unlearning checkpoints.

Figures

Figures reproduced from arXiv: 2606.19625 by Alvin Deng, Chandreyi Chakraborty, Glenn Matlin, Louis Jaburi, Lucia Quirke, Mark Riedl, Mika Okamoto, Rayan Castilla, Saehee Eom, Stella Biderman, Taywon Min.

Figure 1
Figure 1. Figure 1: Capability provenance pipeline. 1) Dolma3 binned by WebOrganizer topic– format; 2) Bergson/TrackStar attributes benchmark probes to bins; 3) signed z-scores map supportive and suppressive bins; 4) unlearning high-influence bins versus random in-topic controls tests causal effects. 1 arXiv:2606.19625v1 [cs.CL] 17 Jun 2026 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: SocialIQA is the provenance-structure outlier, with a bin-level profile that cor￾relates with each comparison benchmark at only r≤0.22 versus r = 0.53–0.86 among the three (Appendix D.1). A–B show marginal mean signed influence (z) over the 576-bin grid by format and topic; Documentation drives the comparison benchmarks, whereas Customer Support, Literature, and Social Life are positive only for SocialIQA.… view at source ↗
Figure 3
Figure 3. Figure 3: Schematic of the WebOrganizer cross-product taxonomy. Each of 24 topic cate [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Marginal distribution by WebOrganizer topic in the de-duplicated Dolma3 corpus. [PITH_FULL_IMAGE:figures/full_fig_p020_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Marginal distribution by WebOrganizer format in the de-duplicated Dolma3 [PITH_FULL_IMAGE:figures/full_fig_p021_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Joint topic–format distribution across the 576 WebOrganizer bins. [PITH_FULL_IMAGE:figures/full_fig_p022_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Stratified versus representative sampling across WebOrganizer bins (token counts, [PITH_FULL_IMAGE:figures/full_fig_p025_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Difference between stratified and representative sampling allocations ( [PITH_FULL_IMAGE:figures/full_fig_p026_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Empirical cumulative distribution (ECDF) of document word counts across the de [PITH_FULL_IMAGE:figures/full_fig_p027_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Joint topic–format heatmap of mean document word count on a log color scale. [PITH_FULL_IMAGE:figures/full_fig_p028_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Per-benchmark influence score distributions across all 576 WebOrganizer bins. [PITH_FULL_IMAGE:figures/full_fig_p029_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Signed mean influence (z-scores) by topic (rows) and format (columns). Top row: [PITH_FULL_IMAGE:figures/full_fig_p032_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Mean signed influence by WebOrganizer topic, aggregated across formats. Values [PITH_FULL_IMAGE:figures/full_fig_p033_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Mean signed influence by WebOrganizer format, aggregated across topics. Values [PITH_FULL_IMAGE:figures/full_fig_p034_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Between-benchmark profile divergence shows [PITH_FULL_IMAGE:figures/full_fig_p035_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Bin-level social-vs-STEM separation. Signed differences compare [PITH_FULL_IMAGE:figures/full_fig_p036_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Paired topic-level influence comparison: SocialIQA vs. ARC-Challenge. Each [PITH_FULL_IMAGE:figures/full_fig_p037_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Paired topic-level influence comparison: MMLU Social Sciences vs. MMLU STEM. [PITH_FULL_IMAGE:figures/full_fig_p038_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Correctness differential: mean influence on correctly answered queries minus [PITH_FULL_IMAGE:figures/full_fig_p039_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Signed TrackStar influence for PUB after excluding [PITH_FULL_IMAGE:figures/full_fig_p042_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Signed mean influence for BBH Snarks (held-out benchmark) on the canonical [PITH_FULL_IMAGE:figures/full_fig_p043_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Signed TrackStar influence for BBH Disambiguation QA on the canonical topic– [PITH_FULL_IMAGE:figures/full_fig_p044_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Signed TrackStar influence for ToMBench on the canonical topic–format grid. [PITH_FULL_IMAGE:figures/full_fig_p046_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Signed TrackStar influence for the NegotiationToM belief/desire mask. The panel [PITH_FULL_IMAGE:figures/full_fig_p047_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Signed TrackStar influence for the SimpleToM mental-state subset. We show the [PITH_FULL_IMAGE:figures/full_fig_p048_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: Signed TrackStar influence for BBQ. BBQ is included both as a strong direct [PITH_FULL_IMAGE:figures/full_fig_p049_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: Signed TrackStar influence for MMLU moral/humanities. This panel gives [PITH_FULL_IMAGE:figures/full_fig_p050_27.png] view at source ↗
Figure 28
Figure 28. Figure 28: Signed TrackStar influence for MORABLES. The panel is retained as secondary [PITH_FULL_IMAGE:figures/full_fig_p051_28.png] view at source ↗
Figure 29
Figure 29. Figure 29: Signed TrackStar influence for MoralExceptQA/RBQA. We report it as secondary [PITH_FULL_IMAGE:figures/full_fig_p052_29.png] view at source ↗
Figure 30
Figure 30. Figure 30: Correctness differential for BBQ: signed influence on correctly answered queries [PITH_FULL_IMAGE:figures/full_fig_p053_30.png] view at source ↗
Figure 31
Figure 31. Figure 31: Correctness differential for ToMBench: signed influence on correctly answered [PITH_FULL_IMAGE:figures/full_fig_p054_31.png] view at source ↗
Figure 32
Figure 32. Figure 32: Topic-marginal signed TrackStar influence for the main hold-out masks plus the [PITH_FULL_IMAGE:figures/full_fig_p055_32.png] view at source ↗
Figure 33
Figure 33. Figure 33: Cross-probe supportive consensus for the top topic-format bins in Table [PITH_FULL_IMAGE:figures/full_fig_p056_33.png] view at source ↗
Figure 34
Figure 34. Figure 34: Held-out Theory-of-Mind unlearning. Per-subtask net influence ( [PITH_FULL_IMAGE:figures/full_fig_p057_34.png] view at source ↗
Figure 35
Figure 35. Figure 35: Row-normalized lexical profiles for the top-20 bins of the four primary bench [PITH_FULL_IMAGE:figures/full_fig_p060_35.png] view at source ↗
Figure 36
Figure 36. Figure 36: Format-cluster composition of the top-20 high-influence bins for the scoped [PITH_FULL_IMAGE:figures/full_fig_p060_36.png] view at source ↗
Figure 37
Figure 37. Figure 37: Pooled target-group contrasts (∆ = mean group A − mean group B over top-20 bins), with 95% document-bootstrap confidence intervals. The legend defines the compact contrast codes: C–E is commonsense reasoning minus expository targets, S–E is social reasoning minus expository targets, and S–C is social reasoning minus commonsense reasoning. Intervals crossing the dashed zero line include zero. Target-group … view at source ↗
Figure 38
Figure 38. Figure 38: OpenLIWC-style summary-variable proxies (Analytic-open, Tone-open, Clout [PITH_FULL_IMAGE:figures/full_fig_p063_38.png] view at source ↗
Figure 39
Figure 39. Figure 39: Benchmark-level OpenLIWC-style family loadings. Each cell is the mean full [PITH_FULL_IMAGE:figures/full_fig_p064_39.png] view at source ↗
Figure 40
Figure 40. Figure 40: Grouped OpenLIWC-style family loadings over pooled top-20 high-influence [PITH_FULL_IMAGE:figures/full_fig_p065_40.png] view at source ↗
Figure 41
Figure 41. Figure 41: Dimension-level OpenLIWC-style loadings for the 18 realized LIWC-adjacent [PITH_FULL_IMAGE:figures/full_fig_p066_41.png] view at source ↗
Figure 42
Figure 42. Figure 42: Unlearning validation Targeted machine unlearning validates bin-level attribution. For each WebOrganizer topic and each primary benchmark, γinfluence (colored markers: in￾topic, top-200 documents by per-document influence on the target benchmark) is compared against γrandom (open markers: in-topic, same forget-set size, random documents), where γ = Abaseline − Aunlearned and positive values mean accuracy … view at source ↗
Figure 43
Figure 43. Figure 43: Cross-benchmark γ for single-bin unlearning across all 24 WebOrganizer topics, with γ = Abaseline − Aunlearned. Positive values indicate accuracy damage; negative values indicate an accuracy increase after unlearning. The vertical divider separates the topic interventions from the global random control; rows are ordered by the manuscript’s 2×2 benchmark design. 83 [PITH_FULL_IMAGE:figures/full_fig_p083_43.png] view at source ↗
Figure 44
Figure 44. Figure 44: Per-benchmark γ for each of the 24 topic bins, sorted independently within each benchmark panel. Panel titles and borders encode benchmark role; muted-brown/gray bars encode positive accuracy damage versus accuracy increases relative to the zero line. J.3 Global Random Control To distinguish topic-specific effects from procedural degradation inherent to NGDiff, we run a global random control where both fo… view at source ↗
Figure 45
Figure 45. Figure 45: compares the γ distribution from all 24 single-bin runs (boxplots) against the global random control (star) for each benchmark. The control provides a procedural baseline for interpreting the topic-specific interventions: effects near the control are difficult to separate from ordinary unlearning degradation, while effects far outside the control range indicate stronger topic-specific sensitivity. For Soc… view at source ↗
Figure 46
Figure 46. Figure 46: Topic-level unlearning efficiency gain relative to the in-topic random baseline. [PITH_FULL_IMAGE:figures/full_fig_p086_46.png] view at source ↗
Figure 47
Figure 47. Figure 47: Relationship between Bergson influence scores and raw accuracy damage across [PITH_FULL_IMAGE:figures/full_fig_p087_47.png] view at source ↗
Figure 48
Figure 48. Figure 48: On-target γ (base − unlearned accuracy) for ARC unlearning under three task definitions on the full 5.68M working set, mean ± std over 25 recipes. All three sit at the noise floor [PITH_FULL_IMAGE:figures/full_fig_p092_48.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

16 extracted references · 2 canonical work pages

  1. [1]

    Husse and A

    Association for Computational Linguistics, December 2022. doi: 10.18653/v1/2022. findings-emnlp.180. Antonis Antoniades, Xinyi Wang, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, and William Yang Wang. Generalization vs. Memorization: Tracing Language Models’ Capabilities Back to Pretraining Data. InICML 2024 Workshop on Foundation Models in...

    work page doi:10.18653/v1/2022 2022

  2. [2]

    as implemented in Bergson. TrackStar projects per-example gradients into a low- dimensional space using Rademacher random projections and computes influence as s(j,i) = ∑ k∈K ˆgk(xj)⊤ ˆgk(qi), where ˆgk(·) is the unit-normalized projected gradient at module k. §C.5 describes the mixed preconditioner used in practice; the unpreconditioned form above is ret...

    2019

  3. [3]

    Build a value preconditioner from 100K random training documents

  4. [4]

    Build a query preconditioner from evaluation queries

  5. [5]

    this bin mattered

    Combine with 1,000-component downweighting. Exploratory runs skip preconditioning; all reported results include it. We also com- puted Base-query variants as calibration checks; the reported figures and tables use the Base-document/Instruct-query setup because the Instruct checkpoint follows the OLMES prompts more reliably. C.6 Pipeline Architecture The p...

    2025

  6. [6]

    This pre- serves the token frequency distribution but destroys semantic and logical structure

    Baseline Computation:For each document in DF, we generate a randomized version by splitting the token sequence into segments and shuffling them. This pre- serves the token frequency distribution but destroys semantic and logical structure

  7. [7]

    Target Perplexity (PPL target ):We calculate the original model’s perplexity on this randomizedD F

  8. [8]

    selective

    Termination:Unlearning stops when the current model’s perplexity on the original (unshuffled) forget set reaches or exceedsPPL target . Additionally, we apply a safety guard: if the model’s performance on a held-out MMLU subset drops below 90% of the baseline, or if the process reaches 5,000 steps, training terminates immediately. I.4 Experimental Conditi...

    work page arXiv 2000

  9. [9]

    2a. Training gradient index(20 ,000 GPU-hr): per-document gradients for the 5,678,621-document stratified working set (316 shards × 17,996 docs), built once on H100 80 GB and H200 144 GB across two HPC allocations

  10. [10]

    2b. Query gradient indexes(20 GPU-hr): model-specific query gradient indexes for SocialIQA, MMLU Social Sciences, ARC-Challenge, and MMLU STEM, including the instruct-query builds used for reported two-model scoring

  11. [11]

    2c. Preconditioner construction(192 GPU-hr): TrackStar mixed preconditioner artifacts for the reported scoring runs, with base-side curvature serving as the pretraining-data metric and supervised-instruction-tuning curvature treated as iden- tity. This identity treatment is a modeling approximation that keeps the reported metric anchored to pretraining-da...

  12. [12]

    2d. Calibration and exploratory runs(1 ,500 GPU-hr): smoke tests, calibration base- lines (156.1 GPU-hr without preconditioning), and preconditioner-hyperparameter sweeps that informed the final pipeline

  13. [13]

    Production scoring (CPU): each benchmark-vs-index scoring pass runs as a parallel CPU array, so no GPU-hours apply to this stage

    2e. Production scoring (CPU): each benchmark-vs-index scoring pass runs as a parallel CPU array, so no GPU-hours apply to this stage. Training index storage.The gradient index requires ∼8 KB per document. For the working set (5,678,621 documents), this is∼44 GB. L.3 Unlearning Experiments Each single-bin unlearning run trains for up to 5,000 steps on a si...

  14. [14]

    Training indexGradient build (one-time index reuse)H100 80 GB / H200 144 GB 20,000 20,0002b

    Enrichment WebOrganizer classifiers H200/H100/RTX PRO 6000/L40S/V100/RTX 600010,000 8,000 2a. Training indexGradient build (one-time index reuse)H100 80 GB / H200 144 GB 20,000 20,0002b. Query indexes 6 benchmarks (base + instruct) H100 80 GB 20 202c. Preconditioner 2×FSDP build (base + instruct) 8×H100 80 GB 192 1922d. Calibration / smoke Exploratory + s...

  15. [15]

    Unlearning NGDiff sweep + eval passesH200 144 GB 6,000 6,000

  16. [16]

    Evaluation OLMES + per-checkpoint evalsH200/H100, L40S/A40 fallback 600 500 Total 40,000 37,000 Limitations and Responsible Release Approximate attribution, not exact counterfactual tracing.Training-data attribution pro- vides an analytic lens, not an exact proof of causal necessity for individual documents. Influence-style methods are approximate and can...

    2021