Pith. sign in

REVIEW 4 major objections 4 minor 44 references

Quality over Quantity: An Effective Large-Scale Data Reduction Strategy Based on Pointwise V-Information

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that ranking training examples by Pointwise V-Information (PVI) reveals which easy instances are redundant, so removing the top 10–30% by PVI preserves accuracy while easy-to-hard training adds about 0.8%.

desk verdict Abstract overclaims by an order of magnitude; the real results show much larger accuracy loss on OCNLI and the progressive-learning algorithm is mis-specified. read the letter →

arxiv 2507.00038 v3 pith:U5BTJUIO submitted 2025-06-19 cs.LG cs.AI

Fei Chen , Wenchi Zhou This is my paper
classification cs.LGcs.AI
keywords datareductionpointwiseV-informationdatasetdifficultydata-centricAInaturallanguageinferencecurriculumlearningChineseNLPprogressive
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a data-reduction strategy built on Pointwise V-Information (PVI), a model-based score that measures how much each input helps a classifier predict its label: high PVI means the instance is easy, low PVI means it is hard. The central claim is that the easiest instances are largely redundant, so deleting the 10–30% of training data with the highest PVI costs almost nothing — reported accuracy drops range from 0.0001% to 0.76% across three Chinese natural-language-inference datasets. A second claim is that training on the remaining examples in order of increasing difficulty, a progressive or curriculum-style schedule, converges faster and can outperform conventional training, with a 0.8% accuracy gain on one dataset. The paper also extends PVI, previously used on English data, to Chinese NLI tasks and multiple base models, arguing that difficulty-based reduction transfers across languages. The authors note that PVI requires the model to output a full probability distribution, so the approach is most natural for classification-style tasks.

What carries the argument

The load-bearing object is Pointwise V-Information (PVI), a per-example version of V-information: $PVI(x\to y) = -\log_2 g[\varnothing](y) + \log_2 g'[x](y)$, where $g'$ and $g$ are two models from the same predictive family, finetuned with and without the input $x$ (the 'null input' is the empty string). PVI converts 'how much usable information does this example carry' into a scalar difficulty score that is model-relative and label-space-independent. The two algorithms built on it are a static reducer (sort by PVI descending, keep the hardest $1-r$ fraction, retrain from scratch) and a progressive learner (feed the sorted list from easy to hard, evaluating after each stage). The same score also powers the dataset-level V-information estimates used to compare the three Chinese NLI datasets.

What would settle it

Recompute the PVI ranking using a model finetuned only on a reduced subset (with, say, 30% of the highest-PVI instances removed) and compare it with the ranking from the full-data model; if the sets of easiest instances differ substantially between the two rankings, the static reduction procedure is chasing a moving target and the reported accuracy preservation would not generalize to new subsets.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that PVI, defined as $PVI(x\to y) = -\log_2 g[\varnothing](y) + \log_2 g'[x](y)$, is a usable difficulty label for large-scale data reduction. Here $g'$ is a model finetuned on standard input $x$, and $g$ is the same architecture finetuned on the empty string $\varnothing$; the score is the gain in log-probability assigned to the correct label $y$ when real input replaces null input. Instances with high PVI are easy for the model and can be removed in bulk; instances with low PVI are hard and carry the remaining training signal. Static removal of the top 10–30% high-PVI instances leaves accuracy essentially intact, while progressive learning over the PVI-sorted data yields a small but consistent performance gain. The authors interpret this as evidence that dataset difficulty is not a fixed property of examples but is relative to the model and task, and that PVI captures this relativity well enough to guide subset selection.

Load-bearing premise

The method assumes that PVI scores computed by a model finetuned on the full training set remain valid as difficulty labels for models trained on reduced subsets; if the rankings shift under the distribution change the reduction itself causes, the selected subset is built on a moving target.

Editorial extensions

If this is right

  • Removing 10–30% of the easiest (highest-PVI) training instances preserves classifier accuracy, with reported drops of 0.0001% to 0.76%, so a meaningful fraction of redundant data can be pruned before training starts.
  • Progressive learning from easy to hard on PVI-sorted data improves OCNLI accuracy by about 0.81% and F1 by about 0.83% over conventional training, with a smaller gain on CINLI.
  • The PVI framework transfers from English to Chinese NLI tasks and multiple base models, indicating that difficulty-based data reduction is not language-specific.
  • The safe reduction ratio is bounded: removing more than roughly 30–50% of high-PVI instances collapses accuracy (for example, CMNLI drops to 17.27% at 90% removal), because easy instances also carry basic inference patterns.
  • Balanced reduction keeps the null-input model near the 33% chance level, showing that label-distribution bias, not only information content, drives part of the accuracy decline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not run: reuse the same PVI-scored subset across many models and hyperparameter settings, which would amortize the one-time scoring cost whenever training is repeated.
  • Because the random-baseline comparison already shows that difficulty alone does not explain all gains, combining PVI with diversity or influence measures could push the safe reduction ratio above 30% while protecting rare hard examples.
  • The same easy-to-hard curriculum is likely portable to other languages and modalities, but only where a meaningful null input exists; for images or audio, defining that baseline is the main transfer barrier.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a data reduction strategy for Chinese natural language inference based on Pointwise V-Information (PVI). PVI scores are computed with a model finetuned on the full training set (standard input) and a null-input model, and instances are ranked by PVI. The static-reduction method removes high-PVI (easy) instances at ratios r=0.1–0.9 and retrains from scratch; the progressive-learning method trains Qwen3-0.6B on data sorted by PVI. Experiments on OCNLI, CMNLI, and CINLI with Chinese-BERT-wwm report accuracy as a function of reduction ratio plus accuracy/F1 for progressive learning. The paper claims that removing 10–30% of data causes only a 0.0001%–0.76% accuracy decline and that PVI-sorted training yields a 0.8% accuracy gain.

Significance. If the headline claims were true, PVI would be a cheap, model-aware data reduction signal for Chinese NLI, with practical value for training efficiency. The paper's strengths are that it operationalizes PVI on three Chinese datasets, reports hyperparameters and a fixed seed, and includes negative results: Table 2 shows that aggressive high-PVI removal can degrade accuracy substantially, especially on OCNLI, and Section 5 candidly lists limitations of the PVI framework. However, the main quantitative claims are contradicted by the paper's own tables, the progressive-learning pseudo-code does not match the described curriculum, and the 'optimal reduction ratios' are chosen post hoc. The contribution as stated is therefore not established.

major comments (4)
  1. [Abstract; §3.2.1, Table 2] The abstract and the contribution bullet in Section 1 state that removing 10%–30% of data causes only a 0.0001%–0.76% accuracy decline. Table 2 contradicts this: at r=0.1, 0.2, and 0.3, OCNLI falls from 69.59 to 68.85, 66.20, and 62.60, i.e., losses of 0.74, 3.39, and 6.99 points; CMNLI loses 0.05, 0.76, and 0.96 points, and CINLI loses 0.38 points at r=0.2 and 2.10 points at r=0.3. The maximum observed loss over the claimed range is 6.99 points, not 0.76 points, and the 0.0001% figure appears nowhere in the experimental tables. The balanced-reduction results in Table A1 also exceed the bound (OCNLI r=0.3: 69.59 to 67.73, a 1.86-point loss). This is a load-bearing discrepancy: the paper's stated central contribution is not supported by the reported experiments.
  2. [§2.2, Algorithm 3] Algorithm 3's pseudo-code is inconsistent with the progressive-learning description. Section 2.2 states that the algorithm sorts instances by descending PVI so that simple (high-PVI) instances are at the head, and that training 'starts from the simplest instances and gradually expands to include more difficult instances.' However, the pseudo-code selects 'the last subset_size instances from D_train sorted', which are the low-PVI (hard) instances, not the easy ones. If the implementation used the first subset_size instances, the pseudo-code is wrong; if it used the last, the experiments in Tables 6 and 8 do not implement the easy-to-hard curriculum described in the paper. Either way, the method is not correctly specified.
  3. [§3.2.2, Tables 6 and 8] Tables 6 and 8 report results under the 'optimal reduction ratio' r=0.1 for OCNLI and r=0.3 for CINLI. These ratios are selected after inspecting test-set performance, so the claimed small declines and the 'safe reduction ratio' recommendations in Section 3.2.1 are fitted to the test data rather than fixed a priori. In addition, no stability analysis is provided for the PVI rankings themselves: the method assumes that PVI scores computed from a model finetuned on the full training set remain valid difficulty labels when subsets are removed, but this assumption is neither tested nor discussed.
  4. [Abstract; §3.2.2, Tables 6–9] The '0.8% accuracy gain' for progressive learning is not a general result. In Table 8 (CINLI), Sort improves over Base by only 91.8676−91.7852=0.0824 accuracy points, and in Table 7 (OCNLI) the three-run means are 69.15±0.29 for Base and 70.32±0.48 for Sort, with overlapping error bars and no significance test. The abstract's 'achieving a 0.8% accuracy gain over conventional training' therefore rests on a single OCNLI run (Table 6) and is not supported by the other reported data.
minor comments (4)
  1. [§2.2, Algorithm 2] Algorithm 2 computes subset_size using 'm1' in the assignment; this should be 'm' to match the notation defined in the text.
  2. [§2.1] The prose describing g and g' is reversed: the text says g is the standard-input model and g' is the null-input model, but Formula (6) and Algorithm 1 use g'[x] for standard input and g[∅] for null input. Please harmonize the notation.
  3. [§3.3, Figure 7] The caption mentions yellow and green dots, but the caption does not explain which panel corresponds to which color; consider labeling the panels directly and defining the color convention.
  4. [Abstract; §1] The 0.0001% lower bound for the accuracy decline appears in the abstract and introduction but is not supported by any table or analysis in the paper; please either provide the source or remove it.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline accuracy-maintenance range is a post hoc selection of per-dataset reduction ratios, not a PVI-derived prediction.

  1. fitted input called prediction [Section 2.1 (Equation 5 discussion); Algorithm 2; Tables 2, 6, 8.]
    "The acceptable removal rate for 𝐷𝑒𝑎𝑠𝑦′ adapts to different tasks and datasets, a principle incorporated into the reduction ratios 𝑟 of Algorithms 2 and 3."

    The paper does not fix the reduction amount from PVI theory or a held-out validation set: Algorithm 2 sweeps r=0.1 to 0.9 and records test accuracy, and Section 2.1 states that the acceptable removal rate 'adapts to different tasks and datasets.' Tables 6 and 8 then label r=0.1 (OCNLI) and r=0.3 (CINLI) as the 'optimal reduction ratio.' The abstract's headline range of 0.0001% to 0.76% decline is therefore a report of the most favorable selected cells after seeing the test results, not an out-of-sample prediction. The same test set used to choose r is used to state the finding, so the central accuracy-maintenance claim reduces to the post hoc choice of r; the instance-level threshold tau is defined the same way ('determined based on the specific task and model performance').

full rationale

There is no definitional circularity in the PVI computation itself: PVI is defined externally by Ethayarajh et al. [18] and built on the V-information theory of Xu et al. [25], and the progressive-learning comparison is run on a different base model (Qwen3-0.6B) using PVI rankings from Chinese-BERT-wwm, so that part is an independent experiment. The circularity is confined to the central empirical claim about accuracy preservation. The 'safe' reduction ratio r is selected adaptively per dataset after sweeping all ratios on the test set, and the best ratio is then presented as the 'optimal reduction ratio' in Tables 6 and 8. Consequently, the abstract's 0.0001%-0.76% accuracy-decline range is a selected outcome, not a prediction derived from PVI. This is aggravated by the paper's own Table 2, where OCNLI loses 6.99 points at r=0.3, far exceeding the claimed ceiling, showing that the range in the abstract corresponds to the most favorable dataset/ratio combinations. No load-bearing self-citation is present; references [18] and [25] are external. The score reflects the fitted nature of the headline reduction claim while acknowledging that the PVI framework and the progressive-learning comparison retain independent content.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new theoretical entities are introduced; PVI and the empty input are taken from prior work. The main free parameter is the reduction ratio r, which is fitted per dataset. The axioms are mostly domain assumptions about the validity of PVI and the null-input baseline.

free parameters (4)
  • reduction ratio r = 0.1 (OCNLI), 0.3 (CINLI) in progressive learning; 0.1-0.9 scanned
    The ratio of high-PVI instances removed is a free knob; the 'optimal' r is chosen per dataset based on best test accuracy (Tables 6 and 8), so the reported small declines are conditional on this fit.
  • noise level = 0.1
    For the noisy-data experiment, 10% of training instances were randomly replaced with low-quality text; this level is chosen by hand.
  • class-balanced constraint = proportional reduction per class
    In balanced experiments, the authors force equal reduction across classes, an additional modeling choice not derived from PVI.
  • threshold tau = not used
    The paper defines a threshold tau for 'simple instances' in Equation (5) but never operationalizes it; difficulty decisions are made via r instead.
assumptions (5)
  • standard math V-information and PVI definitions from Xu et al. (2019) and Ethayarajh et al. (2022) are valid for measuring model-usable information.
    The paper builds directly on these definitions (Equations 1-4) without re-deriving them.
  • domain assumption A model finetuned on null inputs (empty string) provides a valid baseline for predicting labels without input.
    The PVI calculation subtracts null-input log-likelihood; if the null model is poorly trained, PVI values are unreliable. No analysis of null-model sensitivity is given.
  • ad hoc to paper High-PVI (easy) instances are largely redundant or contain annotation artifacts, so removing them should not hurt performance.
    This is the paper's central premise (Section 2.1, Eq. 5). The OCNLI results at r>0.1 contradict it, showing large accuracy drops when easy instances are removed.
  • domain assumption PVI computed from a model finetuned on the full dataset remains a valid difficulty ordering for models trained on reduced subsets.
    Algorithm 2 computes PVI using a full-data model, then trains new models on subsets; the paper does not test whether the ordering is stable across training runs or subset sizes.
  • domain assumption Chinese NLI datasets (OCNLI, CMNLI, CINLI) are representative enough to support the claimed cross-lingual generalization of PVI.
    The paper extends PVI 'to Chinese NLP' but evaluates only three Chinese NLI datasets and two base models; no other languages are tested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quality over Quantity: An Effective Large-Scale Data Reduction Strategy Based on Pointwise V-Information." pith.science (2026). https://pith.science/paper/U5BTJUIO

@misc{pith2026250700038,
  author       = {Pith},
  title        = {Pith review of: Quality over Quantity: An Effective Large-Scale Data Reduction Strategy Based on Pointwise V-Information},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U5BTJUIO}},
  note         = {Machine review of arXiv:2507.00038}
}
read the original abstract

In order to increase the effectiveness of model training, data reduction is essential to data-centric Artificial Intelligence (AI). It achieves this by locating the most instructive examples in massive datasets. To increase data quality and training efficiency, the main difficulty is choosing the best examples rather than the complete datasets. In this paper, we propose an effective data reduction strategy based on Pointwise V-Information (PVI). To enable a static method, we first use PVI to quantify instance difficulty and remove instances with low difficulty. Experiments show that classifier performance is maintained with only a 0.0001% to 0.76% decline in accuracy when 10%-30% of the data is removed. Second, we train the classifiers using a progressive learning strategy on examples sorted by increasing PVI, accelerating convergence and achieving a 0.8% accuracy gain over conventional training. Our findings imply that training a classifier on the chosen optimal subset may improve model performance and increase training efficiency when combined with an efficient data reduction strategy. Furthermore, we have adapted the PVI framework, which was previously limited to English datasets, to a variety of Chinese Natural Language Processing (NLP) tasks and base models, yielding insightful results for faster training and cross-lingual data reduction.

Figures

Figures reproduced from arXiv: 2507.00038 by the authors.

Figure 1
Figure 1. The model architecture of the data reduction strategy. Data Transformation: This module is the preprocessing stage of the entire process and is responsible for converting the original dataset into a variety of input formats re￾quired by subsequent modules. It is an NLI Transformation base class, which defines standard processes for data loading, filtering, and preservation. We have obtained various data transformati… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 20 canonical work pages

  1. [1]

    A Survey on Data Quality Dimensions and Tools for Machine Learning

    Zhou, Y.; Tu, F.; Sha, K.; Ding, J.; Chen, H. A Survey on Data Quality Dimensions and Tools for Machine Learning. CoRR 2024, https://doi.org/10.48550/arXiv.2406.19614

    arXiv doi:10.48550/arxiv.2406.19614 2024

  2. [2]

    In Proceedings of the SmartCom 2024, Pune, India, 12 –13 January 2024 ; Volume 1, SpringerLink

    Smart Trends in Computing and Communications. In Proceedings of the SmartCom 2024, Pune, India, 12 –13 January 2024 ; Volume 1, SpringerLink. Available online: https://link.springer.com/book/10.1007/978-981-97-1320-2 (accessed on 15 June 2025)

    work page doi:10.1007/978-981-97-1320-2 2024

  3. [3]

    CrossWeigh: Training Named Entity Tagger from Imperfect Annotations

    Wang, Z.; Shang, J.; Liu, L.; Lu, L.; Liu, J.; Han, J. CrossWeigh: Training Named Entity Tagger from Imperfect Annotations. In Proceedings of the EMNLP-IJCNLP 2019, Hong Kong, China, 3–7 November 2019. https://doi.org/10.18653/v1/D19-1519

    work page doi:10.18653/v1/d19-1519 2019

  4. [4]

    Confident Learning: Estimating Uncertainty in Dataset Labels

    Northcutt, C.; Jiang, L.; Chuang, I. Confident Learning: Estimating Uncertainty in Dataset Labels. J. Artif. Intell. Res. 2021, 70, 1373–1411. https://doi.org/10.1613/jair.1.12125

    doi:10.1613/jair.1.12125 2021

  5. [5]

    Learning From Noisy Labels With Deep Neural Networks: A Survey

    Song, H.; Kim, M.; Park, D.; Shin, Y.; Lee, J.G. Learning From Noisy Labels With Deep Neural Networks: A Survey. IEEE Trans. Neural Netw. Learn. Syst. 2023, 34, 8135–8153. https://doi.org/10.1109/TNNLS.2022.3152527

    arXiv 2023

  6. [6]

    Data Quality Considerations for Big Data and Machine Learning: Going Beyond Data Clean- ing and Transformations

    Gudivada, V.N.; Apon, A.; Ding, J. Data Quality Considerations for Big Data and Machine Learning: Going Beyond Data Clean- ing and Transformations. Int. J. Adv. Softw. 2017, 10, 1-20

    work page 2017

  7. [7]

    Data-Efficient Contrastive Language-Image Pretraining: Prioritizing Data Quality over Quantity

    Joshi, S.; Jain, A.; Payani, A.; Mirzasoleiman, B. Data-Efficient Contrastive Language-Image Pretraining: Prioritizing Data Qual- ity over Quantity. In Proceedings of the 27th International Conference on Artificial Intelligence and Statistics, Valencia, Spain, 18 April 2024; pp. 1000–1008. https://doi.org/10.48550/arXiv.2403.12267

    work page Pith review arXiv doi:10.48550/arxiv.2403.12267 2024

  8. [8]

    A Data-Centric Approach to Improve Performance of Deep Learning Models

    Bhatt, N.; Bhatt, N.; Prajapati, P.; Sorathiya, V.; Alshathri, S.; El-Shafai, W. A Data-Centric Approach to Improve Performance of Deep Learning Models. Sci. Rep. 2024, 14, 22329. https://doi.org/10.1038/s41598-024-73643-x

    doi:10.1038/s41598-024-73643-x 2024

Show all 44 references
  1. [9]

    An In-Depth Analysis of Data Reduction Methods for Sustainable Deep Learning

    Toscano-Durán, V.; Perera-Lago, J.; Paluzo-Hidalgo, E.; Gonzalez-Diaz, R.; Gutierrez-Naranjo, M.Á.; Rucco, M. An In-Depth Analysis of Data Reduction Methods for Sustainable Deep Learning. Open Res. Eur. 2024, 4, 101. https://doi.org/10.12688/open- reseurope.17554.2

    2024 doi

  2. [10]

    A Comprehensive Survey of Dataset Distillation

    Lei, S.; Tao, D. A Comprehensive Survey of Dataset Distillation. IEEE Trans. Pattern Anal. Mach. Intell. 2024, 46, 17–32. https://doi.org/10.1109/tpami.2023.3322540

    2024

  3. [11]

    Training Compute-Optimal Large Language Models

    Hoffmann, J.; Borgeaud, S.; Mensch, A.; Buchatskaya, E.; Cai, T.; Rutherford, E.; Casas, D.d.L.; Hendricks, L.A.; Welbl, J.; Clark, A.; et al. Training Compute-Optimal Large Language Models. In Proceedings of the 36th International Conference on Neural Information Processing S...

    2022 doi arXiv

  4. [12]

    Dataset Difficulty and the Role of Inductive Bias

    Kwok, D.; Anand, N.; Frankle, J.; Dziugaite, G.K.; Rolnick, D. Dataset Difficulty and the Role of Inductive Bias. CoRR 2024, https://doi.org/10.48550/arXiv.2401.01867

    2024 doi arXiv

  5. [13]

    Deep Learning Through the Lens of Example Difficulty

    Baldock, R.; Maennel, H.; Neyshabur, B. Deep Learning Through the Lens of Example Difficulty. Adv. Neural Inf. Process. Syst. 2021, 34, 10876–10889

    2021

  6. [14]

    Characterising Bias in Compressed Models

    Hooker, S.; Moorosi, N.; Clark, G.; Bengio, S.; Denton, E. Characterising Bias in Compressed Models. arXiv 2020, https://arxiv.org/abs/2010.03058

    2020 arXiv

  7. [15]

    Learning Sample Difficulty from Pre-Trained Models for Reliable Prediction

    Cui, P.; Zhang, D.; Deng, Z.; Dong, Y.; Zhu, J. Learning Sample Difficulty from Pre-Trained Models for Reliable Prediction. Adv. Neural Inf. Process. Syst. 2023, 36, 25390–25408

    2023

  8. [16]

    How Hard Are Computer Vision Datasets? Calibrating Dataset Difficulty to Viewing Time

    Mayo, D.; Cummings, J.; Lin, X.; Gutfreund, D.; Katz, B.; Barbu, A. How Hard Are Computer Vision Datasets? Calibrating Dataset Difficulty to Viewing Time. Adv. Neural Inf. Process. Syst. 2023, 36, 11008–11036

    2023

  9. [17]

    Statistical Dataset Evaluation: Reliability, Difficulty, and Validity

    Wang, C.; Dong, Q.; Wang, X.; Wang, H.; Sui, Z. Statistical Dataset Evaluation: Reliability, Difficulty, and Validity. arXiv 2022, https://arxiv.org/abs/2212.09272. 25 of 26

    2022 arXiv

  10. [18]

    Understanding Dataset Difficulty with V-Usable Information

    Ethayarajh, K.; Choi, Y.; Swayamdipta, S. Understanding Dataset Difficulty with V-Usable Information. In Proceedings of the 39th International Conference on Machine Learning, Baltimore, MD, USA, 17–23 July 2022; 162, pp. 5988–6008

    2022

  11. [19]

    Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models

    Havrilla, A.; Dai, A.; O’Mahony, L.; Oostermeijer, K.; Zisler, V.; Albalak, A.; Milo, F.; Raparthy, S.C.; Gandhi, K.; Abbasi, B.; et al. Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models. arXiv 2024,

    2024

  12. [20]

    Do ImageNet Classifiers Generalize to ImageNet? In Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA , 24 May 2019; pp

    Recht, B.; Roelofs, R.; Schmidt, L.; Shankar, V. Do ImageNet Classifiers Generalize to ImageNet? In Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA , 24 May 2019; pp. 5389–5400. Available online: https://proceedings.mlr.press/v97/recht...

    2019

  13. [21]

    Natural Adversarial Examples

    Hendrycks, D.; Zhao, K.; Basart, S.; Steinhardt, J.; Song, D. Natural Adversarial Examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 11 –15 June 2021; pp. 15262–15271. https://doi.org/10.48550/arXiv.1907.07174

    2021 doi arXiv

  14. [22]

    The Third International Chinese Language Processing Bakeoff: Word Segmentation and Named Entity Recognition

    Levow, G.A. The Third International Chinese Language Processing Bakeoff: Word Segmentation and Named Entity Recognition. In Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing, Sydney, Australia, 22–23 July 2006; pp. 108– 117

    2006

  15. [23]

    CLUENER2020: Fine-Grained Named Entity Recognition Dataset and Benchmark for Chinese

    Xu, L.; Tong, Y.; Dong, Q.; Liao, Y.; Yu, C.; Tian, Y.; Liu, W.; Li, L.; Liu, C.; Zhang, X. CLUENER2020: Fine-Grained Named Entity Recognition Dataset and Benchmark for Chinese. arXiv 2020, https://arxiv.org/abs/2001.04351v4

    2020 arXiv

  16. [24]

    Natural Language Reasoning, A Survey

    Yu, F.; Zhang, H.; Tiwari, P.; Wang, B. Natural Language Reasoning, A Survey. ACM Comput . Surv. 2024, 56, 1–39. https://doi.org/10.1145/3664194

    2024 doi

  17. [25]

    A Theory of Usable Information Under Computational Constraints

    Xu, Y.; Zhao, S.; Song, J.; Stewart, R.; Ermon, S. A Theory of Usable Information Under Computational Constraints. International Conference on Learning Representations. Available online: https:// openreview.net/forum?id=r1eBeyHFDH (accessed on 15 June 2025)

    2025

  18. [26]

    A Mathematical Theory of Communication

    Shannon, C.E. A Mathematical Theory of Communication. Bell Syst. Tech. J. 1948, 27, 379–423. https://doi.org/10.1002/j.1538- 7305.1948.tb01338.x

    1948

  19. [27]

    Pre-Training With Whole Word Masking for Chinese BERT

    Cui, Y.; Che, W.; Liu, T.; Qin, B.; Yang, Z. Pre-Training With Whole Word Masking for Chinese BERT. IEEEACM Trans. Audio Speech Lang. Process 2021, 29, 3504–3514. https://doi.org/10.1109/TASLP.2021.3124365

    2021

  20. [28]

    BERT: Pre-Training of Deep Bidirectional Transformers for Language Under- standing

    Devlin, J.; Chang, M.W.; Lee, K.; Toutanova, K. BERT: Pre-Training of Deep Bidirectional Transformers for Language Under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologi...

    2019 doi

  21. [29]

    Revisiting Pre-Trained Models for Chinese Natural Language Processing

    Cui, Y.; Che, W.; Liu, T.; Qin, B.; Wang, S.; Hu, G. Revisiting Pre-Trained Models for Chinese Natural Language Processing. In Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16 –20 November 2020; pp. 657–668. https://doi...

    2020 doi

  22. [30]

    Progressive Learning: A Deep Learning Framework for Continual Learning

    Fayek, H.M.; Cavedon, L.; Wu, H.R. Progressive Learning: A Deep Learning Framework for Continual Learning. Neural Netw. 2020, 128, 345–357. https://doi.org/10.1016/j.neunet.2020.05.011

    2020 doi

  23. [31]

    Cross-Entropy Loss Functions: Theoretical Analysis and Applications

    Mao, A.; Mohri, M.; Zhong, Y. Cross-Entropy Loss Functions: Theoretical Analysis and Applications. In Proceedings of the 40th International Conference on Machine Learning, Honolulu, HI, USA, 23 –29 July 2023; pp. 23803–23828. Available online: https://proceedings.mlr.press/v20...

    2023

  24. [32]

    Understanding Transformers via N-Gram Statistics

    Nguyen, T. Understanding Transformers via N-Gram Statistics. Adv. Neural Inf. Process Syst. 2024, 37, 98049–98082

    2024

  25. [33]

    Qwen3 Technical Report

    Yang, A.; Li, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Gao, C.; Huang, C.; Lv, C.; et al. Qwen3 Technical Report. arXiv 2025, https://arxiv.org/abs/2505.09388v1

    2025 arXiv

  26. [34]

    OCNLI: Original Chinese Natural Language Inference

    Hu, H.; Richardson, K.; Xu, L.; Li, L.; Kübler, S.; Moss, L.S. OCNLI: Original Chinese Natural Language Inference. arXiv 2020. https://doi.org/10.48550/arXiv.2010.05444

    2020 doi arXiv

  27. [35]

    CLUE: A Chinese Language Understanding Evaluation Benchmark

    Xu, L.; Hu, H.; Zhang, X.; Li, L.; Cao, C.; Li, Y.; Xu, Y.; Sun, K.; Yu, D.; Yu, C.; et al. CLUE: A Chinese Language Understanding Evaluation Benchmark. In Proceedings of the 28th International Conference on Computational Linguistic, Barcelona, Spain, 8– 13 December 2020; pp. ...

    2020 doi

  28. [36]

    XNLI: Evaluating Cross-Lingual Sentence Representations

    Conneau, A.; Rinott, R.; Lample, G.; Williams, A.; Bowman, S.; Schwenk, H.; Stoyanov, V. XNLI: Evaluating Cross-Lingual Sentence Representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brus- sels, Belgium, 2–4 November 2018; ...

    2018 doi

  29. [37]

    A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference

    Williams, A.; Nangia, N.; Bowman, S.R. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. arXiv 2018, https://arxiv.org/abs/1704.05426v4

    2018 arXiv

  30. [38]

    Annotation Artifacts in Natural Language Inference Data

    Gururangan, S.; Swayamdipta, S.; Levy, O.; Schwartz, R.; Bowman, S.; Smith, N.A. Annotation Artifacts in Natural Language Inference Data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational 26 of 26 Linguistics: Human Langua...

    2018 doi

  31. [39]

    Sexism Detection on a Data Diet

    Bandyopadhyay, R.; Assenmacher, D.; Moral, J.M.A.; Wagner, C. Sexism Detection on a Data Diet. In Proceedings of the Com- panion Publication of the 16th ACM Web Science Conference, Stuttgart, Germany, 21 –24 May 2024; pp. 94–102. https://doi.org/10.1145/3630744.3663609

    2024

  32. [40]

    Deep Learning on a Data Diet: Finding Important Examples Early in Training

    Paul, M.; Ganguli, S.; Dziugaite, G.K. Deep Learning on a Data Diet: Finding Important Examples Early in Training. Adv. Neural Inf. Process Syst. 2021, 34, 20596–20607. https://doi.org/10.48550/arXiv.2107.07075

    2021 doi arXiv

  33. [41]

    Estimating Example Difficulty Using Variance of Gradients

    Agarwal, C.; D’souza, D.; Hooker, S. Estimating Example Difficulty Using Variance of Gradients. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 10358–10368. https://doi.org/10.48550/arXiv.2008.11600

    2022 doi arXiv

  34. [42]

    Estimating Training Data Influence by Tracing Gradient Descent

    Pruthi, G.; Liu, F.; Kale, S.; Sundararajan, M. Estimating Training Data Influence by Tracing Gradient Descent. Adv. Neural Inf. Process. Syst. 2020, 33, 19920–19930

    2020

  35. [43]

    Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

    Abdin, M.; Aneja, J.; Awadalla, H.; Awadallah, A.; Awan, A.A.; Bach, N.; Bahree, A.; Bakhtiari, A.; Bao, J.; Behl, H.; et al. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. arXiv 2024, https://arxiv.org/abs/2404.14219

    2024 arXiv

  36. [44]

    Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs

    Microsoft; Abouelenin, A.; Ashfaq, A.; Atkinson, A.; Awadalla, H.; Bach, N.; Bao, J.; Benhaim, A.; Cai, M.; Chaudhary, V.; et al. Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs. arXiv 2025, https://arxiv.org/abs/2503.01743

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.