Pith. sign in

REVIEW 4 major objections 5 minor 31 references

Dual Debiasing for Noisy In-Context Learning for Text Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a Sample Cleanliness Score, computed by dividing conditional per-token NLL by unconditional NLL and by a neighbor-averaged domain-bias estimate, separates clean from noisy demonstrations in text-generation in-context…

desk verdict Novel, empirically strong cleaning score for noisy ICL, but the theory underpinning the 'absolute' claim is loose and the headline results are cherry-picked between two corpora; deserves review with major revision demands. read the letter →

arxiv 2506.00418 v2 pith:QZAVM2J3 submitted 2025-05-31 cs.CL cs.AI

Siqi Liang , Sumyeong Ahn , Paramveer S. Dhillon , Jiayu Zhou This is my paper
classification cs.CLcs.AI
keywords noisyin-contextlearningsamplecleanlinessscoredualdebiasingperplexitytextgenerationlabelnoisedemonstrationselectionLLMpriorknowledge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that perplexity-based noise detection in in-context learning fails at high noise ratios because raw per-token negative log-likelihood is biased twice: the LLM is familiar with the annotation text itself, and the LLM has uneven knowledge across domains. It proposes a Sample Cleanliness Score that removes both biases, dividing the conditional NLL by the unconditional NLL and then by a neighbor-averaged estimate of domain difficulty, so a clean demonstration scores higher than a noisy one whether or not they come from the same topic. The claim is that this score detects noisy demonstrations reliably even when 80 percent of the corpus is noisy, and that in-context learning using only the detected clean subset performs about as well as using a fully clean corpus. A reader should care because noisy annotations are common in real data, and existing methods assume noise is a minority.

What carries the argument

The Sample Cleanliness Score I(x,y)=Phi((x,y))/L_de-int(y|x), where L_de-int(y|x)=L(y|x)/L(y) is the per-token NLL of the annotation given the query divided by the per-token NLL of the annotation alone, and Phi is a neighbor-based estimate of the domain-level average of L_de-int over pairs with the same query and random annotations from a corpus. The mechanism is a ratio: high conditional loss caused by genuine mismatch survives, while high conditional loss caused by a hard domain or by the annotation's own improbability is divided out, making scores comparable across queries, annotations, and knowledge domains.

What would settle it

Take a corpus with 80 percent noisy annotations where the noisy annotations are deliberately short and plausible, such as swapping in other valid answers from the same domain, compute the Sample Cleanliness Score, and check whether the two-component mixture still recovers the true clean subset with the AUC gains reported; the claim would fail if the neighbor average is dominated by noisy annotations and clean samples from hard domains receive scores overlapping the noisy component.

Watch

Extended reading notes

Core claim

The central discovery is a two-step debiasing identity for per-token NLL. First, intrinsic debiasing defines L_de-int(y|x)=L(y|x)/L(y), the conditional per-token loss divided by the unconditional per-token loss of the annotation alone, which removes the LLM's prior familiarity with the annotation string. Second, extrinsic debiasing divides L_de-int by the average of L_de-int over a neighborhood of synthesized pairs that keep the query x but replace the annotation with random annotations sampled from a corpus, estimated by edit-distance-bounded neighbors. The Sample Cleanliness Score I(x,y) is then compared across all demonstrations; a two-component Gaussian mixture model on the scores separates clean from noisy, and discarding the noisy subset before retrieval yields in-context-learning performance the paper reports as comparable to a fully clean demonstration set, across four text-generation datasets, two noise types, three retrievers, and noise ratios up to 0.8.

Load-bearing premise

The score's cross-domain comparability depends on the assumption that the average of L_de-int over pairs formed by pairing each query with random annotations from a corpus faithfully captures the LLM's domain-level difficulty; if the corpus is dominated by noisy annotations or annotation lengths vary widely, that neighborhood average can be contaminated and the central claim of absolute cleanliness would break.

Editorial extensions

If this is right

  • Noise-ratio independence: because the score is normalized per sample rather than ranked within the corpus, clean samples sit on the same side of the mixture split even when noisy samples are the majority; experiments show stability up to 0.8 noise.
  • Deletion instead of repair: once noisy demonstrations are identified, simply discarding them is enough to match clean-corpus in-context-learning performance, removing the need for replacement or correction steps.
  • Model-scale robustness: the same score works when computed with a 1.3B model as with a 7B model, with only negligible in-context-learning performance decline, so noise screening can be done cheaply.
  • Retriever independence: the gain holds across Random, TopK, and DPP retrievers, meaning the cleanliness signal is about the demonstration set, not a particular retrieval strategy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same dual-debiasing ratio could be applied to other per-sample loss-based selection problems, such as filtering instruction-tuning data or selecting demonstrations for a specialized domain, where annotation priors and domain skew play comparable roles.
  • The neighbor-based estimate reuses the same query with random corpus annotations; an extension the paper does not explore is replacing corpus-sampled annotations with LLM-generated or paraphrased ones, which could change the domain-bias estimate for long or rare annotations.
  • The paper's own failure analysis ties detection errors to long annotations; a direct extension would be a length-normalized or length-binned version of the score, since the extrinsic-bias approximation degrades when annotation lengths vary widely.
  • Because the score measures the metric model's perception, a corpus scored by one LLM may not transfer directly to another; a testable corollary is whether a calibration step between metric models is needed before deploying the score on a new inference model.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies noisy annotations in in-context learning (ICL) for text generation. It proposes a Sample Cleanliness Score built from per-token negative log-likelihood after two debiasing steps: an intrinsic step that normalizes L(y|x) by the annotation-only loss L(y), and an extrinsic step that divides by a neighborhood average of the intrinsic-debiased loss. The score is used with a two-component Gaussian mixture model to split the training set into clean and noisy demonstrations; noisy samples are discarded and ICL is performed on the remaining set. The paper evaluates the method on NQ, WebQ, SciQ, and SQuAD, with three retrievers, relevant/irrelevant noise at ratios up to 0.8, and several metric LLMs, reporting performance close to a fully clean corpus and superior to baselines including local perplexity ranking.

Significance. The empirical contribution is substantial if the central claim holds: the method is evaluated broadly and shows consistent gains over Naive ICL, Random deletion, and LPR, with sensitivity analyses for gamma and N_neighbor, a robustness study across metric LLMs, and a failure-case analysis. The use of fixed default hyperparameters and an unsupervised GMM avoids fitting the thresholds to test labels. However, the 'absolute' cross-domain comparability of the score is the load-bearing theoretical claim, and it is neither proven nor convincingly supported by the discrete estimator; the paper itself states that the metric depends on the quality and diversity of neighbor samples. The reported best-of-Cin/Cout selection also requires a practical selection rule before the method can be used as claimed.

major comments (4)
  1. [Section 3.3, Eq. (7) and Eq. (10)] The discrete estimator in Eq. (10) does not estimate the continuous quantity in Eq. (7). Equation (7) averages Lde-int over the joint neighborhood N((x,y))={(x',y'): d_edit(y,y')<eta}, and because the distance function in Eq. (6) ignores x, that neighborhood includes variation in both query and annotation. The construction NDISC in Eq. (9) instead pairs the original query x with annotations y'_z sampled from a corpus C, so Eq. (10) computes a query-conditional average E_{y'|x}[Lde-int(y'|x)] with x held fixed. No argument is given that this conditional average converges to the joint neighborhood average, and the paper's own premise that domain bias varies with x makes it unlikely in general. This mismatch is load-bearing because the cross-domain comparability claim in Eq. (8) depends on extrinsic debiasing removing a domain-level baseline rather than a query-specific baseline.
  2. [Section 3.3, Eq. (8)] The assertion that I(x1,y1)>I(x2,y2) whenever y1 is clean and y2 is noisy, even across domains, is stated without proof. The within-query inequality in Eq. (5) only supports comparisons for a fixed query; it does not imply that the ratio Phi(x,y)/Lde-int(y|x) separates clean from noisy uniformly after dividing by a corpus-dependent Phi. Either prove this inequality under explicit assumptions on the metric space and the sampling distribution of C, or collapse the claim to an empirical observation. As written, this unsupported inequality is the mathematical content of 'absolute sample cleanliness' and must be justified.
  3. [Section 4.1, Implementation details] The paper reports the main results using 'the optimal performance achieved by either Cin or Cout' without specifying how that choice is made. If the better corpus is selected using the test split, the numbers in Table 1 are optimistic and the pipeline as described is not reproducible. A practical method needs a fixed default corpus or a validation-based selection rule; at minimum, results for Cin and Cout should be reported separately in the main table instead of the best of the two.
  4. [Limitations] The Limitations paragraph concedes that the method's effectiveness 'relies heavily on the quality and diversity of the neighbor samples' and may be degraded in domains with limited data. This directly conflicts with the Abstract's claim that the metric 'uncovers absolute sample cleanliness regardless of the overall corpus noise level.' Since Eq. (10) averages over annotations sampled from C, a corpus dominated by noisy annotations contributes noisy values to Phi, making the score dependent on the noise composition of C. The paper should either qualify the absolute claim or provide an experiment that varies the clean/noisy composition of C while holding the scored demonstrations fixed and shows that clean/noisy separation is unchanged.
minor comments (5)
  1. [Section 3.3, Eq. (9)] The text says 'serve as alternation for the domain' where it should be 'alternative'.
  2. [Figure 2d] The caption for Figure 2d reads 'Band vs Number on GPT-Neo-2.7B', while the text in Section 3.3 says a similar issue is observed for Music-Band vs Number under Gemma-2b; the caption and body text disagree.
  3. [Related Work] The sentence 'the efficient approach by Liu et al. that extracts in-context vectors' omits the year and venue of the citation; please supply the full reference or use the numbered citation format consistently.
  4. [Algorithm 1] The comment '/* Metric Calculation For the Whole Training Set */' appears twice in Algorithm 1; the second occurrence, before the ICL inference step, should be replaced with a description of the inference phase.
  5. [Appendix, Table 6] The hyperparameter gmm_part_thres is listed in Table 6 but is never defined or discussed in Section 3.4; please define it and explain how the per-dataset values were chosen.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the debiased score is computed from LLM probabilities without fitting to noise labels, and no load-bearing derivation reduces to its own inputs.

full rationale

The derivation chain is not circular. The Sample Cleanliness Score I(x,y) is defined from per-token NLL values via intrinsic debiasing (Eq. 4: L_de-int(y|x)=L(y|x)/L(y)) and extrinsic debiasing (Eq. 8: I=Phi/L_de-int). Neither step uses the true clean/noisy labels; the GMM in Section 3.4 is unsupervised, and gamma=0.5 and N_neighbor=50 are fixed defaults with sensitivity analysis. The inequality in Eq. 5 is an assumption about clean/noisy separability rather than a fitted prediction. The main weakness is that the continuous-domain neighborhood average in Eq. 7 is not faithfully approximated by the query-fixed discrete estimator in Eq. 9-10; the estimator is a query-conditional average, not the joint-domain average, and the Limitations section explicitly concedes that the method 'relies heavily on the quality and diversity of the neighbor samples used for extrinsic debiasing.' That is a correctness and robustness limitation, not a definitional equivalence to the input. The paper contains no self-citations, so no load-bearing self-citation or imported uniqueness theorem is present. Reporting the better of Cin and Cout is a selective reporting concern, not a fitted-input-called-prediction step. No circular step can be exhibited from the paper's equations or citations.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or ontological entities; the Sample Cleanliness Score is a derived metric. All load-bearing premises are statistical assumptions about LLM probabilities and the validity of neighbor-based normalization.

free parameters (4)
  • GMM separation threshold (gmm_part_thres) = 5.0 (NQ), 4.0 (WebQ), 10.0 (SCIQ), 12.5 (SQuAD)
    Dataset-specific hyperparameters controlling GMM component separation in noise detection (Table 6).
  • Noise-probability threshold gamma = 0.5
    Threshold on GMM posterior for classifying a sample as noisy; fixed default, sensitivity analysis shown in Figure 5.
  • Neighbor count N_neighbor = 50
    Number of random annotations sampled per demonstration for extrinsic debiasing; sensitivity analysis shown in Figure 4.
  • Neighbor radius eta = max(T, Tmax)
    Maximum allowed annotation length for neighbors, bounding the edit-distance radius of the associated domain; set to the maximum annotation length in the observed training set.
assumptions (4)
  • domain assumption LLMs assign higher per-token probability to clean annotations than to noisy ones for a fixed query (Equation 1).
    Foundation of the perplexity-based noise detection paradigm; motivated by prior work but not proven here.
  • domain assumption Dividing by the unconditional token loss L(y) removes the LLM's prior knowledge bias on the annotation, so L_de-int(y|x) isolates query-annotation mismatch.
    Assumes L(y) fully captures intrinsic bias; no formal proof that the ratio is bias-free.
  • ad hoc to paper An extrinsic domain bias exists as a neighborhood average of L_de-int with uniform density over a continuous domain (Equation 7), and the discrete same-query random-annotation average (Equations 9-10) faithfully estimates it.
    The continuous-domain integral is intractable and the discrete approximation is a heuristic; the edit-distance definition of the domain is not actually enforced in sampling.
  • ad hoc to paper The Sample Cleanliness Score satisfies I(x1,y1) > I(x2,y2) whenever y1 is clean and y2 is noisy, even across different domains.
    This cross-domain comparability property is stated without proof in Section 3.3 and is the basis for the global GMM threshold.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dual Debiasing for Noisy In-Context Learning for Text Generation." pith.science (2026). https://pith.science/paper/QZAVM2J3

@misc{pith2026250600418,
  author       = {Pith},
  title        = {Pith review of: Dual Debiasing for Noisy In-Context Learning for Text Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QZAVM2J3}},
  note         = {Machine review of arXiv:2506.00418}
}
read the original abstract

In context learning (ICL) relies heavily on high quality demonstrations drawn from large annotated corpora. Existing approaches detect noisy annotations by ranking local perplexities, presuming that noisy samples yield higher perplexities than their clean counterparts. However, this assumption breaks down when the noise ratio is high and many demonstrations are flawed. We reexamine the perplexity based paradigm for text generation under noisy annotations, highlighting two sources of bias in perplexity: the annotation itself and the domain specific knowledge inherent in large language models (LLMs). To overcome these biases, we introduce a dual debiasing framework that uses synthesized neighbors to explicitly correct perplexity estimates, yielding a robust Sample Cleanliness Score. This metric uncovers absolute sample cleanliness regardless of the overall corpus noise level. Extensive experiments demonstrate our method's superior noise detection capabilities and show that its final ICL performance is comparable to that of a fully clean demonstration corpus. Moreover, our approach remains robust even when noise ratios are extremely high.

Figures

Figures reproduced from arXiv: 2506.00418 by the authors.

Figure 1
Figure 1. Overview of noisy ICL and the proposed method: When noisy information is present in an ICL dataset, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Distribution of Lde-int for both clean and noisy versions of demonstrations for different topics. The Lde-int values are calculated using Gemma-2b and GPT-Neo-2.7B. LLMs show different degrees of familiarity with queries from distinct knowledge domains, which affects the value distributions of Lde-int for sam￾ples from these domains. Consequently, the Lde-int values of different domains are not directly compa￾rable,… view at source ↗
Figure 3
Figure 3. Sample Cleanliness Score distributions on [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Sensitivity on neighbor number Nneighbor un￾der noise ratio 0.6 and TopK retriever. Left: SCIQ; right: NQ. The ‘irrelevant’/‘relevant’ indicates the per￾formance of our method under noise, and ‘clean’ indi￾cates the performance of naive baseline under clean. Computatio…
Figure 5
Figure 5. Figure 5: Sensitivity of γ under noise ratio 0.6 and TopK retriever. Left: SCIQ; right: NQ. The ‘irrele￾vant’/‘relevant’ indicates the performance of our method under noised setting, and ‘clean’ indicates the perfor￾mance of naive ICL baseline under clean setting. that our metho…
Figure 7
Figure 7. Figure 7: Distribution of L(y|x) for both noisy and clean samples on NQ. Heavy overlapping between the distributions of noisy and clean samples. Hyper-parameter The hyper-parameter settings for main table are shown in [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 6
Figure 6. Figure 6: Performance of our method under setting of [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 8
Figure 8. Figure 8: Comparison between using Cin and Cout on NQ under noise ratio 0.6. Left: irrelevant noise; Right: relevant noise [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Comparison between using Cin and Cout on SCIQ under noise ratio 0.6. Left: irrelevant noise; Right: relevant noise. B.4 Dependency on choice of LLMs We conduct further analysis on how different LLMs benefit from our dual-debiasing approach under 40% relevant noise sett…
Figure 10
Figure 10. Figure 10: Annotation sequence length distribution for clean/noisy samples that our method failed/successfully [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 9 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Gabriel Alon and Michael Kamfonas. 2023. Detecting language model attacks with perplexity. arXiv preprint arXiv:2308.14132

    arXiv 2023

  3. [3]

    Udit Arora, William Huang, and He He. 2021. Types of out-of-distribution texts and how to detect them. arXiv preprint arXiv:2109.06827

    work page Pith review arXiv 2021

  4. [4]

    Sid Black, Gao Leo, Phil Wang, Connor Leahy, and Stella Biderman. 2021. https://doi.org/10.5281/zenodo.5297715 GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow . If you use this software, please cite it using these metadata

    doi:10.5281/zenodo.5297715 2021

  5. [5]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  6. [6]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.64 A survey on in-context learning . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1107--1128, Miami, Florid...

    doi:10.18653/v1/2024.emnlp-main.64 2024

  7. [7]

    Yu Fei, Yifan Hou, Zeming Chen, and Antoine Bosselut. 2023. Mitigating label biases for in-context learning. arXiv preprint arXiv:2305.19148

    arXiv 2023

  8. [8]

    Hongfu Gao, Feipeng Zhang, Wenyu Jiang, Jun Shu, Feng Zheng, and Hongxin Wei. 2024. On the noise robustness of in-context learning for text generation

    work page 2024

Show all 31 references
  1. [9]

    Zhizhong Huang, Junping Zhang, and Hongming Shan. 2023. Twin contrastive learning with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11661--11670

    2023

  2. [10]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  3. [11]

    Junyong Kang, Donghyun Son, Hwanjun Song, and Buru Chang. 2024. In-context learning with noisy labels. arXiv preprint arXiv:2411.19581

    2024 arXiv

  4. [12]

    Nazmul Karim, Mamshad Nayeem Rizve, Nazanin Rahnavard, Ajmal Mian, and Mubarak Shah. 2022. Unicon: Combating label noise through uniform selection and contrastive learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9676--9686

    2022

  5. [13]

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for C...

    2019

  6. [14]

    Dividemix: Learning with noisy labels as semi-supervised learning

    Junnan Li, Richard Socher, and Steven CH Hoi. Dividemix: Learning with noisy labels as semi-supervised learning. In International Conference on Learning Representations

  7. [15]

    Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2022. Contrastive decoding: Open-ended text generation as optimization. arXiv preprint arXiv:2210.15097

    2022 arXiv

  8. [16]

    Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, and Xipeng Qiu. 2023. Unified demonstration retriever for in-context learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long P...

    2023

  9. [17]

    In-context vectors: Making in context learning more effective and controllable through latent space steering

    Sheng Liu, Haotian Ye, Lei Xing, and James Y Zou. In-context vectors: Making in context learning more effective and controllable through latent space steering. In Forty-first International Conference on Machine Learning

  10. [18]

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837

    2022 arXiv

  11. [19]

    Sean O'Brien and Mike Lewis. 2023. Contrastive decoding improves reasoning in large language models. arXiv preprint arXiv:2309.09117

    2023 arXiv

  12. [20]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  13. [21]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118

    2024 arXiv

  14. [22]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  15. [23]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  16. [24]

    Zhenyu Wu Wu, Yaoxiang Wang, Jiacheng Ye, Jiangtao Feng, Jingjing Xu, Yu Qiao, and Zhiyong Wu. 2023. Openicl: An open-source framework for in-context learning. arXiv preprint arXiv:2303.02913

    2023 arXiv

  17. [25]

    Jianhao Yan, Pingchuan Yan, Yulong Chen, Jing Li, Xianchao Zhu, and Yue Zhang. 2024. Benchmarking gpt-4 against human translators: A comprehensive evaluation across languages, domains, and expertise levels. arXiv preprint arXiv:2411.13775

    2024 arXiv

  18. [26]

    Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong. 2023. Compositional exemplars for in-context learning. In International Conference on Machine Learning, pages 39818--39833. PMLR

    2023

  19. [27]

    Kang Min Yoo, Junyeob Kim, Hyuhng Joon Kim, Hyunsoo Cho, Hwiyeol Jo, Sang-Woo Lee, Sang-goo Lee, and Taeuk Kim. 2022. Ground-truth labels matter: A deeper look into input-label demonstrations. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Proce...

    2022

  20. [28]

    Zheng Zhao, Emilio Monti, Jens Lehmann, and Haytham Assem. 2024. Enhancing contextual understanding in large language models through contrastive decoding. arXiv preprint arXiv:2405.02750

    2024 arXiv

  21. [29]

    Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. In International conference on machine learning, pages 12697--12706. PMLR

    2021

  22. [30]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  23. [31]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.