Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

This paper proposes early-exit language models with a zero-shot fallback, and certifies that bad in-context examples cost at most a user-chosen amount of expected accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 12:40 UTC pith:YKAEEDCZ

load-bearing objection A genuinely useful risk-control trick for ICL, but the headline safety claim is marginal and the exit-layer selection may break the LTT guarantee. the 3 major comments →

arxiv 2510.02480 v3 pith:YKAEEDCZ submitted 2025-10-02 cs.AI cs.LG

Controlling the Risk of Corrupted Contexts for Language Models via Early-Exiting

Andrea Wynn , Metod Jazbec , Charith Peris , Rinat Khaziev , Anqi Liu , Daniel Khashabi , Eric Nalisnick This is my paper
classification cs.AI cs.LG
keywords in-context learningearly exitingdistribution-free risk controlzero-shot baselineoverthinkingLLM safetycomputational efficiency
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that an LLM can be given in-context demonstrations of unknown quality without risking large performance loss. It defines a 'safe' predictor that either exits at an early layer when confident or falls back to the model's zero-shot answer, and uses distribution-free risk control to pick the confidence threshold. The guarantee is that on new data drawn like the calibration set, the average accuracy loss relative to zero-shot stays below a user-chosen epsilon, even when some demonstrations are wrong. A second contribution adapts the risk-control procedure to preserve helpful demonstrations' negative losses, so the model keeps accuracy and speed gains when context is good. A sympathetic reader would care because it gives a principled, certified mechanism for guarding against corrupted prompts instead of hoping the model ignores them.

Core claim

The central claim is that a single threshold can simultaneously prevent overthinking on harmful demonstrations and preserve gains from helpful ones. The paper's safe ICL model returns an intermediate-layer prediction once confidence exceeds λ, and if no layer reaches λ it returns the zero-shot prediction. Treating the loss of this model minus the full zero-shot loss as the risk, a distribution-free risk-control procedure selects λ that controls expected risk at level ε. The authors prove that rescaling the bounded loss preserves the risk-control guarantee, and empirically show that risk is controlled across eight classification tasks and four models, with roughly 53% fewer layers evaluated t

What carries the argument

The central object is the safe in-context-learning predictor: an LLM that checks layer-by-layer confidence and either exits early or falls back to the zero-shot prediction. The argument runs through the ICL loss — accuracy of the safe predictor with demonstrations minus accuracy of the full zero-shot model — which is positive for harmful context and negative for helpful context. Because this loss is non-monotonic in λ, the procedure uses Learn-then-Test with a domain-preserving rescaling that maps losses in [-1,1] to [0,1] and the risk level epsilon accordingly, preserving the control guarantee while retaining the negative losses that encode helpful demonstrations.

Load-bearing premise

Calibration and deployment inputs, labels, and demonstration sets are drawn from the same distribution, so the mix of helpful and harmful demonstrations seen at test time must match the calibration mix; the bound says nothing about a test set made mostly of harmful demonstrations.

What would settle it

Calibrate on a 50/50 mix of correct and incorrect demonstrations, then evaluate on a test set that contains only incorrect demonstrations from the same task. The paper's own figures show the per-subgroup risk will exceed ε; that is not a violation of the marginal guarantee, but it identifies exactly the scenario under which the title's safety promise would be tested.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Practitioners can deploy in-context learning with a certified cap on average accuracy degradation relative to zero-shot, given a calibration set that reflects deployment demonstration quality.
  • The same early-exit mechanism yields computational savings: at ε=0.05 the method evaluates roughly half the layers compared with loss-clipping baselines.
  • On tasks where correct demonstrations help, the threshold can be chosen so that most of the accuracy gain is retained while harmful demonstrations are neutralized.
  • The risk-control guarantee holds for different mixes of correct and incorrect demonstrations during calibration, as long as the test mix matches the calibration mix.
  • The method works even though the loss is non-monotonic in λ, which rules out simpler monotonicity-based risk-control approaches.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the guarantee is marginal, a deployment where bad demonstrations are concentrated on a vulnerable subgroup could still suffer large harm; class-conditional risk control would be the natural next step.
  • The method inherits the quality of the zero-shot baseline: if zero-shot itself is unsafe or weak, 'safe' is anchored to that weak point; a testable extension would combine the early-exit fallback with a stronger safety-aligned baseline.
  • The efficiency gains depend on models having reliable confidence in mid-layers; on models where early-layer confidence is miscalibrated, the first-exit restriction to the last half of layers may throttle savings — an empirical question across architectures.
  • A direct practical extension is to use the ICL loss to monitor demonstration quality online, retraining λ on streaming calibration to track shifts in the correct/incorrect mix.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a risk-controlled early-exit mechanism for in-context learning. The safe ICL predictor considers all layers of an LLM up to a confidence threshold and falls back to the zero-shot prediction if no intermediate exit exceeds the threshold. The loss is the difference between the ICL-with-context prediction loss and the zero-shot prediction loss (Eq. 3), so negative values correspond to helpful demonstrations. To select the threshold, the authors adapt Learn-then-Test (LTT) by affinely rescaling the bounded loss to [0,1] and prove in Appendix C that controlling the rescaled risk controls the original risk. Experiments over 8 classification tasks and 4 LLMs report that the empirical risk stays below user-chosen epsilon levels while providing efficiency gains relative to loss-clipping.

Significance. If the claimed guarantee is valid, this is a useful contribution: it provides a formal, distribution-free mechanism to bound the average performance degradation caused by user-supplied demonstrations relative to the model's zero-shot behavior, while still exploiting helpful demonstrations. The affine rescaling of a bounded loss for LTT is simple but correct, and the paper is transparent about the marginal nature of the guarantee. The empirical evaluation is extensive — 8 tasks, 4 models, 100 repeated calibration/test splits — and the released code supports reproducibility. The main value is in formalizing a safety-relevant control objective for ICL and connecting it to dynamic early exit.

major comments (3)
  1. [§J.3 and §3.4] LTT's finite-sample guarantee requires the candidate predictor family to be fixed before seeing the calibration data. Appendix J.3 states that the exit layer range was restricted to the last half of the layers based on 'detailed examination' and 'empirical results' (Fig. 20) on the same models/datasets, with no separate model-selection split. If this restriction was inferred from calibration or test data, the calibration losses are not independent of the family, and the LTT guarantee stated in Appendix C does not apply to the reported procedure. The central claim R_ICL(λ̂)≤ε (Eq. 3) is therefore not established unless the layer range is pre-specified before any data inspection, or selected on a separate development split with LTT applied on a fresh calibration set.
  2. [§4.3 and §3.3] The controlled risk is marginal over a fixed data-generating distribution P over (x,y,c), and the calibration mix of correct versus incorrect demonstrations must match the deployment mix. When all contexts are harmful, the average risk over a 50-50 mixture says nothing about the harmful subgroup. The paper acknowledges this in §4.3, and Figs. 14–15 indeed show the incorrect-only risk exceeding ϵ for both scaled and clipped losses. Given the title and abstract emphasize 'corrupted contexts,' the authors should explicitly qualify the headline as average-case control over a calibrated mix, not conditional or worst-case control over harmful demonstrations.
  3. [§4.1 / §3.4] The LTT guarantee holds with probability at least 1−δ, but no δ value is reported in the main text or experiments. Without specifying δ, the reader cannot assess the strength of the claimed risk control. Please report the δ used for calibration, or if the empirical curves average over 100 runs regardless of δ, state that explicitly.
minor comments (4)
  1. [Abstract / §4] The abstract says the approach is evaluated on '9 tasks,' while the main text and Section 4 consistently say 8 tasks. Please reconcile the count.
  2. [§J.3] The sentence 'we cannot have a confidence-based λ threshold which allows us to early-exit while preserving performance' is ambiguous. It should clarify that this is for very early layers, not for the last half.
  3. [Fig. 5 caption] The caption says risk is 'always' controlled. The guarantee is probabilistic and the figure shows means with standard errors; please phrase as 'empirical risk is below ϵ on average across 100 runs.'
  4. [Appendix C] The proof is correct but somewhat verbose. It could be shortened to a two-line equivalence, but no change is required.

Circularity Check

0 steps flagged

No significant circularity: the calibration-selection loop is standard risk control, and the risk transformation is a proven equivalence; acknowledged limitations are scope concerns, not circular steps.

full rationale

The paper's central claim is a Learn-then-Test (LTT) risk-control guarantee for an early-exit predictor with a zero-shot fallback. The threshold lambda is selected on a 50% calibration split and evaluated on the held-out 50% from the same distribution, which is the standard conformal/risk-control loop rather than a fit-then-predict tautology. The ICL loss (Eq. 3) compares the early-exit-with-context prediction to the zero-shot prediction; this defines the risk being controlled, and no parameter is fitted and then relabeled as a prediction. The domain-preserving risk transformation in Sec. 3.4 is supported by a self-contained proof in Appendix C showing R(ell) <= epsilon iff R(ell') <= epsilon', so it is a mathematical equivalence, not a circular reduction. The paper explicitly acknowledges that the marginal guarantee does not extend to class-conditional risk control (Sec. 4.3), which is a limitation narrowing the interpretation of the result, not a circular step. One legitimate methodological concern is noted: Appendix J.3 states that the early-exit range was restricted to the last half of layers after "detailed examination" of the same models and datasets, without describing a separate validation set. If the predictor family was chosen using the calibration or test data, the LTT guarantee may not strictly apply. However, this is a data-dependent design choice that could invalidate the theorem's assumptions, not a case where the derived risk bound is equivalent to the input by construction. The risk bound is not an identity with the layer-range choice. Citations to the authors' prior work (Jazbec et al. 2023, 2024; Fang et al. 2025) are used for background, baselines, or design conventions and are not load-bearing for the central theoretical claim. Therefore, no circular step can be exhibited, and the circularity score is 0.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The method introduces no new entities, forces, or conserved quantities. Its load-bearing structure is the confidence threshold lambda (selected by LTT on calibration data), the zero-shot fallback, and the calibration distribution over demonstration quality. The main uncharged premises are the i.i.d. assumption and the empirical overthinking claim.

free parameters (3)
  • earliest allowed exit layer (L/2 = 16 of 32) = 16
    Risk-control early exit is restricted to the last half of layers; chosen after observing that early layers are overconfident (Appendix J.3, Fig. 20). This hand-picked restriction limits the hypothesis space of reachable lambda and shapes the risk-control results, but it is disclosed and ablation-motivated.
  • calibration mix of correct/incorrect demonstrations = 50/50 in main experiments (10/90, 75/25, 90/10, 95/5 in appendix)
    The selected lambda inherits the calibration mixture because the risk is marginal over it. The user chooses this mix, and different mixes produce different certified thresholds and different safety properties on the incorrect-only subgroup.
  • confidence measure (max-class probability) = argmax of softmax probabilities
    One of three tested confidence scores (Appendix J.2); the ablation shows no significant difference, but it is a methodological choice rather than a derived quantity.
axioms (5)
  • standard math Learn-then-Test with the Hoeffding-Bentkus bound gives a valid finite-sample guarantee for losses bounded in [0,1]
    Invoked in sections 2.2 and 3.4 as the backbone of the threshold selection; the paper relies on this theorem without re-deriving it.
  • domain assumption Calibration and test samples (x, y, c) are i.i.d. from the same distribution P
    Stateed in section 2.2 as the condition for the risk guarantee. This includes the correctness mix of the demonstrations, which is the fragile part of the safety claim.
  • domain assumption Overthinking: with incorrect demonstrations, model accuracy peaks at intermediate layers and degrades in later layers
    Adopted from Halawi et al. 2024 and replicated in this paper (section 3.2, Fig. 17). If this layer-wise behavior does not hold for a new model or task, early exiting will not protect against harmful context.
  • domain assumption The zero-shot model is a safe, stable baseline for all tasks studied
    Section 3.1 defines the safe predictor around the zero-shot fallback. The paper shows in Fig. 21 that on some memorized datasets zero-shot matches or beats in-context learning, which supports but does not guarantee the baseline's safety in general.
  • domain assumption The classification loss is bounded with ell_ICL in [-1,1]
    Section 3.4 uses the 0/1 loss difference so that a=-1, b=1; this makes the affine rescaling valid for the LTT bounds.

pith-pipeline@v1.3.0-alltime-deepseek · 24240 in / 12129 out tokens · 90078 ms · 2026-08-04T12:40:10.111747+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) can be influenced by harmful or irrelevant context, which can significantly harm model performance on downstream tasks. This motivates principled designs in which LLM systems include built-in mechanisms to guard against such "garbage in, garbage out" scenarios. We propose a novel approach to limit the degree to which harmful context can degrade model performance. First, we define a baseline "safe" behavior for the model -- the model's performance given no context at all (zero-shot). Next, we apply distribution-free risk control (DFRC) to control the extent to which the user-provided context can decay performance below this safe zero-shot baseline. We achieve this by leveraging dynamic early exit prediction, ignoring later attention heads that attend the most to the unsafe inputs. Finally, we propose modifications to DFRC that allow it to both control risk for harmful inputs \textit{and} leverage performance and efficiency gains on helpful inputs. We present both theoretical and empirical results across 9 tasks spanning in-context learning and open-ended question answering, showing that our approach can effectively control risk for harmful context and simultaneously achieve substantial computational efficiency gains with helpful context.

Figures

Figures reproduced from arXiv: 2510.02480 by Andrea Wynn, Anqi Liu, Charith Peris, Daniel Khashabi, Eric Nalisnick, Metod Jazbec, Rinat Khaziev.

Figure 2
Figure 2. Figure 2: With an early-exit LLM, we execute every layer until our confidence exceeds the λ threshold, after which we directly make a pre￾diction from the intermediate layer. yˆλ :=    arg max k∈Y p1 (k|x, c) if C1 ≥ λ, arg max k∈Y p2 (k|x, c) else if C2 ≥ λ, . . . . . . arg max k∈Y pL (k|x, c) otherwise. (1) Moreover, we denote a context set of Nc demonstrations as c = {(xi , yi)} Nc i=1. Lastly, a d… view at source ↗
Figure 3
Figure 3. Figure 3: Some choices of λ thresholds can both attain performance gains from correct demonstra￾tions and control overthinking from incorrect demonstrations. The highlighted regions show where λ values exist such that we lose no more than 5% of the accuracy gains from correct demon￾strations while still doing better than the full model given incorrect demonstrations. 3 IN-CONTEXT LEARNING RISK CONTROL VIA EARLY-EXIT… view at source ↗
Figure 4
Figure 4. Figure 4: We show the distribution of our ICL loss on the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Empirical risk vs the user-specified risk level [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: We demonstrate that our risk transformation approach enables much greater efficiency [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: We show that across many of our models and datasets, the risk is non-monotonic in [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: We show that across all tasks and models, our risk transformation approach is less con [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Some choices of λ thresholds can both attain performance gains from correct demon￾strations and control overthinking from incorrect demonstrations. We show the robustness of our approach to different selections of λ by adding error bars that further restrict the choices of λ. Col￾lecting more i.i.d. samples from the dataset will simply yield narrower error bars and thus more choices of λ. 18 [PITH_FULL_IM… view at source ↗
Figure 10
Figure 10. Figure 10: Empirical risk vs the user-specified risk level [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Empirical risk vs the user-specified risk level [PITH_FULL_IMAGE:figures/full_fig_p020_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Empirical risk vs the user-specified risk level [PITH_FULL_IMAGE:figures/full_fig_p020_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Empirical risk vs the user-specified risk level [PITH_FULL_IMAGE:figures/full_fig_p021_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Empirical risk vs the user-specified risk level [PITH_FULL_IMAGE:figures/full_fig_p021_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Empirical risk vs the user-specified risk level [PITH_FULL_IMAGE:figures/full_fig_p022_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Empirical risk vs the user-specified risk level [PITH_FULL_IMAGE:figures/full_fig_p022_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Overthinking occurs across widely varying datasets - demonstrating that, if given incor [PITH_FULL_IMAGE:figures/full_fig_p023_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: We find that calibration is necessary to stabilize accuracy and confidence across layers [PITH_FULL_IMAGE:figures/full_fig_p023_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: The top row shows λ vs risk for different measures of confidence for the TweetEval-Hate dataset with the LayerSkip LLaMA 3 model, showing that different measures of confidence can impact the way that λ-thresholds on confidence affect risk (both with loss-clipping and true relative loss). The risk control plots for all models on TweetEval-Hate are shown in the last three plots; there is no significant impa… view at source ↗
Figure 20
Figure 20. Figure 20: We empirically find that the models become less overconfident and more accurate in [PITH_FULL_IMAGE:figures/full_fig_p026_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Here, we show the accuracy of predictions from each layer of the model. We show [PITH_FULL_IMAGE:figures/full_fig_p026_21.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Where and When to Commit: Candidate-Aware Decoding for Diffusion Language Models

    cs.CL 2026-07 conditional novelty 6.0

    Candidate-aware early exit (CVC) plus block-local acceleration (BWEC) give training-free DLM speedups of 2–18× within 2 pp of full-decode accuracy under one frozen hyperparameter set.

Reference graph

Works this paper leans on

55 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [2]

    Angelopoulos, Stephen Bates, Emmanuel J

    Anastasios N. Angelopoulos, Stephen Bates, Emmanuel J. Candès, Michael I. Jordan, and Lihua Lei. Learn then test: Calibrating predictive algorithms to achieve risk control, 2022. URL https://arxiv.org/abs/2110.01052

    Pith/arXiv arXiv 2022

  2. [3]

    Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster

    Anastasios N. Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster. Conformal risk control, 2023. URL https://arxiv.org/abs/2208.02814

    Pith/arXiv arXiv 2023

  3. [4]

    Bowman, Ethan Perez, Roger Baker Grosse, and David Duvenaud

    Cem Anil, Esin Durmus, Nina Panickssery, Mrinank Sharma, Joe Benton, Sandipan Kundu, Joshua Batson, Meg Tong, Jesse Mu, Daniel Ford, Fracesco Mosconi, Rajashree Agrawal, Rylan Schaeffer, Naomi Bashkansky, Samuel Svenningsen, Mike Lambert, Ansh Radhakrishnan, Carson Denison, Evan J Hubinger, Yuntao Bai, Trenton Bricken, Timothy Maxwell, Nicholas Schiefer, ...

    2024

  4. [5]

    Mortensen, David Yarowsky, Hale Sirin, and Daniel Khashabi

    Niyati Bafna, Tianjian Li, Kenton Murray, David R. Mortensen, David Yarowsky, Hale Sirin, and Daniel Khashabi. The translation barrier hypothesis: Multilingual generation with large language models suffers from implicit translation failure. arXiv preprint arXiv:2506.22724, 2025. URL https://arxiv.org/abs/2506.22724

    arXiv 2025

  5. [6]

    Tweeteval: Unified benchmark and comparative evaluation for tweet classification, 2020

    Francesco Barbieri, Jose Camacho-Collados, Leonardo Neves, and Luis Espinosa-Anke. Tweeteval: Unified benchmark and comparative evaluation for tweet classification, 2020. URL https://arxiv.org/abs/2010.12421

    Pith/arXiv arXiv 2020

  6. [7]

    Distribution-free, risk-controlling prediction sets

    Stephen Bates, Anastasios Angelopoulos, Lihua Lei, Jitendra Malik, and Michael Jordan. Distribution-free, risk-controlling prediction sets. Journal of the ACM (JACM), 2021 a

    2021

  7. [8]

    Distribution-free, risk-controlling prediction sets

    Stephen Bates, Anastasios Angelopoulos, Lihua Lei, Jitendra Malik, and Michael Jordan. Distribution-free, risk-controlling prediction sets. J. ACM, 68 0 (6), September 2021 b . ISSN 0004-5411. doi:10.1145/3478535. URL https://doi.org/10.1145/3478535

    doi:10.1145/3478535 2021

  8. [9]

    On hoeffding's inequalities

    Vidmantas Bentkus. On hoeffding's inequalities. Annals of probability, 2004

    2004

  9. [10]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    Pith/arXiv arXiv 2020

  10. [11]

    Antonello

    Emily Cheng and Richard J. Antonello. Evidence from fmri supports a two-phase abstraction process in language models, 2024. URL https://arxiv.org/abs/2409.05771

    arXiv 2024

  11. [12]

    Bridging information-theoretic and geometric compression in language models, 2023

    Emily Cheng, Corentin Kervadec, and Marco Baroni. Bridging information-theoretic and geometric compression in language models, 2023. URL https://arxiv.org/abs/2310.13620

    Pith/arXiv arXiv 2023

  12. [13]

    Emergence of a high-dimensional abstraction phase in language transformers, 2025

    Emily Cheng, Diego Doimo, Corentin Kervadec, Iuri Macocco, Jade Yu, Alessandro Laio, and Marco Baroni. Emergence of a high-dimensional abstraction phase in language transformers, 2025. URL https://arxiv.org/abs/2405.15471

    Pith/arXiv arXiv 2025

  13. [14]

    Human-interpretable adversarial prompt attack on large language models with situational context, 2024

    Nilanjana Das, Edward Raff, and Manas Gaur. Human-interpretable adversarial prompt attack on large language models with situational context, 2024. URL https://arxiv.org/abs/2407.14644

    Pith/arXiv arXiv 2024

  14. [15]

    Ranking a stream of news

    Gianna Del corso, Antonio Gulli, and Francesco Romani. Ranking a stream of news. pp.\ 97--106, 01 2005. doi:10.1145/1060745.1060764

    arXiv 2005

  15. [16]

    Class-conditional conformal prediction with many classes

    Tiffany Ding, Anastasios Angelopoulos, Stephen Bates, Michael Jordan, and Ryan J Tibshirani. Class-conditional conformal prediction with many classes. Advances in neural information processing systems, 36: 0 64555--64576, 2023

    2023

  16. [17]

    A survey on in-context learning, 2024

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. A survey on in-context learning, 2024. URL https://arxiv.org/abs/2301.00234

    Pith/arXiv arXiv 2024

  17. [18]

    Depth-adaptive transformer, 2020

    Maha Elbayad, Jiatao Gu, Edouard Grave, and Michael Auli. Depth-adaptive transformer, 2020. URL https://arxiv.org/abs/1910.10073

    Pith/arXiv arXiv 2020

  18. [19]

    Layerskip: Enabling early exit inference and self-speculative decoding

    Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed Aly, Beidi Chen, and Carole-Jean Wu. Layerskip: Enabling early exit inference and self-speculative decoding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics...

    doi:10.18653/v1/2024.acl-long.681 2024

  19. [20]

    The llama 3 herd of models, 2024

    Aaron Grattafiori et al. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783

    Pith/arXiv arXiv 2024

  20. [21]

    Llama 2: Open foundation and fine-tuned chat models, 2023

    Hugo Touvron et al. Llama 2: Open foundation and fine-tuned chat models, 2023. URL https://arxiv.org/abs/2307.09288

    Pith/arXiv arXiv 2023

  21. [22]

    ICL Ciphers: Quantifying "Learning'' in In-Context Learning via Substitution Ciphers

    Zhouxiang Fang, Aayush Mishra, Muhan Gao, Anqi Liu, and Daniel Khashabi. ICL Ciphers: Quantifying "Learning'' in In-Context Learning via Substitution Ciphers . In Conference on Empirical Methods in Natural Language Processing EMNLP , 2025. URL https://arxiv.org/abs/2504.19395

    Pith/arXiv arXiv 2025

  22. [23]

    Making pre-trained language models better few-shot learners

    Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), p...

    doi:10.18653/v1/2021.acl-long.295 2021

  23. [24]

    Overthinking the truth: Understanding how language models process false demonstrations, 2024

    Danny Halawi, Jean-Stanislas Denain, and Jacob Steinhardt. Overthinking the truth: Understanding how language models process false demonstrations, 2024. URL https://arxiv.org/abs/2307.09476

    Pith/arXiv arXiv 2024

  24. [25]

    Dynamic neural networks: A survey

    Yizeng Han, Gao Huang, Shiji Song, Le Yang, Honghui Wang, and Yulin Wang. Dynamic neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 44 0 (11): 0 7436--7456, 2021

    2021

  25. [26]

    Multi-scale dense networks for resource efficient image classification

    Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, and Kilian Weinberger. Multi-scale dense networks for resource efficient image classification. International Conference on Learning Representations, 2018

    2018

  26. [27]

    Towards anytime classification in early-exit architectures by enforcing conditional monotonicity

    Metod Jazbec, James Allingham, Dan Zhang, and Eric Nalisnick. Towards anytime classification in early-exit architectures by enforcing conditional monotonicity. Advances in Neural Information Processing Systems, 36: 0 56138--56168, 2023

    2023

  27. [28]

    Fast yet safe: Early-exiting with risk control

    Metod Jazbec, Alexander Timans, Tin Had z i Veljkovi \'c , Kaspar Sakmann, Dan Zhang, Christian Andersson Naesseth, and Eric Nalisnick. Fast yet safe: Early-exiting with risk control. Advances in Neural Information Processing Systems, 37: 0 129825--129854, 2024

    2024

  28. [29]

    Shallow-deep networks: Understanding and mitigating network overthinking

    Yigitcan Kaya, Sanghyun Hong, and Tudor Dumitras. Shallow-deep networks: Understanding and mitigating network overthinking. In International conference on machine learning, pp.\ 3301--3310. PMLR, 2019

    2019

  29. [30]

    Large language models are zero-shot reasoners, 2023

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners, 2023. URL https://arxiv.org/abs/2205.11916

    Pith/arXiv arXiv 2023

  30. [31]

    A comprehensive review on sentiment analysis: Tasks, approaches and applications, 2023

    Sudhanshu Kumar, Partha Pratim Roy, Debi Prosad Dogra, and Byung-Gyu Kim. A comprehensive review on sentiment analysis: Tasks, approaches and applications, 2023. URL https://arxiv.org/abs/2311.11250

    arXiv 2023

  31. [32]

    Learning question classifiers

    Xin Li and Dan Roth. Learning question classifiers. In COLING 2002: The 19th International Conference on Computational Linguistics , 2002. URL https://aclanthology.org/C02-1150/

    2002

  32. [33]

    An open source data contamination report for large language models, 2024

    Yucheng Li, Frank Guerin, and Chenghua Lin. An open source data contamination report for large language models, 2024. URL https://arxiv.org/abs/2310.17589

    Pith/arXiv arXiv 2024

  33. [34]

    Automatic and universal prompt injection attacks against large language models, 2024

    Xiaogeng Liu, Zhiyuan Yu, Yizhe Zhang, Ning Zhang, and Chaowei Xiao. Automatic and universal prompt injection attacks against large language models, 2024. URL https://arxiv.org/abs/2403.04957

    Pith/arXiv arXiv 2024

  34. [35]

    Insights into llm long-context failures: When transformers know but don't tell

    Taiming Lu, Muhan Gao, Kuai Yu, Adam Byerly, and Daniel Khashabi. Insights into llm long-context failures: When transformers know but don't tell. In Conference on Empirical Methods in Natural Language Processing EMNLP - Findings , 2024. URL https://arxiv.org/abs/2406.14673

    Pith/arXiv arXiv 2024

  35. [36]

    Good debt or bad debt: Detecting semantic orientations in economic texts, 2013

    Pekka Malo, Ankur Sinha, Pyry Takala, Pekka Korhonen, and Jyrki Wallenius. Good debt or bad debt: Detecting semantic orientations in economic texts, 2013. URL https://arxiv.org/abs/1307.5336

    Pith/arXiv arXiv 2013

  36. [37]

    First align, then predict: Understanding the cross-lingual ability of multilingual bert, 2021

    Benjamin Muller, Yanai Elazar, Benoît Sagot, and Djamé Seddah. First align, then predict: Understanding the cross-lingual ability of multilingual bert, 2021. URL https://arxiv.org/abs/2101.11109

    Pith/arXiv arXiv 2021

  37. [38]

    What in-context learning "learns" in-context: Disentangling task recognition and task learning, 2023

    Jane Pan, Tianyu Gao, Howard Chen, and Danqi Chen. What in-context learning "learns" in-context: Disentangling task recognition and task learning, 2023. URL https://arxiv.org/abs/2305.09731

    Pith/arXiv arXiv 2023

  38. [39]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 2019. URL https://openai.com/blog/better-language-models/

    2019

  39. [40]

    Tran, Yi Tay, and Donald Metzler

    Tal Schuster, Adam Fisch, Jai Gupta, Mostafa Dehghani, Dara Bahri, Vinh Q. Tran, Yi Tay, and Donald Metzler. Confident adaptive language modeling, 2022. URL https://arxiv.org/abs/2207.07061

    Pith/arXiv arXiv 2022

  40. [41]

    Do multilingual llms think in english?, 2025

    Lisa Schut, Yarin Gal, and Sebastian Farquhar. Do multilingual llms think in english?, 2025. URL https://arxiv.org/abs/2502.15603

    Pith/arXiv arXiv 2025

  41. [42]

    Manning, Andrew Ng, and Christopher Potts

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In David Yarowsky, Timothy Baldwin, Anna Korhonen, Karen Livescu, and Steven Bethard (eds.), Proceedings of the 2013 Conference on Empirical Methods in Natural Langu...

    2013

  42. [43]

    Brown, Adam Santoro, Aditya Gupta, Adri\` a Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W

    Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adri\` a Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Am...

    Pith/arXiv arXiv 2023

  43. [44]

    Branchynet: Fast inference via early exiting from deep neural networks

    Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd international conference on pattern recognition (ICPR), pp.\ 2464--2469. IEEE, 2016

    2016

  44. [45]

    Do llamas work in english? on the latent language of multilingual transformers, 2024

    Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. Do llamas work in english? on the latent language of multilingual transformers, 2024. URL https://arxiv.org/abs/2402.10588

    Pith/arXiv arXiv 2024

  45. [46]

    L ink P rompt: Natural and universal adversarial attacks on prompt-based language models

    Yue Xu and Wenjie Wang. L ink P rompt: Natural and universal adversarial attacks on prompt-based language models. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp.\ 6473--6486, Mex...

    work page doi:10.18653/v1/2024.naacl-long.360 2024

  46. [47]

    S-eval: Towards automated and comprehensive safety evaluation for large language models, 2025

    Xiaohan Yuan, Jinfeng Li, Dongxia Wang, Yuefeng Chen, Xiaofeng Mao, Longtao Huang, Jialuo Chen, Hui Xue, Xiaoxia Liu, Wenhai Wang, Kui Ren, and Jingyi Wang. S-eval: Towards automated and comprehensive safety evaluation for large language models, 2025. URL https://arxiv.org/abs/2405.14191

    Pith/arXiv arXiv 2025

  47. [48]

    Safetybench: Evaluating the safety of large language models, 2024

    Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang. Safetybench: Evaluating the safety of large language models, 2024. URL https://arxiv.org/abs/2309.07045

    Pith/arXiv arXiv 2024

  48. [49]

    Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh

    Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models, 2021. URL https://arxiv.org/abs/2102.09690

    Pith/arXiv arXiv 2021

  49. [50]

    Bert loses patience: Fast and robust inference with early exit

    Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, and Furu Wei. Bert loses patience: Fast and robust inference with early exit. Advances in Neural Information Processing Systems, 2020

    2020

  50. [51]

    Zollo, Todd Morrill, Zhun Deng, Jake C

    Thomas P. Zollo, Todd Morrill, Zhun Deng, Jake C. Snell, Toniann Pitassi, and Richard Zemel. Prompt risk control: A rigorous framework for responsible deployment of large language models, 2024. URL https://arxiv.org/abs/2311.13628

    Pith/arXiv arXiv 2024

  51. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  52. [53]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  53. [54]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  54. [55]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    arXiv

  55. [56]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    2026