Pith. sign in

REVIEW 3 major objections 5 minor 71 references

A Survey of Early Exit Deep Neural Networks in NLP

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey maps how early exit networks adapt NLP inference to input difficulty, claiming gains in speed, efficiency, and adversarial resistance.

desk verdict A useful NLP-focused early-exit survey whose main flaw is a load-bearing but unqualified robustness claim that needs fixing before the paper is used as a reliable map. read the letter →

arxiv 2501.07670 v1 pith:CRPTJYZ6 submitted 2025-01-13 cs.LG cs.CL

Divya Jyoti Bajpai , Manjesh Kumar Hanawal This is my paper · ORCID
classification cs.LGcs.CL
keywords earlyexitdeepneuralnetworksnaturallanguageprocessingadaptiveinferencemodelefficiencyBERTaccelerationdomainadaptationtransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Early exit deep networks attach extra classifiers to intermediate layers so that easy inputs can be classified before they travel through the whole network. This survey argues that this simple mechanism is a practical, input-adaptive way to cut inference cost in NLP, where real datasets mix easy and hard examples. The paper's contribution is a map of the design space: how exit classifiers are trained (separately, jointly, or alternately), what counts as confidence (maximum probability, entropy, prediction consistency, learned distributions, representation similarity, or ensembles), how thresholds are set (static or learned), and where the method has been applied, from text classification to translation, summarization, and image captioning. If the map is right, a practitioner can pick an exit strategy from a known set of trade-offs instead of reinventing one, and researchers can see which open problems—overconfidence, risky early predictions, and exit placement in large models—are still unsolved.

What carries the argument

The load-bearing mechanism is the exit criterion: at layer $i$, an internal classifier produces a distribution $\hat{P}_i(c)$ over classes, and inference stops when a confidence score $C_i$ crosses a threshold $\alpha$. The paper's organizing axis is what $C_i$ should be: the maximum class probability, the entropy of $\hat{P}_i$, patience-based agreement across several consecutive classifiers, a learned distribution $p(i|x)$ over exit layers, the similarity between consecutive hidden representations, or a weighted ensemble of past classifiers. The second half of the machinery is the training loss, $L = \sum_{i=1}^N w_i L_i$ for joint training versus the two-stage separate procedure that freezes the backbone before training exits, with weights $w_i$ chosen by cost, layer index, or learned sigmoid parameters. This pair—confidence definition and training scheme—determines the accuracy-latency trade-off that the whole survey is organized around.

What would settle it

Re-implement the representative methods (single-classifier confidence, patience-based, distribution-based, and similarity-based exits) on the same backbone and benchmark, with matched compute, and compare accuracy-latency curves and adversarial accuracy. If early exit never produces lower latency at matched accuracy, or if the adversarial-resistance advantage disappears once a single-classifier confidence exit is used, the survey's central claims about the benefits and structure of the field would be undercut.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is an organizational insight: the many early exit systems in NLP are variations on one shared architecture—a backbone network with internal classifiers and a stopping rule—and the apparent variety reduces to a few design choices. The paper groups those choices into training strategies (separate fine-tuning of the backbone followed by frozen-backbone exit training; joint optimization; alternating optimization), confidence criteria (single-classifier confidence, patience-based consistency, learned exit distributions, hidden-representation similarity, and ensembles), and threshold mechanisms (static validation-tuned thresholds versus dynamic bandit-learned thresholds). It then reads the applications—BERT-style classification, summarization, sequence labeling, translation, and vision-language models—as instances of those choices, and identifies the open problems: fake confidence on the wrong class, risk from incorrect early predictions, and the parameter explosion of adding a large classifier to every layer of a big language model.

Load-bearing premise

The survey assumes that the cited papers are a representative sample of early exit NLP work and that the advantages it lists—especially greater resistance to adversarial attacks—hold generally rather than only in the specific settings where they were measured.

Editorial extensions

If this is right

  • A practitioner can treat the choice of confidence metric and training strategy as independent design decisions, and expect the accuracy-latency trade-off to shift predictably with the threshold setting.
  • EEDNNs enable edge-cloud co-inference: easy samples exit on the device, hard samples go to the cloud, and bandit-based threshold learning can adapt the split to a new domain without labels.
  • For generative tasks such as summarization and captioning, token-level and similarity-based exits reduce computation, and self-speculative decoding can reuse early layers as a draft model.
  • The survey's robustness discussion ties adversarial resistance to criteria that aggregate multiple classifiers, so patience-based and ensemble methods are the ones to use when inputs may be attacked.
  • Overconfidence and 'fake confidence' samples are identified as the main unresolved accuracy risk, motivating calibration or out-of-distribution detection inside exit classifiers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the survey's taxonomy implies a combinatorics of untested combinations—for example, patience-based exiting with joint training and a bandit-tuned threshold has not been systematically compared, and could dominate current methods if the benefits are additive.
  • Editorial inference: the parameter-explosion argument for large models suggests that exit classifiers must shrink to linear probes or hash-based assignments as models grow, so the field's center of gravity may shift from richer confidence criteria to cheaper ones.
  • Editorial inference: the robustness claim rests mainly on one patience-based method, so a reader should not assume that every early exit variant inherits that property; a controlled comparison of robustness across confidence metrics would settle it.
  • Editorial inference: the survey treats domain generalization as a threshold or feature adaptation problem, which suggests a testable extension: combining threshold adaptation with feature-invariance learning could handle shifts that change both confidence distributions and representations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper surveys early exit (EE) mechanisms for deep neural networks with a focus on NLP. It motivates EE through inference latency, resource constraints, and the overthinking problem; lists seven claimed advantages; presents a taxonomy of training methods (separate vs. joint), confidence metrics (individual, patience-based, distribution-based, similarity-based, ensemble), and threshold choices; and reviews applications in text classification, NLI, summarization, sequence labeling, translation, and vision-language tasks. It then discusses domain generalization, additional applications (OOD detection, reinforcement learning, self-speculative decoding, distributed inference), and future directions, including a parameter-accounting argument for exit placement in large models. The stated goal is to provide a comprehensive and reliable map of early-exit methods for NLP.

Significance. If the survey's characterizations are accurate, it fills a genuine gap: existing surveys cover dynamic neural networks or edge-cloud inference more broadly, but a focused NLP-oriented early-exit survey is missing. The paper's taxonomy of confidence metrics and training strategies is a useful organizing device, and the parameter calculation in §7.1 makes a concrete design point about the cost of attaching classifiers to large models. The paper is descriptive and does not provide experiments or code, so its value depends on the precision and representativeness of its claims. The most valuable parts are the concrete method summaries and the identification of open problems such as overconfidence and exit placement. However, the survey's reliability is weakened by at least one overgeneralized benefit claim and a few unsupported statements about common practice, which need correction before the paper can serve as the dependable reference it promises to be.

major comments (3)
  1. [Abstract and §2, Advantage 5] The sentence 'EE models demonstrate increased robustness against adversarial attacks' appears in the abstract and is repeated as Advantage 5 in §2, with Zhou et al. (2020) as the only citation. However, the survey itself later explains that PABEE's robustness comes from patience-based agreement across multiple classifiers (§3.3) and states that it 'does not rely on a single classifier to decide exiting' (§4.1). Individual-confidence, entropy-threshold, similarity-based, and distribution-based exits are placed in different categories in §3.3, and no evidence is cited that the PABEE result transfers to those families. As written, the claim can mislead practitioners into expecting adversarial robustness from any early-exit architecture. Please qualify the claim to the methods actually evaluated in the cited work (e.g., patience/ensemble-based exits) or supply additional evidence for each exit family.
  2. [§3.2, training methods] The sentence 'Most of the methods replace wi = i with a justification that more emphasis should be given to deeper layers' is a specific empirical claim about the literature but is given without citation. It is also ambiguous whether 'wi = i' means the loss weight is set to the layer index or something else. Since the loss definitions in this subsection (Eq. (1) and the summed loss L = Σ wi Li) are the formal basis for the training taxonomy, this unsupported statement weakens the survey's accuracy. Please either provide supporting references for the claimed common practice, remove the claim, or rephrase it as an observation limited to the cited methods.
  3. [§5, Domain Generalization in EE Models] The Domain Generalization section is almost entirely a summary of CeeBERT (Bajpai and Hanawal, 2024b) and DAdEE (Bajpai and Hanawal, 2024c), both from the same group. The opening claim that EEDNNs inherit the generalization capability of DNNs is asserted without evidence, and no external domain-adaptation or threshold-adaptation work is compared. For a survey whose stated aim is to be comprehensive and to 'highlight the interconnections between different subareas,' this imbalance should be addressed: either add and compare other methods known in the literature, or explicitly state that these are the only works the authors are aware of on this subarea.
minor comments (5)
  1. [§1, Introduction] There are several typos in the introduction, including 'adpation' (for 'adaptation') and 'at the at shallower layers' (for 'at the shallower layers'). These should be corrected in a final revision.
  2. [§2 and §7.3, Figure 2] Figure 2 is described as plotting average confidence values over the true class, but the text does not specify whether this is a post-hoc analysis on labeled data or something observable at inference time. If true labels are used to define 'fake confidence,' this should be stated explicitly, and the claim that the figure can help detect OOD samples should be qualified accordingly.
  3. [§3.3 and §4.1] The description of PALBERT's exit distribution in §3.3 and the sentence in §4.1 about 'generalized geometric distribution on the of exiting' contain unclear or incomplete wording. Please rewrite these passages for precision.
  4. [References] Some references are formatted inconsistently or incompletely: for example, the ViT entry is listed as 'Dosovitskiy Alexey' and the survey by Rahmath P et al. is cited in running text without standard author formatting. Please normalize all entries against the target journal's style.
  5. [§7.2] The typo 'EEDNNS' appears in the first sentence, and the phrase 'it has very less insights' should read 'it provides very little insight.' These are minor but should be cleaned up.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: this is a survey with no predictive derivation, and its self-citations are descriptive summaries rather than load-bearing evidence.

full rationale

This paper is a literature survey, not a derivation or empirical study, so the circularity failure modes involving fitted parameters, predicted quantities reducing to inputs, or ansatz smuggled in via citation do not apply. The central claim is to provide a comprehensive overview of early-exit methods for NLP; that claim rests on coverage and characterization of the cited literature, not on any equation or construction that re-imports its conclusion. The paper frequently cites the authors' own CeeBERT, DAdEE, SplitEE, UCBEE, and UEEUCB, but these citations are descriptive summaries of those methods and applications; they are not used as the sole justification for a theorem, do not forbid alternative approaches through a self-cited uniqueness result, and are not load-bearing in the sense of making the survey's organizational conclusions equivalent to its inputs. Even the priority statement 'CeeBERT (Bajpai and Hanawal, 2024b) is the first work that tries to solve the issue of domain adaptation in EEDNNs' is a factual priority claim, not a circular derivation. The unqualified robustness advantage in the abstract and Section 2, supported only by Zhou et al. (2020), is an overgeneralization and a correctness risk, but overgeneralizing an external result is not circularity. No specific circular step can be exhibited, so the score is 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The survey introduces no free parameters or invented entities. It rests on the assumptions that its literature selection is representative and that benefits demonstrated in specific papers apply generally.

assumptions (2)
  • domain assumption The selected papers cover the full scope of early exit in NLP.
    The survey's usefulness depends on the completeness of its literature coverage; gaps would make its comprehensiveness claim hollow.
  • domain assumption Claims of EE benefits (e.g., robustness) from specific cited papers transfer to the general early exit framework.
    Section 2 generalizes results from a few model-task combinations (e.g., Zhou et al. 2020) to all EEDNNs without a meta-analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Survey of Early Exit Deep Neural Networks in NLP." pith.science (2026). https://pith.science/paper/CRPTJYZ6

@misc{pith2026250107670,
  author       = {Pith},
  title        = {Pith review of: A Survey of Early Exit Deep Neural Networks in NLP},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CRPTJYZ6}},
  note         = {Machine review of arXiv:2501.07670}
}
read the original abstract

Deep Neural Networks (DNNs) have grown increasingly large in size to achieve state of the art performance across a wide range of tasks. However, their high computational requirements make them less suitable for resource-constrained applications. Also, real-world datasets often consist of a mixture of easy and complex samples, necessitating adaptive inference mechanisms that account for sample difficulty. Early exit strategies offer a promising solution by enabling adaptive inference, where simpler samples are classified using the initial layers of the DNN, thereby accelerating the overall inference process. By attaching classifiers at different layers, early exit methods not only reduce inference latency but also improve the model robustness against adversarial attacks. This paper presents a comprehensive survey of early exit methods and their applications in NLP.

Figures

Figures reproduced from arXiv: 2501.07670 by the authors.

Figure 1
Figure 1. Difference between the DNN and EEDNN. (Kaya et al., 2019; Michel et al., 2019; Zhou et al., 2020). This unnecessary deep processing can overfit irrelevant features, resulting in poor gen￾eralization and wasted computation. Additionally, overthinking contributes to the models’ suscepti￾bility to adversarial attacks (Zhou et al., 2020). To address these issues, recent research has fo￾cused on accelerating DNN inferenc… view at source ↗
Figure 2
Figure 2. The figure shows the average of the confi [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Separate training vs Joint Training the intermediate classifiers. Additionally, attach￾ing classifiers at multiple layers introduces more parameters to the model, which raises the ques￾tion of how to strategically place these exits across the network to avoid excessive model size, particu￾larly for very large models (Zhu, 2021; Zhou et al., 2020; Wang et al., 2019; Xin et al., 2021). 3) Generalization of EEDNNs: Whi… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Inference methods: 1) Max Probability: confidence is the maximum output of an individual classifier. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

71 extracted references · 31 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Dosovitskiy Alexey. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv: 2010.11929

    arXiv 2020

  4. [4]

    Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jing Jin, Xin Jiang, Qun Liu, Michael Lyu, and Irwin King. 2020. Binarybert: Pushing the limit of bert quantization. arXiv preprint arXiv:2012.15701

    arXiv 2020

  5. [5]

    Divya J Bajpai, Vivek K Trivedi, Sohan L Yadav, and Manjesh K Hanawal. 2023. Splitee: Early exit in deep neural networks with split computing. arXiv preprint arXiv:2309.09195

    arXiv 2023

  6. [6]

    Divya Jyoti Bajpai and Manjesh Kumar Hanawal. 2024 a . Capeen: Image captioning with early exits and knowledge distillation. arXiv preprint arXiv:2410.04433

    arXiv 2024

  7. [7]

    Divya Jyoti Bajpai and Manjesh Kumar Hanawal. 2024 b . Ceebert: Cross-domain inference in early exit bert. In To appear in proceedings of the 62nd conference of the Association for computational linguistics: Findings Volume

    work page 2024

  8. [8]

    Divya Jyoti Bajpai and Manjesh Kumar Hanawal. 2024 c . Dadee: Unsupervised domain adaptation in early exit plms. arXiv preprint arXiv:2410.04424

    work page Pith review arXiv 2024

Show all 71 references
  1. [9]

    Divya Jyoti Bajpai, Aastha Jaiswal, and Manjesh Kumar Hanawal. 2024. I-splitee: Image classification in split computing dnns with early exits. arXiv preprint arXiv:2401.10541

    2024 arXiv

  2. [10]

    Nikita Balagansky and Daniil Gavrilov. 2022. Palbert: Teaching albert to ponder. Advances in Neural Information Processing Systems, 35:14002--14012

    2022

  3. [11]

    Ankur Bapna, Naveen Arivazhagan, and Orhan Firat. 2020. Controlling computation versus quality for neural sequence models. arXiv preprint arXiv:2002.07106

    2020 arXiv

  4. [12]

    Joud Chataoui, Mark Coates, et al. 2023. Jointly-learned exit and inference for a dynamic neural network. In The Twelfth International Conference on Learning Representations

    2023

  5. [13]

    Xin Dai, Xiangnan Kong, and Tian Guo. 2020. Epnet: Learning to exit with flexible multi-branch network. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 235--244

    2020

  6. [14]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  7. [15]

    Maha Elbayad, Jiatao Gu, Edouard Grave, and Michael Auli. 2019. Depth-adaptive transformer. arXiv preprint arXiv:1910.10073

    2019 arXiv

  8. [16]

    Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, et al. 2024. Layer skip: Enabling early exit inference and self-speculative decoding. arXiv preprint arXiv:2404.16710

    2024 arXiv

  9. [17]

    Angela Fan, Edouard Grave, and Armand Joulin. 2019. Reducing transformer depth on demand with structured dropout. arXiv preprint arXiv:1909.11556

    2019 arXiv

  10. [18]

    Biyi Fang, Xiao Zeng, Faen Zhang, Hui Xu, and Mi Zhang. 2020. Flexdnn: Input-adaptive on-device deep learning for efficient mobile vision. In 2020 IEEE/ACM Symposium on Edge Computing (SEC), pages 84--95. IEEE

    2020

  11. [19]

    Zhengcong Fei, Xu Yan, Shuhui Wang, and Qi Tian. 2022. Deecap: Dynamic early exiting for efficient image captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12216--12226

    2022

  12. [20]

    Xiangxiang Gao, Yue Liu, Tao Huang, and Zhongyu Hou. 2023. Pf-berxit: Early exiting for bert with parameter-efficient fine-tuning and flexible early exiting strategy. Neurocomputing, 558:126690

    2023

  13. [21]

    Shijie Geng, Peng Gao, Zuohui Fu, and Yongfeng Zhang. 2021. Romebert: Robust training of multi-exit bert. arXiv preprint arXiv:2101.09755

    2021 arXiv

  14. [22]

    Yizeng Han, Gao Huang, Shiji Song, Le Yang, Honghui Wang, and Yulin Wang. 2021. Dynamic neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):7436--7456

    2021

  15. [23]

    Manjesh K Hanawal, Avinash Bhardwaj, et al. 2022. Unsupervised early exit in dnns with multiple exits. arXiv preprint arXiv:2209.09480

    2022 arXiv

  16. [24]

    Jianing He, Qi Zhang, Hongyun Zhang, Xuanjing Huang, Usman Naseem, and Duoqian Miao. 2024. Cosee: Consistency-oriented signal-based early exiting via calibrated sample weighting mechanism. arXiv preprint arXiv:2412.13236

    2024 arXiv

  17. [25]

    Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Multi-scale dense networks for resource efficient image classification. arXiv preprint arXiv:1703.09844

    2017 arXiv

  18. [26]

    Metod Jazbec, Alexander Timans, Tin Had z i Veljkovi \'c , Kaspar Sakmann, Dan Zhang, Christian A Naesseth, and Eric Nalisnick. 2024. Fast yet safe: Early-exiting with risk control. arXiv preprint arXiv:2405.20915

    2024 arXiv

  19. [27]

    Yixin Ji, Jikai Wang, Juntao Li, Qiang Chen, Wenliang Chen, and Min Zhang. 2023. Early exit with disentangled representation and equiangular tight frame. In Findings of the Association for Computational Linguistics: ACL 2023, pages 14128--14142

    2023

  20. [28]

    Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2019. Tinybert: Distilling bert for natural language understanding. arXiv preprint arXiv:1909.10351

    2019 arXiv

  21. [29]

    Weiyu Ju, Wei Bao, Liming Ge, and Dong Yuan. 2021. Dynamic early exit scheduling for deep neural network inference through contextual bandits. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 823--832

    2021

  22. [30]

    Yigitcan Kaya, Sanghyun Hong, and Tudor Dumitras. 2019. Shallow-deep networks: Understanding and mitigating network overthinking. In International conference on machine learning, pages 3301--3310. PMLR

    2019

  23. [31]

    Selfxit: An unsupervised early exit mechanism for deep neural networks

    Hossein KhademSohi, Mohammadamin Abedi, Yani Ioannou, Steve Drew, Pooyan Jamshidi, and Hadi Hemmati. Selfxit: An unsupervised early exit mechanism for deep neural networks. Transactions on Machine Learning Research

  24. [32]

    Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. 2021. I-bert: Integer-only bert quantization. In International conference on machine learning, pages 5506--5518. PMLR

    2021

  25. [33]

    Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942

    2019 arXiv

  26. [34]

    Stefanos Laskaridis, Stylianos I Venieris, Mario Almeida, Ilias Leontiadis, and Nicholas D Lane. 2020. Spinn: synergistic progressive inference of neural networks over device and cloud. In Proceedings of the 26th annual international conference on mobile computing and networki...

    2020

  27. [35]

    En Li, Liekang Zeng, Zhi Zhou, and Xu Chen. 2019 a . Edge ai: On-demand accelerating deep neural network inference via edge computing. IEEE Transactions on Wireless Communications, 19(1):447--457

    2019

  28. [36]

    Hao Li, Hong Zhang, Xiaojuan Qi, Ruigang Yang, and Gao Huang. 2019 b . Improved techniques for training adaptive deep networks. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1891--1900

    2019

  29. [37]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730--19742. PMLR

    2023

  30. [38]

    Kaiyuan Liao, Yi Zhang, Xuancheng Ren, Qi Su, Xu Sun, and Bin He. 2021. A global past-future early exit method for accelerating inference of pre-trained language models. In Proceedings of the 2021 conference of the north american chapter of the association for computational li...

    2021

  31. [39]

    Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Haotang Deng, and Qi Ju. 2020. Fastbert: a self-distilling bert with adaptive inference time. arXiv preprint arXiv:2004.02178

    2020 arXiv

  32. [40]

    Xiangyang Liu, Tianxiang Sun, Junliang He, Lingling Wu, Xinyu Zhang, Hao Jiang, Zhao Cao, Xuanjing Huang, and Xipeng Qiu. 2021. https://arxiv.org/abs/2110.07038 Towards efficient NLP: A standard evaluation and A strong baseline

    2021 arXiv

  33. [41]

    Yoshitomo Matsubara, Marco Levorato, and Francesco Restuccia. 2022. Split computing and early exiting for deep learning applications: Survey and research challenges. ACM Computing Surveys, 55(5):1--30

    2022

  34. [42]

    Ruijie Miao, Yihan Yan, Xinshuo Yao, and Tong Yang. 2024. An efficient inference framework for early-exit large language models. arXiv preprint arXiv:2407.20272

    2024 arXiv

  35. [43]

    Paul Michel, Omer Levy, and Graham Neubig. 2019. Are sixteen heads really better than one? Advances in neural information processing systems, 32

    2019

  36. [44]

    Roberto G Pacheco, Divya J Bajpai, Mark Shifrin, Rodrigo S Couto, Daniel S Menasch \'e , Manjesh K Hanawal, and Miguel Elias M Campista. 2024. Ucbee: A multi armed bandit approach for early-exit in neural networks. IEEE Transactions on Network and Service Management

    2024

  37. [45]

    Mary Phuong and Christoph H Lampert. 2019. Distillation-based training for multi-exit architectures. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1355--1364

    2019

  38. [46]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  39. [47]

    Haseena Rahmath P, Vishal Srivastava, Kuldeep Chaurasia, Roberto G Pacheco, and Rodrigo S Couto. 2024. Early-exit deep neural network-a comprehensive survey. ACM Computing Surveys, 57(3):1--37

    2024

  40. [48]

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108

    2019 arXiv

  41. [49]

    Tianxiang Sun, Xiangyang Liu, Wei Zhu, Zhichao Geng, Lingling Wu, Yilong He, Yuan Ni, Guotong Xie, Xuanjing Huang, and Xipeng Qiu. 2022. A simple hash-based early exiting approach for language understanding and generation. arXiv preprint arXiv:2203.01670

    2022 arXiv

  42. [50]

    Tianxiang Sun, Yunhua Zhou, Xiangyang Liu, Xinyu Zhang, Hao Jiang, Zhao Cao, Xuanjing Huang, and Xipeng Qiu. 2021. Early exiting with ensemble internal classifiers. arXiv preprint arXiv:2105.13792

    2021 arXiv

  43. [51]

    Peng Tang, Pengkai Zhu, Tian Li, Srikar Appalaraju, Vijay Mahadevan, and R Manmatha. 2023 a . Deed: Dynamic early exit on decoder for accelerating encoder-decoder transformer models. arXiv preprint arXiv:2311.08623

    2023 arXiv

  44. [52]

    Shengkun Tang, Yaqing Wang, Caiwen Ding, Yi Liang, Yao Li, and Dongkuan Xu. 2023 b . Deediff: Dynamic uncertainty-aware early exiting for accelerating diffusion model generation. arXiv preprint arXiv:2309.17074

    2023 arXiv

  45. [53]

    Shengkun Tang, Yaqing Wang, Zhenglun Kong, Tianchi Zhang, Yao Li, Caiwen Ding, Yanzhi Wang, Yi Liang, and Dongkuan Xu. 2023 c . You need multiple exiting: Dynamic early exiting for accelerating unified vision language model. In Proceedings of the IEEE/CVF Conference on Compute...

    2023

  46. [54]

    Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. 2016. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd International Conference on Pattern Recognition (ICPR), pages 2464--2469. IEEE

    2016

  47. [55]

    Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. 2017. Distributed deep neural networks over the cloud, the edge and end devices. In 2017 IEEE 37th international conference on distributed computing systems (ICDCS), pages 328--339. IEEE

    2017

  48. [56]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  49. [57]

    Chih-Shuo Tsai, Ying-Hong Chan, and Yao-Chung Fan. 2022. Hierarchical cache transformer: Dynamic early exit for language translation. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1--9. IEEE

    2022

  50. [58]

    Meiqi Wang, Jianqiao Mo, Jun Lin, Zhongfeng Wang, and Li Du. 2019. Dynexit: A dynamic early-exit strategy for deep residual networks. In 2019 IEEE International Workshop on Signal Processing Systems (SiPS), pages 178--183. IEEE

    2019

  51. [59]

    Yue Wang, Lijun Wu, Juntao Li, Xiaobo Liang, and Min Zhang. 2023. Are the bert family zero-shot learners? a study on their potential and limitations. Artificial Intelligence, page 103953

    2023

  52. [60]

    Zizhao Wang, Wei Bao, Dong Yuan, Liming Ge, Nguyen H Tran, and Albert Zomaya. 2020. Accelerating on-device dnn inference during service outage through scheduling early exit. Computer Communications, 162:69--82

    2020

  53. [61]

    Maciej Wo czyk, Bartosz W \'o jcik, Klaudia Ba azy, Igor T Podolak, Jacek Tabor, Marek \'S mieja, and Tomasz Trzcinski. 2021. Zero time waste: Recycling predictions in early exit neural networks. Advances in Neural Information Processing Systems, 34:2516--2528

    2021

  54. [62]

    Ji Xin, Raphael Tang, Jaejun Lee, Yaoliang Yu, and Jimmy Lin. 2020. Deebert: Dynamic early exiting for accelerating bert inference. arXiv preprint arXiv:2004.12993

    2020 arXiv

  55. [63]

    Ji Xin, Raphael Tang, Yaoliang Yu, and Jimmy Lin. 2021. Berxit: Early exiting for bert with better fine-tuning and extension to regression. In Proceedings of the 16th conference of the European chapter of the association for computational linguistics: Main Volume, pages 91--104

    2021

  56. [64]

    Canwen Xu and Julian McAuley. 2022. A survey on dynamic neural networks for natural language processing. arXiv preprint arXiv:2202.07101

    2022 arXiv

  57. [65]

    Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, 32

    2019

  58. [66]

    Jun Zhang, Jue Wang, Huan Li, Lidan Shou, Ke Chen, Gang Chen, and Sharad Mehrotra. 2023. Draft & verify: Lossless large language model acceleration via self-speculative decoding. arXiv preprint arXiv:2309.08168

    2023 arXiv

  59. [67]

    Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, and Qun Liu. 2020. Ternarybert: Distillation-aware ultra-low bit bert. arXiv preprint arXiv:2009.12812

    2020 arXiv

  60. [68]

    Zhen Zhang, Wei Zhu, Jinfan Zhang, Peng Wang, Rize Jin, and Tae-Sun Chung. 2022. Pcee-bert: Accelerating bert inference via patient and confident early exiting. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 327--338

    2022

  61. [69]

    Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, and Furu Wei. 2020. Bert loses patience: Fast and robust inference with early exit. Advances in Neural Information Processing Systems, 33:18330--18341

    2020

  62. [70]

    Yunhua Zhou, Jianqiang Yang, Pengyu Wang, and Xipeng Qiu. 2023. Two birds one stone: Dynamic ensemble for ood intent classification. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10659--10673

    2023

  63. [71]

    Wei Zhu. 2021. Leebert: Learned early exit for bert with cross-level optimization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pag...

    2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.