Pith. sign in

REVIEW 6 cited by

Depth-Adaptive Transformer

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1910.10073 v4 pith:O6V6VIEB submitted 2019-10-22 cs.CL cs.LG

Maha Elbayad , Jiatao Gu , Edouard Grave , Michael Auli This is my paper · ORCID
classification cs.CLcs.LG
keywords computationdifferentlayerstransformermodelssequencewellaccuracy
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

State of the art sequence-to-sequence models for large scale tasks perform a fixed number of computations for each input sequence regardless of whether it is easy or hard to process. In this paper, we train Transformer models which can make output predictions at different stages of the network and we investigate different ways to predict how much computation is required for a particular sequence. Unlike dynamic computation in Universal Transformers, which applies the same set of layers iteratively, we apply different layers at every step to adjust both the amount of computation as well as the model capacity. On IWSLT German-English translation our approach matches the accuracy of a well tuned baseline Transformer while using less than a quarter of the decoder layers.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adaptive Depth in Looped Transformers: Diagnosing Learned Halting Gates and Trajectory Readouts

    cs.LG 2026-07 conditional novelty 6.0 of 10

    In looped transformers, halting-gate failures come mainly from how gate training reshapes the trajectory; fixed-prior depth supervision plus simple confidence readouts yields better accuracy per unit of compute.

  2. Controlling the Risk of Corrupted Contexts for Language Models via Early-Exiting

    cs.AI 2025-10 conditional novelty 6.0 of 10

    An early-exit rule with a zero-shot fallback, calibrated by Learn-then-Test risk control, keeps the average loss from corrupted in-context demonstrations under a preset bound.

  3. System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcuts

    cs.CL 2025-05 conditional novelty 6.0 of 10

    System-1.5 Reasoning lets LLMs reason in latent space with early exits and step-skipping, matching chain-of-thought accuracy at over 20x speedup on GSM8K and StrategyQA.

  4. Extracting Visual Facts from Intermediate Layers for Mitigating Hallucinations in Multimodal Large Language Models

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Selecting the intermediate layer where image-conditioned and text-only predictions diverge most, and adding that layer's contrastive visual signal back to the final logits, reduces object hallucinations in four large ...

  5. Learning to Skip the Middle Layers of Transformers

    cs.LG 2025-06 conditional novelty 5.0 of 10

    A gated 'skip the middle' Transformer fails to beat fewer-layer dense baselines on the cross-entropy/FLOPs trade-off at 12 layers and 10B tokens.

  6. Change of Thought: Adaptive Test-Time Computation

    cs.LG 2025-07 reject novelty 4.0 of 10

    A transformer layer that iteratively refines its attention matrix to a fixed point is claimed to improve accuracy with no extra parameters, but the benchmark evidence is not reproducible.

Pith tools