REVIEW 3 major objections 4 minor 35 references
Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus
T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read The paper claims that in hybrid linear-attention LLMs, massive activations are organized by full-attention placement: spikes form right before full-attention layers and merge into plateaus as attention density grows, recovering the…
desk verdict A solid empirical mapping of per-token activation peaks in hybrid linear attention LLMs, but the 'massive activation' label is not yet backed by sparsity evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central instrument is attention-sink-guided tracing: for each input the paper computes a consensus sink token from attention probabilities averaged over full-attention layers and heads, then follows that token's maximum absolute hidden-state entry across depth. Two derived metrics, the sink-spike alignment rate and the inter-spike retention score, convert the trajectories into quantitative claims. The explanatory machinery is a cancellation-timing lifecycle: a pre-attention layer writes a large signed outlier, the token acts as an attention sink during full attention, and a prompt opposite-signed update cancels the outlier to form a sharp spike; delayed cancellation leaves the outlier elevated through the intervening layers, forming a plateau.
What would settle it
Compute the sink-spike alignment metric in a hybrid model after masking or removing the first token so the first position is no longer a candidate attention sink. If the pre-attention spike disappears or scatters to random layers, the PAS pattern is an artifact of the chosen single-token anchor; if it stays at the same pre-full-attention layer carried by another token, the architecture-aligned claim is supported.
Extended reading notes
Core claim
Massive activations in hybrid linear-attention LLMs are not scattered outliers but form a layerwise pattern set by full-attention placement. Specifically, the maximum absolute hidden-state activation of a consensus attention-sink token spikes immediately before every full-attention layer, giving pre-attention spikes; as full attention becomes denser, these spikes persist through the intervening linear-attention layers as inter-spike plateaus; and at the pure full-attention limit the two morphologies merge into the stable, plateau-like MA profile already known in full-attention transformers. The paper reports this recurrence across five linear attention backbones, six hybridization configurations, five input domains, and open-source hybrid models from 1.2B to 397B parameters. Controlled pretraining of gated-delta-rule hybrids shows that both morphologies emerge early during training, that gating the full-attention output strongly attenuates their magnitudes without deleting their layerwise organization, and that removing the linear mixer's own output gates gives only a modest amplification. Mechanistically, the paper attributes the pattern to a shared write-sink-cancel lifecycle whose outcome depends on cancellation timing: prompt cancellation localizes a spike, delayed cancellation sustains a plateau, and the full-attention limit is the endpoint of progressively deferred cancellation.
Load-bearing premise
The load-bearing premise is that a single consensus sink token, usually the first token, can stand in for all massive activations when tracing their layerwise organization; if that anchor is not representative, the pre-attention-spike and inter-spike-plateau pattern may be an artifact of where the analysis looks.
Editorial extensions
If this is right
- Massive-activation positions can be read off the layer schedule: every full-attention layer's predecessor is a spike site, regardless of input domain or linear-attention backbone.
- The spike-to-plateau transition is monotone in full-attention density: the inter-spike retention score rises from sparse to dense hybrids across every architecture-scale pair evaluated.
- Output gating of full attention suppresses the size of spikes and plateaus but does not remove their organization, while deleting the linear mixer's gates has only a modest effect, so full attention is the primary organizer of MA dynamics.
- PAS and ISP are learned structures, not checkpoint artifacts: they appear early in controlled pretraining and consolidate as training proceeds.
- The full-attention MA morphology is the same write-sink-cancel lifecycle seen at the long-delay endpoint, so hybrid and full-attention massive activations form one continuum of outlier persistence.
Reading between the lines
- If the paper is right, one could deliberately choose the full-attention schedule to place massive-activation spikes where a downstream quantizer or pruner can handle them, or where they best support attention-sink behavior.
- A testable extension would be to perturb residual-stream scaling or gating to shorten or lengthen cancellation delay and see whether spikes turn into plateaus and back without moving their layerwise position; that would confirm cancellation timing as the causal knob.
- The single-sink tracing result invites a check under long-context or streaming inference, where the fixed-size recurrent state may change sink behavior; the paper does not claim its morphology persists there.
- One could also test whether spike positions predict task-relevant behavior by ablating the pre-attention spike at a specific layer and measuring changes in retrieval, reasoning, or next-token accuracy.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents the first systematic study of massive activations (MAs) in layer-interleaved hybrid linear attention (HLA) LLMs. It introduces an attention-sink-guided tracing procedure that follows a consensus sink token across depth and documents two layerwise morphologies: pre-attention spikes (PAS), in which activation magnitude peaks immediately before full attention layers, and inter-spike plateaus (ISP), in which elevated activation persists between successive PAS. Using a controlled suite of five linear attention architectures across hybridization ratios and model scales, twelve large-scale pretrained hybrid checkpoints, five input domains, and controlled pretraining of GDN-based models, the paper argues that full attention placement is the primary organizer of MA morphology. It also proposes a write–sink–cancel lifecycle account, with PAS attributed to localized cancellation and ISP to delayed cancellation, and claims that increasing full attention density recovers the stable MA morphology of full attention LLMs.
Significance. If the morphological claims hold, this is a useful descriptive contribution to the interpretability of hybrid linear attention architectures, which are increasingly deployed in production LLMs. The paper's strengths include a controlled model suite, evaluation across diverse large-scale checkpoints and domains, quantitative metrics with paired bootstrap confidence intervals, explicit token and layer controls, and publicly available analysis code and checkpoints. The metrics are descriptive and involve no fitted parameters, so circularity is not a concern. The main risk is that the central object being measured is a token-level activation envelope rather than the sparse-outlier phenomenon that defines MAs; this is fixable with additional sparsity-oriented measurements and does not invalidate the descriptive morphology itself.
major comments (3)
- [Section 3.3, Eq. (6); Section 3.4, Eq. (7)] The central empirical signal is m(l)_{x,t} = ||X(l)_{x,t,:}||_inf, the maximum absolute activation across features for one token. This is a token-level envelope, not a measure of the sparse-outlier property that defines MAs (entries exceeding typical activations by orders of magnitude and sparse across features). A pre-attention peak in m could arise if the entire hidden vector of the sink token is uniformly scaled up before a full attention layer, with no single outlier feature. The fixed-coordinate verification in Section 4.1 and Appendix E.1 traces one coordinate in one 1.3B GDN model, and the cross-model analyses in Appendix E.2 again report layerwise dominant-feature magnitudes rather than sparsity or outlierness. The paper should add a sparsity/outlierness measure (e.g., fraction of features exceeding a threshold, max-to-median ratio, top-k energy share) applied to the same suite and show that PAS and ISP are carried by a small number of outlier features. Without this, the claims are about activation magnitude spikes, not yet about massive activations as defined in the paper.
- [Section 3.2, Eq. (5)] All quantitative results (Align, ISR, and all trajectory plots) trace the single consensus sink token selected by the argmax in Eq. (5). The controls in Appendix B.2 and B.5 show that non-sink tokens behave differently and that the first token is a common sink, but they do not provide metrics-level evidence that the PAS–ISP organization is insensitive to the specific choice among the top-k consensus sinks. I ask for Align and ISR recomputed with an average over the top-k consensus sinks and with an 'any of the top-k sinks' variant. If the authors believe the existing controls are sufficient, the text should state explicitly why a single argmax anchor cannot distort the measured layerwise morphology.
- [Section 3.4; Section 4.2] The claim that increasing full attention density 'recovers' the stable MA morphology of full attention LLMs is supported visually by trajectory plots and by the monotonic increase of ISR, but no quantitative similarity measure compares the layerwise HLA profiles at each hybridization ratio with the full attention baseline profile. Adding a simple quantitative comparison (e.g., Pearson correlation or normalized L2 distance between the first-token magnitude profiles, evaluated across inputs and domains) would substantiate the continuum claim that PAS, ISP, and full attention MAs form a single progression.
minor comments (4)
- [Tables 2 and 7] Bootstrap confidence intervals are reported in Appendix B.5 but not in the main tables; adding CI columns or a compact uncertainty summary would make the main quantitative claims easier to assess.
- [Abstract and Section 7] The abstract states that code is available at a repository, while the Reproducibility Statement says the authors 'will release' the analysis code; please clarify the actual availability status.
- [Figure 2] The two panels of Figure 2 use bubble size and color in ways that are not defined in the caption; a short legend or explicit description of the bubble encoding would improve readability.
- [Section 3.2] The text says 'attention distribution determines only which token is tracked; whether and how strongly that token exhibits an MA remains determined by its activation magnitude.' This is a useful clarification, but it highlights exactly the gap addressed by Major Comment 1: the measured magnitude is not tested for the sparse-outlier property.
Circularity Check
No significant circularity: PAS/ISP are descriptive statistics over independent activation traces, not fit-derived predictions.
full rationale
The paper's central empirical claims (PAS, ISP, and their density-dependent merging) are measured directly from hidden states via Align (Eq. 6) and ISR (Eq. 7); neither metric contains fitted parameters, and neither is defined from the claim it supports. The sink-guided tracing in Eq. 5 selects a token from attention probabilities, but the spike location is then read off from that token's max-absolute activation; the selection does not force layer f-1 to be the argmax over the preceding block, and the non-sink and random-layer controls in Appendix B.5 provide a genuine contrast. The controlled pretraining in Section 3.6 and Appendix D manipulates full-attention placement and output gating while holding other factors fixed, so the observed gating asymmetry is an empirical outcome rather than a consequence of the definitions. The write-sink-cancel account in Section 4 is an interpretive overlay on fixed-coordinate traces (Appendix E.1), not the source of the observations. Prior MA/attention-sink work, including self-citations by the authors, is used as background and as a borrowed analysis framework, not as a load-bearing uniqueness or existence theorem; no quantity is fitted and then reported as a prediction. The main legitimate concern is measurement validity: m = ||X||_inf is a token-level envelope and by itself does not certify sparse outlierness relative to typical activations. That is an operationalization concern, not circularity, because the paper does not define 'massive activation' as 'max-abs activation of a chosen sink token' and does not use the MA definition to construct the metric.
Assumptions & free parameters
assumptions (4)
- domain assumption Massive activations in full attention LLMs are coupled to attention sinks and can be detected by magnitude ranking.
- domain assumption The residual stream follows a pre-normalized Mixer+FFN block structure with full attention layers indexed by I_FA (Equations 1-3).
- ad hoc to paper The consensus sink token defined by Equation 5, using averaged attention mass over full attention layers, is a stable and representative anchor for tracing MA dynamics across depth.
- ad hoc to paper The maximum absolute activation of a single token, m(l)_x,t = ||X(l)_x,t,:||_inf, is a sufficient scalar to represent the presence and magnitude of massive activations at a layer.
Cite this review
Pith. "Pith review of Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus." pith.science (2026). https://pith.science/paper/EL2FETRT
@misc{pith2026260812149,
author = {Pith},
title = {Pith review of: Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus},
year = {2026},
howpublished = {\url{https://pith.science/paper/EL2FETRT}},
note = {Machine review of arXiv:2608.12149}
}
read the original abstract
We present the first systematic study of Massive activations (MAs) in layer-interleaved HLA LLMs and uncover two architecture-aligned morphologies: MAs consistently spike immediately before full attention layers, forming pre-attention spikes (PAS), and can persist through intervening linear attention layers, giving rise to inter-spike plateaus (ISP). As full attention becomes denser, successive PAS become increasingly connected through ISP, ultimately recovering the stable MA morphology of full attention LLMs. We establish the recurrence of this organization across five linear attention architectures, six hybridization configurations, five data domains, and representative open-source hybrid models spanning 1.2B to 397B total parameters. Controlled pretraining of GDN-based hybrids at scales up to 1.3B shows that both morphologies emerge early and respond asymmetrically to output gating: full attention output gating strongly attenuates their absolute magnitudes without eliminating their layerwise organization, whereas removing GDN gates yields comparatively modest amplification. Mechanistically, our systematic-outlier analysis supports a shared lifecycle account governed by the timing of MA cancellation. PAS follows a localized write-sink-cancel process, while the extended persistence of ISP is consistent with delayed cancellation. At the full attention limit, this account recovers the stable MA morphology characteristic of full attention LLMs. Our code is available at https://github.com/StartluxLabs/Massive-Activations-HLA.
Figures
Figures from the paper (32 more)
Reference graph
Works this paper leans on
-
[1]
Systematic outliers in large language models.arXiv preprint arXiv:2502.06415,
Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. Systematic outliers in large language models.arXiv preprint arXiv:2502.06415,
-
[4]
Qwen3-coder-next technical report.arXiv preprint arXiv:2603.00729,
Ruisheng Cao, Mouxiang Chen, Jiawei Chen, Zeyu Cui, Yunlong Feng, Binyuan Hui, Yuheng Jing, Kaixin Li, Mingze Li, Junyang Lin, et al. Qwen3-coder-next technical report.arXiv preprint arXiv:2603.00729,
-
[5]
We compare it with two token controls: the first token and the average over eligible tokens outside the top three consensus sinks. We also include a random-layer baseline, which selects uniformly among the 11 layers in each preceding linear attention block and therefore has an expected alignment rate of1/11 = 9.1%. Because the binary alignment rate record...
work page 2025
-
[8]
No language left behind: Scaling human-centered machine translation.arXiv preprint arXiv:2207.04672,
Marta R Costa-Juss`a, James Cross, Onur C ¸ elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. No language left behind: Scaling human-centered machine translation.arXiv preprint arXiv:2207.04672,
-
[10]
The zamba2 suite: Technical report.arXiv preprint arXiv:2411.15242,
Paolo Glorioso, Quentin Anthony, Yury Tokpanov, Anna Golubeva, Vasudev Shyam, James Whit- tington, Jonathan Pilault, and Beren Millidge. The zamba2 suite: Technical report.arXiv preprint arXiv:2411.15242,
-
[12]
Summer is warm. Winter is cold
The standard model develops a pronounced PAS. Adding an output gate to the full attention layer markedly attenuates the spike without eliminating it, whereas removing the native GDN output gates moderately increases its magnitude. layer 12 produces a pronounced spike and layer 20 yields the strongest PAS. Within these controlled configurations, deeper ful...
work page 2026
-
[13]
Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. Codesearchnet challenge: Evaluating the state of semantic code search.arXiv preprint arXiv:1909.09436,
arXiv 1909
-
[15]
Jamba: A hybrid transformer- mamba language model.arXiv preprint arXiv:2403.19887,
Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedigos, Erez Safahi, Shaked Meirom, Yonatan Belinkov, Shai Shalev-Shwartz, et al. Jamba: A hybrid transformer- mamba language model.arXiv preprint arXiv:2403.19887,
Show all 35 references
-
[16]
Openceres: When open information extrac- tion meets the semi-structured web
Colin Lockard, Prashant Shiralkar, and Xin Luna Dong. Openceres: When open information extrac- tion meets the semi-structured web. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, ...
2019
-
[17]
Pointer sentinel mixture models.arXiv preprint arXiv:1609.07843,
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models.arXiv preprint arXiv:1609.07843,
-
[18]
A single layer to ex- plain them all: Understanding massive activations in large language models.arXiv preprint arXiv:2605.08504,
Zeru Shi, Zhenting Wang, Fan Yang, Qifan Wang, and Ruixiang Tang. A single layer to ex- plain them all: Understanding massive activations in large language models.arXiv preprint arXiv:2605.08504,
-
[19]
Kvsink: Understanding and enhancing the preservation of attention sinks in kv cache quantization for llms
Zunhai Su and Kehong Yuan. Kvsink: Understanding and enhancing the preservation of attention sinks in kv cache quantization for llms. InSecond Conference on Language Modeling (COLM 2025),
2025
-
[20]
Rotatekv: Accurate and robust 2-bit kv cache quantization for llms via outlier-aware adaptive rotations
12 Preprint Zunhai Su, Zhe Chen, Wang Shen, Hanyu Wei, Linge Li, Huangqi Yu, and Kehong Yuan. Rotatekv: Accurate and robust 2-bit kv cache quantization for llms via outlier-aware adaptive rotations. In Proceedings of the Thirty-Fourth International Joint Conference on Artifici...
2026 arXiv
-
[21]
The spike, the sparse and the sink: Anatomy of massive activations and attention sinks.arXiv preprint arXiv:2603.05498,
Shangwen Sun, Alfredo Canziani, Yann LeCun, and Jiachen Zhu. The spike, the sparse and the sink: Anatomy of massive activations and attention sinks.arXiv preprint arXiv:2603.05498,
-
[22]
Retentive network: A successor to transformer for large language models.arXiv preprint arXiv:2307.08621,
Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, and Furu Wei. Retentive network: A successor to transformer for large language models.arXiv preprint arXiv:2307.08621,
-
[23]
Kimi linear: An expressive, efficient attention architecture
Kimi Team, Yu Zhang, Zongyu Lin, Xingcheng Yao, Jiaxi Hu, Fanqing Meng, Chengyin Liu, Xin Men, Songlin Yang, Zhiyuan Li, et al. Kimi linear: An expressive, efficient attention architecture. arXiv preprint arXiv:2510.26692, 2025a. Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao,...
-
[25]
Efficient streaming language models with attention sinks
13 Preprint Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. InInternational Conference on Learning Representations, volume 2024, pp. 21875–21895,
2024
-
[26]
Exploring layer-wise information effectiveness for post- training quantization in small language models
He Xiao, Qingyao Yang, Dirui Xie, Wendong Xu, Zunhai Su, Runming Yang, Haobo Liu, Wenyong Zhou, Zhengwu Liu, and Ngai Wong. Exploring layer-wise information effectiveness for post- training quantization in small language models. InFindings of the Association for Computational ...
2026
-
[27]
Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025a
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025a. Songlin Yang and Yu Zhang. FLA: A triton-based library for hardware-efficient im- plemen...
-
[28]
Songlin Yang, Bailin Wang, Yikang Shen, Rameswar Panda, and Yoon Kim
GitHub repository. Songlin Yang, Bailin Wang, Yikang Shen, Rameswar Panda, and Yoon Kim. Gated linear attention transformers with hardware-efficient training.arXiv preprint arXiv:2312.06635,
-
[29]
Gated delta networks: Improving mamba2 with delta rule
Songlin Yang, Jan Kautz, and Ali Hatamizadeh. Gated delta networks: Improving mamba2 with delta rule. InInternational Conference on Learning Representations, volume 2025, pp. 29687– 29707, 2025b. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag...
2025
-
[30]
Beyond outliers: A data-free layer-wise mixed- precision quantization approach driven by numerical and structural dual-sensitivity.Under review, 2026a
Hengyuan Zhang, Xinrong Chen, Zunhai Su, Xiao Liang, Jing Xiong, Wendong Xu, He Xiao, Chaofan Tao, Wei Zhang, Ruobing Xie, et al. Beyond outliers: A data-free layer-wise mixed- precision quantization approach driven by numerical and structural dual-sensitivity.Under review, 20...
2025
-
[31]
Summer is warm. Winter is cold
using implementations from the Flash Linear Attention library (Yang & Zhang, 2024). Within each parameter scale, all architectures and hybridization configurations share the same training corpus and optimization recipe, minimizing variation unrelated to the sequence- mixing ar...
2024
-
[32]
Pronounced MAs concentrate at attention-sink positions, particu- larly the initial token, “Summer,” and the first period
Red dashed lines mark layers immediately preceding full attention, while green dotted lines mark full attention layers. Pronounced MAs concentrate at attention-sink positions, particu- larly the initial token, “Summer,” and the first period. These tokens exhibit the same ratio...
2016
-
[34]
Summer is warm. Winter is cold
For each checkpoint, six trajectories cover the running example,“Summer is warm. Winter is cold. ”, and representative inputs from five domains: general prose from WikiText-103, scientific writing from Scientific Papers, mathematical reasoning from GSM8K, Python code from Code...
2025
-
[2017]
A systematic analysis of hybrid linear attention.arXiv preprint arXiv:2507.06457,
Dustin Wang, Rui-Jie Zhu, Steven Abreu, Yong Shan, Taylor Kergan, Yuqi Pan, Yuhong Chou, Zheng Li, Jibin Wu, Ge Zhang, et al. A systematic analysis of hybrid linear attention.arXiv preprint arXiv:2507.06457,
-
[2018]
Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,
-
[2019]
Minimax-01: Scaling foundation models with lightning attention
Aonian Li, Bangwei Gong, Bo Yang, Boji Shan, Chang Liu, Cheng Zhu, Chunhao Zhang, Congchao Guo, Da Chen, Dong Li, et al. Minimax-01: Scaling foundation models with lightning attention. arXiv preprint arXiv:2501.08313,
-
[2020]
Nemotron-h: A family of accurate and efficient hybrid mamba-transformer models.arXiv preprint arXiv:2504.03624,
Aaron Blakeman, Aarti Basant, Abhinav Khattar, Adithya Renduchintala, Akhiad Bercovich, Alek- sander Ficek, Alexis Bjorlin, Ali Taghibakhshi, Amala Sanjay Deshmukh, Ameya Sunil Maha- baleshwarkar, et al. Nemotron-h: A family of accurate and efficient hybrid mamba-transformer m...
-
[2021]
A discourse-aware attention model for abstractive summarization of long documents
Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. A discourse-aware attention model for abstractive summarization of long documents. InProceedings of the 2018 Conference of the North American Chapter of the Asso- ciation ...
2018
-
[2022]
Hidden dynamics of massive activations in transformer training.arXiv preprint arXiv:2508.03616,
Jorge Gallego-Feliciano, S Aaron McClendon, Juan Morinelli, Stavros Zervoudakis, and Antonios Saravanos. Hidden dynamics of massive activations in transformer training.arXiv preprint arXiv:2508.03616,
-
[2023]
Ruler: What’s the real context size of your long-context language models?arXiv preprint arXiv:2404.06654,
Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. Ruler: What’s the real context size of your long-context language models?arXiv preprint arXiv:2404.06654,
-
[2024]
Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752,
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752,
-
[2025]
Just read twice: closing the recall gap for recurrent language models.arXiv preprint arXiv:2407.05483,
Simran Arora, Aman Timalsina, Aaryan Singhal, Benjamin Spector, Sabri Eyuboglu, Xinyi Zhao, Ashish Rao, Atri Rudra, and Christopher R´e. Just read twice: closing the recall gap for recurrent language models.arXiv preprint arXiv:2407.05483,
-
[2026]
Think you have solved question answering? try arc, the ai2 reasoning challenge
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457,
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.