The Benchmark Illusion: Pruned LLMs Can Pass Multiple Choice but Fail to Answer

Jiayang Liu; Lu Sun; Rui Wen; Tianshuo Cong; Zesheng Xu; Zheng Li

The Benchmark Illusion: Pruned LLMs Can Pass Multiple Choice but Fail to Answer

T0 review · 0 major / 3 minor · reviewed 2026-06-27 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pruned LLMs can still pick the right multiple-choice answer while failing to generate it in open output.

desk verdict Pruning creates a generation vs recognition gap on identical questions, with answers demoted rather than lost. read the letter →

arxiv 2606.17609 v1 pith:OYRGHDZ6 submitted 2026-06-16 cs.CL

Rui Wen , Lu Sun , Jiayang Liu , Zesheng Xu , Tianshuo Cong , Zheng Li This is my paper · ORCID

classification cs.CL

keywords prunedLLMsmultiplechoiceevaluationopengenerationmodelcompressionbenchmarkillusionWandapruningQArecognitionversus

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tracks identical multilingual QA questions before and after pruning and finds that high-sparsity methods, especially Wanda, produce many cases where greedy generation yields the wrong answer yet multiple-choice scoring still selects the correct one. In these recognition-only failures the correct answer is typically demoted in the output distribution rather than erased, so it often reappears under beam search, sampling, or a single in-context example. The result is a benchmark illusion: standard multiple-choice tests can overstate how usable a compressed model actually is for tasks that require producing the answer. The authors conclude that compressed models must be evaluated on what they generate, not only on what they can recognize.

What carries the argument

Tracking performance on the exact same questions before and after pruning, comparing greedy open generation against multiple-choice selection to isolate demotion versus erasure of answers.

What would settle it

Run the identical pruned model on the same QA items and measure whether the rank of the correct answer in the full output distribution drops after pruning while the multiple-choice accuracy stays high.

Watch

Extended reading notes

Core claim

Under high-sparsity pruning, models frequently fail in greedy open generation on the same questions they still answer correctly under multiple-choice scoring; the correct answer is usually not removed from the model but demoted in rank, and can be recovered by changing the decoding strategy or adding one in-context example.

Load-bearing premise

That comparing the same questions before and after pruning cleanly isolates the effect of pruning on generation versus recognition.

Editorial extensions

If this is right

Multiple-choice benchmarks can overstate the practical performance of heavily pruned models.
Pruned models should be tested on open generation rather than recognition alone.
The correct answer often remains recoverable with beam search or light prompting after pruning.
Wanda pruning at high sparsity produces more of these recognition-only errors than other methods examined.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The pattern suggests pruning primarily disrupts output ranking rather than removing stored knowledge.
Real-world deployment of compressed models may need generation-specific checks even when MCQ scores look acceptable.
Similar demotion effects could appear under other compression techniques that alter token probabilities.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The main thing here is that high-sparsity pruning, especially Wanda, makes models fail at greedy open generation on questions they still answer correctly in multiple choice. The correct answer is usually still present but ranked lower, and it surfaces again with beam search, sampling, or one in-context example.

The paper tracks the exact same multilingual QA questions before and after pruning. That pre/post design is a direct way to measure what changes, and the recovery tests support the demotion claim over outright erasure.

It does a clean job showing why multiple-choice scores can overstate how usable a compressed model actually is for real generation tasks. The multilingual angle adds some breadth without overcomplicating the core comparison.

The soft spot is that the abstract gives almost no experimental details on datasets, controls, or how decoding was handled across conditions. Without those, it's hard to judge whether the gap is as general as claimed or tied to specific choices in the multilingual setup. If the full paper has ablations and error bars, the result holds up better.

This is useful for anyone working on model compression or evaluation. It flags a real blind spot in current benchmarks. The work deserves peer review so the methods can be checked and the scope clarified.

Referee Report

0 major / 3 minor

Summary. The paper claims that high-sparsity pruning (especially Wanda) induces a 'benchmark illusion' in LLMs: models fail greedy open-ended generation on multilingual QA questions while still selecting the correct answer under multiple-choice scoring. The correct answer is typically demoted rather than erased, as it can be recovered via beam search, sampling, or one-shot ICL. The design tracks identical questions pre- and post-pruning to isolate the effect, concluding that MC benchmarks overstate usability of compressed models and that open-generation tests are needed.

Significance. If the empirical results hold, the work identifies a practically important evaluation gap for compressed LLMs. The pre/post tracking on identical questions, combined with recovery experiments distinguishing demotion from erasure, provides direct evidence that standard MC benchmarks can mask production failures. This is a clear, falsifiable contribution to the compression and evaluation literature.

minor comments (3)

[Experimental setup] The multilingual QA setup is integral to the design, but the manuscript should explicitly discuss whether language-specific tokenization or script differences could interact with pruning in ways that affect generation vs. recognition (e.g., in the experimental setup section).
[§4] Clarify the exact decoding parameters (temperature, top-p, beam width) used in the recovery experiments and whether they were held constant across all models and sparsity levels.
[Figures 2-4] Figure captions and axis labels should explicitly state the number of questions per language and the exact pruning ratios tested to allow direct replication.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive assessment of the paper's significance and the recommendation for minor revision. No specific major comments were listed in the report.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity

full rationale

The paper is a direct empirical comparison of MC vs. open-generation performance on identical questions before/after pruning, with recovery experiments via beam search, sampling, and ICL. No equations, derivations, fitted parameters, or load-bearing self-citations appear; the design isolates pruning effects through pre/post tracking without reducing any claim to its own inputs by construction. This is self-contained against external benchmarks.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Empirical observation paper with no mathematical model, free parameters, axioms, or invented entities.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The Benchmark Illusion: Pruned LLMs Can Pass Multiple Choice but Fail to Answer." pith.science (2026). https://pith.science/paper/OYRGHDZ6

@misc{pith2026260617609,
  author       = {Pith},
  title        = {Pith review of: The Benchmark Illusion: Pruned LLMs Can Pass Multiple Choice but Fail to Answer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OYRGHDZ6}},
  note         = {Machine review of arXiv:2606.17609}
}

read the original abstract

Compressing large language models reduces memory use and inference cost, but it can also create failures that standard benchmarks miss. A pruned model may still perform well on multiple-choice evaluations, yet fail to answer the same question in open generation. We ask what pruning changes: does it erase the correct answer, or does it make the answer harder to produce as the top output? We study this question with multilingual question answering, tracking the same questions before and after pruning. We find a benchmark illusion. Under high-sparsity pruning, especially Wanda, models often fail in greedy open generation while still selecting the correct answer under multiple-choice scoring. In these recognition-only errors, the answer is usually not gone, but demoted: it often reappears with beam search, sampling, or one in-context example. Overall, multiple-choice benchmarks can overstate the usability of compressed LLMs, creating an evaluation blind spot. Compressed models should be tested on what they can produce, not only on what they can recognize.

Figures

Figures reproduced from arXiv: 2606.17609 by the authors.

**Figure 1.** Many producible answers become recognition-only after pruning. The pruned model fails in greedy open generation but still selects the gold answer from candidates. The full four-way fate breakdown is shown in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

**Figure 3.** Pruning demotes gold answers rather than erasing them. On both-correct items, recognition-only answers stay near the top of the next-token distribution after pruning: the gold first token has a median rank of 3.5, which is close enough to remain reachable but no longer selected by greedy decoding. distractors-only, and 32.1% with unrelated candidates. By contrast, when the prompt includes the gold answer together w… view at source ↗

**Figure 4.** Missed answers remain reachable from the distribution. Pruning lowers greedy top-1 accuracy but barely changes whether the answer appears under broader decoding. Among items where pruned greedy decoding fails, beam-5 and 10-sample recovery is substantial for several languages but low for low resource languages. result comes from the MC-only items: when pruning breaks open generation but preserves multiplechoice co… view at source ↗

Figures from the paper (3 more)

**Figure 5.** Figure 5: The dissociation generalizes across models and datasets. For two additional model families and a second dataset (XQuAD), the recognition-only error rate remains substantial, and the production-recognition gap ∆PR is positive in every setting. Qwen3-8B has a recognition…

**Figure 7.** Figure 7: Calibration lifts both formats but leaves the gap. Mean open-generation and MC accuracy under English-only vs. balanced multilingual calibration (9 languages). pruning process exclusively on English text. If the pruning mask simply discards the “wrong” languages, then…

**Figure 8.** Figure 8: Full four-way fate of uncompressed-bothcorrect items after pruning (WANDA sp = 0.5), the breakdown summarized by [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation
cs.LG 2026-07 conditional novelty 5.0 of 10

A repetition-gated, short-to-long rollout schedule lets on-policy distillation recover pruned LLMs' generation quality with 71% fewer rollout tokens than a fixed 8192-token horizon, matching its score within two points.

Reference graph

Works this paper leans on

300 extracted references · cited by 1 Pith paper

[1]

Ahmed Salem and Yang Zhang and Mathias Humbert and Pascal Berrang and Mario Fritz and Michael Backes , title =
[2]

Inken Hagestedt and Yang Zhang and Mathias Humbert and Pascal Berrang and Haixu Tang and XiaoFeng Wang and Michael Backes , title =
[3]

Michael Backes and Mathias Humbert and Jun Pang and Yang Zhang , title =
[4]

Yang Zhang and Mathias Humbert and Tahleen Rahman and Cheng-Te Li and Jun Pang and Michael Backes , title =
[5]

Pascal Berrang and Mathias Humbert and Yang Zhang and Irina Lehmann and Roland Eils and Michael Backes , title =
[6]

Michael Backes and Pascal Berrang and Mathias Humbert and Praveen Manoharan , title =
[7]

Brendan McMahan and Sarvar Patel and Daniel Ramage and Aaron Segal and Karn Seth , title =

Keith Bonawitz and Vladimir Ivanov and Ben Kreuter and Antonio Marcedone and H. Brendan McMahan and Sarvar Patel and Daniel Ramage and Aaron Segal and Karn Seth , title =
[8]

Raphael Bost and Raluca Ada Popa and Stephen Tu and Shafi Goldwasser , title =

Show all 300 references

[9]

Nicholas Carlini and David Wagner , title =
[10]

David Evans and Yan Huang and Jonathan Katz and Lior Malka , title =
[11]

Matt Fredrikson and Eric Lantz and Somesh Jha and Simon Lin and David Page and Thomas Ristenpart , title =
[12]

Jinyuan Jia and Neil Zhenqiang Gong , title =
[13]

Apostolos Pyrgelis and Carmela Troncoso and Emiliano De Cristofaro , title =
[14]

Reza Shokri and Marco Stronati and Congzheng Song and Vitaly Shmatikov , title =
[15]

Yuxiao Dong and Jie Tang and Sen Wu and Jilei Tian and Nittiest V Chawla and Jinghai Rao and Huanhuan Cao , title =
[16]

Aditya Grover and Jure Leskovec , title =
[17]

Simon Oya and Carmela Troncoso and Fernando P
[18]

Yelena Mejova and Sofiane Abbar and Hamed Haddadi , title =
[19]

Flavio Souza and Diego de Las Casas and Vinicius Flores and SunBum Youn and Meeyoung Cha and Daniele Quercia and Virgilio Almeida , title =
[20]

Myers and Jure Leskovec , title =

Eunjoon Cho and Seth A. Myers and Jure Leskovec , title =
[21]

Beatrice Perez and Mirco Musolesi and Gianluca Stringhini , title =
[22]

Tomas Mikolov and Kai Chen and Greg Corrado and Jeffrey Dean , title =
[23]

Corrado and Jeffrey Dean , title =

Tomas Mikolov and Ilya Sutskever and Kai Chen and Greg S. Corrado and Jeffrey Dean , title =
[24]

Jun Pang and Yang Zhang , title =
[25]

Reza Shokri and Georgios Theodorakopoulos and Jean-Yves Le Boudec and Jean-Pierre Hubaux , title =
[26]

Hamilton and Jure Leskovec and Dan Jurafsky , title =

William L. Hamilton and Jure Leskovec and Dan Jurafsky , title =
[27]

Bryan Perozzi and Rami Al-Rfou and Steven Skiena , title =
[28]

Dmitry David and Oren Tsur and Ari Rappoport , title =
[29]

Romero and Brendan Meeder and Jon Kleinberg , title =

Daniel M. Romero and Brendan Meeder and Jon Kleinberg , title =
[30]

Essa , title =

Julia Deeb-Swihart and Christopher Polack and Eric Gilbert and Irfan A. Essa , title =
[31]

Raphael Ottoni and Diego Las Casas and Joao Paulo Pesce and Wagner Meira Jr and Christo Wilson and Alan Mislove and Virgilio Almeida , title =
[32]

Gummadi , title =

Parantapa Bhattacharya and Muhammad Bilal Zafar and Niloy Ganguly and Saptarshi Ghosh and Krishna P. Gummadi , title =
[33]

Kiran Garimella and Gianmarco De Francisci Morales and Aristides Gionis and Michael Mathioudakis , title =
[34]

Conover and Mark Meiss and Bruno Goncalves and Alessandro Flammini and Filippo Menczer , title =

Jacob Ratkiewicz and Michael D. Conover and Mark Meiss and Bruno Goncalves and Alessandro Flammini and Filippo Menczer , title =
[35]

Janette Lehmann and Bruno Goncalves and Jos
[36]

Salvatore Scellato and Anastasios Noulas and Cecilia Mascolo , title =
[37]

Ruth Garcia Gavilanes and Neil O'Hare and Luca Maria Aiello and Alejandro Jaimes , title =
[38]

Daniele Quercia and Luca Maria Aiello and Rossano Schifanella and Adam Davies , title =
[39]

Kevin Tang and Manohar Paluri and Fei-Fei Li and Rob Fergus and Lubomir Bourdev , title =
[40]

Shih and Dongwon Lee , title =

Jin Yea Jang and Kyungsik Han and Patrick C. Shih and Dongwon Lee , title =
[41]

Jian Tang and Meng Qu and Mingzhe Wang and Ming Zhang and Jun Yan and Qiaozhu Mei , title =
[42]

Zhao and Haitao Zheng , title =

Qingyun Liu and Shiliang Tang and Xinyi Zhang and Xiaohan Zhao and Ben Y. Zhao and Haitao Zheng , title =
[43]

Huy Pham and Cyrus Shahabi and Yan Liu , title =
[44]

Hongjian Wang and Zhenhui Li and Wang-Chien Lee , title =
[45]

Gummadi , title =

Bimal Viswanath and Alan Mislove and Meeyoung Cha and Krishna P. Gummadi , title =
[46]

Zhao , title =

Gang Wang and Manish Mohanlal and Christo Wilson and Xiao Wang and Miriam Metzger and Haitao Zheng and Ben Y. Zhao , title =
[47]

Zhao , title =

Gang Wang and Bolun Wang and Tianyi Wang and Ana Nika and Haitao Zheng and Ben Y. Zhao , title =
[48]

Julian McAuley and Jure Leskovec , title =
[49]

Manuel Egele and Gianluca Stringhini and Christopher Kruegel and Giovanni Vigna , title =
[50]

Hao Ma and Dengyong Zhou and Chao Liu and Michael R Lyu and Irwin King , title =
[51]

David Kempe and Jon Kleinberg and
[52]

Kun Liu and Evimaria Terzi , title =
[53]

Zhao , title =

Alessandra Sala and Xiaohan Zhao and Christo Wilson and Haitao Zheng and Ben Y. Zhao , title =
[54]

Prateek Mittal and Charalampos Papamanthou and Dawn Song , title =
[55]

Arvind Narayanan and Vitaly Shmatikov , title =
[56]

Shouling Ji and Weiqing Li and Prateek Mittal and Xin Hu and Raheem Beyah , title =
[57]

Mudhakar Srivatsa and Mike Hicks , title =
[58]

Lars Backstrom and Cynthia Dwork and Jon Kleinberg , title =
[59]

Qian Xiao and Rui Chen and Kian-Lee Tan , title =
[60]

Bin Zhou and Jian Pei , title =
[61]

Davide Proserpio and Sharon Goldberg and Frank McSherry , title =
[62]

Ahmad Bashir and Mark Crovella and Saikat Guha and Krishna P

Bimal Viswanath and M. Ahmad Bashir and Mark Crovella and Saikat Guha and Krishna P. Gummadi and Balachander Krishnamurthy and Alan Mislove , title =
[63]

Michael Backes and Pascal Berrang and Anne Hecksteden and Mathias Humbert and Andreas Keller and Tim Meyer , title =
[64]

Andrew McGregor and Ilya Mironov and Toniann Pitassi and Omer Reingold and Kunal Talwar and Salil Vadhan , title =
[65]

Jinyuan Jia and Binghui Wang and Le Zhang and Neil Zhenqiang Gong , title =
[66]

Shirin Nilizadeh and Apu Kapadia and Yong-Yeol Ahn , title =
[67]

Shouling Ji and Weiqing Li and Mudhakar Srivatsa and Raheem Beyah , title =
[68]

Kumar Sharad and George Danezis , title =
[69]

Gummadi and Nishanth Sastry , title =

Giridhari Venkatadri and Oana Goga and Changtao Zhong and Bimal Viswanath and Krishna P. Gummadi and Nishanth Sastry , title =
[70]

Carley , title =

Fred Morstatter and Jurgen Pfeffer and Huan Liu and Kathleen M. Carley , title =
[71]

Daniele Quercia and Rossano Schifanella and Luca Maria Aiello , title =
[72]

Vivek Kulkarni and Rami Al-Rfou and Bryan Perozzi and Steven Skiena , title =
[73]

Yu-Ru Lin and Drew Margolin and Brian Keegan and Andrea Baronchelli and David Lazer , title =
[74]

Jisun An and Ingmar Weber , title =
[75]

Ferda Ofli and Yusuf Aytar and Ingmar Weber and Raggi al Hammouri and Antonio Torralba , title =
[76]

Bichen Shi and Georgiana Ifrim and Neil Hurley , title =
[77]

Mohsen Jamali and Martin Ester , title =
[78]

Enrico Mariconti and Jeremiah Onaolapo and Syed Sharique Ahmad and Nicolas Nikiforou and Manuel Egele and Nick Nikiforakis and Gianluca Stringhini , title =
[79]

Lisa Singh and Grace Hui Yang and Micah Sherr and Andrew Hian-Cheong and Kevin Tian and Janet Zhu and Sicong Zhang , title =
[80]

Mazurek and Michael K

Lujo Bauer and Lorrie Faith Cranor and Saranga Komanduri and Michelle L. Mazurek and Michael K. Reiter and Manya Sleeper and Blase Ur , title =
[81]

T Ryan Hoens and Marina Blanton and Nitesh V Chawla , title =
[82]

Kurt Thomas and Damon McCoy and Chris Grier and Alek Kolcz and Vern Paxson , title =
[83]

Brian Thompson and Danfeng Yao , title =
[84]

Leyla Bilge and Thorsten Strufe and Davide Balzarotti and Engin Kirda , title =
[85]

Selcuk Uluagac and Raheem A

Xiaojing Liao and A. Selcuk Uluagac and Raheem A. Beyah , title =
[86]

Paul Marinescu and Chad Parry and Marjori Pomarole and Yuan Tian and Patrick Tague and Ioannis Papagiannis , title =
[87]

James Cheng and Ada Wai-chee Fu and Jia Liu , title =
[88]

Lars Backstrom and Jure Leskovec , title =
[89]

Luca Maria Aiello and Nicola Barbieri , title =
[90]

Efthymios Kouloumpis and Theresa Wilson and Johanna Moore , title =
[91]

Michael Conover and Jacob Ratkiewicz and Matthew R Francisco and Bruno Goncalves and Alessandro Flammini and Filippo Menczer , title =
[92]

Oren Tsur and Ari Rappoport , title =
[93]

Paolo Ferragina and Francesco Piccinno and Roberto Santoro , title =
[94]

Anqi Cui and Min Zhang and Yiqun Liu and Shaoping Ma and Kuo Zhang , title =
[95]

Zongyang Ma and Aixin Sun and Gao Cong , title =
[96]

Lei Yang and Tao Sun and Ming Zhang and Qiaozhu Mei , title =
[97]

Shoubin Kong and Qiaozhu Mei and Ling Feng and Fei Ye and Zhe Zhao , title =
[98]

Lisa Posch and Claudia Wagner and Philipp Singer and Markus Strohmaier , title =
[99]

Edgar Meij and Wouter Weerkamp and Maarten de Rijke , title =
[100]

Yi Chang and Jiliang Tang and Dawei Yin and Makoto Yamada and Yan Liu , title =
[101]

Su Mon Kywe and Tuan-Anh Hoang and Ee-Peng Lim and Feida Zhu , title =
[102]

Cheng-Te Li and Chien-Yuan Wang and Chien-Lin Tseng and Shou-De Lin , title =
[103]

Xiao Huang and Qingquan Song and Jundong Li and Xia Hu , title =
[104]

Yao Ma and Zhaochun Ren and Ziheng Jiang and Jiliang Tang and Dawei Yin , title =
[105]

Jiezhong Qiu and Yuxiao Dong and Hao Ma and Jian Li and Kuansan Wang and Jie Tang , title =
[106]

Suhas Ranganath and Ghazaleh Beigi and Huan Liu , title =
[107]

Meng Qu and Jian Tang and Jiawei Han , title =
[108]

Hongwei Wang and Fuzheng Zhang and Min Hou and Xing Xie and Minyi Guo and Qi Liu , title =
[109]

Ming Gao and Leihui Chen and Xiangnan He and Aoying Zhou , title =
[110]

Battista Biggio and Igino Corona and Davide Maiorca and Blaine Nelson and Nedim Srndic and Pavel Laskov and Giorgio Giacinto and Fabio Roli , title =
[111]

Samuel Yeom and Irene Giacomelli and Matt Fredrikson and Somesh Jha , title =
[112]

Nicolas Papernot and Patrick McDaniel and Arunesh Sinha and Michael Wellman , title =
[113]

Shai Avidan and Moshe Butman , title =
[114]

Yossi Adi and Carsten Baum and Moustapha Cisse and Benny Pinkas and Joseph Keshet , title =
[115]

Stoecklin and Heqing Huang and Ian Molloy , title =

Jialong Zhang and Zhongshu Gu and Jiyong Jang and Hui Wu and Marc Ph. Stoecklin and Heqing Huang and Ian Molloy , title =
[116]

Aditi Raghunathan and Jacob Steinhardt and Percy Liang , title =
[117]

Battista Biggio and Blaine Nelson and Pavel Laskov , title =
[118]

Xiaowei Huang and Marta Kwiatkowska and Sen Wang and Min Wu , title =
[119]

Shike Mei and Xiaojin Zhu , title =
[120]

Daniel Lowd and Christopher Meek , title =
[121]

Binghui Wang and Neil Zhenqiang Gong , title =
[122]

Seong Joon Oh and Max Augustin and Bernt Schiele and Mario Fritz , title =
[123]

Olga Ohrimenko and Felix Schuster and Cedric Fournet and Aastha Mehta and Sebastian Nowozin and Kapil Vaswani and Manuel Costa , title =
[124]

Sangho Lee and Ming-Wei Shih and Prasun Gera and Taesoo Kim and Hyesoon Kim and Marcus Peinado , title =
[125]

Manuel Barbosa and Bernardo Portela and Guillaume Scerri and Bogdan Warinschi , title =
[126]

Ben Fisch and Dhinakaran Vinayagamurthy and Dan Boneh and Sergey Gorbunov , title =
[127]

Rafael Pass and Elaine Shi and Florian Tram
[128]

Victor Costan and Ilia Lebedev and Srinivas Devadas , title =
[129]

Ernie Brickell and Jiangtao Li , title =
[130]

JP Aumasson and Luis Merino , title =
[131]

Raoul Strackx and Frank Piessens , title =
[132]

Sinisa Matetic and Mansoor Ahmed and Kari Kostiainen and Aritra Dhar and David Sommer and Arthur Gervais and Ari Juels and Srdjan Capkun , title =
[133]

Gunter , title =

Wenhao Wang and Guoxing Chen and Xiaorui Pan and Yinqian Zhang and XiaoFeng Wang and Vincent Bindschaedler and Haixu Tang and Carl A. Gunter , title =
[134]

Ferdinand Brasser and Urs Muller and Alexandra Dmitrienko and Kari Kostiainen and Srdjan Capkun and Ahmad-Reza Sadeghi , title =
[135]

Matt Fredrikson and Somesh Jha and Thomas Ristenpart , title =
[136]

Ian Goodfellow and Jonathon Shlens and Christian Szegedy , title =
[137]

Bo Li and Yevgeniy Vorobeychik , title =
[138]

Yevgeniy Vorobeychik and Bo Li , title =
[139]

McDaniel and Ian Goodfellow and Somesh Jha and Z

Nicolas Papernot and Patrick D. McDaniel and Ian Goodfellow and Somesh Jha and Z. Berkay Celik and Ananthram Swami , title =
[140]

McDaniel and Somesh Jha and Matt Fredrikson and Z

Nicolas Papernot and Patrick D. McDaniel and Somesh Jha and Matt Fredrikson and Z. Berkay Celik and Ananthram Swami , title =
[141]

Weilin Xu and David Evans and Yanjun Qi , title =
[142]

Asokan , title =

Jian Liu and Mika Juuti and Yao Lu and N. Asokan , title =
[143]

Yan Huang and Lior Malka and David Evans and Jonathan Katz , title =
[144]

Michael Backes and Pascal Berrang and Matthias Bieg and Roland Eils and Carl Herrmann and Mathias Humbert and Irina Lehmann , title =
[145]

Payman Mohassel and Yupeng Zhang , title =
[146]

Seong Joon Oh and Mario Fritz and Bernt Schiele , title =
[147]

Congzheng Song and Thomas Ristenpart and Vitaly Shmatikov , title =
[148]

Matthew Jagielski and Alina Oprea and Battista Biggio and Chang Liu and Cristina Nita-Rotaru and Bo Li , title =
[149]

Milad Nasr and Reza Shokri and Amir Houmansadr , title =
[150]

Xiaoyu Cao and Neil Zhenqiang Gong , title =
[151]

Hamilton and Zhitao Ying and Jure Leskovec , title =

William L. Hamilton and Zhitao Ying and Jure Leskovec , title =
[152]

Yuheng Hu and Lydia Manikonda and Subbarao Kambhampati , title =
[153]

Minyue Ni and Yang Zhang and Weili Han and Jun Pang , title =
[154]

Wenisch and Yuval Yarom and Raoul Strackx , title =

Jo Van Bulck and Marina Minkin and Ofir Weisse and Daniel Genkin and Baris Kasikci and Frank Piessens and Mark Silberstein and Thomas F. Wenisch and Yuval Yarom and Raoul Strackx , title =
[155]

Erick Bauman and Huibo Wang and Mingwei Zhang and Zhiqiang Lin , title =
[156]

Ran Gilad-Bachrach and Nathan Dowlin and Kim Laine and Kristin Lauter and Michael Naehrig and John Wernsing , title =
[157]

Briland Hitaj and Giuseppe Ateniese and Fernando Perez-Cruz , title =
[158]

Luca Melis and Congzheng Song and Emiliano De Cristofaro and Vitaly Shmatikov , title =
[159]

Gunter and Nikita Borisov , title =

Karan Ganju and Qi Wang and Wei Yang and Carl A. Gunter and Nikita Borisov , title =
[160]

Alex Dow and Jon Kleinberg and Jure Leskovec , title =

Justin Cheng and Lada Adamic and P. Alex Dow and Jon Kleinberg and Jure Leskovec , title =
[161]

Martin Abadi and Andy Chu and Ian Goodfellow and Brendan McMahan and Ilya Mironov and Kunal Talwar and Li Zhang , title =
[162]

Reza Shokri and Vitaly Shmatikov , title =
[163]

Kamalika Chaudhuri and Claire Monteleoni , title =
[164]

Ian Goodfellow and Jean Pouget-Abadie and Mehdi Mirza and Bing Xu and David Warde-Farley and Sherjil Ozair and Aaron Courville and Yoshua Bengio , title =
[165]

Andrew Brock and Jeff Donahue and Karen Simonyan , title =
[166]

Tribhuvanesh Orekondy and Bernt Schiele and Mario Fritz , title =
[167]

Dingdong Yang and Seunghoon Hong and Yunseok Jang and Tianchen Zhao and Honglak Lee , title =
[168]

Ana-Andreea Stoica and Christopher Riederer and Augustin Chaintreau , title =
[169]

Friedler and John Moeller and Carlos Scheidegger and Suresh Venkatasubramanian , title =

Michael Feldman and Sorelle A. Friedler and John Moeller and Carlos Scheidegger and Suresh Venkatasubramanian , title =
[170]

Gummadi , title =

Muhammad Bilal Zafar and Isabel Valera and Manuel Gomez Rodriguez and Krishna P. Gummadi , title =
[171]

Joy Buolamwini and Timnit Gebru , title =
[172]

Moritz Hardt and Eric Price and Nati Srebro , title =
[173]

Faisal Kamiran and Toon Calders and Mykola Pechenizkiy , title =
[174]

Toshihiro Kamishima and Shotaro Akaho and Hideki Asoh and Jun Sakuma , title =
[175]

Meike Zehlike and Francesco Bonchi and Carlos Castillo and Sara Hajian and Mohamed Megahed and Ricardo Baeza-Yates , title =
[176]

Jian Tang and Meng Qu and Qiaozhu Mei , title =
[177]

Michael Kearns and Seth Neel and Aaron Roth and Zhiwei Steven Wu , title =
[178]

Cynthia Dwork and Moritz Hardt and Toniann Pitassi and Omer Reingold and Richard Zemel , title =
[179]

Xiao Zhang and David Evans , title =
[180]

Wagner , title =

Anish Athalye and Nicholas Carlini and David A. Wagner , title =
[181]

Wenbo Guo and Dongliang Mu and Jun Xu and Purui Su and Gang Wang abd Xinyu Xing , title =
[182]

Zhao , title =

Bolun Wang and Yuanshun Yao and Bimal Viswanath and Haitao Zheng and Ben Y. Zhao , title =
[183]

Zhao , title =

Yuanshun Yao and Bimal Viswanath and Jenna Cryan and Haitao Zheng and Ben Y. Zhao , title =
[184]

Congzheng Song and Vitaly Shmatikov , title =
[185]

Nicholas Carlini and Chang Liu and
[186]

Qixue Xiao and Yufei Chen and Chao Shen and Yu Chen and Kang Li , title =
[187]

Yingqi Liu and Shiqing Ma and Yousra Aafer and Wen-Chuan Lee and Juan Zhai and Weihang Wang and Xiangyu Zhang , title =
[188]

Raef Bassily and Adam Smith and Abhradeep Thakurta , title =
[189]

Xiang Cai and Xin Cheng Zhang and Brijesh Joshi and Rob Johnson , title =
[190]

Aylin Caliskan and Fabian Yamaguchi and Edwin Dauber and Richard Harang and Konrad Rieck and Rachel Greenstadt and Arvind Narayanan , title =
[191]

Abdelberi Chaabane and Gergely Acs and Mohamed Ali Kaafar , title =
[192]

Cynthia Dwork and Frank McSherry and Kobbi Nissim and Adam Smith , title =
[193]

Neil Zhenqiang Gong and Bin Liu , title =
[194]

Dominik Herrmann and Rolf Wendolsky and Hannes Federrath , title =
[195]

Near and Dawn Xiaodong Song and Om Dipakbhai Thakkar and Abhradeep Thakurta and Lun Wang , title =

Roger Iyengar and Joseph P. Near and Dawn Xiaodong Song and Om Dipakbhai Thakkar and Abhradeep Thakurta and Lun Wang , title =
[196]

Bargav Jayaraman and David Evans , title =
[197]

Marc Juarez and Sadia Afroz and Gunes Acar and Claudia Diaz and Rachel Greenstadt , title =
[198]

Daniel Kifer and Adam Smith and Abhradeep Thakurta , title =
[199]

Aleksander Madry and Aleksandar Makelov and Ludwig Schmidt and Dimitris Tsipras and Adrian Vladu , title =
[200]

Dongyu Meng and Hao Chen , title =
[201]

Seyed-Mohsen Moosavi-Dezfooli and Alhussein Fawzi and Omar Fawzi and Pascal Frossard , title =
[202]

Seyed-Mohsen Moosavi-Dezfooli and Alhussein Fawzi and Pascal Frossard , title =
[203]

Paskov and Neil Zhenqiang Gong and John Bethencourt and Emil Stefanov and Eui Chul Richard Shin and Dawn Song , title =

Arvind Narayanan and Hristo S. Paskov and Neil Zhenqiang Gong and John Bethencourt and Emil Stefanov and Eui Chul Richard Shin and Dawn Song , title =
[204]

Andriy Panchenko and Lukas Niessen and Andreas Zinnen and Thomas Engel , title =
[205]

McDaniel and Xi Wu and Somesh Jha and Ananthram Swami , title =

Nicolas Papernot and Patrick D. McDaniel and Xi Wu and Somesh Jha and Ananthram Swami , title =
[206]

Erwin Quiring and Alwin Maier and Konrad Rieck , title =
[207]

Liwei Song and Reza Shokri and Prateek Mittal , title =
[208]

Sarwate , title =

Shuang Song and Kamalika Chaudhuri and Anand D. Sarwate , title =
[209]

Di Wang and Minwei Ye and Jinhui Xu , title =
[210]

Tao Wang and Xiang Cai and Rishab Nithyanand and Rob Johnson and Ian Goldberg , title =
[211]

Lei Yu and Ling Liu and Calton Pu and Mehmet Emre Gursoy and Stacey Truex , title =
[212]

Reiter and Yinqian Zhang , title =

Xiaokuan Zhang and Jihun Hamm and Michael K. Reiter and Yinqian Zhang , title =
[213]

Reiter and Thomas Ristenpart , title =

Yinqian Zhang and Ari Juels and Michael K. Reiter and Thomas Ristenpart , title =
[214]

Yang Zhang , title =
[215]

Tahleen Rahman and Bartlomiej Surma and Michael Backes and Yang Zhang , title =
[216]

Jinyuan Jia and Ahmed Salem and Michael Backes and Yang Zhang and Neil Zhenqiang Gong , title =
[217]

Yang Zhang and Mathias Humbert and Bartlomiej Surma and Praveen Manoharan and Jilles Vreeken and Michael Backes , title =
[218]

Zheng Li and Chengyu Hu and Yang Zhang and Shanqing Guo , title =
[219]

Ahmed Salem and Apratim Bhattacharya and Michael Backes and Mario Fritz and Yang Zhang , title =
[220]

Fanghua Zhao and Linan Gao and Yang Zhang and Zeyu Wang and Bo Wang and Shanqing Guo , title =
[221]

Adi Akavia and Dan Feldman and Hayim Shaul , title =
[222]

Hao Chen and Kyoohyung Han , title =
[223]

Craig Gentry and Amit Sahai and Brent Waters , title =
[224]

Roman Smolensky , title =
[225]

Craig Gentry , title =
[226]

Dawn Xiaodong Song and David Wagner and Adrian Perrig , title =
[227]

Dan Boneh and Giovanni Di Crescenzo and Rafail Ostrovsky and Giuseppe Persiano , title =
[228]

Jung Hee Cheon and Miran Kim and Kristin Lauter , title =
[229]

Kristin Lauter and Adriana L
[230]

Masaya Yasuda and Takeshi Shimoyama and Jun Kogure and Kazuhiro Yokoyama and Takeshi Koshiba , title =
[231]

Craig Gentry and Shai Halevi and Nigel P Smart , title =
[232]

Shai Halevi and Victor Shoup , title =
[233]

Vladimir Kolesnikov and Ranjit Kumaresan and Mike Rosulek and Ni Trieu , title =
[234]

Hao Chen and Kim Laine and Peter Rindal , title =
[235]

Andrew Chi-Chih Yao , title =
[236]

Oded Goldreich and Silvio Micali and Avi Wigderson , title =
[237]

Benny Pinkas and Mike Rosulek and Ni Trieu and Avishay Yanai , title =
[238]

Zhao , title =

Yuanshun Yao and Huiying Li and Haitao Zheng and Ben Y. Zhao , title =
[239]

Zhao , title =

Bolun Wang and Yuanshun Yao and Shawn Shan and Huiying Li and Bimal Viswanath and Haitao Zheng and Ben Y. Zhao , title =
[240]

Yingqi Liu and Wen-Chuan Lee and Guanhong Tao and Shiqing Ma and Yousra Aafer and Xiangyu Zhang , title =
[241]

Ali Shafahi and W Ronny Huang and Mahyar Najibi and Octavian Suciu and Christoph Studer and Tudor Dumitras and Tom Goldstein , title =
[242]

Yansong Gao and Change Xu and Derui Wang and Shiping Chen and Damith C Ranasinghe and Surya Nepal , title =
[243]

Karen Simonyan and Andrew Zisserman , title =
[244]

James Atwood and Don Towsley , title =
[245]

Petar Velickovic and Guillem Cucurull and Arantxa Casanova and Adriana Romero and Pietro Li
[246]

Varun Chandrasekaran and Kamalika Chaudhuri and Irene Giacomelli and Somesh Jha and Songbai Yan , title =
[247]

Yinzhi Cao and Junfeng Yang , title =
[248]

Theo Bertram and Elie Bursztein and Stephanie Caro and Hubert Chao and Rutledge Chin and FemanPeter Fleischer and Albin Gustafsson and Jess Hemerly and Chris Hibbert and Luca InvernizziLanah and Kammourieh Donnelly and Jason Ketover and Jay Laefer and Paul Nicholas and Yuan Ni...
[249]

Kaiming He and Xiangyu Zhang and Shaoqing Ren and Jian Sun , title =
[250]

Honggang Yu and Kaichen Yang and Teng Zhang and Yun-Yun Tsai and Tsung-Yi Ho and Yier Jin , title =
[251]

Reza Shokri and Martin Strobel and Yair Zick , title =
[252]

Lussier and Nitesh V

Ryan Lichtenwalter and Jake T. Lussier and Nitesh V. Chawla , title =
[253]

Aditya Krishna Menon and Charles Elkan , title =
[254]

Hanjun Dai and Hui Li and Tian Tian and Xin Huang and Lin Wang and Jun Zhu and Le Song , title =
[255]

Guolei Yang and Neil Zhenqiang Gong and and Ying Cai , title =
[256]

Hinton , title =

Alex Krizhevsky and Ilya Sutskever and Geoffrey E. Hinton , title =
[257]

Gomez and Lukasz Kaiser and Illia Polosukhin , title =

Ashish Vaswani and Noam Shazeer and Niki Parmar and Jakob Uszkoreit and Llion Jones and Aidan N. Gomez and Lukasz Kaiser and Illia Polosukhin , title =
[258]

Schoenholz and Patrick F

Justin Gilmer and Samuel S. Schoenholz and Patrick F. Riley and Oriol Vinyals and George E. Dahl , title =
[259]

Federico Errica and Marco Podda and Davide Bacciu and Alessio Micheli , title =
[260]

Keyulu Xu and Weihua Hu and Jure Leskovec and Stefanie Jegelka , title =
[261]

Wenqi Fan and Yao Ma and Qing Li and Yuan He and Yihong Eric Zhao and Jiliang Tang and Dawei Yin , title =
[262]

Jiezhong Qiu and Jian Tang and Hao Ma and Yuxiao Dong and Kuansan Wang and Jie Tang , title =
[263]

Dzmitry Bahdanau and Kyunghyun Cho and Yoshua Bengio , title =
[264]

Le , title =

Ilya Sutskever and Oriol Vinyals and Quoc V. Le , title =
[265]

Xiangnan He and Hanwang Zhang and Min
[266]

Le Wu and Peijie Sun and Yanjie Fu and Richang Hong and Xiting Wang and Meng Wang , title =
[267]

Weihua Hu and Bowen Liu and Joseph Gomes and Marinka Zitnik and Percy Liang and Vijay Pande and Jure Leskovec , title =
[268]

Dingyuan Zhu and Ziwei Zhang and Peng Cui and Wenwu Zhu , title =
[269]

Aleksandar Bojchevski and Stephan G
[270]

Min Du and Zhi Chen and Chang Liu and Rajvardhan Oak and Dawn Song , title =
[271]

Yinzhi Cao and Alexander Fangxiao Yu and Andrew Aday and Eric Stahl and Jon Merwine and Junfeng Yang , title =
[272]

Ginart and Melody Y

Antonio A. Ginart and Melody Y. Guan and Gregory Valiant and James Zou , title =
[273]

Christian Szegedy and Wojciech Zaremba and Ilya Sutskever and Joan Bruna and Dumitru Erhan and Ian Goodfellow and Rob Fergus , title =
[274]

Huijun Wu and Chen Wang and Yuriy Tyshetskiy and Andrew Docherty and Kai Lu and Liming Zhu , title =
[275]

Matthew Jagielski and Nicholas Carlini and David Berthelot and Alex Kurakin and Nicolas Papernot , title =
[276]

Alabdulmohsin and Xin Gao and Xiangliang Zhang , title =

Ibrahim M. Alabdulmohsin and Xin Gao and Xiangliang Zhang , title =
[277]

Yusuke Uchida and Yuki Nagai and Shigeyuki Sakazawa and Shin'ichi Satoh , title =
[278]

Asokan , title =

Mika Juuti and Sebastian Szyller and Samuel Marchal and N. Asokan , title =
[279]

Savvas Zannettou and Tristan Caulfield and Jeremy Blackburn and Emiliano De Cristofaro and Michael Sirivianos and Gianluca Stringhini and Guillermo Suarez-Tangil , title =
[280]

Savvas Zannettou and Joel Finkelstein and Barry Bradlyn and Jeremy Blackburn , title =
[281]

Xiang Ling and Shouling Ji and Jiaxu Zou and Jiannan Wang and Chunming Wu and Bo Li and Ting Wang , title =
[282]

Yujie Ji and Xinyang Zhang and Shouling Ji and Xiapu Luo and Ting Wang , title =
[283]

Yizheng Chen and Shiqi Wang and Dongdong She and Suman Jana , title =
[284]

Dongdong She and Yizheng Chen and Abhishek Shah and Baishakhi Ray and Suman Jana , title =
[285]

Jinyuan Jia and Binghui Wang and Xiaoyu Cao and Neil Zhenqiang Gong , title =
[286]

Yizheng Chen and Yacin Nadji and Athanasios Kountouras and Fabian Monrose and Roberto Perdisci and Manos Antonakakis and Nikolaos Vasiloglou , title =
[287]

Congzheng Song and Reza Shokri , title =
[288]

Hamilton and Jure Leskovec , title =

Rex Ying and Jiaxuan You and Christopher Morris and Xiang Ren and William L. Hamilton and Jure Leskovec , title =
[289]

Nicolas Papernot and Martin Abadi and Ulfar Erlingsson and Ian Goodfellow and Kunal Talwar , title =
[290]

Hamilton and Jure Leskovec , title =

Jiaxuan You and Rex Ying and Xiang Ren and William L. Hamilton and Jure Leskovec , title =
[291]

Dingfan Chen and Ning Yu and Yang Zhang and Mario Fritz , title =
[292]

Maas and Raymond E

Andrew L. Maas and Raymond E. Daly and Peter T. Pham and Dan Huang and Andrew Y. Ng and Christopher Potts , title =
[293]

Manning and Andrew Y

Richard Socher and Alex Perelygin and Jean Wu and Jason Chuang and Christopher D. Manning and Andrew Y. Ng and Christopher Potts , title =
[294]

McAuley , title =

Jianmo Ni and Jiacheng Li and Julian J. McAuley , title =
[295]

Wei Jin and Yao Ma and Xiaorui Liu and Xianfeng Tang and Suhang Wang and Jiliang Tang , title =
[296]

Jung Hee Cheon and Andrey Kim and Miran Kim and Yongsoo Song , title =
[297]

Jack L. H. Crawford and Craig Gentry and Shai Halevi and Daniel Platt and Victor Shoup , title =
[298]

Jung Hee Cheon and Dongwoo Kim and Duhyeong Kim and Hun Hee Lee and Keewoo Lee , title =
[299]

Reed and Dragomir Anguelov and Dumitru Erhan and Vincent Vanhoucke and Andrew Rabinovich , title =

Christian Szegedy and Wei Liu and Yangqing Jia and Pierre Sermanet and Scott E. Reed and Dragomir Anguelov and Dumitru Erhan and Vincent Vanhoucke and Andrew Rabinovich , title =
[300]

Goodfellow and Wojciech Zaremba and Vicki Cheung and Alec Radford and Xi Chen , title =

Tim Salimans and Ian J. Goodfellow and Wojciech Zaremba and Vicki Cheung and Alec Radford and Xi Chen , title =

Pith tools

Reviewed June 27, 2026 · model on record in the stance chip above.

[1] [1]

Ahmed Salem and Yang Zhang and Mathias Humbert and Pascal Berrang and Mario Fritz and Michael Backes , title =

[2] [2]

Inken Hagestedt and Yang Zhang and Mathias Humbert and Pascal Berrang and Haixu Tang and XiaoFeng Wang and Michael Backes , title =

[3] [3]

Michael Backes and Mathias Humbert and Jun Pang and Yang Zhang , title =

[4] [4]

Yang Zhang and Mathias Humbert and Tahleen Rahman and Cheng-Te Li and Jun Pang and Michael Backes , title =

[5] [5]

Pascal Berrang and Mathias Humbert and Yang Zhang and Irina Lehmann and Roland Eils and Michael Backes , title =

[6] [6]

Michael Backes and Pascal Berrang and Mathias Humbert and Praveen Manoharan , title =

[7] [7]

Brendan McMahan and Sarvar Patel and Daniel Ramage and Aaron Segal and Karn Seth , title =

Keith Bonawitz and Vladimir Ivanov and Ben Kreuter and Antonio Marcedone and H. Brendan McMahan and Sarvar Patel and Daniel Ramage and Aaron Segal and Karn Seth , title =

[8] [8]

Raphael Bost and Raluca Ada Popa and Stephen Tu and Shafi Goldwasser , title =