Pith. sign in

REVIEW 5 major objections 4 minor 59 references

Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM's Instruction-Following Capabilities

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that instruction-following in LLMs is carried by a sparse set of instruction-specific neurons and experts, and that fine-tuning improves it by refining these components and adding new ones while leaving the layer-wise…

desk verdict Useful descriptive work with a good balanced dataset and a novel MoE angle, but the 'critical role' claim outruns the evidence and the identification method needs stronger baselines before the core findings can be trusted. read the letter →

arxiv 2505.21191 v1 pith:MTTKZMIE submitted 2025-05-27 cs.CL cs.LG

Junyan Zhang , Yubo Gao , Yibo Yan , Jungang Li , Zhaorui Hou , Sicheng Tao , Shuliang Liu , Song Dai
show 3 more authors
Yonghua Hei Junzhuo Li Xuming Hu
This is my paper
classification cs.CLcs.LG
keywords instruction-specificneuronsexpertsmixture-of-expertsmechanisticinterpretabilityinstructiontuningsparsecomponentsfine-tuninganalysisLLM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks where instruction-following lives inside a large language model. It tries to establish that the ability is carried by a small set of 'instruction-specific' components: individual neurons in dense models like LLaMA and Mistral, and both neurons and experts in mixture-of-experts models like Qwen-MoE. To make the case, the authors build a balanced instruction dataset across six categories and a framework that identifies the neurons and experts that activate most consistently for each instruction, then measure how those sets overlap across same-type and different-type instructions and how they shift after fine-tuning. Their conclusion is that fine-tuning works by refining existing instruction-specific components and adding new ones, while the overall layer-wise logic of instruction processing stays unchanged. If true, this means instruction-following can be localized, inspected, and potentially edited in sparse parts of the network.

What carries the argument

The load-bearing object is the activation-frequency set $S(I)$: for an instruction $I$, each neuron's activation frequency $p_{ij}(I)$ is the fraction of tokens where that neuron's post-activation output is positive, and $S(I)$ selects the top $\epsilon$ percentile of these frequencies across all layers. This set is the unit of analysis for everything else. ISN-level claims are computed as Jaccard similarities between $S(I)$ sets for pairs of instructions; ISE-level claims are computed as Pearson correlations between flattened expert-activation-frequency vectors $F(I)$. The comparison before and after fine-tuning uses the same metrics between a vanilla model and its chat version. The paper also introduces a three-stage mechanistic reading of the layer distribution: many ISNs in early layers (shallow encoding of instructions), few in middle layers (generalized understanding), and many again in late layers (decoding output tokens), with MoE models showing a longer early increase. The HEXAINST dataset, balanced across six categories and two sources, is the controlled stimulus that lets overlap patterns be attributed to instruction type rather than data source.

What would settle it

Ablate or amplify the union of instruction-specific neurons $S(I)$ for one category (say code) in LLaMA-2-Chat-7B and measure whether code-instruction performance drops (or rises) far more than performance on other categories; the paper's identification criterion predicts a selective effect. A cheaper partial check is to recompute the Jaccard similarity matrices under a grid of $\epsilon$ values and see whether the same-type diagonal remains darker, since the reported patterns would be artifacts if they flip when $\epsilon$ moves.

Watch

Extended reading notes

Core claim

The central claim is that instruction tuning reconfigures sparse components rather than the whole network. For each instruction, SPARCOM ranks every neuron by the fraction of instruction tokens on which it activates above zero, and labels the top $\epsilon$ percentile as Instruction-Specific Neurons (ISNs); in MoE models, the experts selected most often are labelled Instruction-Specific Experts (ISEs). Using the balanced HEXAINST dataset (six categories, 100 human-curated and 100 AI-generated instructions each), the paper reports that Jaccard similarity between ISN sets is higher for instructions of the same type than for different types, especially for classification, code, and math, which it takes as evidence that these components are functionally unique, while cross-type overlap indicates general instruction components. Comparing vanilla and fine-tuned model pairs, the layer-wise distribution of ISNs is roughly preserved, but fine-tuned models have more specialized ISNs, concentrated in early parsing and late generation layers; the same instruction activates substantially different neurons after fine-tuning (Jaccard similarity roughly 0.4–0.6 across model families), whereas in MoE models expert-routing correlation stays high (0.91–0.94). The paper concludes that fine-tuning refines existing instruction-specific components, adds new ones, and does so without changing the model's fundamental layer-wise instruction-processing structure.

Load-bearing premise

Everything downstream rests on the assumption that the neurons and experts that fire most often while an instruction is being processed are the components actually executing that instruction, rather than bystanders that happen to activate alongside it; the paper never fixes the percentile cutoff $\epsilon$ or tests how sensitive its results are to that choice.

Editorial extensions

If this is right

  • Instruction-following behavior can be localized to a small fraction of neurons (and experts in MoE models), so auditing or interpreting a chat model could focus on these sparse components instead of the full network.
  • Fine-tuning improves instruction following mainly by refining existing instruction-specific components and adding new ones, without re-architecting the layer-wise processing logic, so supervised fine-tuning can be understood as a targeted edit of a stable substrate.
  • Same-type instructions share specialized components, which implies that an intervention on one instruction category's ISNs could affect an entire category of instructions rather than a single prompt.
  • In MoE models, expert routing is almost unchanged by fine-tuning (correlation around 0.92), so the improvement from fine-tuning is carried mostly by the neurons inside experts rather than by rerouting which experts are used.
  • The three-stage layer pattern (parse, generalize, decode) identifies where instruction-specific computation concentrates, suggesting layer-selective fine-tuning or editing strategies that preserve the stable structure while updating the specialized components.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A causal test would be to ablate or amplify $S(I)$ for one instruction type and measure whether performance on that type changes selectively; the paper's identification criterion is correlational, so this is the natural next step.
  • If expert routing stays so stable under fine-tuning, one could fine-tune only the neurons inside experts while freezing the router, which would cut training cost if the claimed mechanism holds.
  • Because HEXAINST pairs AI-generated with human-curated instructions, the same framework could separate semantic instruction specificity from surface-form confounds by testing whether rephrased versions of the same instruction recruit the same ISNs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper introduces HEXAINST, a small balanced instruction dataset spanning six categories (classification, code, general QA, generation, math, summarization), and SPARCOM, a three-step framework for identifying Instruction-Specific Neurons (ISNs) in dense LLMs and both ISNs and Instruction-Specific Experts (ISEs) in MoE models. ISNs are defined as neurons whose positive-activation frequency over instruction tokens falls in the top epsilon percentile (Eqs. 5-7); ISEs are defined as the top-k routed experts. The framework then measures same-type versus cross-type overlap via Jaccard similarity (Eq. 10) and expert activation correlation (Eqs. 13-14), and compares ISN/ISE distributions before and after fine-tuning (Eqs. 15-16). Based on experiments on LLaMA-2, Mistral-7B, and Qwen1.5-MoE-A2.7B, the authors claim functional generality and uniqueness of these sparse components and a 'critical role' in instruction execution, with fine-tuning refining existing and adding new ISNs while preserving overall layer-wise structure.

Significance. If the central claims hold, the paper would provide a sparse-component account of instruction-following behavior, complementing prior work on language-specific and skill-specific neurons and offering a practical target for future interpretability and fine-tuning interventions. The paper's strengths are its cross-model scope (four model families/sizes, dense and MoE), its explicit, computable formulas for activation frequency and overlap, and the construction of a balanced category-level instruction dataset with both natural and AI-generated instances. The framework is simple enough to be reproduced, which is a virtue. However, the empirical content is correlational: the identification step is a thresholded percentile selection with an unspecified epsilon, no baseline controls are run against lexical or template overlap, and no significance testing or error bars accompany the central figures. The dataset is small (1,200 instructions, 200 per category) and the paper states that code and data will be released only upon acceptance. I therefore see this as a promising but currently under-supported set of findings rather than a demonstrated mechanism.

major comments (5)
  1. [Section 3.1, Eq. (7)] The value of the percentile threshold epsilon in Eq. (7) is never specified in the paper, and no sensitivity analysis is provided. Since S(I) is the input to every downstream overlap and correlation metric (Eqs. 10, 14-16, Tables 1-2, Figure 5), the quantitative results are uninterpretable until epsilon is given a concrete value and shown not to drive the main conclusions. A sweep over a range of percentiles, with a report of the stability of Figures 3-4, is necessary.
  2. [Section 3.1, Eqs. (5)-(7) and Section 4.1] The ISN identification operates on raw instruction tokens. The six HEXAINST categories necessarily share category-level vocabulary and template words (e.g., 'classify', 'code', 'summarize', 'matrix', 'median'), so neurons selected for high positive-activation frequency may be responding to surface lexical features rather than to instruction-specific computation. The paper itself concedes in Section 5.1 that 'overlap in certain vocabulary is inevitable', but it provides no control: there is no token-shuffled baseline, no matched non-instruction text, and no permutation test over random neuron sets. Without such a control, the same-type/cross-type separation in Figure 3 and the fine-tuning shifts in Table 1 could be artifacts of lexical overlap.
  3. [Section 3.2, Eqs. (10)-(14) and Figures 3-4] All conclusions about same-type versus cross-type separation are based on point estimates with no error bars, confidence intervals, or significance tests. Appendix D says the similarity cells use 300 randomly sampled instances, but the paper does not report the variance or the number of sampled neurons. A permutation or bootstrap test is required to substantiate claims that the diagonal is 'notably darker' than chance, especially given that the thresholding procedure itself guarantees a sparse set of ISNs for every instruction.
  4. [Section 3.1, 'ISEs Identification' and Eqs. (8)-(9)] The ISE identification step is not a selection procedure: every expert in the top-k routing set is labeled an Instruction-Specific Expert. This makes the subsequent correlation analysis in Eqs. (13)-(14) a description of overall routing behavior rather than a property of instruction-specific sparse components. A criterion for selectivity across instruction types (e.g., an expert is more frequently routed for one type than others) is needed before the paper can claim unique or general ISEs.
  5. [Section 5.3 and Conclusion] The abstract and conclusion assert that ISNs and ISEs play a 'critical role' in instruction execution, but the paper performs no causal intervention such as activation patching, ablations, or neuron editing. The Limitations section explicitly defers 'developing effective strategies to leverage these components' to future work, which is inconsistent with the strong causal claim. The evidence is correlational, so the language should be softened to 'associated with' or 'co-occur with' until causal evidence is provided.
minor comments (4)
  1. [Abstract and Introduction] The text contains numerous typos and spacing errors (e.g., 'theun-derlying', 'posesspecific neuronscrucial', 'HEXAINST' vs. 'HexaInst' inconsistency), which should be corrected before publication.
  2. [Section 3.2, Eq. (13)] Equation (13) is typeset incorrectly; the square root symbols and fraction structure make the formula difficult to read, and the placement of the indices in the numerator should be fixed.
  3. [Figure 5] The figure caption says 'ISN Numbers' but the figure panels are labeled 'Finetuned Models' and 'Vanilla Models'; the caption should explain what is plotted on the y-axis and how the layer index is defined.
  4. [Appendix D] The statement that '300 randomly sampled instances' are used for the Jaccard and Pearson calculations should appear in the main text near Figures 3 and 4, because it affects the interpretation of those figures, and the precise sampling procedure (seeds, replacement) should be reported.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: ISN identification is a definitional operationalization, but the reported overlap and fine-tuning results are empirical and not entailed by the defining equations; the only self-citations are peripheral.

full rationale

The paper's derivation chain starts with an activation-frequency definition (Eqs. 5-7) that designates the top-epsilon percentile of positively activated neurons as Instruction-Specific Neurons. This is an operational definition, not a derived prediction: the claim that such a sparse set exists is true by construction, but the paper does not present sparsity as a discovered result. The subsequent empirical content resides in the Jaccard overlaps (Eq. 10) and Pearson correlations (Eqs. 13-14) across same-type versus cross-type instructions, and in the before/after fine-tuning comparisons (Eqs. 15-16). None of these quantities is algebraically forced by the definition of S(I); they depend on actual model activations. The same-type/cross-type separation could be inflated by shared category-level vocabulary because the identification is computed on raw instruction tokens and no token-matched or shuffled control is reported; that is a validity threat for the 'uniqueness' interpretation, not a circularity. Likewise, the abstract's 'critical role' language is stronger than the correlational evidence, and the Limitations section concedes that leveraging these components is future work; this is an evidential gap, not a self-referential derivation. Self-citations (e.g., Huo et al. 2024 aligning with the three-phase interpretation) are peripheral and not load-bearing for the central identification or overlap measurements. Overall, no step reduces to its own input by construction, so the circularity score is low.

Assumptions & free parameters 2 free parameters · 6 assumptions · 3 invented entities

The ledger counts one hand-chosen threshold (epsilon) and one sampling choice (300 instances), plus six domain assumptions. The central empirical results depend on the activation-frequency proxy and on overlap and correlation measures being interpreted as functional specialization. No new physical entities are introduced, but the ISN and ISE labels are constructs whose independent evidence is weak.

free parameters (2)
  • epsilon (top-percentile threshold for ISN selection) = not specified
    Determines which neurons count as instruction-specific; no value or sensitivity analysis is given, so all reported overlap numbers depend on an unreported hand-chosen threshold.
  • sample size for similarity calculations = 300
    Appendix D states that Jaccard and Pearson calculations used 300 randomly sampled instances out of 1200; this sampling choice affects the estimates in Figures 3 and 4 and no confidence intervals are reported.
assumptions (6)
  • domain assumption Activation frequency is a valid proxy for functional relevance
    Eqs. (5)-(6) define ISNs by positive-activation frequency; no causal or ablation evidence links high activation frequency to functional necessity.
  • domain assumption Top-epsilon percentile isolates sparse, meaningful components
    Eq. (7) selects top epsilon percentile as ISNs; epsilon is unspecified and the sparsity claim is built into the selection rule.
  • domain assumption Jaccard overlap between ISN sets measures shared function
    Eq. (10) interprets same-type versus cross-type overlap as functional specialization, but overlap could reflect shared vocabulary or instruction templates, a confound the authors acknowledge for general ISNs.
  • domain assumption Pearson correlation of expert activation vectors measures expert similarity
    Eqs. (13)-(14) use correlation to compare expert distributions across instructions; high linear correlation is treated as functional similarity without a baseline.
  • domain assumption The six-category balanced dataset is representative of instructions
    Section 4 constructs HEXAINST from six categories; the limitations section explicitly acknowledges the limited category set and generalizability concerns.
  • domain assumption Synthetic instructions are not contaminated by pretraining data
    Section 4 states that synthetic instructions were cross-checked against public training data, but the checking method and contamination metrics are not described.
invented entities (3)
  • Instruction-Specific Neurons (ISNs)
    purpose: Analytic label for neurons claimed to be responsible for processing and executing instructions
    Defined by the paper's own activation-frequency criterion; no external validation, intervention, or falsifiable prediction links these neurons to instruction-following behavior.
  • Instruction-Specific Experts (ISEs)
    purpose: Analytic label for MoE experts claimed to be specialized for instruction types
    Identified simply as the experts activated under top-k routing; Appendix F shows the top five experts are largely shared across categories, so instruction-type specificity is not clearly established.
  • General and unique ISN/ISE subtypes
    purpose: Conceptual categories used to explain why same-type instructions overlap more than cross-type instructions
    Inferred from visual inspection of Figures 3 and 4 without statistical tests or a null baseline controlling for token overlap.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM's Instruction-Following Capabilities." pith.science (2026). https://pith.science/paper/MTTKZMIE

@misc{pith2026250521191,
  author       = {Pith},
  title        = {Pith review of: Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM's Instruction-Following Capabilities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MTTKZMIE}},
  note         = {Machine review of arXiv:2505.21191}
}
read the original abstract

The finetuning of Large Language Models (LLMs) has significantly advanced their instruction-following capabilities, yet the underlying computational mechanisms driving these improvements remain poorly understood. This study systematically examines how fine-tuning reconfigures LLM computations by isolating and analyzing instruction-specific sparse components, i.e., neurons in dense models and both neurons and experts in Mixture-of-Experts (MoE) architectures. In particular, we introduce HexaInst, a carefully curated and balanced instructional dataset spanning six distinct categories, and propose SPARCOM, a novel analytical framework comprising three key contributions: (1) a method for identifying these sparse components, (2) an evaluation of their functional generality and uniqueness, and (3) a systematic comparison of their alterations. Through experiments, we demonstrate functional generality, uniqueness, and the critical role of these components in instruction execution. By elucidating the relationship between fine-tuning-induced adaptations and sparse computational substrates, this work provides deeper insights into how LLMs internalize instruction-following behavior for the trustworthy LLM community.

Figures

Figures reproduced from arXiv: 2505.21191 by the authors.

Figure 1
Figure 1. Comparison of research focuses between Language-Specific Neurons (a) and Instruction-Specific Neurons & Experts in dense LLMs & MoE models (b). Prior works in neuron-level interpretation (Tang et al., 2024; Huo et al., 2024; Kojima et al., 2024; Wang et al., 2022b; Huang et al., 2024) has success￾fully identified X-specific neurons crucial for stor￾ing factual knowledge (Dai et al., 2021), processing specific langua… view at source ↗
Figure 2
Figure 2. The SPARCOM framework, which comprises three elements, aims for the identification & evaluation of sparse components. ISNs and ISEs denote Instruction-Specific Neurons and Instruction-Specific Experts. processed by different parts of the model. This allows for scalable and efficient model growth (Shi et al., 2024; Li et al., 2025a; Yuan et al., 2025; Yang et al., 2025). LLM Fine-tuning While foundational pre￾trained… view at source ↗
Figure 3
Figure 3. Overlaps and differences in ISNs distribu [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Overlaps and differences in ISEs distribu [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Hierarchy distribution of ISNs across different layers. The upper part includes LLaMA-2-Chat-7B, [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Venn-bar diagram illustrating the distribu [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 3 canonical work pages

  1. [1]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609

    arXiv 2023

  2. [2]

    Ahsan Bilal, David Ebert, and Beiyu Lin. 2025. Llms for explainable ai: A comprehensive survey. arXiv preprint arXiv:2504.00125

    arXiv 2025

  3. [3]

    Weilin Cai, Juyong Jiang, Fan Wang, Jing Tang, Sunghun Kim, and Jiayi Huang. 2025. A survey on mixture of experts in large language models. IEEE Transactions on Knowledge and Data Engineering

    2025

  4. [4]

    Erik Cambria, Lorenzo Malandri, Fabio Mercorio, Navid Nobani, and Andrea Seveso. 2024. Xai meets llms: A survey of the relation between explainable ai and large language models. arXiv preprint arXiv:2407.15248

    arXiv 2024

  5. [5]

    Jianhui Chen, Xiaozhi Wang, Zijun Yao, Yushi Bai, Lei Hou, and Juanzi Li. 2024. Finding safety neurons in large language models. arXiv preprint arXiv:2406.14144

    arXiv 2024

  6. [6]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

    arXiv 2021

  7. [7]

    Brown, Miljan Martic, Shane Legg, and Dario Amodei

    Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2023. https://arxiv.org/abs/1706.03741 Deep reinforcement learning from human preferences . Preprint, arXiv:1706.03741

    arXiv 2023

  8. [8]

    Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2021. Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2104.08696

    arXiv 2021

Show all 59 references
  1. [9]

    Yunkai Dang, Mengxi Gao, Yibo Yan, Xin Zou, Yanggan Gu, Aiwei Liu, and Xuming Hu. 2024. Exploring response uncertainty in mllms: An empirical evaluation under misleading scenarios. arXiv preprint arXiv:2411.02708

    2024 arXiv

  2. [10]

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608

    2024 arXiv

  3. [11]

    Kaichen Huang, Jiahao Huo, Yibo Yan, Kun Wang, Yutao Yue, and Xuming Hu. 2024. Miner: Mining the underlying pattern of modality-specific neurons in multimodal large language models. arXiv preprint arXiv:2410.04819

    2024 arXiv

  4. [12]

    Zihao Huang, Qiyang Min, Hongzhi Huang, Defa Zhu, Yutao Zeng, Ran Guo, and Xun Zhou. 2025. https://arxiv.org/abs/2411.12364 Ultra-sparse memory network . Preprint, arXiv:2411.12364

    2025 arXiv

  5. [13]

    Jiahao Huo, Yibo Yan, Boren Hu, Yutao Yue, and Xuming Hu. 2024. Mmneuron: Discovering neuron-level domain-specific interpretation in multimodal large language model. arXiv preprint arXiv:2406.11193

    2024 arXiv

  6. [14]

    Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. 1991. Adaptive mixtures of local experts. Neural computation, 3(1):79--87

    1991

  7. [15]

    Jiaming Ji, Boyuan Chen, Hantao Lou, Donghai Hong, Borong Zhang, Xuehai Pan, Tianyi Alex Qiu, Juntao Dai, and Yaodong Yang. 2024. Aligner: Efficient alignment by learning to correct. Advances in Neural Information Processing Systems, 37:90853--90890

    2024

  8. [16]

    Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L'elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, T...

    2023 arXiv

  9. [17]

    Michael I Jordan and Robert A Jacobs. 1994. Hierarchical mixtures of experts and the em algorithm. Neural computation, 6(2):181--214

    1994

  10. [18]

    Mandar Joshi , Eunsol Choi , Daniel Weld , and Luke Zettlemoyer . 2017. https://arxiv.org/abs/1705.03551 triviaqa: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension . arXiv e-prints, arXiv:1705.03551

    2017 arXiv

  11. [19]

    Zixuan Ke, Fangkai Jiao, Yifei Ming, Xuan-Phi Nguyen, Austin Xu, Do Xuan Long, Minzhi Li, Chengwei Qin, Peifeng Wang, Silvio Savarese, et al. 2025. A survey of frontiers in llm reasoning: Inference scaling, learning to reason, and agentic systems. arXiv preprint arXiv:2504.09037

    2025

  12. [20]

    Takeshi Kojima, Itsuki Okimura, Yusuke Iwasawa, Hitomi Yanaka, and Yutaka Matsuo. 2024. On the multilingual ability of decoder-based pre-trained language models: Finding and controlling language-specific neurons. arXiv preprint arXiv:2404.02431

    2024 arXiv

  13. [21]

    Komal Kumar, Tajamul Ashraf, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, Phillip HS Torr, Fahad Shahbaz Khan, and Salman Khan. 2025. Llm post-training: A deep dive into reasoning large language models. arXiv preprint arXiv:2502.21321

    2025 arXiv

  14. [22]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating S...

    2023

  15. [23]

    Yunxin Li, Shenyuan Jiang, Baotian Hu, Longyue Wang, Wanqi Zhong, Wenhan Luo, Lin Ma, and Min Zhang. 2025 a . Uni-moe: Scaling unified multimodal llms with mixture of experts. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2025

  16. [24]

    Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. 2025 b . From system 1 to system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502.17419

    2025 arXiv

  17. [25]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. arXiv preprint arXiv:2305.20050

    2023 arXiv

  18. [26]

    Bin Lin, Zhenyu Tang, Yang Ye, Jiaxi Cui, Bin Zhu, Peng Jin, Jinfa Huang, Junwu Zhang, Yatian Pang, Munan Ning, et al. 2024. Moe-llava: Mixture of experts for large vision-language models. arXiv preprint arXiv:2401.15947

    2024 arXiv

  19. [27]

    Le, Barret Zoph, Jason Wei, and Adam Roberts

    Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023. https://arxiv.org/abs/2301.13688 The flan collection: Designing data and methods for effective instruction tuning . Preprint, arXiv:2301.13688

    2023 arXiv

  20. [28]

    Samuel Marks, Can Rager, Eric J Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. 2024. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. arXiv preprint arXiv:2403.19647

    2024 arXiv

  21. [29]

    Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. 2023. Circuit component reuse across tasks in transformer language models. arXiv preprint arXiv:2310.08744

    2023 arXiv

  22. [30]

    Fuseini Mumuni and Alhassan Mumuni. 2025. Explainable artificial intelligence (xai): from inherent explainability to large language models. arXiv preprint arXiv:2501.09967

    2025 arXiv

  23. [31]

    Nikhil Prakash, Tamar Rott Shaham, Tal Haklay, Yonatan Belinkov, and David Bau. 2024. Fine-tuning enhances existing mechanisms: A case study on entity tracking. arXiv preprint arXiv:2402.14811

    2024 arXiv

  24. [32]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2024. https://arxiv.org/abs/2305.18290 Direct preference optimization: Your language model is secretly a reward model . Preprint, arXiv:2305.18290

    2024 arXiv

  25. [33]

    Junhong Shen, Neil Tenenholtz, James Brian Hall, David Alvarez-Melis, and Nicolo Fusi. 2024. Tag-llm: Repurposing general-purpose llms for specialized domains. arXiv preprint arXiv:2402.05140

    2024 arXiv

  26. [34]

    Xiaoming Shi, Shiyu Wang, Yuqi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. 2024. Time-moe: Billion-scale time series foundation models with mixture of experts. arXiv preprint arXiv:2409.16040

    2024 arXiv

  27. [35]

    Ran Song, Shizhu He, Shuting Jiang, Yantuan Xian, Shengxiang Gao, Kang Liu, and Zhengtao Yu. 2024. Does large language model contain task-specific neurons? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 7101--7113

    2024

  28. [36]

    Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano

    Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. 2022. https://arxiv.org/abs/2009.01325 Learning to summarize from human feedback . Preprint, arXiv:2009.01325

    2022 arXiv

  29. [37]

    Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. Language-specific neurons: The key to multilingual capabilities in large language models. arXiv preprint arXiv:2402.16438

    2024 arXiv

  30. [38]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca

    2023

  31. [39]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  32. [40]

    Jiahao Wang, Bolin Zhang, Qianlong Du, Jiajun Zhang, and Dianhui Chu. 2024 a . A survey on data selection for llm instruction tuning. arXiv preprint arXiv:2402.05123

    2024 arXiv

  33. [41]

    Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2022 a . Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. arXiv preprint arXiv:2211.00593

    2022 arXiv

  34. [42]

    Luping Wang, Sheng Chen, Linnan Jiang, Shu Pan, Runze Cai, Sen Yang, and Fei Yang. 2024 b . Parameter-efficient fine-tuning in large models: A survey of methodologies. arXiv preprint arXiv:2410.19878

    2024 arXiv

  35. [43]

    Weixuan Wang, Barry Haddow, Minghao Wu, Wei Peng, and Alexandra Birch. 2024 c . Sharing matters: Analysing neurons across languages and tasks in llms. arXiv preprint arXiv:2406.09265

    2024

  36. [44]

    Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, and Juanzi Li. 2022 b . Finding skill neurons in pre-trained transformer-based language models. arXiv preprint arXiv:2211.07349

    2022 arXiv

  37. [45]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. https://arxiv.org/abs/2212.10560 Self-instruct: Aligning language models with self-generated instructions . Preprint, arXiv:2212.10560

    2023 arXiv

  38. [46]

    Zhichao Wang, Bin Bi, Shiva Kumar Pentyala, Kiran Ramnath, Sougata Chaudhuri, Shubham Mehrotra, Xiang-Bo Mao, Sitaram Asur, et al. 2024 d . A comprehensive survey of llm alignment techniques: Rlhf, rlaif, ppo, dpo and more. arXiv preprint arXiv:2407.16216

    2024 arXiv

  39. [47]

    Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M

    Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. https://arxiv.org/abs/2109.01652 Finetuned language models are zero-shot learners . Preprint, arXiv:2109.01652

    2022 arXiv

  40. [48]

    Ting-Ruen Wei, Haowei Liu, Xuyang Wu, and Yi Fang. 2025. A survey on feedback-based multi-step reasoning for large language models on mathematics. arXiv preprint arXiv:2502.14333

    2025 arXiv

  41. [49]

    Xuansheng Wu, Haiyan Zhao, Yaochen Zhu, Yucheng Shi, Fan Yang, Tianming Liu, Xiaoming Zhai, Wenlin Yao, Jundong Li, Mengnan Du, et al. 2024. Usable xai: 10 strategies towards exploiting explainability in the llm era. arXiv preprint arXiv:2403.08946

    2024 arXiv

  42. [50]

    Yibo Yan, Jiamin Su, Jianxiang He, Fangteng Fu, Xu Zheng, Yuanhuiyi Lyu, Kun Wang, Shen Wang, Qingsong Wen, and Xuming Hu. 2024 a . A survey of mathematical reasoning in the era of multimodal large language model: Benchmark, method & challenges. arXiv preprint arXiv:2412.11936

    2024 arXiv

  43. [51]

    Yibo Yan, Shen Wang, Jiahao Huo, Hang Li, Boyan Li, Jiamin Su, Xiong Gao, Yi-Fan Zhang, Tianlong Xu, Zhendong Chu, et al. 2024 b . Errorradar: Benchmarking complex mathematical reasoning of multimodal large language models via error detection. arXiv preprint arXiv:2410.04509

    2024 arXiv

  44. [52]

    Yibo Yan, Shen Wang, Jiahao Huo, Jingheng Ye, Zhendong Chu, Xuming Hu, Philip S Yu, Carla Gomes, Bart Selman, and Qingsong Wen. 2025. Position: Multimodal large language models can significantly advance scientific reasoning. arXiv preprint arXiv:2502.02871

    2025 arXiv

  45. [53]

    Haoqi Yang, Luohe Shi, Qiwei Li, Zuchao Li, Ping Wang, Bo Du, Mengjia Shen, and Hai Zhao. 2025. Faster moe llm inference for extremely large models. arXiv preprint arXiv:2505.03531

    2025 arXiv

  46. [54]

    Yunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang, Ziwen Xu, Shumin Deng, and Huajun Chen. 2024. Knowledge circuits in pretrained transformers. arXiv preprint arXiv:2405.17969

    2024 arXiv

  47. [55]

    Yichao Yuan, Lin Ma, and Nishil Talati. 2025. Moe-lens: Towards the hardware limit of high-throughput moe llm serving under resource constraints. arXiv preprint arXiv:2504.09345

    2025 arXiv

  48. [56]

    Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. 2023. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792

    2023

  49. [57]

    Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing. 2024. How do large language models handle multilingualism? arXiv preprint arXiv:2402.18815

    2024 arXiv

  50. [58]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  51. [59]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.