REVIEW 3 major objections 5 minor 1 cited by
A Survey of Large Language Models in Discipline-specific Research: Challenges, Methods and Opportunities
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Large language models adapted for research fall into two families—training the model's own knowledge or connecting it to external tools—and this survey maps that split across five disciplines.
desk verdict A serviceable survey of LLMs across disciplines; the taxonomy is a loose organizing device and the 'comprehensive' claim needs methodology, but newcomers will find it helpful. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The organizing device is the two-branch taxonomy 'Internal Knowledge Optimization versus External Interaction and Collaboration.' The internal branch contains three techniques—continued pre-training, supervised fine-tuning, and RLHF—that modify the model's parameters or training objective; the external branch contains four—prompt engineering, retrieval-augmented generation, agent-based methods, and tool-use integration—that modify the model's inputs, context, or environment. The taxonomy carries the argument by converting a large collection of discipline-specific systems into a small set of recurring adaptation moves, which is what allows the survey to compare mathematics, physics, chemistry, biology, and the humanities and social sciences on common ground.
What would settle it
Find a widely used discipline-specific LLM method that improves task performance through neither weight updates nor external interaction—for example a pure decoding-time reasoning strategy, a latent-space steering method, or an architecture change—and the taxonomy loses its exhaustiveness. A more direct check is to take a complete collection of LLM-for-science papers and test whether every method can be placed in one of the seven named subcategories; any substantial residue invalidates the comprehensiveness claim.
Extended reading notes
Core claim
The paper's central claim is that the scattered applications of LLMs across disciplines are not a random collection but fall into a principled taxonomy with two branches. The first branch, Internal Knowledge Optimization, groups techniques that change the model itself: continued pre-training on domain corpora, supervised fine-tuning on labeled or instructional data, and RLHF to align outputs with human preference. The second branch, External Interaction and Collaboration, groups techniques that keep the model fixed and extend it through prompt engineering, retrieval-augmented generation, agent-based methods, and tool-use integration. The survey asserts that this taxonomy 'elucidates how these models enhance research across different disciplines,' and it supports the claim by placing representative systems—from theorem provers in mathematics to laboratory agents in chemistry to social simulations in the social sciences—into the relevant cells. The paper further claims that the field's main obstacles are data quality, user barriers for non-AI experts, absent standardized benchmarks, and computational cost, and that these are where future work should concentrate.
Load-bearing premise
The load-bearing premise is that every significant way of adapting an LLM for research falls into one of two buckets—changing the model's internal knowledge or connecting it to outside sources and tools—and that this split gives an exhaustive and accurate picture of the field.
Editorial extensions
If this is right
- A researcher entering a new discipline can pick an adaptation strategy by asking whether the gap is missing knowledge inside the model (train it) or missing access to data, tools, or environment (connect it).
- Technique families recur across all five disciplines, so a method that works in one field—such as retrieval-augmented generation for physics question answering—is a plausible candidate for transfer to another knowledge-intensive field.
- The paper's challenge list implies that the next bottleneck for LLM-assisted research is not raw model scale but the availability of high-quality domain datasets, standard evaluation benchmarks, and tools usable by non-AI specialists.
- If the taxonomy is accepted, new methods can be located by their position on the internal/external axis, making future surveys and comparisons more structured.
Reading between the lines
- The two branches are better understood as complementary axes than as exclusive categories: real deployments often fine-tune a model and then wrap it in RAG or an agent loop, so the taxonomy's main use may be to describe combinations rather than disjoint classes.
- A testable extension would be to sample a large corpus of LLM-for-science papers and classify each method by this taxonomy; the fraction of systems that straddle both branches would measure whether the binary split needs refinement.
- The disciplinary pattern the survey displays—mathematics favoring internal optimization, chemistry and biology favoring external tools and agents—could be turned into a quantitative test by counting technique usage per discipline across a systematic corpus.
- Because the survey does not claim to have proven the completeness of its example selection, its map is best used as a scaffold for organizing the literature rather than as an exhaustive census of published methods.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper is a survey of recent applications of large language models to discipline-specific research. It proposes a taxonomy of adaptation techniques divided into 'Internal Knowledge Optimization' (continued pre-training, supervised fine-tuning, RLHF) and 'External Interaction and Collaboration' (prompt engineering, RAG, agent-based methods, tool-use integration), then reviews applications in mathematics, physics, chemistry, biology, and the humanities/social sciences, and closes with challenges and future directions. The abstract and conclusion claim comprehensiveness and a 'novel principled taxonomy.'
Significance. If the taxonomy were rigorous and the coverage systematically selected, the survey would be a useful entry point for researchers mapping LLM-for-science methods. The paper collects a broad set of recent works and connects them to per-discipline tasks, and its discussion of challenges (data quality, evaluation benchmarks, computational cost) is sensible. Its main value is as a structured bibliography. However, the absence of a documented selection methodology and the looseness of the central taxonomy currently cap its usefulness; the 'comprehensive' and 'principled' claims are not yet supported. No code or data accompanies the paper; strengths are breadth of recent citations and explicit enumeration of challenges.
major comments (3)
- [§3.2.1 / §3 (taxonomy)] The classification of prompt engineering as External Interaction and Collaboration is not as cleanly wrong as a purely external-resource reading would suggest, because §3.2's definition explicitly includes 'user-provided information,' and prompts are user-provided. But under that broad reading the internal/external split becomes weak: ordinary task instructions also qualify as external interaction, and ICL/CoT are inference-time mechanisms that elicit reasoning from the model's internal parameters. The paper neither defends the exhaustiveness of the two top-level categories nor discusses excluded alternatives (e.g., architecture modifications, decoding-time methods such as self-consistency). Since Section 6 calls the taxonomy 'novel principled,' the organizing criterion needs to be sharpened or the claim qualified.
- [§1, §6, Figure 1] The abstract and conclusion assert a 'comprehensive overview,' but the paper provides no search strategy, inclusion/exclusion criteria, time window, or quality filter for the surveyed literature. Consequently, Figure 1's per-discipline lists read as illustrative rather than exhaustive, and the reader cannot verify the comprehensiveness claim. Please add a methodology subsection or change the claim to reflect a curated selection.
- [§4.2.1] The sentence 'In quantum mechanics, fine-tuned LLMs can generate hypotheses about quantum entanglement or propose experiments to test current models, opening new avenues for scientific discovery' is presented as a statement about existing work but is given no citation. This is a specific capability claim; please cite supporting work or remove the claim.
minor comments (5)
- [§3.2.3] The citation '(Park et al., 2023; Li, G. et al., 2023)(Park et al., 2023; Li et al., 2023a)' is duplicated and should be a single citation in a consistent format.
- [References] Fang et al. 2024a and 2024b are the same Mol-Instructions paper under two keys; these should be merged.
- [References] Two different papers (CulturePark and X-instruction) share the key 'Li et al. 2024b', which makes the in-text citation 'Li et al., 2024a,b' in §4.5.1 ambiguous.
- [§4.3.3] The paper uses 'ChemCrow (M. Bran et al., 2024)' here but 'Bran et al., 2023' in §3.2.4 and Figure 1; use one consistent citation for this work.
- [General language] The verb 'feature' is used in nonstandard ways (e.g., 'they also feature significant challenges'); consider using 'face' or 'present' for clarity.
Circularity Check
No significant circularity: the paper is a descriptive survey whose taxonomy is stipulated, not derived, and no prediction reduces to its inputs.
full rationale
This is a literature survey, not a derivational paper: it contains no fitted parameters, no equations, no predictive claims that could reduce to its inputs, and no uniqueness theorem imported from prior work. The central organizational claim—the two-part taxonomy of Internal Knowledge Optimization versus External Interaction and Collaboration (Section 3, Figure 1)—is a stipulated categorization of known, externally documented techniques (CPT, SFT, RLHF, prompt engineering, RAG, agents, tools), so the taxonomy can be assessed for adequacy or consistency, but it is not circular: the examples are not derived from the taxonomy, and the taxonomy is not derived from the examples. The paper's self-citations (e.g., Zhao et al. 2023c in Section 2.1; Li et al. 2024a and Zhang et al. 2025a,b in Section 4.5.1) are illustrative and appear alongside independent primary citations such as Vaswani (2017); none is load-bearing for a central claim. The survey's conclusion that LLMs support discipline-specific research is backed by numerous external applications and benchmarks, not by the survey's own definitions. Accordingly, no circular step can be exhibited from the manuscript text.
Assumptions & free parameters
assumptions (2)
- domain assumption The cited papers are accurately summarized and their findings are reliable.
- domain assumption The binary taxonomy (Internal Knowledge Optimization vs. External Interaction and Collaboration) is a valid and exhaustive organization of LLM adaptation techniques.
Cite this review
Pith. "Pith review of A Survey of Large Language Models in Discipline-specific Research: Challenges, Methods and Opportunities." pith.science (2026). https://pith.science/paper/H2QVRDCV
@misc{pith2026250708425,
author = {Pith},
title = {Pith review of: A Survey of Large Language Models in Discipline-specific Research: Challenges, Methods and Opportunities},
year = {2026},
howpublished = {\url{https://pith.science/paper/H2QVRDCV}},
note = {Machine review of arXiv:2507.08425}
}
read the original abstract
Large Language Models (LLMs) have demonstrated their transformative potential across numerous disciplinary studies, reshaping the existing research methodologies and fostering interdisciplinary collaboration. However, a systematic understanding of their integration into diverse disciplines remains underexplored. This survey paper provides a comprehensive overview of the application of LLMs in interdisciplinary studies, categorising research efforts from both a technical perspective and with regard to their applicability. From a technical standpoint, key methodologies such as supervised fine-tuning, retrieval-augmented generation, agent-based approaches, and tool-use integration are examined, which enhance the adaptability and effectiveness of LLMs in discipline-specific contexts. From the perspective of their applicability, this paper explores how LLMs are contributing to various disciplines including mathematics, physics, chemistry, biology, and the humanities and social sciences, demonstrating their role in discipline-specific tasks. The prevailing challenges are critically examined and the promising research directions are highlighted alongside the recent advances in LLMs. By providing a comprehensive overview of the technical developments and applications in this field, this survey aims to serve as an invaluable resource for the researchers who are navigating the complex landscape of LLMs in the context of interdisciplinary studies.
Figures
Forward citations
Cited by 1 Pith paper
-
Enough is as good as a feast: A Comprehensive Analysis of How Reinforcement Learning Mitigates Task Conflicts in LLMs
RL-trained LLMs keep most of their skills after weight merging, while SFT-trained LLMs drop about 19% on average, because RL keeps parameter updates smaller and more task-compatible.
Reference graph
Works this paper leans on
-
[4]
From Generalist to Specialist: A Survey of Large Language Models for Chemistry
From generalist to specialist: A survey of large language models for chemistry. arXiv preprint arXiv:2412.19994. Joy He-Yueya, Gabriel Poesia, Rose E Wang, and Noah D Goodman. 2023. Solving math word prob- lems by combining language models with symbolic solvers. arXiv preprint arXiv:2304.09102. Michael Heseltine and Bernhard Clemm von Hohenberg
work page Pith review arXiv 2023
-
[5]
Research & Politics, 11(1):20531680241236239
Large language models as a substitute for human experts in annotating political text. Research & Politics, 11(1):20531680241236239. Kaixuan Huang, Yuanhao Qu, Henry Cousins, William A Johnson, Di Yin, Mihir Shah, Denny Zhou, Russ Altman, Mengdi Wang, and Le Cong. 2024a. Crispr-gpt: An llm agent for automated de- sign of gene-editing experiments. arXiv pre...
-
[6]
arXiv preprint arXiv:2412.16720
Openai o1 system card. arXiv preprint arXiv:2412.16720. Raj Jaiswal, Dhruv Jain, Harsh Parimal Popat, Avinash Anand, Abhishek Dharmadhikari, Atharva Marathe, and Rajiv Ratn Shah. 2024. Improving physics rea- soning in large language models using mixture of refinement agents. arXiv preprint arXiv:2412.00821. Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Sh...
arXiv 2024
-
[8]
Document image machine translation with dynamic multi-pre-trained models assembling. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7084–7095. Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Deng...
arXiv 2024
-
[9]
Nature Machine Intelligence, pages 1–11
Augmenting large language models with chem- istry tools. Nature Machine Intelligence, pages 1–11. Pingchuan Ma, Tsun-Hsuan Wang, Minghao Guo, Zhiqing Sun, Joshua B. Tenenbaum, Daniela Rus, Chuang Gan, and Wojciech Matusik. 2024. Llm and simulation as bilevel optimizers: a new paradigm to advance physical scientific discovery. In Proceed- ings of the 41st ...
arXiv 2024
-
[10]
arXiv preprint arXiv:2405.14734
Simpo: Simple preference optimization with a reference-free reward. arXiv preprint arXiv:2405.14734. Ali Najafi and Onur Varol. 2024. Turkishbertweet: Fast and reliable large language model for social media analysis. Expert Systems with Applications , 255:124737. Tarek Naous, Michael J Ryan, Alan Ritter, and Wei Xu. 2023. Having beer after prayer? measuri...
arXiv 2024
-
[11]
Astrollama: Towards specialized foundation models in astronomy. In Proceedings of the Second Workshop on Information Extraction from Scientific Publications, pages 49–55, Bali, Indonesia. Associa- tion for Computational Linguistics. Odhran O’Donoghue, Aleksandar Shtedritski, John Gin- ger, Ralph Abboud, Ali Ghareeb, and Samuel Ro- driques. 2023. BioPlanne...
arXiv 2023
-
[12]
Astromlab 2: Astrollama-2-70b model and benchmarking specialised llms for astronomy. In SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Stor- age and Analysis, pages 87–96. IEEE. Xinyu Pang, Ruixin Hong, Zhanke Zhou, Fangrui Lv, Xinwei Yang, Zhilong Liang, Bo Han, and Chang- shui Zhang. 2024. Physics reasoner:...
arXiv 2024
Show all 15 references
-
[13]
Research Notes of the AAS, 8(1):7
Astrollama-chat: Scaling astrollama with con- versational and diverse datasets. Research Notes of the AAS, 8(1):7. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie...
2024
-
[14]
Advances in Neu- ral Information Processing Systems, 36
Direct preference optimization: Your language model is secretly a reward model. Advances in Neu- ral Information Processing Systems, 36. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented...
2023 arXiv
-
[15]
Advances in Neural Information Processing Systems, 36
Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in langua...
2023 arXiv
-
[2022]
Advances in neural information processing systems, 35:26337–26349
Hypertree proof search for neural theorem proving. Advances in neural information processing systems, 35:26337–26349. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et a...
2022 arXiv
-
[2023]
Political Analysis, 31(3):337–351
Out of one, many: Using language mod- els to simulate human samples. Political Analysis, 31(3):337–351. Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q Jiang, Jia Deng, Stella Biderman, and Sean Welleck. 2023. Llemma: An open la...
2023 arXiv
-
[2024]
Journal of Machine Learning Research, 25(70):1–53
Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53. Ioana Ciuc˘a, Yuan-Sen Ting, Sandor Kruk, and Kartheik Iyer. 2023. Harnessing the power of adversarial prompting and large language models for robust hy- pothesis generation in as...
2023 arXiv
-
[2025]
In Proceedings of the 31st Inter- national Conference on Computational Linguistics, pages 354–379, Abu Dhabi, UAE
InstructMol: Multi-modal integration for build- ing a versatile and reliable molecular assistant in drug discovery. In Proceedings of the 31st Inter- national Conference on Computational Linguistics, pages 354–379, Abu Dhabi, UAE. Association for Computational Linguistics. He ...
2024 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.