Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

A Survey of Large Language Models in Discipline-specific Research: Challenges, Methods and Opportunities

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read Large language models adapted for research fall into two families—training the model's own knowledge or connecting it to external tools—and this survey maps that split across five disciplines.

desk verdict A serviceable survey of LLMs across disciplines; the taxonomy is a loose organizing device and the 'comprehensive' claim needs methodology, but newcomers will find it helpful. read the letter →

arxiv 2507.08425 v1 pith:H2QVRDCV submitted 2025-07-11 cs.CL

classification cs.CL
keywords largelanguagemodelsinterdisciplinaryresearchtaxonomyretrieval-augmentedgenerationLLMagentstoolusedomainadaptationsurvey
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey aims to establish a usable map of how large language models are being adapted for research in mathematics, physics, chemistry, biology, and the humanities and social sciences. Its central proposal is a two-branch taxonomy: methods that optimize the model's internal knowledge through continued pre-training, supervised fine-tuning, or reinforcement learning from human feedback, and methods that leave the model's weights alone and instead give it prompts, retrieved evidence, agents, or tools. The paper reads the field's many systems as instances of these two moves and uses the taxonomy to compare how disciplines adopt LLMs. It also identifies four shared bottlenecks—domain data quality, usability for non-AI researchers, missing evaluation standards, and computational cost—and argues these, not model capability alone, now limit interdisciplinary progress. A sympathetic reader would take away a structured lens for placing any LLM-for-science method and seeing what problem it addresses.

What carries the argument

The organizing device is the two-branch taxonomy 'Internal Knowledge Optimization versus External Interaction and Collaboration.' The internal branch contains three techniques—continued pre-training, supervised fine-tuning, and RLHF—that modify the model's parameters or training objective; the external branch contains four—prompt engineering, retrieval-augmented generation, agent-based methods, and tool-use integration—that modify the model's inputs, context, or environment. The taxonomy carries the argument by converting a large collection of discipline-specific systems into a small set of recurring adaptation moves, which is what allows the survey to compare mathematics, physics, chemistry, biology, and the humanities and social sciences on common ground.

What would settle it

Find a widely used discipline-specific LLM method that improves task performance through neither weight updates nor external interaction—for example a pure decoding-time reasoning strategy, a latent-space steering method, or an architecture change—and the taxonomy loses its exhaustiveness. A more direct check is to take a complete collection of LLM-for-science papers and test whether every method can be placed in one of the seven named subcategories; any substantial residue invalidates the comprehensiveness claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that the scattered applications of LLMs across disciplines are not a random collection but fall into a principled taxonomy with two branches. The first branch, Internal Knowledge Optimization, groups techniques that change the model itself: continued pre-training on domain corpora, supervised fine-tuning on labeled or instructional data, and RLHF to align outputs with human preference. The second branch, External Interaction and Collaboration, groups techniques that keep the model fixed and extend it through prompt engineering, retrieval-augmented generation, agent-based methods, and tool-use integration. The survey asserts that this taxonomy 'elucidates how these models enhance research across different disciplines,' and it supports the claim by placing representative systems—from theorem provers in mathematics to laboratory agents in chemistry to social simulations in the social sciences—into the relevant cells. The paper further claims that the field's main obstacles are data quality, user barriers for non-AI experts, absent standardized benchmarks, and computational cost, and that these are where future work should concentrate.

Load-bearing premise

The load-bearing premise is that every significant way of adapting an LLM for research falls into one of two buckets—changing the model's internal knowledge or connecting it to outside sources and tools—and that this split gives an exhaustive and accurate picture of the field.

Editorial extensions

If this is right

  • A researcher entering a new discipline can pick an adaptation strategy by asking whether the gap is missing knowledge inside the model (train it) or missing access to data, tools, or environment (connect it).
  • Technique families recur across all five disciplines, so a method that works in one field—such as retrieval-augmented generation for physics question answering—is a plausible candidate for transfer to another knowledge-intensive field.
  • The paper's challenge list implies that the next bottleneck for LLM-assisted research is not raw model scale but the availability of high-quality domain datasets, standard evaluation benchmarks, and tools usable by non-AI specialists.
  • If the taxonomy is accepted, new methods can be located by their position on the internal/external axis, making future surveys and comparisons more structured.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The two branches are better understood as complementary axes than as exclusive categories: real deployments often fine-tune a model and then wrap it in RAG or an agent loop, so the taxonomy's main use may be to describe combinations rather than disjoint classes.
  • A testable extension would be to sample a large corpus of LLM-for-science papers and classify each method by this taxonomy; the fraction of systems that straddle both branches would measure whether the binary split needs refinement.
  • The disciplinary pattern the survey displays—mathematics favoring internal optimization, chemistry and biology favoring external tools and agents—could be turned into a quantitative test by counting technique usage per discipline across a systematic corpus.
  • Because the survey does not claim to have proven the completeness of its example selection, its map is best used as a scaffold for organizing the literature rather than as an exhaustive census of published methods.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper is a survey of recent applications of large language models to discipline-specific research. It proposes a taxonomy of adaptation techniques divided into 'Internal Knowledge Optimization' (continued pre-training, supervised fine-tuning, RLHF) and 'External Interaction and Collaboration' (prompt engineering, RAG, agent-based methods, tool-use integration), then reviews applications in mathematics, physics, chemistry, biology, and the humanities/social sciences, and closes with challenges and future directions. The abstract and conclusion claim comprehensiveness and a 'novel principled taxonomy.'

Significance. If the taxonomy were rigorous and the coverage systematically selected, the survey would be a useful entry point for researchers mapping LLM-for-science methods. The paper collects a broad set of recent works and connects them to per-discipline tasks, and its discussion of challenges (data quality, evaluation benchmarks, computational cost) is sensible. Its main value is as a structured bibliography. However, the absence of a documented selection methodology and the looseness of the central taxonomy currently cap its usefulness; the 'comprehensive' and 'principled' claims are not yet supported. No code or data accompanies the paper; strengths are breadth of recent citations and explicit enumeration of challenges.

major comments (3)
  1. [§3.2.1 / §3 (taxonomy)] The classification of prompt engineering as External Interaction and Collaboration is not as cleanly wrong as a purely external-resource reading would suggest, because §3.2's definition explicitly includes 'user-provided information,' and prompts are user-provided. But under that broad reading the internal/external split becomes weak: ordinary task instructions also qualify as external interaction, and ICL/CoT are inference-time mechanisms that elicit reasoning from the model's internal parameters. The paper neither defends the exhaustiveness of the two top-level categories nor discusses excluded alternatives (e.g., architecture modifications, decoding-time methods such as self-consistency). Since Section 6 calls the taxonomy 'novel principled,' the organizing criterion needs to be sharpened or the claim qualified.
  2. [§1, §6, Figure 1] The abstract and conclusion assert a 'comprehensive overview,' but the paper provides no search strategy, inclusion/exclusion criteria, time window, or quality filter for the surveyed literature. Consequently, Figure 1's per-discipline lists read as illustrative rather than exhaustive, and the reader cannot verify the comprehensiveness claim. Please add a methodology subsection or change the claim to reflect a curated selection.
  3. [§4.2.1] The sentence 'In quantum mechanics, fine-tuned LLMs can generate hypotheses about quantum entanglement or propose experiments to test current models, opening new avenues for scientific discovery' is presented as a statement about existing work but is given no citation. This is a specific capability claim; please cite supporting work or remove the claim.
minor comments (5)
  1. [§3.2.3] The citation '(Park et al., 2023; Li, G. et al., 2023)(Park et al., 2023; Li et al., 2023a)' is duplicated and should be a single citation in a consistent format.
  2. [References] Fang et al. 2024a and 2024b are the same Mol-Instructions paper under two keys; these should be merged.
  3. [References] Two different papers (CulturePark and X-instruction) share the key 'Li et al. 2024b', which makes the in-text citation 'Li et al., 2024a,b' in §4.5.1 ambiguous.
  4. [§4.3.3] The paper uses 'ChemCrow (M. Bran et al., 2024)' here but 'Bran et al., 2023' in §3.2.4 and Figure 1; use one consistent citation for this work.
  5. [General language] The verb 'feature' is used in nonstandard ways (e.g., 'they also feature significant challenges'); consider using 'face' or 'present' for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a descriptive survey whose taxonomy is stipulated, not derived, and no prediction reduces to its inputs.

full rationale

This is a literature survey, not a derivational paper: it contains no fitted parameters, no equations, no predictive claims that could reduce to its inputs, and no uniqueness theorem imported from prior work. The central organizational claim—the two-part taxonomy of Internal Knowledge Optimization versus External Interaction and Collaboration (Section 3, Figure 1)—is a stipulated categorization of known, externally documented techniques (CPT, SFT, RLHF, prompt engineering, RAG, agents, tools), so the taxonomy can be assessed for adequacy or consistency, but it is not circular: the examples are not derived from the taxonomy, and the taxonomy is not derived from the examples. The paper's self-citations (e.g., Zhao et al. 2023c in Section 2.1; Li et al. 2024a and Zhang et al. 2025a,b in Section 4.5.1) are illustrative and appear alongside independent primary citations such as Vaswani (2017); none is load-bearing for a central claim. The survey's conclusion that LLMs support discipline-specific research is backed by numerous external applications and benchmarks, not by the survey's own definitions. Accordingly, no circular step can be exhibited from the manuscript text.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The survey relies on accurate reporting of cited works and on the completeness of its proposed taxonomy; neither is empirically verified within the paper.

assumptions (2)
  • domain assumption The cited papers are accurately summarized and their findings are reliable.
    The survey's descriptions of prior work rely on the trustworthiness of the original papers; no independent verification is attempted.
  • domain assumption The binary taxonomy (Internal Knowledge Optimization vs. External Interaction and Collaboration) is a valid and exhaustive organization of LLM adaptation techniques.
    The paper does not justify why this split covers all relevant techniques or why other categorization schemes are inferior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Survey of Large Language Models in Discipline-specific Research: Challenges, Methods and Opportunities." pith.science (2026). https://pith.science/paper/H2QVRDCV

@misc{pith2026250708425,
  author       = {Pith},
  title        = {Pith review of: A Survey of Large Language Models in Discipline-specific Research: Challenges, Methods and Opportunities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H2QVRDCV}},
  note         = {Machine review of arXiv:2507.08425}
}
read the original abstract

Large Language Models (LLMs) have demonstrated their transformative potential across numerous disciplinary studies, reshaping the existing research methodologies and fostering interdisciplinary collaboration. However, a systematic understanding of their integration into diverse disciplines remains underexplored. This survey paper provides a comprehensive overview of the application of LLMs in interdisciplinary studies, categorising research efforts from both a technical perspective and with regard to their applicability. From a technical standpoint, key methodologies such as supervised fine-tuning, retrieval-augmented generation, agent-based approaches, and tool-use integration are examined, which enhance the adaptability and effectiveness of LLMs in discipline-specific contexts. From the perspective of their applicability, this paper explores how LLMs are contributing to various disciplines including mathematics, physics, chemistry, biology, and the humanities and social sciences, demonstrating their role in discipline-specific tasks. The prevailing challenges are critically examined and the promising research directions are highlighted alongside the recent advances in LLMs. By providing a comprehensive overview of the technical developments and applications in this field, this survey aims to serve as an invaluable resource for the researchers who are navigating the complex landscape of LLMs in the context of interdisciplinary studies.

Figures

Figures reproduced from arXiv: 2507.08425 by the authors.

Figure 1
Figure 1. Taxonomy of LLM techniques for different disciplines. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Enough is as good as a feast: A Comprehensive Analysis of How Reinforcement Learning Mitigates Task Conflicts in LLMs

    cs.CL 2026-07 conditional novelty 6.0 of 10

    RL-trained LLMs keep most of their skills after weight merging, while SFT-trained LLMs drop about 19% on average, because RL keeps parameter updates smaller and more task-compatible.

Reference graph

Works this paper leans on

15 extracted references · 2 canonical work pages · cited by 1 Pith paper

  1. [4]

    From Generalist to Specialist: A Survey of Large Language Models for Chemistry

    From generalist to specialist: A survey of large language models for chemistry. arXiv preprint arXiv:2412.19994. Joy He-Yueya, Gabriel Poesia, Rose E Wang, and Noah D Goodman. 2023. Solving math word prob- lems by combining language models with symbolic solvers. arXiv preprint arXiv:2304.09102. Michael Heseltine and Bernhard Clemm von Hohenberg

  2. [5]

    Research & Politics, 11(1):20531680241236239

    Large language models as a substitute for human experts in annotating political text. Research & Politics, 11(1):20531680241236239. Kaixuan Huang, Yuanhao Qu, Henry Cousins, William A Johnson, Di Yin, Mihir Shah, Denny Zhou, Russ Altman, Mengdi Wang, and Le Cong. 2024a. Crispr-gpt: An llm agent for automated de- sign of gene-editing experiments. arXiv pre...

  3. [6]

    arXiv preprint arXiv:2412.16720

    Openai o1 system card. arXiv preprint arXiv:2412.16720. Raj Jaiswal, Dhruv Jain, Harsh Parimal Popat, Avinash Anand, Abhishek Dharmadhikari, Atharva Marathe, and Rajiv Ratn Shah. 2024. Improving physics rea- soning in large language models using mixture of refinement agents. arXiv preprint arXiv:2412.00821. Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Sh...

  4. [8]

    Document image machine translation with dynamic multi-pre-trained models assembling. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7084–7095. Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Deng...

  5. [9]

    Nature Machine Intelligence, pages 1–11

    Augmenting large language models with chem- istry tools. Nature Machine Intelligence, pages 1–11. Pingchuan Ma, Tsun-Hsuan Wang, Minghao Guo, Zhiqing Sun, Joshua B. Tenenbaum, Daniela Rus, Chuang Gan, and Wojciech Matusik. 2024. Llm and simulation as bilevel optimizers: a new paradigm to advance physical scientific discovery. In Proceed- ings of the 41st ...

  6. [10]

    arXiv preprint arXiv:2405.14734

    Simpo: Simple preference optimization with a reference-free reward. arXiv preprint arXiv:2405.14734. Ali Najafi and Onur Varol. 2024. Turkishbertweet: Fast and reliable large language model for social media analysis. Expert Systems with Applications , 255:124737. Tarek Naous, Michael J Ryan, Alan Ritter, and Wei Xu. 2023. Having beer after prayer? measuri...

  7. [11]

    In Proceedings of the Second Workshop on Information Extraction from Scientific Publications, pages 49–55, Bali, Indonesia

    Astrollama: Towards specialized foundation models in astronomy. In Proceedings of the Second Workshop on Information Extraction from Scientific Publications, pages 49–55, Bali, Indonesia. Associa- tion for Computational Linguistics. Odhran O’Donoghue, Aleksandar Shtedritski, John Gin- ger, Ralph Abboud, Ali Ghareeb, and Samuel Ro- driques. 2023. BioPlanne...

  8. [12]

    In SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Stor- age and Analysis, pages 87–96

    Astromlab 2: Astrollama-2-70b model and benchmarking specialised llms for astronomy. In SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Stor- age and Analysis, pages 87–96. IEEE. Xinyu Pang, Ruixin Hong, Zhanke Zhou, Fangrui Lv, Xinwei Yang, Zhilong Liang, Bo Han, and Chang- shui Zhang. 2024. Physics reasoner:...

Show all 15 references
  1. [13]

    Research Notes of the AAS, 8(1):7

    Astrollama-chat: Scaling astrollama with con- versational and diverse datasets. Research Notes of the AAS, 8(1):7. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie...

  2. [14]

    Advances in Neu- ral Information Processing Systems, 36

    Direct preference optimization: Your language model is secretly a reward model. Advances in Neu- ral Information Processing Systems, 36. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented...

  3. [15]

    Advances in Neural Information Processing Systems, 36

    Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in langua...

  4. [2022]

    Advances in neural information processing systems, 35:26337–26349

    Hypertree proof search for neural theorem proving. Advances in neural information processing systems, 35:26337–26349. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et a...

  5. [2023]

    Political Analysis, 31(3):337–351

    Out of one, many: Using language mod- els to simulate human samples. Political Analysis, 31(3):337–351. Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q Jiang, Jia Deng, Stella Biderman, and Sean Welleck. 2023. Llemma: An open la...

  6. [2024]

    Journal of Machine Learning Research, 25(70):1–53

    Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53. Ioana Ciuc˘a, Yuan-Sen Ting, Sandor Kruk, and Kartheik Iyer. 2023. Harnessing the power of adversarial prompting and large language models for robust hy- pothesis generation in as...

  7. [2025]

    In Proceedings of the 31st Inter- national Conference on Computational Linguistics, pages 354–379, Abu Dhabi, UAE

    InstructMol: Multi-modal integration for build- ing a versatile and reliable molecular assistant in drug discovery. In Proceedings of the 31st Inter- national Conference on Computational Linguistics, pages 354–379, Abu Dhabi, UAE. Association for Computational Linguistics. He ...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.