Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Research

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AI can automate the study of science itself by testing hypotheses in simulated research societies.

desk verdict A reasonable agenda paper whose preliminary demo overclaims: the simulated correlations may be inherited from OAG features that overlap the validation window, so treat the proof-of-concept as illustrative until leakage is controlled. read the letter →

arxiv 2505.12039 v1 pith:XSVPVLB4 submitted 2025-05-17 cs.AI cs.CLphysics.soc-ph

Renqi Chen , Haoyang Su , Shixiang Tang , Zhenfei Yin , Qi Wu , Hui Li , Ye Sun , Nanqing Dong
show 2 more authors
Wanli Ouyang Philip Torr
This is my paper · ORCID
classification cs.AIcs.CLphysics.soc-ph
keywords ScienceofAIformulti-agentsimulationlargelanguagemodelsresearchpatterndiscoveryscientificcollaborationcitationanalysisautomationhierarchy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that artificial intelligence can transform the field known as the Science of Science—the study of how research communities form, collaborate, and produce knowledge—by automating the whole pipeline of data processing, pattern analysis, simulation, and validation. The authors propose a five-level hierarchy of automation, from fully human-driven statistics to fully autonomous AI societies that design and run their own experiments, and they claim that AI offers a 'sandbox' in which hypotheses about science can be tested before being applied to real-world policy. As a proof of concept, they build a multi-agent system of one million large-language-model scientists that choose collaborators, discuss topics, write abstracts, and undergo peer review. After 40 simulated epochs, the citation counts produced by this synthetic community reproduce two real-world patterns: papers with more ethnically diverse author teams attract more citations, and papers from higher-ranked institutions attract fewer citations per output. The result is a first step, in the authors' view, toward making the study of science an experimentally testable discipline.

What carries the argument

The central mechanism is a large-scale multi-agent system in which each scientist is a language-model-driven agent that can communicate, retrieve papers from a shared reference database, and write in natural language. Agents are initialized from a real academic graph—names, affiliations, inferred ethnicity, citation histories, co-author lists, disciplines, and research topics—and the simulation proceeds through collaborator selection, topic discussion, idea generation, novelty assessment, abstract generation, and peer review, with accepted papers entering the database and updating citation counts. The system runs asynchronously so that a society of one million agents can be simulated in about a week, and team sizes are drawn from an exponential distribution fitted to historical data. This machinery is what lets the authors compare simulated citation patterns against real-world patterns and claim that the system can replicate and uncover research dynamics.

What would settle it

A decisive test would be to run the same simulation with author features scrambled—for example, randomly permuting ethnicity labels and affiliation rankings across agents—or with agents that retrieve and cite references by random similarity rather than by their learned judgments. If the ethnicity–citation and ranking–citation correlations persist nearly unchanged, they are an artifact of the input data; if they weaken or vanish, the agents' interactions are what generate the patterns.

Watch

Extended reading notes

Core claim

The central claim is that AI can become the foundation of next-generation Science of Science research, not merely as a tool for crunching bibliometric data but as a way to observe research processes in action and validate hypotheses in a virtual world. The paper defines AI for SoS (AI4SoS) as a meta-level enterprise distinct from using AI to solve domain problems: the object of study is the scientific ecosystem itself. The supporting empirical contribution is a preliminary multi-agent system in which scientist agents, initialized with real author characteristics from a large academic graph, form teams, generate ideas, write abstracts, go through peer review, and update citations in a shared database. The authors report that the simulated citation counts reproduce the real-world positive correlation with ethnic diversity and the negative correlation with affiliation ranking observed in 2010 and 2011 data, while the affiliation-diversity correlation is positive but not statistically significant. They take this as evidence that AI-driven simulations have the potential to replicate known patterns and, ultimately, to uncover new ones.

Load-bearing premise

The load-bearing assumption is that the simulated correlations arise from the agents' own simulated behavior rather than being inherited from the real author characteristics (such as ethnicity, affiliation rank, and past citation counts) that were pre-loaded into the agents from historical data; if those injected features alone explain the correlations, the proof-of-concept would only be echoing its inputs, not discovering anything.

Editorial extensions

If this is right

  • Researchers in the Science of Science could run controlled experiments on funding rules, team-size distributions, reviewer thresholds, or policy interventions inside a million-agent society before trying them in the real world.
  • The five-level automation hierarchy gives the field a shared vocabulary and a roadmap, making it clear which stages of research are already automatable and which still require human oversight.
  • The successful reproduction of the ethnicity-diversity and affiliation-ranking correlations suggests that LLM-driven agents can capture at least some emergent social dynamics of science, opening the door to discovering patterns that are invisible in retrospective statistics.
  • The failure of the affiliation-diversity correlation to reach significance in the simulation provides a concrete benchmark: any improved AI4SoS system should be expected to reproduce all three real-world correlations, not just the first two.
  • If fully realized, automated SoS discovery could make science-foresight tools—trend analysis, collaboration recommendations, policy evaluation—available to individual researchers and smaller institutions, not only to large labs with dedicated data teams.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that the same leakage concern applies to their scalability claim: if a million-agent simulation merely replays historical co-authorship structures, the 'emergent' patterns are not emergent in a behavioral sense, so the strongest defense of the sandbox idea will require perturbation experiments that change agent incentives and show aggregate patterns shi
  • The simulation could be extended to probe causal questions that retrospective SoS cannot answer, such as whether the diversity–citation link is driven by team composition itself or by the institutional contexts where diverse teams form; by blocking one pathway in the simulation while controlling the other, one could generate testable hypotheses for real-world data.
  • A natural next experiment would be to add explicit funding and career-advancement mechanisms to the agents, since the paper names these as missing; measuring how the ethnicity-diversity correlation responds would reveal how sensitive the reproduced pattern is to resource allocation.
  • The automation hierarchy could be repurposed as an evaluation instrument for the broader AI-for-science movement, mapping existing autonomous-science systems to levels 0–4 and exposing where the real bottleneck lies—simulation realism, validation metrics, or explainability—rather than treating each system in isolation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This manuscript is a forward-looking perspective on using AI to automate Science of Science (SoS) research. It defines AI4SoS, distinguishes it from AI for Science, proposes a five-level autonomy hierarchy (Level 0 through Level 4), surveys open problems in forecasting research trends and understanding research-society dynamics, and discusses challenges such as data bias, system construction, evaluation, and explainability. The paper's empirical core is a proof-of-concept in Sec. 5: a large-scale multi-agent system built on OAG data and LLaMA3.1-8B that simulates one million scientist-agents over 40 epochs. The authors claim that the simulation replicates real-world correlations between citation counts and ethnicity diversity, affiliation diversity, and affiliation ranking, thereby demonstrating that AI can automate pattern discovery and provide a sandbox for SoS experiments. Sections 6-8 provide alternative views, outlook, and conclusions.

Significance. If the central claim were fully supported, the manuscript would make a useful contribution by offering a conceptual framework for AI4SoS and by showing that LLM-based agent simulations can scale to a million agents while producing plausible research-society dynamics. The five-level autonomy hierarchy and the discussion of evaluation and causality are reasonable organizing contributions, and the engineering achievement of a million-agent asynchronous LLM simulation is nontrivial. However, the proof-of-concept's validation is the load-bearing part of the claim that the system can 'replicate and uncover key patterns in scientific research,' and that validation is currently inadequate because of feature-vs-validation-window leakage and the absence of control experiments. The paper also responsibly acknowledges limitations such as missing career trajectories and funding/policy influences, and it includes a peer-review prompt in Appendix B, but these do not compensate for the missing leakage controls. No code or data is provided, so the empirical results are not independently checkable.

major comments (3)
  1. [§5.1, Table 4; §5.2; §5.3] The claimed replication is vulnerable to target leakage. Table 4 states that each agent's citation count, co-author list, ethnicity, affiliation, affiliation ranking, discipline, and research topic are extracted from OAG papers published between 2010 and 2020, while the validation patterns are measured on real papers from 2010 and 2011 (Sec. 5.1 and Sec. 5.2). Because the agent features include the validation period and even future collaborations, the simulated correlations between citation counts and ethnicity/affiliation diversity could be inherited from the input joint distribution rather than generated by agent behavior. The co-author lists are especially problematic: the 'Collaborator Selection' stage can reassemble teams from the real 2010-2020 collaboration network, and the simulated papers' citation counts may then reflect the pre-assigned citation impact of those co-authors. The paper provides no leakage-control experiment, no holdout-based feature extraction, and no permutation or null baseline to rule out this inheritance. Without such a control, the abstract's statement that the system 'showcases AI's ability to replicate real-world research patterns' is not supported.
  2. [§5.3, Figs. 6-7] The statistical basis for the replication claim is too weak. The paper reports only qualitative agreement between real and simulated scatter plots, plus a single p-value for affiliation diversity, and that p-value is greater than 0.05. There are no confidence intervals, no R-squared values, no correlation coefficients with uncertainty, and no statement of how many simulated papers the scatter plots are based on. The caption of Fig. 6 refers to 'Strong correlations' in real data, but the text says the simulated correlations are 'slightly weaker,' and no quantitative comparison is given. The authors should report effect sizes and uncertainty for all three relationships, and should state the number of papers and the exact statistical test used for each comparison.
  3. [§5.1, Table 4; §5.2, Fig. 5; §5.3] The simulation's ability to 'uncover key patterns' is not distinguished from the reproduction of correlations already present in the input features. Ethnicity diversity, affiliation diversity, and average university ranking are attributes of the agents rather than outcomes of the simulated research process, and the team-size distribution is fit to OAG data (Fig. 5). A minimal control would be to randomize the ethnicity and affiliation labels, or to permute the citation-count assignments, and then verify that the simulated correlations disappear. The authors should also report a sensitivity analysis over the free parameters (team-size distribution parameters, peer review threshold, agent interaction hyperparameters, simulation scale and duration) to show that the observed correlations are not artifacts of specific parameter choices. Without such controls, the claim that the system 'replicate[s] and uncover[s] key patterns' conflates input-feature correlations with emergent AI-driven discovery.
minor comments (6)
  1. [Sec. 1] There is a typo in the phrase 'W e take the position' near the end of the introduction; it should read 'We take the position.'
  2. [§5.1, Table 5] The table says the year of initial-database papers is set to -1, but the text in Sec. 5.1 says the reference database contains papers from 2002 to 2009. Please clarify what -1 means in the simulation timeline and how it interacts with the epoch-based calendar.
  3. [Fig. 4] The x-axis labels '1060 320 1000' are unclear; they appear to be comma-separated numbers rendered without separators. Please label the axis clearly and state the units for all three values.
  4. [§5.3] The text says 'the pattern observed in the simulation is not statistically significant' but does not state which test was used or which of the three correlations it refers to. Please clarify the test, the sample size, and the direction of the non-significant result.
  5. [General] No code, configuration files, or simulation output data are provided, which makes the empirical results difficult to assess or reproduce. I encourage the authors to release the agent-initialization pipeline and the analysis scripts.
  6. [Fig. 6 caption] The caption says 'Strong correlations observed in real data are partially reproduced,' but the text in Sec. 5.3 says the simulated correlations are 'slightly weaker.' Please make the caption consistent with the reported effect sizes.

Circularity Check

1 steps flagged · score 5.0 of 10

Simulated pattern replication may be inherited from OAG features spanning the 2010-2011 validation window.

  1. fitted input called prediction [Sec. 5.1 Table 4, Sec. 5.1 environment construction, Sec. 5.2 experiments, Sec. 5.3 simulation results]
    "Citation: Extract the author’s published papers between 2010 to 2020 and calculate the total number of citations for the papers; ... Co-author: Extract the author’s published papers between 2010 to 2020 and record the collaborators in the papers ... We use papers from 2002 to 2009 as the reference database and papers from 2010 to 2011 as the validation database ... we analyze the citation counts of agent-generated papers to assess whether the system can replicate patterns observed in real-world data from the years 2010 to 2011."

    The agent-author features that initialize the simulation are extracted from OAG publications in 2010-2020, a range that includes the 2010-2011 validation window. Each agent's citation feature is the total citations of papers published in 2010-2020, so historical citation impact already encodes information about papers from the validation period; the co-author feature likewise imports the collaboration network that generates the 2010-2011 papers. Consequently, the simulated citation counts can inherit the joint distribution between ethnicity/affiliation and citation impact directly from the input features rather than from the agents' simulated research process.

full rationale

The paper is primarily a perspective piece, and its definitions, five-level autonomy hierarchy, and qualitative arguments are not derivations, so they do not raise circularity concerns. The only load-bearing empirical claim is the proof-of-concept in Sec. 5, and there the circularity issue is real but partial: the simulation initializes agents with citation counts, co-author lists, ethnicity, and affiliation drawn from OAG data covering 2010-2020, while the validation targets are real-world patterns measured on OAG papers from 2010-2011. This temporal overlap means the simulated correlations can be inherited from the input features instead of emerging from agent behavior; in particular, the pre-assigned citation counts and co-author networks already contain information about the validation period. The paper acknowledges only the non-significant affiliation-diversity result and missing model components, not this leakage. Because the simulation pipeline is complex and the paper does not formally fit the validation correlations, the circularity is not total, but the absence of any leakage control makes the central supporting claim only weakly supported. No load-bearing self-citation chain was found; citations such as [22] are used for inspiration and context rather than to force a conclusion.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The simulation depends on two classes of imported structure: fitted free parameters (team-size distribution, review threshold, hyperparameters) and domain assumptions about LLM realism, OAG accuracy, and the validity of the benchmark correlations. The central perspective claims do not rest on a physical model, so no physical entities are introduced; the only new conceptual artifact is the AI4SoS framing itself.

free parameters (4)
  • Team-size exponential distribution parameters = 0.339 * exp(-0.335 * (x-3)) + 0.002
    Fitted to team sizes of over 1,000,000 OAG papers from 2002-2009 (Fig. 5) and used to generate agent team sizes; this injects real-world team structure into the simulation.
  • Peer review acceptance threshold = score > 5
    Hand-set in Sec. 5.1; controls which agent papers enter the reference database and become citable, directly shaping citation counts and the observed correlations.
  • Agent interaction hyperparameters = up to 9 references, memory 5 entries, 3 reviewers, up to 3 teams per agent
    Chosen in Sec. 5.2 without sensitivity analysis; these constraints determine how papers are generated and cited in the simulation.
  • Simulation scale and duration = 1,000,000 agents, 40 epochs
    Set in Sec. 5.2; no demonstration that results are stable across scales or epoch counts.
assumptions (5)
  • domain assumption LLM agents (LLaMA3.1-8b) can behave sufficiently like real scientists for SoS pattern inference.
    Sec. 5.2 assigns all roles (discussion, idea generation, novelty, review) to the same 8B model; if agent behavior is not scientist-like, the reproduced correlations are artifacts.
  • domain assumption OAG-derived author and paper features are accurate enough for simulation.
    Table 4 uses a name-ethnicity classifier, THE rankings, and GPT-4 discipline classification; errors in these fields propagate into diversity and ranking metrics.
  • domain assumption The real-world correlations in [11] and the 2010-2011 OAG subset are valid ground truth for validation.
    Sec. 5.3 treats the real data correlations as the target pattern; if those patterns are dataset artifacts, the simulation reproduction is meaningless.
  • domain assumption Simulated citation counts measure research impact.
    Sec. 5.2 follows [11, 30, 82] in using citations as the impact metric, although the simulated citation process is a simplified retrieval-and-review loop.
  • ad hoc to paper LLM peer review scores with threshold 5 are a reasonable filter for scientific quality.
    Appendix B adapts NeurIPS review guidelines without validating that the LLM review distribution matches real peer review; threshold and scales are chosen by hand.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Research." pith.science (2026). https://pith.science/paper/XSVPVLB4

@misc{pith2026250512039,
  author       = {Pith},
  title        = {Pith review of: AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Research},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XSVPVLB4}},
  note         = {Machine review of arXiv:2505.12039}
}
read the original abstract

The Science of Science (SoS) explores the mechanisms underlying scientific discovery, and offers valuable insights for enhancing scientific efficiency and fostering innovation. Traditional approaches often rely on simplistic assumptions and basic statistical tools, such as linear regression and rule-based simulations, which struggle to capture the complexity and scale of modern research ecosystems. The advent of artificial intelligence (AI) presents a transformative opportunity for the next generation of SoS, enabling the automation of large-scale pattern discovery and uncovering insights previously unattainable. This paper offers a forward-looking perspective on the integration of Science of Science with AI for automated research pattern discovery and highlights key open challenges that could greatly benefit from AI. We outline the advantages of AI over traditional methods, discuss potential limitations, and propose pathways to overcome them. Additionally, we present a preliminary multi-agent system as an illustrative example to simulate research societies, showcasing AI's ability to replicate real-world research patterns and accelerate progress in Science of Science research.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AI4Research: A Survey of Artificial Intelligence for Scientific Research

    cs.CL 2025-07 conditional novelty 4.0 of 10

    A survey that organizes AI-for-research work into five tasks, comprehension, survey, discovery, writing, and peer review, and compiles associated tools and benchmarks.

Reference graph

Works this paper leans on

114 extracted references · 62 canonical work pages · cited by 1 Pith paper

  1. [1]

    Science359(6379), 0185 (2018)

    Fortunato, S., Bergstrom, C.T., B¨ orner, K., Evans, J.A., Helbing, D., Milojevi´ c, S., Petersen, A.M., Radicchi, F., Sinatra, R., Uzzi, B.,et al.: Science of science. Science359(6379), 0185 (2018)

    2018

  2. [2]

    Journal of Informetrics 3(3), 210–221 (2009)

    Bettencourt, L.M., Kaiser, D.I., Kaur, J.: Scientific discovery and topo- logical transitions in collaboration networks. Journal of Informetrics 3(3), 210–221 (2009)

    2009

  3. [3]

    Social Networks43, 73–85 (2015)

    Shi, F., Foster, J.G., Evans, J.A.: Weaving the fabric of science: Dynamic network models of science’s unfolding structure. Social Networks43, 73–85 (2015)

    2015

  4. [4]

    Klavans, R., Boyack, K.W.: Which type of citation analysis generates the most accurate taxonomy of scientific and technical knowledge? Journal of the Association for Information Science and Technology68(4), 984–998 (2017)

    2017

  5. [5]

    In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020, pp

    Wang, D., Liu, L.: The science of science. In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020, pp. 563–564 (2020)

    2020

  6. [6]

    Nature 610(7930), 120–127 (2022)

    Wapman, K.H., Zhang, S., Clauset, A., Larremore, D.B.: Quantify- ing hierarchy and dynamics in us faculty hiring and retention. Nature 610(7930), 120–127 (2022)

    2022

  7. [7]

    Advances in Neural Information Processing Systems36(2024)

    Gao, B., Qiang, B., Tan, H., Jia, Y., Ren, M., Lu, M., Liu, J., Ma, W.-Y., Lan, Y.: Drugclip: Contrasive protein-molecule representation learn- ing for virtual screening. Advances in Neural Information Processing Systems36(2024)

    2024

  8. [8]

    Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A.J., Bambrick, J., et al.: Ac- curate structure prediction of biomolecular interactions with alphafold

Show all 114 references
  1. [9]

    Nature Communications 15(1), 2323 (2024)

    Chang, J., Ye, J.C.: Bidirectional generation of structure and properties through a single molecular foundation model. Nature Communications 15(1), 2323 (2024)

    2024

  2. [10]

    Proceedings of the National Academy of Sciences112(47), 14569–14574 (2015)

    Rzhetsky, A., Foster, J.G., Foster, I.T., Evans, J.A.: Choosing exper- iments to accelerate collective discovery. Proceedings of the National Academy of Sciences112(47), 14569–14574 (2015)

    2015

  3. [11]

    Nature communications9(1), 5163 24AI4SoS (2018)

    AlShebli, B.K., Rahwan, T., Woon, W.L.: The preeminence of ethnic diversity in scientific collaboration. Nature communications9(1), 5163 24AI4SoS (2018)

    2018

  4. [12]

    Nature559(7714), 396–399 (2018)

    Liu, L., Wang, Y., Sinatra, R., Giles, C.L., Song, C., Wang, D.: Hot streaks in artistic, cultural, and scientific careers. Nature559(7714), 396–399 (2018)

    2018

  5. [13]

    Nature575(7781), 190–194 (2019)

    Yin, Y., Wang, Y., Evans, J.A., Wang, D.: Quantifying the dynamics of failure across science, startups and security. Nature575(7781), 190–194 (2019)

    2019

  6. [14]

    Nature Communications14, 1641 (2023)

    Shi, F., Evans, J.: Surprising combinations of research contents and con- texts are related to impact and emerge with scientific outsiders from distant disciplines. Nature Communications14, 1641 (2023)

    2023

  7. [15]

    Journal of chemical information and modeling59(9), 3692–3702 (2019)

    Weston, L., Tshitoyan, V., Dagdelen, J., Kononova, O., Trewartha, A., Persson, K.A., Ceder, G., Jain, A.: Named entity recognition and normal- ization applied to large-scale information extraction from the materials science literature. Journal of chemical information and model...

    2019

  8. [16]

    Science advances1(1), 1400005 (2015)

    Clauset, A., Arbesman, S., Larremore, D.B.: Systematic inequality and hierarchy in faculty hiring networks. Science advances1(1), 1400005 (2015)

    2015

  9. [17]

    In: Ijcai, pp

    Xiao, S., Yan, J., Li, C., Jin, B., Wang, X., Yang, X., Chu, S.M., Zha, H.: On modeling and predicting individual paper citation count over time. In: Ijcai, pp. 2676–2682 (2016)

    2016

  10. [18]

    Proceedings of the National Academy of Sciences117(4), 1910–1916 (2020)

    Krenn, M., Zeilinger, A.: Predicting research trends with semantic and neural networks with an application in quantum physics. Proceedings of the National Academy of Sciences117(4), 1910–1916 (2020)

    2020

  11. [19]

    Scientometrics111, 1053–1070 (2017)

    Van Eck, N.J., Waltman, L.: Citation-based clustering of publica- tions using citnetexplorer and vosviewer. Scientometrics111, 1053–1070 (2017)

    2017

  12. [20]

    Proceedings of the National Academy of Sciences117(17), 9284–9291 (2020)

    Hofstra, B., Kulkarni, V.V., Munoz-Najar Galvez, S., He, B., Juraf- sky, D., McFarland, D.A.: The diversity–innovation paradox in science. Proceedings of the National Academy of Sciences117(17), 9284–9291 (2020)

    2020

  13. [21]

    arXiv preprint arXiv:2409.05556 (2024)

    Ghafarollahi, A., Buehler, M.J.: Sciagents: Automating scientific dis- covery through multi-agent intelligent graph reasoning. arXiv preprint arXiv:2409.05556 (2024)

    2024 arXiv

  14. [22]

    arXiv preprint arXiv:2410.09403 (2024)

    Su, H., Chen, R., Tang, S., Zheng, X., Li, J., Yin, Z., Ouyang, W., Dong, N.: Two heads are better than one: A multi-agent system AI4SoS25 has the potential to improve scientific idea generation. arXiv preprint arXiv:2410.09403 (2024)

    2024 arXiv

  15. [23]

    arXiv preprint arXiv:2411.11581 (2024)

    Yang, Z., Zhang, Z., Zheng, Z., Jiang, Y., Gan, Z., Wang, Z., Ling, Z., Chen, J., Ma, M., Dong, B., et al.: Oasis: Open agents social interac- tion simulations on one million agents. arXiv preprint arXiv:2411.11581 (2024)

    2024 arXiv

  16. [24]

    Materials Today Bio, 101014 (2024)

    Li, Z., Song, P., Li, G., Han, Y., Ren, X., Bai, L., Su, J.: Ai energized hydrogel design, optimization and application in biomedicine. Materials Today Bio, 101014 (2024)

    2024

  17. [25]

    Telematics and Informatics Reports, 100127 (2024)

    Ofosu-Ampong, K.: Artificial intelligence research: A review on dominant themes, methods, frameworks and future research directions. Telematics and Informatics Reports, 100127 (2024)

    2024

  18. [26]

    Physical review letters120(4), 048301 (2018)

    Iacopini, I., Milojevi´ c, S., Latora, V.: Network dynamics of innovation processes. Physical review letters120(4), 048301 (2018)

    2018

  19. [27]

    PLoS biology19(7), 3001313 (2021)

    Bolt, T., Nomi, J.S., Bzdok, D., Uddin, L.Q.: Educating the future gen- eration of researchers: A cross-disciplinary survey of trends in analysis methods. PLoS biology19(7), 3001313 (2021)

    2021

  20. [28]

    Proceedings of the National Academy of Sciences115(50), 12573–12581 (2018)

    B¨ orner, K., Rouse, W.B., Trunfio, P., Stanley, H.E.: Forecasting innova- tions in science, technology, and education. Proceedings of the National Academy of Sciences115(50), 12573–12581 (2018)

    2018

  21. [29]

    AI open1, 57–81 (2020)

    Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., Sun, M.: Graph neural networks: A review of methods and applications. AI open1, 57–81 (2020)

    2020

  22. [30]

    Nature communications10(1), 4331 (2019)

    Wang, Y., Jones, B.F., Wang, D.: Early-career setback and future career impact. Nature communications10(1), 4331 (2019)

    2019

  23. [31]

    Nature566, 378–382 (2019)

    Wu, L., Wang, D., Evans, J.A.: Large teams develop and small teams disrupt science and technology. Nature566, 378–382 (2019)

    2019

  24. [32]

    Science316(5827), 1036–1039 (2007)

    Wuchty, S., Jones, B.F., Uzzi, B.: The increasing dominance of teams in production of knowledge. Science316(5827), 1036–1039 (2007)

    2007

  25. [33]

    Proceedings of the National Academy of Sciences119(36), 2200841119 (2022)

    Yang, Y., Tian, T.Y., Woodruff, T.K., Jones, B.F., Uzzi, B.: Gender- diverse teams produce more novel and higher-impact scientific ideas. Proceedings of the National Academy of Sciences119(36), 2200841119 (2022)

    2022

  26. [34]

    Proceedings of the National Academy of Sciences120(13), 2215324120 (2023)

    Liu, F., Rahwan, T., AlShebli, B.: Non-white scientists appear on fewer 26AI4SoS editorial boards, spend more time under review, and receive fewer ci- tations. Proceedings of the National Academy of Sciences120(13), 2215324120 (2023)

    2023

  27. [35]

    Nature Human Behaviour6(7), 919–929 (2022)

    Gomez, C.J., Herman, A.C., Parigi, P.: Leading countries in global science increasingly receive more citations than other countries doing similar research. Nature Human Behaviour6(7), 919–929 (2022)

    2022

  28. [36]

    Science308(5722), 697–702 (2005)

    Guimera, R., Uzzi, B., Spiro, J., Amaral, L.A.N.: Team assembly mecha- nisms determine collaboration network structure and team performance. Science308(5722), 697–702 (2005)

    2005

  29. [37]

    Complex & Intelligent Systems3, 105–120 (2017)

    Fern´ andez, A., del R´ ıo, S., Chawla, N.V., Herrera, F.: An insight into imbalanced big data classification: outcomes and challenges. Complex & Intelligent Systems3, 105–120 (2017)

    2017

  30. [38]

    ACM computing surveys (CSUR)52(4), 1–36 (2019)

    Kaur, H., Pannu, H.S., Malhi, A.K.: A systematic review on imbalanced data challenges in machine learning: Applications and solutions. ACM computing surveys (CSUR)52(4), 1–36 (2019)

    2019

  31. [39]

    Journal of Big Data5(1), 1–30 (2018)

    Leevy, J.L., Khoshgoftaar, T.M., Bauder, R.A., Seliya, N.: A survey on addressing high-class imbalance in big data. Journal of Big Data5(1), 1–30 (2018)

    2018

  32. [40]

    Journal of big data6(1), 1–54 (2019)

    Johnson, J.M., Khoshgoftaar, T.M.: Survey on deep learning with class imbalance. Journal of big data6(1), 1–54 (2019)

    2019

  33. [41]

    Nature human behaviour7(7), 1046–1058 (2023)

    Liu, L., Jones, B.F., Uzzi, B., Wang, D.: Data, measurement and em- pirical methods in the science of science. Nature human behaviour7(7), 1046–1058 (2023)

    2023

  34. [42]

    arXiv preprint arXiv:2303.08774 (2023)

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    2023 arXiv

  35. [44]

    5 technical report

    Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., et al.: Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115 (2024)

    2024 arXiv

  36. [45]

    ACM computing surveys (CSUR)54(6), 1–35 (2021) AI4SoS27

    Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning. ACM computing surveys (CSUR)54(6), 1–35 (2021) AI4SoS27

    2021

  37. [46]

    In: Proceedings of the 24th International Conference on World Wide Web, pp

    Sinha, A., Shen, Z., Song, Y., Ma, H., Eide, D., Hsu, B.-J., Wang, K.: An overview of microsoft academic service (mas) and applications. In: Proceedings of the 24th International Conference on World Wide Web, pp. 243–246 (2015)

    2015

  38. [47]

    IEEE Transactions on Knowledge and Data Engineering35(9), 9225–9239 (2022)

    Zhang, F., Liu, X., Tang, J., Dong, Y., Yao, P., Zhang, J., Gu, X., Wang, Y., Kharlamov, E., Shao, B.,et al.: Oag: Linking entities across large- scale heterogeneous knowledge graphs. IEEE Transactions on Knowledge and Data Engineering35(9), 9225–9239 (2022)

    2022

  39. [48]

    Scientific Data10(1), 315 (2023)

    Lin, Z., Yin, Y., Liu, L., Wang, D.: Sciscinet: A large-scale open data lake for the science of science research. Scientific Data10(1), 315 (2023)

    2023

  40. [49]

    Scatiggio, V.: Tackling the issue of bias in artificial intelligence to de- sign ai-driven fair and inclusive service systems. how human biases are breaching into ai algorithms, with severe impacts on individuals and so- cieties, and what designers can do to face this phenomen...

    2020

  41. [50]

    arXiv preprint arXiv:2110.05699 (2021)

    Prabhakaran, V., Davani, A.M., Diaz, M.: On releasing annotator-level labels and information in datasets. arXiv preprint arXiv:2110.05699 (2021)

    2021 arXiv

  42. [51]

    Simulating Social Complexity: A Handbook, 61–79 (2017)

    Norling, E., Edmonds, B., Meyer, R.: Informal approaches to developing simulation models. Simulating Social Complexity: A Handbook, 61–79 (2017)

    2017

  43. [52]

    Humanities and Social Sciences Communications11(1), 1–24 (2024)

    Gao, C., Lan, X., Li, N., Yuan, Y., Ding, J., Zhou, Z., Xu, F., Li, Y.: Large language models empowered agent-based modeling and sim- ulation: A survey and perspectives. Humanities and Social Sciences Communications11(1), 1–24 (2024)

    2024

  44. [53]

    In: 2023 IEEE International Conference on Smart Computing (SMARTCOMP), pp

    Khaleghian, S., Neema, H., Sartipi, M., Tran, T., Sen, R., Dubey, A.: Calibrating real-world city traffic simulation model using vehicle speed data. In: 2023 IEEE International Conference on Smart Computing (SMARTCOMP), pp. 303–308 (2023). IEEE

    2023

  45. [54]

    Journal of Artificial Societies and Social Simulation20(2) (2017)

    Schulze, J., M¨ uller, B., Groeneveld, J., Grimm, V.: Agent-based mod- elling of social-ecological systems: achievements, challenges, and a way forward. Journal of Artificial Societies and Social Simulation20(2) (2017)

    2017

  46. [55]

    Ecological Modelling457, 109685 (2021) 28AI4SoS

    An, L., Grimm, V., Sullivan, A., Turner Ii, B., Malleson, N., Heppenstall, A., Vincenot, C., Robinson, D., Ye, X., Liu, J.,et al.: Challenges, tasks, and opportunities in modeling agent-based complex systems. Ecological Modelling457, 109685 (2021) 28AI4SoS

    2021

  47. [56]

    Ebadi, A., Schiffauerova, A.: How to receive more funding for your re- search? get connected to the right people! PloS one10(7), 0133061 (2015)

    2015

  48. [57]

    Scientometrics116(1), 363–383 (2018)

    Ronda-Pupo, G.A., Pham, T.: The evolutions of the rich get richer and the fit get richer phenomena in scholarly networks: The case of the strategic management journal. Scientometrics116(1), 363–383 (2018)

    2018

  49. [58]

    Science as Culture29(4), 475–502 (2020)

    Katz, Y., Matter, U.: Metrics of inequality: The concentration of re- sources in the us biomedical elite. Science as Culture29(4), 475–502 (2020)

    2020

  50. [59]

    In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pp

    Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., Xu, J., Li, D., Liu, Z., Sun, M.: Chatdev: Com- municative agents for software development. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics...

    2024

  51. [60]

    arXiv preprint arXiv:2402.01680 (2024)

    Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N.V., Wiest, O., Zhang, X.: Large language model based multi-agents: A survey of progress and challenges. arXiv preprint arXiv:2402.01680 (2024)

    2024 arXiv

  52. [61]

    Aldrich, C., Auret, L.: Unsupervised Process Monitoring and Fault Diagnosis with Machine Learning Methods vol. 16. Springer, ??? (2013)

    2013

  53. [62]

    Pin and Wang, Dashun, The Philosopher’s Stone for Science–The Catalyst Change of AI for Scientific Creativity (March 5, 2024) (2024)

    Chen, Q., Ho, Y.-J.I., Sun, P., Wang, D.: The philosopher’s stone for science–the catalyst change of ai for scientific creativity. Pin and Wang, Dashun, The Philosopher’s Stone for Science–The Catalyst Change of AI for Scientific Creativity (March 5, 2024) (2024)

    2024

  54. [63]

    International Journal of Intelligent Systems2024 (2024)

    Balasubramaniam, S., Chirchi, V., Kadry, S., Agoramoorthy, M., Guru- rama, S.P., Satheesh, K.K., Sivakumar, T.: The road ahead: Emerging trends, unresolved issues, and concluding remarks in generative ai—a comprehensive review. International Journal of Intelligent Systems2024 (2024)

    2024

  55. [64]

    Challenges, and Ethical Considerations (September 02, 2024) (2024)

    Lissack, M., Meagher, B.: Navigating the future of large language models in scientific research: Opportunities, challenges, and ethical consider- ations. Challenges, and Ethical Considerations (September 02, 2024) (2024)

    2024

  56. [65]

    arXiv preprint arXiv:2408.06292 (2024)

    Lu, C., Lu, C., Lange, R.T., Foerster, J., Clune, J., Ha, D.: The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292 (2024)

    2024 arXiv

  57. [66]

    arXiv preprint arXiv:2408.01460 (2024)

    Meadows, G.I., Lau, N.W.L., Susanto, E.A., Yu, C.L., Paul, A.: Localval- uebench: A collaboratively built and extensible benchmark for evaluating AI4SoS29 localized value alignment and ethical safety in large language models. arXiv preprint arXiv:2408.01460 (2024)

    2024 arXiv

  58. [67]

    arXiv preprint arXiv:2406.04428 (2024)

    Ji, J., Chen, Y., Jin, M., Xu, W., Hua, W., Zhang, Y.: Moralbench: Moral evaluation of llms. arXiv preprint arXiv:2406.04428 (2024)

    2024 arXiv

  59. [68]

    Cognitive Computation16(1), 45–74 (2024)

    Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., Hussain, A.: Interpret- ing black-box models: a review on explainable artificial intelligence. Cognitive Computation16(1), 45–74 (2024)

    2024

  60. [69]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Reddy, C.K., Shojaee, P.: Towards scientific discovery with generative ai: Progress, opportunities, and challenges. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, pp. 28601–28609 (2025)

    2025

  61. [70]

    Sonnenwald, D.H.: Scientific collaboration. Annu. Rev. Inf. Sci. Technol. 41(1), 643–681 (2007)

    2007

  62. [71]

    Epidemiol- ogy25(3), 418–426 (2014)

    Petersen, M.L., van der Laan, M.J.: Causal models and learning from data: integrating causal modeling and statistical estimation. Epidemiol- ogy25(3), 418–426 (2014)

    2014

  63. [72]

    Nature Medicine30(4), 958–968 (2024)

    Feuerriegel, S., Frauen, D., Melnychuk, V., Schweisthal, J., Hess, K., Curth, A., Bauer, S., Kilbertus, N., Kohane, I.S., van der Schaar, M.: Causal machine learning for predicting treatment outcomes. Nature Medicine30(4), 958–968 (2024)

    2024

  64. [73]

    ACM Computing Surveys55(9), 1–33 (2023)

    Dwivedi, R., Dave, D., Naik, H., Singhal, S., Omer, R., Patel, P., Qian, B., Wen, Z., Shah, T., Morgan, G.,et al.: Explainable ai (xai): Core ideas, techniques, and solutions. ACM Computing Surveys55(9), 1–33 (2023)

    2023

  65. [74]

    Information Fusion106, 102301 (2024)

    Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Del Ser, J., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A.,et al.: Explain- able artificial intelligence (xai) 2.0: A manifesto of open challenges and interdisciplinary research directions. Information Fusi...

    2024

  66. [75]

    Unpublished manuscript, Institute for Quantitative Social Science, Harvard Univer- sity, Cambridge, MA (2011)

    King, G., Nielsen, R., Coberley, C., Pope, J.E., Wells, A.: Compara- tive effectiveness of matching methods for causal inference. Unpublished manuscript, Institute for Quantitative Social Science, Harvard Univer- sity, Cambridge, MA (2011)

    2011

  67. [76]

    Cambridge university press, ??? (2015) 30AI4SoS

    Imbens, G.W., Rubin, D.B.: Causal Inference in Statistics, Social, and Biomedical Sciences. Cambridge university press, ??? (2015) 30AI4SoS

    2015

  68. [77]

    American International Journal of Contemporary Research4(1), 6–11 (2014)

    De Carvalho, J., Chima, F.O.: Applications of structural equation mod- eling in social sciences research. American International Journal of Contemporary Research4(1), 6–11 (2014)

    2014

  69. [78]

    Internet Research29(3), 407–429 (2019)

    Khan, G.F., Sarstedt, M., Shiau, W.-L., Hair, J.F., Ringle, C.M., Fritze, M.P.: Methodological research on partial least squares struc- tural equation modeling (pls-sem) an analysis based on social network approaches. Internet Research29(3), 407–429 (2019)

    2019

  70. [79]

    Science Advances8(42), 1942 (2022)

    Leist, A.K., Klee, M., Kim, J.H., Rehkopf, D.H., Bordas, S.P., Muniz- Terrera, G., Wade, S.: Mapping of machine learning approaches for description, prediction, and causal inference in the social and health sciences. Science Advances8(42), 1942 (2022)

    2022

  71. [80]

    arXiv preprint arXiv:2407.08940 (2024)

    Qi, B., Zhang, K., Tian, K., Li, H., Chen, Z.-R., Zeng, S., Hua, E., Jin- fang, H., Zhou, B.: Large language models as biomedical hypothesis gen- erators: A comprehensive evaluation. arXiv preprint arXiv:2407.08940 (2024)

    2024 arXiv

  72. [81]

    In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp

    Ambekar, A., Ward, C., Mohammed, J., Male, S., Skiena, S.: Name- ethnicity classification from open sources. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 49–58 (2009)

    2009

  73. [82]

    Nature communications 10(1), 5170 (2019)

    Li, W., Aste, T., Caccioli, F., Livan, G.: Early coauthorship with top scientists predicts success in academic careers. Nature communications 10(1), 5170 (2019)

    2019

  74. [83]

    In: Conference on Fairness, Accountability and Transparency, pp

    Binns, R.: Fairness in machine learning: Lessons from political philoso- phy. In: Conference on Fairness, Accountability and Transparency, pp. 149–159 (2018). PMLR

    2018

  75. [84]

    In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp

    Raji, I.D., Buolamwini, J.: Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial ai products. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 429–435 (2019)

    2019

  76. [85]

    Nature627(8002), 49–58 (2024)

    Messeri, L., Crockett, M.: Artificial intelligence and illusions of under- standing in scientific research. Nature627(8002), 49–58 (2024)

    2024

  77. [86]

    1–16 (2019)

    Holstein, K., Wortman Vaughan, J., Daum´ e III, H., Dudik, M., Wallach, H.: Improving fairness in machine learning systems: What do indus- try practitioners need? In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–16 (2019)

    2019

  78. [87]

    Draft NIST Special AI4SoS31 Publication1270(2021)

    Schwartz, R., Down, L., Jonas, A., Tabassi, E.: A proposal for identi- fying and managing bias in artificial intelligence. Draft NIST Special AI4SoS31 Publication1270(2021)

    2021

  79. [88]

    Schwartz, R., Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., Hall, P.: Towards a Standard for Identifying and Managing Bias in Artificial Intelligence vol. 3. US Department of Commerce, National Institute of Standards and Technology, ??? (2022)

    2022

  80. [89]

    Nature621(7980), 672–675 (2023)

    Van Noorden, R., Perkel, J.M.: Ai and science: what 1,600 researchers think. Nature621(7980), 672–675 (2023)

    2023

  81. [90]

    Neurocomputing410, 237–270 (2020)

    G´ orriz, J.M., Ram´ ırez, J., Ortiz, A., Martinez-Murcia, F.J., Segovia, F., Suckling, J., Leming, M., Zhang, Y.-D., ´Alvarez-S´ anchez, J.R., Bologna, G.,et al.: Artificial intelligence within the interplay between natural and artificial computation: Advances in data science...

    2020

  82. [91]

    Journal of product innovation management 37(3), 212–227 (2020)

    Verganti, R., Vendraminelli, L., Iansiti, M.: Innovation and design in the age of artificial intelligence. Journal of product innovation management 37(3), 212–227 (2020)

    2020

  83. [92]

    Computers, Materials & Continua81(2) (2024)

    Madanchian, M., Taherdoost, H.: Ai-powered innovations in high-tech re- search and development: From theory to practice. Computers, Materials & Continua81(2) (2024)

    2024

  84. [93]

    arXiv preprint arXiv:1510.02855 (2015)

    Wallach, I., Dzamba, M., Heifets, A.: Atomnet: a deep convolutional neu- ral network for bioactivity prediction in structure-based drug discovery. arXiv preprint arXiv:1510.02855 (2015)

    2015 arXiv

  85. [94]

    Wiley Interdisciplinary Reviews: Computational Molecular Science 12(2), 1568 (2022)

    Staszak, M., Staszak, K., Wieszczycka, K., Bajek, A., Roszkowski, K., Tylkowski, B.: Machine learning in drug design: Use of artificial intelligence to explore the chemical structure–biological activity relation- ship. Wiley Interdisciplinary Reviews: Computational Molecular S...

    2022

  86. [95]

    Annual Review of Materials Research50(1), 1–25 (2020)

    Suh, C., Fare, C., Warren, J.A., Pyzer-Knapp, E.O.: Evolving the ma- terials genome: How machine learning is fueling the next generation of materials discovery. Annual Review of Materials Research50(1), 1–25 (2020)

    2020

  87. [96]

    Chemical Science (2024)

    Kim, H., Choi, H., Kang, D., Lee, W.B., Na, J.: Materials discovery with extreme properties via reinforcement learning-guided combinatorial chemistry. Chemical Science (2024)

    2024

  88. [97]

    In: NeurIPS 2023 Foundation Models for Decision Making Workshop (2023) 32AI4SoS

    Light, J., Cai, M., Shen, S., Hu, Z.: Avalonbench: Evaluating llms playing the game of avalon. In: NeurIPS 2023 Foundation Models for Decision Making Workshop (2023) 32AI4SoS

    2023

  89. [98]

    arXiv preprint arXiv:2406.08979 (2024)

    Du, Z., Qian, C., Liu, W., Xie, Z., Wang, Y., Dang, Y., Chen, W., Yang, C.: Multi-agent software development through cross-team collaboration. arXiv preprint arXiv:2406.08979 (2024)

    2024 arXiv

  90. [99]

    Journal of machine learning research 21(140), 1–67 (2020)

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research 21(140), 1–67 (2020)

    2020

  91. [100]

    CoRR (2023)

    OpenAI: GPT-4 technical report. CoRR (2023)

    2023

  92. [101]

    arXiv preprint arXiv:2407.21783 (2024)

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al.: The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  93. [102]

    In: Proceedings of naacL-HLT, vol

    Kenton, J.D.M.-W.C., Toutanova, L.K.: Bert: Pre-training of deep bidi- rectional transformers for language understanding. In: Proceedings of naacL-HLT, vol. 1, p. 2 (2019). Minneapolis, Minnesota

    2019

  94. [103]

    Bioinformatics37(15), 2112–2120 (2021)

    Ji, Y., Zhou, Z., Liu, H., Davuluri, R.V.: Dnabert: pre-trained bidirec- tional encoder representations from transformers model for dna-language in genome. Bioinformatics37(15), 2112–2120 (2021)

    2021

  95. [104]

    arXiv preprint arXiv:2402.06852 (2024)

    Zhang, D., Liu, W., Tan, Q., Chen, J., Yan, H., Yan, Y., Li, J., Huang, W., Yue, X., Zhou, D., et al.: Chemllm: A chemical large language model. arXiv preprint arXiv:2402.06852 (2024)

    2024 arXiv

  96. [105]

    Advances in Neural Information Processing Systems36(2024)

    Liu, J., Zhou, P., Hua, Y., Chong, D., Tian, Z., Liu, A., Wang, H., You, C., Guo, Z., Zhu, L., et al.: Benchmarking large language models on cmexam-a comprehensive chinese medical exam dataset. Advances in Neural Information Processing Systems36(2024)

    2024

  97. [106]

    In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pp

    Zhang, J., Xu, X., Zhang, N., Liu, R., Hooi, B., Deng, S.: Exploring collaboration mechanisms for LLM agents: A social psychology view. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pp. 14544–14607 (2024)

    2024

  98. [107]

    arXiv preprint arXiv:1901.04085 (2019)

    Nogueira, R., Cho, K.: Passage re-ranking with bert. arXiv preprint arXiv:1901.04085 (2019)

    2019 arXiv

  99. [108]

    Applied Sciences12(17), 8805 (2022)

    Mars, M.: From word embeddings to pre-trained language models: A state-of-the-art walkthrough. Applied Sciences12(17), 8805 (2022)

    2022

  100. [109]

    Arabian Journal for Science and Engineering47(2), 2499–2511 (2022) AI4SoS33

    Alsayat, A.: Improving sentiment analysis for social media applications using an ensemble deep learning language model. Arabian Journal for Science and Engineering47(2), 2499–2511 (2022) AI4SoS33

    2022

  101. [110]

    Petukhova, A., Matos-Carvalho, J.P., Fachada, N.: Text clustering with llm embeddings. arXiv preprint arXiv:2403.15112 (2024) 34AI4SoS A Related Work A.1 AI for Science In recent years, AI has become increasingly common in science and is expected to become the center of resear...

    2024 arXiv

  102. [111]

    This is not the place to critique the paper

    Summary: Provide a brief summary of the paper and its contributions. This is not the place to critique the paper. The authors should generally agree with a well-written summary, which reflects an accurate understand- ing of their work from a multidisciplinary perspective

  103. [112]

    Strengths and Weaknesses: Please provide a thorough assessment of the strengths and weaknesses of the paper, touching on each of the following dimensions: - Originality: Are the tasks or methods novel within each of the rel- evant disciplines? Does the work represent an innova...

  104. [113]

    Responses from the authors could change your opinion or address areas of confusion

    Questions: Please list any questions or suggestions that could help clarify the paper’s limitations or improve its quality. Responses from the authors could change your opinion or address areas of confusion. This feedback can be critical for the rebuttal and discussion phase w...

  105. [114]

    Ensure any ethical issues related to research design, data usage, or broader implications are addressed

    Ethical Concerns: Flag any ethical concerns, particularly those that may arise from interdisciplinary collaboration. Ensure any ethical issues related to research design, data usage, or broader implications are addressed

  106. [115]

    Overall Score: Provide a final score based on the paper’s strengths and weaknesses. Use the following scale: - 10: Award Quality: A technically flawless paper with groundbreaking impact across one or more disciplines, with exceptionally strong evaluation, reproducibility, and ...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.