Non-synchronism in Global Usage of Research Methods in Library and Information Science from 1990 to 2019

Chengzhi Zhang; Liang Tian

arxiv: 2607.01833 · v1 · pith:B2ZAECN6new · submitted 2026-07-02 · 💻 cs.DL · cs.CL· cs.CY

Non-synchronism in Global Usage of Research Methods in Library and Information Science from 1990 to 2019

Chengzhi Zhang , Liang Tian This is my paper

Pith reviewed 2026-07-03 02:22 UTC · model grok-4.3

classification 💻 cs.DL cs.CLcs.CY

keywords research methodslibrary and information sciencecross-country comparisondeep learning classificationcontent analysisdiscipline developmentinternational trends

0 comments

The pith

Research methods in library and information science differ across countries but these differences have narrowed over the past 30 years.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines 5,281 research papers from 81 countries published in LIS journals between 1990 and 2019. Authors manually labeled methods in a subset of articles and trained a deep learning model to classify methods across the full set. This enabled comparison showing each country maintains a distinct profile of methods used. Even studies on identical topics employ different methods depending on the country. National patterns deviate from international ones, yet the size of those deviations has declined steadily across the three decades.

Core claim

Analysis of the classified corpus shows that countries maintain unique distributions of research methods in library and information science. These distributions differ from the global aggregate even when the research topic is held constant. The magnitude of national-to-international divergence has decreased across the thirty-year window.

What carries the argument

Deep learning model trained on manually annotated articles to classify research methods across the full 5,281-paper corpus for cross-country comparison.

If this is right

Each country exhibits a unique profile and distribution of research methods.
Research methods differ between countries even when the topic under investigation is the same.
Differences between national and international distributions of methods have decreased over the past 30 years.
The observed patterns can guide discipline development at the national level.
The patterns supply information useful for promoting collaboration and understanding between countries.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same classification approach could be applied to other disciplines to test whether non-synchronism is field-specific.
Convergence might be linked to changes in international publishing or training practices that future studies could measure directly.
National research profiles could be used to identify method gaps when designing graduate curricula or funding priorities.

Load-bearing premise

The deep learning model trained on the manually annotated subset accurately classifies research methods across the full corpus with error rates low enough not to distort the reported country-level differences.

What would settle it

Human re-annotation of a random sample of model-classified papers shows classification errors large enough to reverse the reported direction or size of country differences.

Figures

Figures reproduced from arXiv: 2607.01833 by Chengzhi Zhang, Liang Tian.

**Figure 1.** Figure 1: Distribution of research methods in original and extended datasets 3.3 Automatic classification of research methods 3.3.1 Full-text cognitive-based classification model In some cases, academic article abstracts may not provide a complete description of the research methods used, prompting the need for the full text to be used for automatic classification. However, the full text of academic articles can be … view at source ↗

**Figure 3.** Figure 3: shows the cumulative percentage of articles in various countries, sorted by the number of articles. The statistics are presented for all authors, first authors, and corresponding authors, from left to right. We observe that the 20 countries with the highest number of articles, for all authors, first authors, or corresponding authors, account for over 90% of the total articles. The United States has the lar… view at source ↗

**Figure 11.** Figure 11: Distribution of research topics among the 20 countries with the highest publication volume [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗

**Figure 12.** Figure 12: The Jensen- divergence between the distribution of research methods used in “information behavior research” and international distribution for the top five countries 0 0.2 0.4 0.6 0.8 1 1990-1999 2000-2009 2010-2019 Jensen-Shannon divergence United states United kingdom China Australia Finland [PITH_FULL_IMAGE:figures/full_fig_p019_12.png] view at source ↗

read the original abstract

The global development of Library and Information Science (LIS) is influenced by various factors such as the economy, society, culture, discipline, tradition, and more. Consequently, the research methods of LIS vary greatly among countries. To better understand these differences, we conducted a study of 5,281 research papers from 81 countries published in internationally representative journals over the past thirty years. We manually annotated the research methods used in some articles through content analysis, and subsequently developed and trained a deep learning model for automatic classification of research methods. Using this method, we conducted a comparative analysis of the usage of research methods in different countries. Our findings reveal that there are differences in the research methods used across countries, with each country having its unique research profile and distribution of research methods. Even when investigating the same topic, research methods can differ between countries. Our study also uncovers that there are differences between the national and international distribution of research methods, these differences have decreased over the past 30 years. By highlighting the characteristics of discipline development in various countries from the perspective of research methods, our study can help guide discipline development at the national level. This study provides insights into the usage trends of research methods across different countries and highlights the unique characteristics of discipline development in each country. This information can be valuable in promoting collaboration and understanding between countries and in guiding discipline development at the national level.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper maps LIS method differences across 81 countries and shows some convergence over 30 years, but the deep learning classifier has no reported accuracy or error analysis, so the country profiles rest on an unchecked assumption.

read the letter

The main takeaway is an empirical map of research methods in library and information science across 81 countries from 1990 to 2019, built from 5281 papers. They manually labeled a subset, trained a deep learning model on it, and used the model to label the rest, then compared national profiles and tracked how those profiles moved toward the global distribution.

The scale is the part that stands out. Covering that many countries and that long a period with a mix of hand coding and automation is a step beyond most earlier LIS bibliometric work. The claim that even papers on the same topic use different methods by country, and that national-international gaps narrowed, is the concrete result they put forward.

The soft spot is exactly where the stress-test note points: no performance numbers for the classifier appear in the abstract, and the full text would need to show accuracy, per-class F1, and whether errors are uniform across methods or countries. Without that, any reported national differences or the convergence trend could be driven by uneven misclassification rather than real patterns. Journal selection rules and the statistical tests for the trends are also missing from the description, which makes it hard to judge how stable the findings are.

This is for readers who work on bibliometrics, national research policy in the social sciences, or LIS history. Someone looking for a broad descriptive picture of method use by country could get value if the validation holds. It is worth sending to peer review because the question is straightforward and the dataset is large, but only after the authors supply the classifier metrics and error breakdown. Without those, the central claims stay provisional.

Referee Report

3 major / 1 minor

Summary. The manuscript examines non-synchronism in the use of research methods in Library and Information Science (LIS) across 81 countries from 1990 to 2019. It analyzes a corpus of 5,281 papers from internationally representative journals, manually annotates a subset for research methods, trains a deep learning model for classification, and reports country-specific method profiles, topic-dependent differences, and a decrease in national-international differences over time.

Significance. Should the deep learning classifier prove sufficiently accurate, the findings would provide empirical evidence of global variations and convergence in LIS research practices, offering guidance for discipline development and international collaboration.

major comments (3)

[Methods] The description of the deep learning model provides no performance metrics such as accuracy, per-class F1 scores, or country-stratified error rates on held-out data, leaving open the possibility that non-uniform misclassification rates distort the reported national profiles and 30-year convergence trend.
[Results] No details are supplied on the statistical tests (e.g., chi-square, ANOVA, or regression) used to establish the significance of cross-country differences or the temporal decrease in national-international divergence.
[Data Collection] Selection criteria for the 81 countries and the 'internationally representative journals' yielding the 5,281-paper corpus are not stated, preventing evaluation of sampling bias that could affect the generalizability of the method-distribution claims.

minor comments (1)

[Abstract] The abstract refers to 'content analysis' for manual annotation but omits annotation guidelines, number of annotators, or inter-annotator agreement statistics.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive comments. We address each major point below and will revise the manuscript to incorporate the requested details.

read point-by-point responses

Referee: [Methods] The description of the deep learning model provides no performance metrics such as accuracy, per-class F1 scores, or country-stratified error rates on held-out data, leaving open the possibility that non-uniform misclassification rates distort the reported national profiles and 30-year convergence trend.

Authors: We agree that performance metrics are necessary for validating the classifier and assessing potential bias in the national profiles. The revised manuscript will include overall accuracy, per-class precision/recall/F1 scores, and details of the held-out validation set. If country-stratified performance data from our experiments can be reported without violating privacy or sample-size constraints, we will add those as well. revision: yes
Referee: [Results] No details are supplied on the statistical tests (e.g., chi-square, ANOVA, or regression) used to establish the significance of cross-country differences or the temporal decrease in national-international divergence.

Authors: The referee correctly notes the absence of statistical-test descriptions. Cross-country differences were evaluated with chi-square tests on method distributions, and the 30-year convergence trend was assessed via linear regression on yearly divergence scores. The revision will explicitly state the tests employed, software used, p-values, and any multiple-comparison corrections. revision: yes
Referee: [Data Collection] Selection criteria for the 81 countries and the 'internationally representative journals' yielding the 5,281-paper corpus are not stated, preventing evaluation of sampling bias that could affect the generalizability of the method-distribution claims.

Authors: We will add a dedicated subsection on data collection. Journals were drawn from Web of Science and Scopus categories for Library and Information Science with explicit international scope; countries were retained only if they contributed at least ten papers to ensure stable profiles. The revision will list the journals, state the minimum-paper threshold, and discuss how these choices affect generalizability. revision: yes

Circularity Check

0 steps flagged

Empirical classification study with no load-bearing derivations or self-referential reductions

full rationale

The paper performs manual content analysis on a subset of papers, trains a deep learning classifier on the annotations, applies the model to the remaining corpus, and reports country-level distributions and trends. No equations, fitted parameters, or predictions are defined in terms of the target quantities themselves. No self-citation chains, uniqueness theorems, or ansatzes are invoked to justify the central claims. The workflow is a standard supervised classification pipeline whose outputs are independent of the input annotations by construction; any accuracy limitations are a separate validation concern, not circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claims rest on two unverified premises visible in the abstract: that the chosen international journals adequately represent global LIS output and that the deep-learning classifier generalizes reliably from the hand-labeled subset.

axioms (2)

domain assumption The selected internationally representative journals capture the global distribution of LIS research methods.
Abstract states papers were drawn from such journals but provides no selection criteria or coverage check.
ad hoc to paper The deep learning model produces classifications whose error distribution does not materially alter the reported country-level differences.
Abstract describes training the model but reports no accuracy, confusion matrix, or robustness test.

pith-pipeline@v0.9.1-grok · 5788 in / 1405 out tokens · 30506 ms · 2026-07-03T02:22:00.917944+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

6 extracted references · 6 canonical work pages

[1]

Angelov, D. (2020). Top2Vec: Distributed Representations of Topics. ArXiv:2008.09470 [Cs, Stat] . http://arxiv.org/abs/2008.09470 Beltagy, I., Lo, K., & Cohan, A. (2019). SciBERT: A Pretrained Language Model for Scientific Text. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferenc...

work page arXiv 2020
[2]

Collection Management, 19(1–2), 1–42. Cano, V . (1999). Bibliometric overview of library and information science research in Spain. Journal of the American Society for Information Science, 50(8), 675–680. Carbonell, J., & Goldstein, J. (1998). The use of MMR, diversity -based reranking for reordering documents and producing summaries. Proceedings of the 2...

work page doi:10.1016/j.lisr.2014.09.003 1999
[3]

N., Crist, E., & Pluye, P

Granikov, V ., Hong, Q. N., Crist, E., & Pluye, P. (2020). Mixed methods research in library and information science: A methodological review. Library & Information Science Res earch, 42(1), 101003. https://doi.org/10.1016/j.lisr.2020.101003 Hider, P., & Pymm, B. (2008). Empirical research methods reported in high-profile LIS journal literature. Library &...

work page doi:10.1016/j.lisr.2020.101003 2020
[4]

https://doi.org/10.1108/JD-10- 2020-0171 Ma, J., & Lund, B

Journal of Documentation, 77(5), 1196–1208. https://doi.org/10.1108/JD-10- 2020-0171 Ma, J., & Lund, B. (2021). The evolution and shift of research topics and methods in library and information science. Journal of the Association for Inf ormation Science and Technology , asi.24474. https://doi.org/10.1002/asi.24474 McInnes, L., Healy, J., Saul, N., & Gros...

work page doi:10.1108/jd-10- 2020
[5]

Menéndez, M., Pardo, J., Pardo, L., & Pardo, M. (1997). The jensen -shannon divergence. Journal of the Franklin Institute, 334(2), 307–318. Ngulube, P., & Ukwoma, S. C. (2021). Prevalence of methodological transparency in the use of mixed methods research in library and in formation science research in South Africa and Nigeria, 2009 –2015. Library & Infor...

work page doi:10.1613/jair.1.12125 1997
[6]

Rochester, M. K. (1995). Library and Information Science Research in Australia 1985–1994: A Content Analysis of Research Articles in The Australian Library Journal and Australian Academic & Research Libraries . Australian Academic & Research Libraries , 26(3), 163 –170. https://doi.org/10.1080/00048623.1995.10754930 Rochester, M., & Vakkari, P. (1998). In...

work page doi:10.1080/00048623.1995.10754930 1995

[1] [1]

Angelov, D. (2020). Top2Vec: Distributed Representations of Topics. ArXiv:2008.09470 [Cs, Stat] . http://arxiv.org/abs/2008.09470 Beltagy, I., Lo, K., & Cohan, A. (2019). SciBERT: A Pretrained Language Model for Scientific Text. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferenc...

work page arXiv 2020

[2] [2]

Collection Management, 19(1–2), 1–42. Cano, V . (1999). Bibliometric overview of library and information science research in Spain. Journal of the American Society for Information Science, 50(8), 675–680. Carbonell, J., & Goldstein, J. (1998). The use of MMR, diversity -based reranking for reordering documents and producing summaries. Proceedings of the 2...

work page doi:10.1016/j.lisr.2014.09.003 1999

[3] [3]

N., Crist, E., & Pluye, P

Granikov, V ., Hong, Q. N., Crist, E., & Pluye, P. (2020). Mixed methods research in library and information science: A methodological review. Library & Information Science Res earch, 42(1), 101003. https://doi.org/10.1016/j.lisr.2020.101003 Hider, P., & Pymm, B. (2008). Empirical research methods reported in high-profile LIS journal literature. Library &...

work page doi:10.1016/j.lisr.2020.101003 2020

[4] [4]

https://doi.org/10.1108/JD-10- 2020-0171 Ma, J., & Lund, B

Journal of Documentation, 77(5), 1196–1208. https://doi.org/10.1108/JD-10- 2020-0171 Ma, J., & Lund, B. (2021). The evolution and shift of research topics and methods in library and information science. Journal of the Association for Inf ormation Science and Technology , asi.24474. https://doi.org/10.1002/asi.24474 McInnes, L., Healy, J., Saul, N., & Gros...

work page doi:10.1108/jd-10- 2020

[5] [5]

Menéndez, M., Pardo, J., Pardo, L., & Pardo, M. (1997). The jensen -shannon divergence. Journal of the Franklin Institute, 334(2), 307–318. Ngulube, P., & Ukwoma, S. C. (2021). Prevalence of methodological transparency in the use of mixed methods research in library and in formation science research in South Africa and Nigeria, 2009 –2015. Library & Infor...

work page doi:10.1613/jair.1.12125 1997

[6] [6]

Rochester, M. K. (1995). Library and Information Science Research in Australia 1985–1994: A Content Analysis of Research Articles in The Australian Library Journal and Australian Academic & Research Libraries . Australian Academic & Research Libraries , 26(3), 163 –170. https://doi.org/10.1080/00048623.1995.10754930 Rochester, M., & Vakkari, P. (1998). In...

work page doi:10.1080/00048623.1995.10754930 1995