Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning

Jiachuan Wang; Lei Chen; Shuangyin Li; Tong Li; Yongqi Zhang

REVIEW 3 major objections 5 minor 75 references

Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Citss claims to achieve state-of-the-art citation classification by adding self-supervised contrastive learning, built from sentence-level cropping and keyphrase perturbation, to the fine-tuning of both SciBERT and Llama3-8B.

desk verdict Solid empirical recipe for contrastive fine-tuning on citation classification across both encoder and decoder PLMs; the keyphrase-preservation assumption is under-tested but the gains are real. read the letter →

arxiv 2505.14471 v2 pith:EMSPFI47 submitted 2025-05-20 cs.CL

Tong Li , Jiachuan Wang , Yongqi Zhang , Shuangyin Li , Lei Chen This is my paper

classification cs.CL

keywords citationclassificationintentself-supervisedcontrastivelearningsentence-levelcroppingkeyphraseperturbationscientifictypedkeyphrasesparameter-efficientfine-tuningpretrainedlanguagemodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that citation classification—automatically deciding why an author cites a particular work—can be improved by supplementing supervised fine-tuning with self-supervised contrastive learning. Its framework, Citss, constructs label-free positive pairs through two transformations: sentence-level cropping (SC), which randomly cuts the context window around the target citation, and keyphrase perturbation (KP), which replaces or anonymizes scientific keyphrases and swaps residue words for synonyms. A contrastive InfoNCE loss on these pairs is added to the supervised classification loss, with the goal of making the model attend to the argumentative logic around a citation rather than to exact wording or spurious keyphrase-label correlations. The paper reports that Citss outperforms prior state-of-the-art methods on ACL-ARC, FOCAL, and ACT2 using both the SciBERT encoder and the Llama3-8B decoder with LoRA, and describes it as the first effective fine-tuning of a decoder-only LLM for citation classification. If the claims hold, the framework gives a practical recipe for squeezing more signal out of very small labeled datasets by engineering task-aware self-supervision.

What carries the argument

The central object is the contrastive pair, generated without labels and specialized to the citation task. SC treats the context as a sequence of sentences $\langle s_{-l}^i,\dots,s_0^i,\dots,s_l^i \rangle$ centered on the citance $s_0^i$, and in each epoch randomly draws a subsequence that keeps $s_0^i$ but varies the numbers of preceding and following sentences; these crops are fed through the same model and pulled together in an InfoNCE loss with temperature $\tau_1$. KP first extracts scientific typed keyphrases (STKs) from each context and then, with probability $\beta$, perturbs each keyphrase by replacing it with a same-type keyphrase from the corpus, replacing it with a same-type keyphrase from the same context, or abstracting it to its type name, followed by WordNet synonym replacement on the residue with probability $\gamma$; the perturbed text is the positive pair for a second InfoNCE loss with temperature $\tau_2$. The two contrastive losses are added to the cross-entropy classification loss with weights $\lambda_1$ and $\lambda_2$, and an MLP adapter maps the PLM's last-layer hidden states into a lower-dimensional space where similarity and classification operate. For decoder-only LLMs, the prompt includes an explicit 'output one word' instruction so the final hidden state carries the classification signal, and LoRA keeps the trainable parameter count small.

What would settle it

A direct test is to take a sample of held-out citation contexts from ACL-ARC and FOCAL, apply the three KP operations (global replacement, local replacement, abstraction) at the paper's default probabilities, and have independent annotators label both the original and the perturbed texts. If a non-negligible fraction of perturbed texts receive a different citation-intention label, the contrastive objective is training on invalid positive pairs and the mechanism underlying Citss would not survive; if labels are preserved at near-ceiling level, the assumption is supported.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that two task-specific transformation strategies—sentence-level cropping (SC) and keyphrase perturbation (KP)—make contrastive learning work for citation classification and can be applied to both encoder-based and decoder-based pretrained language models. SC randomly crops the context window so that the representation must stay consistent around the target citation even when noisy surrounding sentences are included or removed. KP replaces or anonymizes scientific typed keyphrases (STKs) and applies synonym substitution to the residue, so the model is pushed to encode citation intention from the remaining rhetorical structure instead of from keyphrase-label correlations. Citss combines an InfoNCE contrastive loss for each strategy with the cross-entropy classification loss, and the paper reports consistent gains over state-of-the-art baselines—including the earlier prompting-based PET method—on three datasets, with Llama3-8B plus LoRA outperforming SciBERT on ACL-ARC.

Load-bearing premise

The load-bearing premise is that changing the scientific keyphrases in a citation context does not change the citation intention, so the perturbed text is a valid positive pair for contrastive learning; the paper demonstrates this with one example and a qualitative STK study rather than a quantitative label-preservation test.

Editorial extensions

If this is right

Fine-tuning on citation classification no longer needs a large annotated corpus: the two transformations generate supervision from the unlabeled parts of the same training examples, lowering the annotation barrier for new citation-intent datasets.
Longer context windows become usable: because SC makes the representation invariant to which surrounding sentences are present, models can look beyond the citance without being hijacked by irrelevant text.
Predictions depend less on surface-level terminology: KP pushes the model to classify by the rhetorical structure of the context, which should improve generalization to papers on unfamiliar topics.
Decoder-only LLMs can be fine-tuned for classification-style tasks despite small datasets: Citss with LoRA improves over both zero-shot prompting and plain LoRA fine-tuning on the three benchmarks.
On ACL-ARC the paper reports that fine-tuned Llama3-8B beats fine-tuned SciBERT, suggesting that large general-purpose pretraining pays off once the fine-tuning recipe handles citation-specific noise.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If keyphrase perturbation's label-preservation assumption holds, the same recipe could transfer to other classification tasks whose labels depend on discourse function rather than topic words, such as relation detection or illocutionary-force classification.
The paper's one-shot LLM-based STK extraction means the cost and quality of KP are tied to the extraction model; a cheaper or more controllable extraction method would be the main practical lever for deploying Citss at scale.
Sentence-level cropping is a generic robustness prior: imposing invariance to context-window position could help other long-document tasks where a target mention is embedded in noisy surroundings.
A testable extension would be to replace WordNet synonym replacement with paraphrase-level perturbation and measure whether the KP loss continues to help, since the paper only varies the synonym-replacement probability $\gamma$.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a solid, honest empirical paper. It shows that self-supervised contrastive augmentations help fine-tuning for citation classification on both SciBERT and Llama3-8B, and the gains are consistent across three datasets. The obvious worry—keyphrase perturbation may not preserve the citation label—is real, but it's not a deal-breaker. The ablations show the method helps even if some positives are wrong.

What's new: the two augmentations, sentence-level cropping and keyphrase perturbation, are tailored to the task, and this is the first work I know to fine-tune a decoder-only LLM for citation classification with real gains. The framework is simple: InfoNCE, a lightweight adapter, LoRA for the LLM. Experiments are broad: three datasets, two backbone families, ablations, significance tests, and efficiency numbers. Code is available. That's solid work.

Where it's soft: the label-preservation assumption behind KP isn't directly validated. Example 4.1 is illustrative, and the STK extraction evaluation (Figure 5) measures precision/recall of extraction, not whether a perturbed context keeps its gold label. If many positives are wrong, the contrastive loss pulls different classes together. But KP is auxiliary (lambda2=0.1–0.2), and the ablation shows gains even with imperfect positives, so the method works as a regularizer regardless of mechanism. A quick human-label agreement study on perturbed contexts would settle it, but I wouldn't block acceptance on it. The ACT2 discrepancy is minor: with SciBERT, Macro-F1 is slightly below IREL and PET, but accuracy is best and they're upfront about the balance. They tune many hyperparameters per dataset, but report means over 3 runs.

Bottom line: this paper deserves serious review. It gives the community a practical, reproducible recipe for fine-tuning LLMs on citation classification with limited data. Send it to referees. The KP assumption is a follow-up, not a fatal flaw.

Referee Report

3 major / 5 minor

Summary. The paper proposes Citss, a framework for citation classification that augments supervised fine-tuning of pretrained language models with two self-supervised contrastive objectives: sentence-level cropping (SC), which generates positive pairs by randomly cropping the surrounding context, and keyphrase perturbation (KP), which replaces typed scientific keyphrases and applies synonym substitution. The method is applied to encoder-based SciBERT and decoder-based Llama3-8B with LoRA, and evaluated on ACL-ARC, FOCAL, and ACT2. The paper reports state-of-the-art or competitive results, and includes ablations, hyperparameter sensitivity analyses, an efficiency study, and a qualitative analysis of extracted keyphrases. The code is publicly available.

Significance. If the empirical claims hold, the paper makes a useful practical contribution: it demonstrates a recipe for fine-tuning a decoder-only LLM with limited labeled data for citation classification, and it proposes two domain-motivated contrastive transformations. The release of code and the coverage of three datasets and two backbone families are strengths. However, the central mechanism of KP rests on a label-preservation assumption that is not directly tested, and several aggregate claims of superiority are not supported by the reported significance tests and ablations on ACT2.

major comments (3)

[Section 4.3, Eq. (8), Algorithm 1] The KP objective treats a perturbed context as a positive pair under the assumption that replacing or anonymizing typed scientific keyphrases and applying synonym replacement does not change the citation-intention label. The manuscript supports this with a single illustrative example (Example 4.1) and a qualitative STK-extraction study (Section 5.4, Figure 5) that measures extraction precision/recall, not label preservation. If a non-negligible fraction of perturbed contexts change label, the InfoNCE loss actively pulls representations of different classes together. I request a direct validation: sample original and perturbed contexts, have annotators assign citation labels (or use the gold labels as a proxy with perturbation-specific agreement statistics), and report label-preservation rates per operation (Gr, Lr, Ab) and per dataset. This is load-bearing for the paper's claim that KP mitigates spurious keyphrase correlations.
[Section 5.3, Table 3] The text states that 'with all datasets and backbone models, using a single strategy is better than no strategy,' but the ACT2/SciBERT rows contradict this. For Macro-F1, the no-strategy baseline is 0.262, while λ2=0 (SC only) gives 0.246 and λ1=0 (KP only) gives 0.260; both single strategies are worse. Even the full model in Table 2 (0.254) is below the no-strategy Macro-F1, and below IREL (0.262) and PET (0.258). This undermines the blanket claim of 'consistent superiority' and the specific '5 out of 6 metrics' phrasing should be qualified, since ACT2 Macro-F1 is not improved. Please re-analyze and either temper the claims or provide an explanation (e.g., accuracy-improvement trade-off) supported by statistical testing.
[Section 5.2.1, Table 2] The asterisk notation indicates p<0.05 by t-test against other baselines with the same backbone, but on ACT2 no metric for Citss is starred for either backbone, and on FOCAL with SciBERT only Accuracy is starred, not Macro-F1. The abstract's claim of 'superiority' and the RQ1 conclusion of 'consistent superiority' are therefore not supported by the significance tests on a third of the experimental conditions. I recommend reporting effect sizes, confidence intervals, or a paired test across runs, and explicitly discussing which comparisons are not significant.

minor comments (5)

[Throughout] There are several typos: 'detials' (Section 5.1.4), 'synoynym' (Section 5.1.4), 'sub-par' and 'consequent' (Introduction). A careful proofreading pass is needed.
[Notation] The paper uses 'Lora' and 'LoRA' inconsistently (e.g., Section 1 and Section 5.1.2). Please unify to 'LoRA'.
[Figure 5] The qualitative analysis of STK extraction does not assess label preservation; consider making this explicit in the caption so that the reader understands the scope of the validation.
[Appendix D] The one-shot example in the STK extraction prompt contains a typo ('Polgu re' instead of 'Polguère'). Please fix.
[Section 5.2.1, Table 2] The IFP baselines are reported as point estimates without variance; since other methods report standard deviations across three runs, adding variance for IFP would improve comparability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Citss is an empirical contrastive fine-tuning framework evaluated on held-out test sets, with no self-citation chain or fitted-input-as-prediction reduction.

full rationale

The paper's contributions are empirical: sentence-level cropping and keyphrase perturbation generate positive pairs from the same training sample via cropping and keyphrase replacement, and the contrastive losses (Eqs. 7 and 8) are auxiliary regularizers added to the supervised classification loss (Eq. 5), all optimized on training data and evaluated on held-out test splits. No parameter is fitted to the test labels; hyperparameters are tuned on validation splits, and the reported improvements are measured on unseen test sets rather than derived from the transformed samples themselves. The Llama3-70B-based STK extraction is a preprocessing step, not a reuse of the target classification result. No load-bearing self-citations appear: the cited works are external prior art. The keyphrase-perturbation label-preservation assumption is a modeling assumption supported only by an illustrative example and a qualitative extraction study, but it is not circular: perturbed contexts are not defined in terms of the predicted labels, and the claimed gains are empirically measured rather than entailed by the assumption. The paper is therefore self-contained with respect to its evaluation, and no circular step is exhibited.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The method is empirical and rests on standard deep learning assumptions plus two task-specific assumptions: sentence-level cropping and keyphrase perturbation preserve the citation label. The main free parameters are loss weights, temperatures, augmentation probabilities, context length, and LoRA settings, all tuned on validation splits. No new entities are introduced.

free parameters (8)

lambda1, lambda2 (contrastive loss weights) = Per dataset/backbone; e.g., ACL-ARC SciBERT: 0.2, 0.1; ACL-ARC Llama3-8B: 0.1, 0.2; FOCAL: 0.2, 0.1; ACT2: 0.1, 0.2
Tuned on validation set; control the contribution of the SC and KP contrastive losses.
tau1, tau2 (InfoNCE temperatures) = ACL-ARC: 1, 1; FOCAL: 5, 1; ACT2: 0.1, 10
Tuned on validation set; adjust the sharpness of the contrastive similarity distribution.
beta (keyphrase perturbation probability) = 0.3 to 0.7 depending on dataset and backbone
Tuned on validation set; probability of perturbing each extracted STK.
gamma (synonym replacement ratio) = 0.1
Fixed at 0.1; probability of synonym replacement for residue words.
l (context range) = 3
Number of surrounding sentences included on each side of the citance; sets the maximum crop window.
LoRA rank r and alpha = r=16 for ACL-ARC/FOCAL, r=8 for ACT2; alpha=16
Tuned on validation set; control capacity of the parameter-efficient adapter for Llama3-8B.
Adapter dimensions d, dz = e.g., d=1024, dz=256 for ACL-ARC; varies per dataset and backbone
MLP hidden and output sizes of the projection head, tuned on validation set.
Batch size |B| = 4 or 16 depending on dataset and backbone
Tuned on validation set; affects contrastive pair statistics.

assumptions (6)

domain assumption Pretrained language models provide transferable linguistic knowledge for citation classification.
Foundation of the fine-tuning approach, extensively supported by prior work (Sections 1 and 2).
domain assumption Sentence-level cropping yields positive pairs that share the same citation label as the original context.
Assumed in Section 4.2; motivates the SC contrastive loss.
ad hoc to paper Keyphrase perturbation preserves citation intention labels.
Core assumption of KP in Section 4.3; only supported by Example 4.1 and qualitative analysis, not by label-preservation statistics.
domain assumption The LLM (Llama3-70B) extracts scientific typed keyphrases with sufficient quality for perturbation.
The paper reports type-agnostic precision 76% and recall 87% on 20 sampled ACL-ARC contexts (Section 5.4), but quality on other datasets is not measured.
standard math InfoNCE contrastive learning improves task-specific representations in low-data regimes.
Standard technique from prior literature, accepted as a working assumption.
domain assumption The null prompt P1 for SciBERT and instruction prompt P2 for Llama3-8B are appropriate task prompts.
Prompt choices follow prior work (PET, Meta-task prompting), but optimality is not guaranteed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning." pith.science (2026). https://pith.science/paper/EMSPFI47

@misc{pith2026250514471,
  author       = {Pith},
  title        = {Pith review of: Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EMSPFI47}},
  note         = {Machine review of arXiv:2505.14471}
}

read the original abstract

Citation classification, which identifies the intention behind academic citations, is pivotal for scholarly analysis. Previous works suggest fine-tuning pretrained language models (PLMs) on citation classification datasets, reaping the reward of the linguistic knowledge they gained during pretraining. However, directly fine-tuning for citation classification is challenging due to labeled data scarcity, contextual noise, and spurious keyphrase correlations. In this paper, we present a novel framework, Citss, that adapts the PLMs to overcome these challenges. Citss introduces self-supervised contrastive learning to alleviate data scarcity, and is equipped with two specialized strategies to obtain the contrastive pairs: sentence-level cropping, which enhances focus on target citations within long contexts, and keyphrase perturbation, which mitigates reliance on specific keyphrases. Compared with previous works that are only designed for encoder-based PLMs, Citss is carefully developed to be compatible with both encoder-based PLMs and decoder-based LLMs, to embrace the benefits of enlarged pretraining. Experiments with three benchmark datasets with both encoder-based PLMs and decoder-based LLMs demonstrate our superiority compared to the previous state of the art. Our code is available at: github.com/LITONG99/Citss

Figures

Figures reproduced from arXiv: 2505.14471 by the authors.

**Figure 2.** Ablation study of SC with different 𝑇𝑖 . The x-axis is 𝑙, and 𝑙 = 0 corresponds to the citance. overall training time to Citss with SciBERT. (4) For LLM baselines, while IFP is training-free, its inference speed is significantly slower due to the sequential decoding, requiring roughly 5 and 15 seconds per sample for Llama3-8B and Llama3-70B, respectively. This limits its practical applicability. LoRA fine-tuning, on… view at source ↗

**Figure 3.** Performance with varying 𝛽 (x-axis) and 𝛾. The dashed line is the performance without KP. Macro-f1 Acc. 0.5 0.6 0.7 0.8 ACL-ARC Mixed Gr, SciBERT Gr, Llama3-8B Lr, SciBERT Lr, Llama3-8B Ab, SciBERT Ab, Llama3-8B Macro-f1 Acc. 0.5 0.6 0.7 0.8 FOCAL Macro-f1 Acc. 0.2 0.3 0.4 0.5 0.6 ACT2 [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Performance with perturbation operation Op. scores with SciBERT in [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 6.** Figure 6: Accuracy with varying 𝛽 (x-axis) and 𝛾. The dashed line is the performance without KP. Empirical Methods in Natural Language Processing (EMNLP), Alessandro Moschitti, Bo Pang, and Walter Daelemans (Eds.). Association for Computational Linguistics, Doha, Qatar, 1532–154…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

75 extracted references · 51 canonical work pages

[1]

Riaz Ahmad and Muhammad Tanvir Afzal. 2018. CAD: An algorithm for citation- anchors detection in research papers. Scientometrics 117 (2018), 1405–1423

work page 2018
[2]

Riaz Ahmad, Muhammad Tanvir Afzal, and Muhammad Abdul Qadir. 2017. Pat- tern analysis of citation-anchors in citing documents for accurate identification of in-text citations. IEEE Access 5 (2017), 5819–5828

work page 2017
[3]

Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciBERT: A Pretrained Language Model for Scientific Text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . 3615–3620

work page 2019
[4]

Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long- document transformer. arXiv preprint arXiv:2004.05150 (2020)

arXiv 2020
[5]

Dan Berrebbi, Nicolas Huynh, and Oana Balalau. 2022. GraphCite: citation intent classification in scientific publications via graph embeddings. In Companion Proceedings of the Web Conference 2022 . 779–783

work page 2022
[6]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

2020
[7]

Martin Juan José Bucher and Marco Martini. 2024. Fine-Tuned’Small’LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classification. arXiv preprint arXiv:2406.08660 (2024)

arXiv 2024
[8]

Jingqiang Chen and Hai Zhuge. 2019. Automatic generation of related work through summarizing citations. Concurrency and Computation: Practice and Experience 31, 3 (2019), e4261

work page 2019

Show all 75 references

[9]

Liang Chen, Zekun Wang, Shuhuai Ren, Lei Li, Haozhe Zhao, Yunshui Li, Zefan Cai, Hongcheng Guo, Lei Zhang, Yizhe Xiong, et al. 2024. Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey. arXiv preprint arXiv:2412.18619 (2024)

2024 arXiv
[10]

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Interna- tional conference on machine learning . PMLR, 1597–1607

2020
[11]

Arman Cohan, Waleed Ammar, Madeleine van Zuylen, and Field Cady. 2019. Structural Scaffolds for Citation Intent Classification in Scientific Publications. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Huma...

2019
[12]

Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel S Weld
[13]

Arman Cohan and Nazli Goharian. 2018. Scientific document summarization via citation contextualization and scientific discourse. International Journal on Digital Libraries 19 (2018), 287–303

2018
[14]

Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2018. Supervised Learning of Universal Sentence Representations from Natural Language Inference Data. arXiv:1705.02364 [cs.CL] https://arxiv.org/ abs/1705.02364

2018 arXiv
[15]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR abs/1810.04805 (2018). arXiv:1810.04805 http://arxiv.org/abs/1810.04805

2018 arXiv
[16]

Terrance DeVries. 2017. Improved Regularization of Convolutional Neural Net- works with Cutout. arXiv preprint arXiv:1708.04552 (2017)

2017 arXiv
[17]

Jiayuan Ding, Tong Xiang, Zijing Ou, Wangyang Zuo, Ruihui Zhao, Chenhua Lin, Yefeng Zheng, and Bang Liu. 2022. Tell me how to survey: literature review made simple with automatic reading path generation. In 2022 IEEE 38th international conference on data engineering (ICDE) . I...

2022
[18]

Felix Grezes, Thomas Allen, Tirthankar Ghosal, and Sergi Blanco-Cuaresma. 2023. Function of citation in astrophysics literature (focal): Findings of the shared task. In Proceedings of the Second Workshop on Information Extraction from Scientific Publications. 143–147

2023
[19]

Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024. Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey. arXiv:2403.14608 [cs.LG] https://arxiv.org/abs/2403.14608

2024 arXiv
[20]

Qianyue Hao, Jingyang Fan, Fengli Xu, Jian Yuan, and Yong Li. 2024. HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

2024
[21]

Saeed-Ul Hassan, Iqra Safder, Anam Akram, and Faisal Kamiran. 2018. A novel machine-learning approach to measuring scientific knowledge flows using cita- tion context analysis. Scientometrics 116 (2018), 973–996

2018
[22]

Qi He, Jian Pei, Daniel Kifer, Prasenjit Mitra, and Lee Giles. 2010. Context-aware citation recommendation. In Proceedings of the 19th international conference on World wide web. 421–430

2010
[23]

Myriam Hernández-Alvarez and José M Gomez. 2016. Survey about citation context analysis: Tasks, techniques, and resources. Natural Language Engineering 22, 3 (2016), 327–349

2016
[24]

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-Efficient Transfer Learning for NLP. arXiv:1902.00751 [cs.LG] https: //arxiv.org/abs/1902.00751

2019 arXiv
[25]

Rahul Jha, Amjad-Abu Jbara, Vahed Qazvinian, and Dragomir R Radev. 2017. NLP-driven citation analysis for scientometrics. Natural Language Engineering 23, 1 (2017), 93–130

2017
[26]

Ting Jiang, Shaohan Huang, Zhongzhi Luan, Deqing Wang, and Fuzhen Zhuang
[27]

Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Xinyang Zhang, Qi Zhu, and Jiawei Han. 2023. PATTON: Language Model Pretraining on Text-Rich Networks. In 61st Annual Meeting of the Association for Computational Linguistics, ACL 2023 . Association for Computational Linguistics (ACL...

2023
[28]

David Jurgens, Srijan Kumar, Raine Hoover, Dan McFarland, and Dan Jurafsky
[29]

Gary King and Langche Zeng. 2001. Logistic regression in rare events data. Political analysis 9, 2 (2001), 137–163

2001
[30]

Suchetha N Kunnath, Drahomira Herrmannova, David Pride, and Petr Knoth
[31]

Suchetha N Kunnath, David Pride, Drahomira Herrmannova, and Petr Knoth
[32]

Suchetha Nambanoor Kunnath, David Pride, and Petr Knoth. 2022. Dynamic Context Extraction for Citation Classification. InProceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference o...

2022
[33]

Suchetha N Kunnath, David Pride, and Petr Knoth. 2023. Prompting strategies for citation classification. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management . 1127–1137

2023
[34]

Suchetha Nambanoor Kunnath, Valentin Stauber, Ronin Wu, David Pride, Vik- tor Botev, and Petr Knoth. 2022. ACT2: A multi-disciplinary semi-structured dataset for importance and purpose classification of citations. In Proceedings of the Thirteenth Language Resources and Evaluat...

2022
[35]

Avishek Lahiri, Pratyay Sarkar, Medha Sen, Debarshi Sanyal, and Imon Mukherjee
[36]

Association for Computational Linguistics

Overview of the 2021 SDP 3C citation context classification shared task. Association for Computational Linguistics

2021
[37]

Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics 12 (2024), 157–173

2024
[38]

Robert Logan IV, Ivana Balažević, Eric Wallace, Fabio Petroni, Sameer Singh, and Sebastian Riedel. 2022. Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models. In Findings of the Association for Computational Linguistics: ACL 2022. 2824–2835

2022
[39]

Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

2017 arXiv
[40]

Kelvin Luu, Xinyi Wu, Rik Koncel-Kedziorski, Kyle Lo, Isabel Cachola, and Noah A. Smith. 2021. Explaining Relationships Between Scientific Documents. In Pro- ceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint...

2021 doi
[41]

Himanshu Maheshwari, Bhavyajeet Singh, and Vasudeva Varma. 2021. SciBERT sentence representation for citation context classification. In Proceedings of the Second Workshop on Scholarly Document Processing . 130–133

2021
[42]

Yibin Lei, Di Wu, Tianyi Zhou, Tao Shen, Yu Cao, Chongyang Tao, and Andrew Yates. 2024. Meta-task prompting elicits embedding from large language models. arXiv preprint arXiv:2402.18458 (2024)

2024 arXiv
[43]

Nicholson, Milo Mordaunt, Patrice Lopez, Ashish Uppala, Domenic Rosati, Neves P

Josh M. Nicholson, Milo Mordaunt, Patrice Lopez, Ashish Uppala, Domenic Rosati, Neves P. Rodrigues, Peter Grabitz, and Sean C. Rife. 2021. scite: A smart citation index that displays the context of citations and classi- fies their intent using deep learning. Quantitative Scien...

2021 doi
[44]

Jeppe Nicolaisen. 2007. Citation analysis. Annual review of information science and technology 41, 1 (2007), 609–641

2007
[45]

Kai Nishikawa and Hitoshi Koshiba. 2024. Exploring the applicability of large language models to citation context analysis. Scientometrics 129, 11 (2024), 6751– 6777

2024
[46]

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)

2018 arXiv
[47]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning KDD ’25, August 3...

2014
[48]

George A Miller. 1995. WordNet: a lexical database for English. Commun. ACM 38, 11 (1995), 39–41

1995
[49]

Georgios Petasis, Alessandro Cucchiarelli, Paola Velardi, Georgios Paliouras, Vangelis Karkaletsis, and Constantine D Spyropoulos. 2000. Automatic adapta- tion of proper noun dictionaries through cooperation of machine learning and probabilistic methods. In Proceedings of the ...

2000
[50]

Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer

Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv:1802.05365 [cs.CL] https://arxiv.org/abs/1802.05365

2018 arXiv
[51]

David Pride and Petr Knoth. 2020. An authoritative approach to citation classifi- cation. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in

2020
[52]

David Pride, Petr Knoth, and Jozef Harag. 2019. Act: An annotation platform for citation typing at scale. In 2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL). IEEE, 329–330

2019
[53]

Xiang Ren, Jialu Liu, Xiao Yu, Urvashi Khandelwal, Quanquan Gu, Lidan Wang, and Jiawei Han. 2014. Cluscite: Effective citation recommendation by information network-based clustering. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data...

2014
[54]

Julien Perier-Camby, Marc Bertin, Iana Atanassova, and Frédéric Armetta. 2019. A preliminary study to compare deep learning with rule-based approaches for citation classification. In 8th international workshop on bibliometric-enhanced in- formation retrieval (bir) co-located w...

2019
[55]

Zeren Shui, Petros Karypis, Daniel S Karls, Mingjian Wen, Saurav Manchanda, Ellad B Tadmor, and George Karypis. 2024. Fine-Tuning Language Models on Mul- tiple Datasets for Citation Intention Classification.arXiv preprint arXiv:2410.13332 (2024)

2024 arXiv
[56]

Yizhou Sun, Hongzhi Yin, and Xiang Ren. 2017. Recommendation in context-rich environment: An information network analysis approach. In Proceedings of the 26th International Conference on World Wide Web Companion . 941–945

2017
[57]

Shahbaz Syed, Ahmad Hakimi, Khalid Al Khatib, and Martin Potthast. 2023. Citance-Contextualized Summarization of Scientific Papers. In Findings of the Association for Computational Linguistics: EMNLP 2023 . 8551–8568

2023
[58]

Hong-Jin Tsai, An-Zi Yen, Hen-Hsen Huang, and Hsin-Hsi Chen. 2023. Citation intent classification and its supporting evidence extraction for citation graph con- struction. In Proceedings of the 32nd ACM international conference on information and knowledge management. 2472–2481

2023
[59]

Marco Valenzuela, Vu Ha, and Oren Etzioni. 2015. Identifying meaningful cita- tions. In Workshops at the twenty-ninth AAAI conference on artificial intelligence

2015
[60]

Muhammad Roman, Abdul Shahid, Shafiullah Khan, Anis Koubaa, and Lisu Yu
[61]

Ieee Access 9 (2021), 9982–9995

Citation intent classification using word embedding. Ieee Access 9 (2021), 9982–9995

2021
[62]

Wenhao Yu, Mengxia Yu, Tong Zhao, and Meng Jiang. 2020. Identifying referential intention with heterogeneous contexts. In Proceedings of The Web Conference

2020
[63]

Yu Yu, Chao-Han Huck Yang, Jari Kolehmainen, Prashanth G Shivakumar, Yile Gu, Sungho Ryu Roger Ren, Qi Luo, Aditya Gourav, I-Fan Chen, Yi-Chieh Liu, et al
[64]

Fanjin Zhang, Kun Cao, Yukuo Cen, Jifan Yu, Da Yin, and Jie Tang. 2024. PST- Bench: Tracing and Benchmarking the Source of Publications. arXiv preprint arXiv:2402.16009 (2024)

2024 arXiv
[65]

Yu Zhang, Hao Cheng, Zhihong Shen, Xiaodong Liu, Ye-Yi Wang, and Jianfeng Gao. 2023. Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding. In Findings of the Association for Computational Lin- guistics: EMNLP 2023. 12259–12275

2023
[66]

Type-agnostic

He Zhao, Zhunchen Luo, Chong Feng, and Yuming Ye. 2019. A context-based framework for resource citation classification in scientific literatures. In Proceed- ings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1041–1044. A ...

2019
[67]

Jason Wei and Kai Zou. 2019. EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language P...

2019
[68]

Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, and Hao Ma
[69]

arXiv preprint arXiv:2012.15466 (2020)

Clear: Contrastive learning for sentence representation. arXiv preprint arXiv:2012.15466 (2020)

2020 arXiv
[72]

In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

Low-rank adaptation of large language model rescoring for parameter- efficient speech recognition. In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 1–8

2023
[2018]

Transactions of the Association for Computational Linguistics 6 (2018), 391–406

Measuring the evolution of a scientific field through citation frames. Transactions of the Association for Computational Linguistics 6 (2018), 391–406

2018
[2020]

In Proceedings of the 58th Annual Meeting of the As- sociation for Computational Linguistics

SPECTER: Document-level Representation Learning using Citation- informed Transformers. In Proceedings of the 58th Annual Meeting of the As- sociation for Computational Linguistics . 2270–2282
[2021]

Quantitative science studies 2, 4 (2021), 1170–1215

A meta-analysis of semantic classification of citations. Quantitative science studies 2, 4 (2021), 1170–1215

2021
[2023]

arXiv preprint arXiv:2307.16645 (2023)

Scaling sentence embeddings with large language models. arXiv preprint arXiv:2307.16645 (2023)

2023 arXiv
[2024]

In Findings of the Association for Computational Linguistics: NAACL 2024

Few-TK: A Dataset for Few-shot Scientific Typed Keyphrase Recognition. In Findings of the Association for Computational Linguistics: NAACL 2024. 4011–4025

2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Riaz Ahmad and Muhammad Tanvir Afzal. 2018. CAD: An algorithm for citation- anchors detection in research papers. Scientometrics 117 (2018), 1405–1423

work page 2018

[2] [2]

Riaz Ahmad, Muhammad Tanvir Afzal, and Muhammad Abdul Qadir. 2017. Pat- tern analysis of citation-anchors in citing documents for accurate identification of in-text citations. IEEE Access 5 (2017), 5819–5828

work page 2017

[3] [3]

Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciBERT: A Pretrained Language Model for Scientific Text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . 3615–3620

work page 2019

[4] [4]

Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long- document transformer. arXiv preprint arXiv:2004.05150 (2020)

arXiv 2020

[5] [5]

Dan Berrebbi, Nicolas Huynh, and Oana Balalau. 2022. GraphCite: citation intent classification in scientific publications via graph embeddings. In Companion Proceedings of the Web Conference 2022 . 779–783

work page 2022

[6] [6]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

2020

[7] [7]

Martin Juan José Bucher and Marco Martini. 2024. Fine-Tuned’Small’LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classification. arXiv preprint arXiv:2406.08660 (2024)

arXiv 2024

[8] [8]

Jingqiang Chen and Hai Zhuge. 2019. Automatic generation of related work through summarizing citations. Concurrency and Computation: Practice and Experience 31, 3 (2019), e4261

work page 2019