arxiv: 2604.19335 · v1 · submitted 2026-04-21 · 💻 cs.LG

Recognition: unknown

When Active Learning Falls Short: An Empirical Study on Chemical Reaction Extraction

Simin Yu, Sufia Fathima

Pith reviewed 2026-05-10 03:45 UTC · model grok-4.3

classification 💻 cs.LG

keywords active learningchemical reaction extractiontransformer-CRFuncertainty samplingdiversity samplingproduct extractionrole labelinginformation extraction

0 comments

The pith

Conventional active learning shows unstable and task-dependent gains when extracting chemical reactions from text.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper runs a systematic empirical study of active learning for pulling reaction details out of chemical literature, where expert labels are costly to obtain. It pairs six standard uncertainty and diversity selection methods with pretrained transformer models that use CRF decoding, then tests them on identifying products and assigning roles in reactions. Several strategies reach near full-dataset performance with far fewer labels, yet the gains arrive in fits and starts rather than steady progress and differ sharply between the two tasks. The authors trace the instability to the strength of pretraining, the structured decoding step, and the sparsity of the labels themselves. Readers should care because these limits affect how quickly usable datasets can be built for reaction prediction and drug design.

Core claim

When six uncertainty- and diversity-based active learning strategies are combined with pretrained transformer-CRF models on product extraction and role labeling, the resulting learning curves are frequently non-monotonic and vary by task. Strong pretraining already yields solid baseline performance, structured CRF decoding alters how uncertainty signals are interpreted, and sparse labels reduce the informativeness of selected examples. Consequently conventional active learning fails to deliver stable improvements with reduced annotation budgets.

What carries the argument

The integration of uncertainty- and diversity-based query strategies with pretrained transformer-CRF architectures for product extraction and role labeling tasks.

If this is right

Certain active learning methods can reach performance close to a full dataset while using substantially fewer labeled examples.
Performance improvements under active learning are often non-monotonic rather than steadily increasing.
Effectiveness differs markedly between product extraction and role labeling tasks.
Strong pretraining, CRF decoding, and label sparsity each reduce the reliability of standard active learning.
Standard strategies require caution or modification before use in chemical information extraction.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Similar stability problems may appear in other scientific information extraction settings that rely on large-scale pretraining.
New query strategies that explicitly handle strong pretrained representations or structured outputs could be needed for this domain.
Hybrid approaches that adjust selection criteria after initial fine-tuning might mitigate the observed non-monotonic behavior.
Evaluating the same methods on broader reaction datasets would test whether the current task dependence is general.

Load-bearing premise

The assumption that the non-monotonic curves and task-dependent patterns seen on product extraction and role labeling will hold for other chemical reaction extraction settings.

What would settle it

A new experiment that obtains steady, monotonic performance gains and consistent results across tasks when the same active learning strategies are applied to a larger or more varied set of chemical reaction annotations.

Figures

Figures reproduced from arXiv: 2604.19335 by Simin Yu, Sufia Fathima.

**Figure 1.** Figure 1: Pool-based active learning framework applied to product extraction based on ChemBERT and role labeling based on [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

**Figure 2.** Figure 2: The learning curves of each active learning strategy. The dash line refers to the reported passive learning results [3]. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Figure 3: Visualization of sample selection process of Core-set and CLUSTER+ strategies for product extraction across rounds. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

read the original abstract

The rapid growth of chemical literature has generated vast amounts of unstructured data, where reaction information is particularly valuable for applications such as reaction predictions and drug design. However, the prohibitive cost of expert annotation has led to a scarcity of training data, severely hindering the performance of automatic reaction extraction. In this work, we conduct a systematic study of active learning for chemical reaction extraction. We integrate six uncertainty- and diversity-based strategies with pretrained transformer-CRF architectures, and evaluate them on product extraction and role labeling task. While several methods approach full-data performance with fewer labeled instances, learning curves are often non-monotonic and task-dependent. Our analysis shows that strong pretraining, structured CRF decoding, and label sparsity limit the stability of conventional active learning strategies. These findings provide practical insights for the effective use of active learning in chemical information extraction.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Active learning curves turn non-monotonic on chemical reaction tasks with pretrained transformer-CRF models, but the paper does not isolate whether pretraining, CRF, or sparsity actually drive the instability.

read the letter

The paper's main point is that six standard active learning strategies, when paired with a pretrained transformer-CRF, often produce non-monotonic learning curves on product extraction and role labeling, even though some eventually near full-supervision performance. The authors conclude that strong pretraining, structured CRF decoding, and label sparsity are responsible for the instability and task dependence. This is a straightforward empirical observation worth noting for anyone who assumes active learning will reliably cut annotation costs in chemistry NLP. The work is new in its focused comparison on these two tasks and in documenting the practical limits rather than claiming broad success. It does the field a service by showing the curves explicitly and by giving a clear caution that conventional strategies can be unreliable here. The citation pattern is appropriate and draws on relevant prior work in both active learning and chemical information extraction. The datasets and tasks are described enough to make the results usable as a reference point. The central limitation is the missing controls. Every reported run keeps the same pretrained transformer-CRF architecture, so there is no direct test of whether dropping pretraining, switching to a flat classifier, or using denser labels would restore monotonic improvement. Without those ablations the attribution remains suggestive; the non-monotonic behavior could equally come from the query strategies themselves, optimization details, or dataset properties. The abstract also omits statistical tests and full baseline numbers, which leaves the strength of the evidence moderate. This paper is aimed at chemoinformatics researchers who curate data for reaction prediction models and at NLP people working on low-resource structured extraction. A reader who needs to decide whether to invest in active learning for similar tasks will get actionable warnings. It is coherent on its own terms and shows honest engagement with the literature, so it deserves a serious referee even though the causal claims would need tightening through added experiments.

Referee Report

1 major / 2 minor

Summary. The paper conducts a systematic empirical study on active learning for chemical reaction extraction, integrating six uncertainty- and diversity-based strategies with pretrained transformer-CRF models and evaluating them on product extraction and role labeling tasks. It reports that several methods approach full-data performance with fewer labels but that learning curves are frequently non-monotonic and task-dependent. The central analysis concludes that strong pretraining, structured CRF decoding, and label sparsity limit the stability of conventional active learning strategies, providing practical insights for chemical information extraction.

Significance. If the attribution of non-monotonicity holds after proper controls, the work would offer valuable domain-specific guidance on the limitations of active learning in scientific NLP, particularly where pretrained models and structured prediction are involved. This could help practitioners avoid ineffective labeling strategies and motivate more robust AL variants for low-resource extraction tasks.

major comments (1)

[Abstract and experimental analysis] The central claim (Abstract) that strong pretraining, structured CRF decoding, and label sparsity limit AL stability is not isolated by controls. All experiments use the same pretrained transformer-CRF architecture; no ablations (random initialization, non-CRF flat classifiers, or denser label regimes) are reported to test whether monotonic curves are restored when these factors are removed. Without them the non-monotonicity could stem from the AL strategies, optimization, or dataset properties instead.

minor comments (2)

[Experimental setup] Dataset descriptions, including sizes, annotation protocols, and quantitative measures of label sparsity, are needed for reproducibility and to assess how representative the tasks are of broader chemical literature extraction.
[Results] Learning curves and performance tables would be strengthened by statistical tests or error bars to confirm that observed non-monotonicity exceeds noise.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive review and the suggestion to strengthen the attribution of non-monotonic learning curves through additional controls. We address the major comment below and outline planned revisions.

read point-by-point responses

Referee: [Abstract and experimental analysis] The central claim (Abstract) that strong pretraining, structured CRF decoding, and label sparsity limit AL stability is not isolated by controls. All experiments use the same pretrained transformer-CRF architecture; no ablations (random initialization, non-CRF flat classifiers, or denser label regimes) are reported to test whether monotonic curves are restored when these factors are removed. Without them the non-monotonicity could stem from the AL strategies, optimization, or dataset properties instead.

Authors: We agree that the absence of ablations leaves the attribution of non-monotonicity partially observational rather than fully isolated. Our study focuses on the practical, state-of-the-art setting for chemical reaction extraction, where pretrained transformer-CRF models are standard due to the structured nature of the task and the benefits of pretraining on limited data. The non-monotonic curves appear consistently across six AL strategies and two tasks when compared to random sampling, which suggests the behavior is not an artifact of any single AL method. Nevertheless, we acknowledge that alternative explanations (e.g., optimization dynamics or dataset characteristics) cannot be ruled out without controls. To address this, we will add a dedicated limitations subsection and, where computationally feasible, include a small-scale ablation comparing the pretrained CRF to a randomly initialized flat classifier on a subset of the data. We will also revise the abstract and conclusion to use more cautious phrasing such as 'are associated with' rather than 'limit'. revision: partial

Circularity Check

0 steps flagged

No significant circularity; purely empirical reporting

full rationale

This is an empirical study that integrates active learning strategies with transformer-CRF models and reports observed learning curves and performance metrics on product extraction and role labeling tasks. No mathematical derivations, equations, fitted parameters renamed as predictions, or self-referential claims appear in the provided text. The central observations about non-monotonic curves and task dependence are presented as direct experimental outcomes without reduction to inputs by construction or load-bearing self-citations. The paper is self-contained against its own benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

As an empirical study, the central claims rest on assumptions about the representativeness of the chosen tasks and models rather than mathematical derivations or new entities.

axioms (1)

domain assumption The product extraction and role labeling tasks are representative of the general challenges in chemical reaction extraction from literature.
The study draws general conclusions about active learning limitations from evaluations on these two specific tasks.

pith-pipeline@v0.9.0 · 5435 in / 1304 out tokens · 50258 ms · 2026-05-10T03:45:51.072780+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

50 extracted references · 9 canonical work pages · 1 internal anchor

[1]

A graph-convolutional neural network model for the prediction of chemical reactivity,

C. W. Coley, W. Jin, L. Rogers, T. F. Jamison, T. S. Jaakkola, W. H. Green, R. Barzilay, and K. F. Jensen, “A graph-convolutional neural network model for the prediction of chemical reactivity,”Chemical science, vol. 10, no. 2, pp. 370–377, 2019

2019
[2]

Computational approaches stream- lining drug discovery,

A. V . Sadybekov and V . Katritch, “Computational approaches stream- lining drug discovery,”Nature, vol. 616, no. 7958, pp. 673–685, 2023

2023
[3]

Automated chemical reaction extraction from scientific literature,

J. Guo, A. S. Ibanez-Lopez, H. Gao, V . Quach, C. W. Coley, K. F. Jensen, and R. Barzilay, “Automated chemical reaction extraction from scientific literature,”Journal of chemical information and modeling, vol. 62, no. 9, pp. 2035–2045, 2021

2035
[4]

https://www.elsevier

Reaxys – An expert-curated chemistry database. https://www.elsevier. com/products/reaxys
[5]

https://www.cas.org/ solutions/cas-scifinder-discovery-platform

CAS SciFinder - Chemical Compound Database. https://www.cas.org/ solutions/cas-scifinder-discovery-platform
[6]

The making of reaxys—towards unobstructed access to relevant chemistry information,

A. J. Lawson, J. Swienty-Busch, T. G ´eoui, and D. Evans, “The making of reaxys—towards unobstructed access to relevant chemistry information,” inThe Future of the History of Chemical Information, pp. 127–148, ACS Publications, 2014

2014
[7]

Chemi- caltagger: A tool for semantic text-mining in chemistry,

L. Hawizy, D. M. Jessop, N. Adams, and P. Murray-Rust, “Chemi- caltagger: A tool for semantic text-mining in chemistry,”Journal of cheminformatics, vol. 3, no. 1, p. 17, 2011

2011
[8]

D. M. Lowe,Extraction of chemical structures and reactions from the literature. PhD thesis, 2012

2012
[9]

Chemspot: a hybrid system for chemical named entity recognition,

T. Rockt ¨aschel, M. Weidlich, and U. Leser, “Chemspot: a hybrid system for chemical named entity recognition,”Bioinformatics, vol. 28, no. 12, pp. 1633–1640, 2012

2012
[10]

tmchem: a high performance approach for chemical named entity recognition and normalization,

R. Leaman, C.-H. Wei, and Z. Lu, “tmchem: a high performance approach for chemical named entity recognition and normalization,” Journal of cheminformatics, vol. 7, no. Suppl 1, p. S3, 2015

2015
[11]

Extracting chemical reactions from text using snorkel,

E. K. Mallory, M. de Rochemonteix, A. Ratner, A. Acharya, C. Re, R. A. Bright, and R. B. Altman, “Extracting chemical reactions from text using snorkel,”BMC bioinformatics, vol. 21, no. 1, p. 217, 2020

2020
[12]

Snorkel: Rapid training data creation with weak supervision,

A. Ratner, S. H. Bach, H. Ehrenberg, J. Fries, S. Wu, and C. R ´e, “Snorkel: Rapid training data creation with weak supervision,” in Proceedings of the VLDB endowment. International conference on very large data bases, vol. 11, p. 269, 2017

2017
[13]

Domain-specific language model pretraining for biomedical natural language processing,

Y . Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, and H. Poon, “Domain-specific language model pretraining for biomedical natural language processing,”ACM Transactions on Computing for Healthcare (HEALTH), vol. 3, no. 1, pp. 1–23, 2021

2021
[14]

Fine-tuning large language models for chemical text mining,

W. Zhang, Q. Wang, X. Kong, J. Xiong, S. Ni, D. Cao, B. Niu, M. Chen, Y . Li, R. Zhang,et al., “Fine-tuning large language models for chemical text mining,”Chemical science, vol. 15, no. 27, pp. 10600–10611, 2024

2024
[15]

Bert: Pre-training of deep bidirectional transformers for language understanding,

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” inPro- ceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pp. 4171–4186, 2019

2019
[16]

Active learning literature survey,

B. Settles, “Active learning literature survey,” 2009

2009
[17]

Introduction to the conll-2003 shared task: Language-independent named entity recognition,

E. F. Sang and F. De Meulder, “Introduction to the conll-2003 shared task: Language-independent named entity recognition,”arXiv preprint cs/0306050, 2003

work page internal anchor Pith review arXiv 2003
[18]

Ontonotes release 4.0,

R. Weischedel, S. Pradhan, L. Ramshaw, M. Palmer, N. Xue, M. Marcus, A. Taylor, C. Greenberg, E. Hovy, R. Belvin,et al., “Ontonotes release 4.0,”LDC2011T03, Philadelphia, Penn.: Linguistic Data Consortium, vol. 17, 2011

2011
[19]

A survey on named en- tity recognition—datasets, tools, and methodologies,

B. Jehangir, S. Radhakrishnan, and R. Agarwal, “A survey on named en- tity recognition—datasets, tools, and methodologies,”Natural Language Processing Journal, vol. 3, p. 100017, 2023

2023
[20]

Deep active learning for named entity recognition,

Y . Shen, H. Yun, Z. C. Lipton, Y . Kronrod, and A. Anandkumar, “Deep active learning for named entity recognition,”arXiv preprint arXiv:1707.05928, 2017

work page arXiv 2017
[21]

A survey of deep active learning,

P. Ren, Y . Xiao, X. Chang, P.-Y . Huang, Z. Li, B. B. Gupta, X. Chen, and X. Wang, “A survey of deep active learning,”ACM computing surveys (CSUR), vol. 54, no. 9, pp. 1–40, 2021

2021
[22]

Active learning using uncertainty information,

Y . Yang and M. Loog, “Active learning using uncertainty information,” in2016 23rd International Conference on Pattern Recognition (ICPR), pp. 2646–2651, IEEE, 2016

2016
[23]

Cost-effective active learning for deep image classification,

K. Wang, D. Zhang, Y . Li, R. Zhang, and L. Lin, “Cost-effective active learning for deep image classification,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 27, no. 12, pp. 2591–2600, 2016

2016
[24]

Deep bayesian active learning with image data,

Y . Gal, R. Islam, and Z. Ghahramani, “Deep bayesian active learning with image data,” inInternational conference on machine learning, pp. 1183–1192, PMLR, 2017

2017
[25]

Active learning for sequence tagging with deep pre-trained models and bayesian uncertainty estimates,

A. Shelmanov, D. Puzyrev, L. Kupriyanova, D. Belyakov, D. Larionov, N. Khromov, O. Kozlova, E. Artemova, D. V . Dylov, and A. Panchenko, “Active learning for sequence tagging with deep pre-trained models and bayesian uncertainty estimates,”arXiv preprint arXiv:2101.08133, 2021

work page arXiv 2021
[26]

Active Learning for Convolutional Neural Networks: A Core-Set Approach

O. Sener and S. Savarese, “Active learning for convolutional neural networks: A core-set approach,”arXiv preprint arXiv:1708.00489, 2017

work page Pith review arXiv 2017
[27]

Liu and Z

J. Liu and Z. S. Wong, “Utilizing active learning strategies in machine- assisted annotation for clinical named entity recognition: a compre- hensive analysis considering annotation costs and target effectiveness,” Journal of the American Medical Informatics Association, vol. 31, no. 11, pp. 2632–2640, 2024

2024
[28]

Ud bbc: Named entity recognition in social network combined bert-bilstm-crf with active learning,

W. Li, Y . Du, X. Li, X. Chen, C. Xie, H. Li, and X. Li, “Ud bbc: Named entity recognition in social network combined bert-bilstm-crf with active learning,”Engineering Applications of Artificial Intelligence, vol. 116, p. 105460, 2022

2022
[29]

A cybersecurity named entity recognition model based on active learning and self-learning,

Z. Liu, K. Jiang, Z. Liu, and T. Qin, “A cybersecurity named entity recognition model based on active learning and self-learning,” in2024 36th Chinese Control and Decision Conference (CCDC), pp. 4505–4510, IEEE, 2024

2024
[30]

Chemdataextractor: a toolkit for automated extraction of chemical information from the scientific literature,

M. C. Swain and J. M. Cole, “Chemdataextractor: a toolkit for automated extraction of chemical information from the scientific literature,”Journal of chemical information and modeling, vol. 56, no. 10, pp. 1894–1904, 2016

1904
[31]

The chemdner corpus of chemicals and drugs and its annotation principles,

M. Krallinger, O. Rabal, F. Leitner, M. Vazquez, D. Salgado, Z. Lu, R. Leaman, Y . Lu, D. Ji, D. M. Lowe,et al., “The chemdner corpus of chemicals and drugs and its annotation principles,”Journal of cheminformatics, vol. 7, no. Suppl 1, p. S2, 2015

2015
[32]

Oscar4: a flexible architecture for chemical text-mining,

D. M. Jessop, S. E. Adams, E. L. Willighagen, L. Hawizy, and P. Murray- Rust, “Oscar4: a flexible architecture for chemical text-mining,”Journal of cheminformatics, vol. 3, no. 1, p. 41, 2011

2011
[33]

Khabsa,Towards better accessibility of scholarly data

M. Khabsa,Towards better accessibility of scholarly data. The Penn- sylvania State University, 2015

2015
[34]

Chemical named entities recognition: a review on approaches and applications,

S. Eltyeb and N. Salim, “Chemical named entities recognition: a review on approaches and applications,”Journal of cheminformatics, vol. 6, no. 1, p. 17, 2014

2014
[35]

Information retrieval and text mining technologies for chemistry,

M. Krallinger, O. Rabal, A. Lourenco, J. Oyarzabal, and A. Valencia, “Information retrieval and text mining technologies for chemistry,” Chemical reviews, vol. 117, no. 12, pp. 7673–7761, 2017

2017
[36]

Neural architectures for named entity recognition,

G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. Dyer, “Neural architectures for named entity recognition,”arXiv preprint arXiv:1603.01360, 2016

work page arXiv 2016
[37]

Matscibert: A materials domain language model for text mining and information extraction,

T. Gupta, M. Zaki, N. A. Krishnan, and Mausam, “Matscibert: A materials domain language model for text mining and information extraction,”npj Computational Materials, vol. 8, no. 1, p. 102, 2022

2022
[38]

Seyone Chithrananda, Gabriel Grand, Bharath Ramsun- dar, et al

S. Chithrananda, G. Grand, and B. Ramsundar, “Chemberta: large-scale self-supervised pretraining for molecular property prediction,”arXiv preprint arXiv:2010.09885, 2020

work page arXiv 2010
[39]

A chemical reaction entity recognition method based on a natural language data augmentation strategy,

X. Zhang, Y . Li, C. Li, J. Zhu, Z. Gan, L. Wang, X. Sun, and H. You, “A chemical reaction entity recognition method based on a natural language data augmentation strategy,”Chemical Communications, vol. 60, no. 71, pp. 9610–9613, 2024

2024
[40]

Entity-based synthetic data generation for named entity recognition in low-resource domains,

T. A. Dao, H. Teranishi, Y . Matsumoto, and A. Aizawa, “Entity-based synthetic data generation for named entity recognition in low-resource domains,” inJSAI International Symposium on Artificial Intelligence, pp. 210–225, Springer, 2025

2025
[41]

k-means++: The advantages of careful seeding,

D. Arthur and S. Vassilvitskii, “k-means++: The advantages of careful seeding,” tech. rep., Stanford, 2006

2006
[42]

A study of deep active learning methods to reduce labelling efforts in biomedical relation extraction,

C. Nachtegael, J. De Stefani, and T. Lenaerts, “A study of deep active learning methods to reduce labelling efforts in biomedical relation extraction,”PloS one, vol. 18, no. 12, p. e0292356, 2023

2023
[43]

Ltp: a new active learning strategy for bert-crf based named entity recognition,

M. Liu, Z. Tu, Z. Wang, and X. Xu, “Ltp: a new active learning strategy for bert-crf based named entity recognition,”arXiv preprint arXiv:2001.02524, 2020

work page arXiv 2001
[44]

Active learning approach using a modified least confidence sampling strategy for named entity recognition,

A. Agrawal, S. Tripathi, and M. Vardhan, “Active learning approach using a modified least confidence sampling strategy for named entity recognition,”Progress in Artificial Intelligence, vol. 10, no. 2, pp. 113– 128, 2021

2021
[45]

Active hidden markov models for information extraction,

T. Scheffer, C. Decomain, and S. Wrobel, “Active hidden markov models for information extraction,” inInternational symposium on intelligent data analysis, pp. 309–318, Springer, 2001. 9

2001
[46]

Bayesian active learning for classiﬁcation and preferenc e learning,

N. Houlsby, F. Husz ´ar, Z. Ghahramani, and M. Lengyel, “Bayesian active learning for classification and preference learning,”arXiv preprint arXiv:1112.5745, 2011

work page arXiv 2011
[47]

Dropout as a bayesian approximation: Representing model uncertainty in deep learning,

Y . Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” ininternational conference on machine learning, pp. 1050–1059, PMLR, 2016

2016
[48]

Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning,

A. Kirsch, J. Van Amersfoort, and Y . Gal, “Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning,”Advances in neural information processing systems, vol. 32, 2019

2019
[49]

The speed-up factor: A quanti- tative multi-iteration active learning performance metric,

H. Kath, T. S. Gouv ˆea, and D. Sonntag, “The speed-up factor: A quanti- tative multi-iteration active learning performance metric,”Transactions on Machine Learning Research, 2026

2026
[50]

Visualizing data using t-sne.,

L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.,” Journal of machine learning research, vol. 9, no. 11, 2008. 10 APPENDIX TABLE III: Test precision (%) across rounds for product extraction. The best value in each strategy is in bold. The passive learning baseline is 84.62. Strategy\Round 1 2 3 4 5 6 7 8 9 10 Core-set 0.00 66.67 74.65 77.22...

work page arXiv 2008