Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

MatSKRAFT: A framework for large-scale materials knowledge extraction from scientific tables

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A specialized graph-neural-network pipeline extracts materials data from scientific tables with higher F1 scores than leading large language models, while running far faster on a single GPU.

desk verdict A solid, carefully evaluated table-extraction system with genuine engineering value, but the database-scale claims run ahead of the evidence. read the letter →

arxiv 2509.10448 v2 pith:3D7JGU4E submitted 2025-09-12 cs.IR cond-mat.mtrl-sci

Kausik Hira , Mohd Zaki , Mausam , N. M. Anoop Krishnan This is my paper · ORCID
classification cs.IRcond-mat.mtrl-sci
keywords materialsinformaticsscientifictableextractiongraphneuralnetworkspropertycompositionlargelanguagemodelsknowledgebaseconstructiondistantsupervision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MatSKRAFT is a five-stage pipeline that converts scientific tables into graphs and uses specialized graph neural networks with rule-based post-processing to extract material compositions and physical properties. The paper reports F1 scores of 88.68 for property extraction and 71.35 for composition extraction on manually annotated test sets, beating several frontier LLMs by 15–17 F1 points while processing each table in 0.22–0.39 seconds. Applied to nearly 69,000 tables from more than 47,000 papers, the framework reportedly produced a knowledge base of over 535,000 entries, including more than 104,000 compositions absent from established databases. If these claims hold, the practical significance is that corpus-scale literature mining could become a routine, inexpensive task for a single research group rather than a costly LLM-based operation.

What carries the argument

The load-bearing object is the table-as-graph representation fed to a two-layer graph attention network (GAT). Nodes are cells, row/column headers, and the caption; edges link cells to their row and column headers and feed caption information into headers. Four differentiable constraints—material-property association, material-identifier exclusivity, property exclusivity, and identifier uniqueness—are added to the cross-entropy loss with weight λ = 50 so the network learns scientifically coherent table readings. A separate modular post-processing layer then performs unit extraction (including over 15 electrical-conductivity unit variants), physical-range checks, exponent reconstruction for values like thermal expansion, and disambiguation of overloaded symbols. This combination, rather than any single component, carries the argument: removing post-processing costs 9.38 F1 points, and removing the annotation algorithms costs 9.02 F1 points.

What would settle it

Manually annotate a random sample of, say, 300 tables from the 68,933-table corpus and run the released pipeline on them; if observed F1 falls substantially below the reported 88.68 for properties or 71.35 for compositions on the held-out test sets, the extrapolation fails. A second check is to take 100 of the claimed 104,000 “new” compositions and verify by reading the source paper that the composition truly appears and belongs to the linked property record.

Watch

Extended reading notes

Core claim

The paper's central claim is that a deliberately specialized, constraint-driven extraction architecture can outperform generalist large language models on scientific table understanding at a fraction of the compute cost. The framework represents each table as a graph with nodes for cells, headers, and captions, and trains graph attention networks under soft constraints that encode conventions of materials tables—for example, that a material identifier and its properties should occupy the same orientation, and that only one header should identify the material. Domain rules then post-process the network's predictions, validating units, physical ranges, and ambiguous notation such as “n” standing for refractive index or Poisson's ratio. On expert-annotated test sets the authors report F1 88.68 across 18 properties, with density at 96.50 and glass transition temperature at 93.00, and F1 71.35 for compositions, with single-cell and multi-cell tables at 78.62 and 75.99. They also run the pipeline over the large corpus, building 100,481 linked composition–property pairs and a knowledge base of 535,643 entries, with the large-scale database still pending manual validation.

Load-bearing premise

The whole-scale database and discovery claims rest on the assumption that the manually annotated dev and test tables are representative of the roughly 69,000 tables in the large-scale run, so the measured F1, precision, and recall transfer to the full corpus.

Editorial extensions

If this is right

  • If the reported F1 scores transfer to the full corpus, researchers can extract structured composition–property data from entire journal collections on a single GPU in hours rather than weeks.
  • The reported 104,000 compositions absent from existing databases would substantially widen the known inorganic composition space, especially for rare-earth elements, transition metals, and semiconductor dopants.
  • The pipeline's 0.22–0.39 seconds per table makes iterative, corpus-wide information extraction economical enough for periodic re-runs as new literature is published.
  • The ablation results suggest that the same design—constraint-driven GNN plus domain post-processing—could be ported to other fields whose tables follow systematic reporting conventions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The largest untested step is the leap from 368 and 737 manually annotated test tables to roughly 69,000 deployed tables; a random manual audit of even 200 deployed tables would tell whether the measured F1 holds outside the dev/test distribution.
  • Because the framework is fully automated and works without commercial database supervision (the no-distant-supervision ablation reaches 93.04 precision), a natural extension is retraining it for battery, polymer, or catalysis tables, where property vocabularies differ.
  • The 104,000 “new” compositions are defined as absent from INTERGLAD and SciGlass; some may appear in other aggregated databases or in text-only form in the same papers, so checking against further databases would clarify how much genuinely new composition space is covered.
  • Because evaluation requires exact value+unit+name matches, the reported entity-level F1 is conservative; however, linked composition–property pairs reach only 78.08 precision, so downstream users should treat pair-level claims with more caution than single-entity extraction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces Matskraft, a pipeline for extracting materials compositions and properties from scientific tables. The method converts tables into graphs processed by constraint-driven graph attention networks, with automated training-data generation via distant supervision, property-specific annotation algorithms, and power-law-guided augmentation. The authors report F1 scores of 88.68 for property extraction and 71.35 for composition extraction on manually annotated test sets, outperform five LLM baselines, and claim a 19–496x speed advantage. They then apply the pipeline to roughly 69,000 tables from more than 47,000 articles, constructing a knowledge base of about 535,000 entries, including more than 104,000 compositions reported as absent from INTERGLAD and SciGlass, and demonstrate several materials-discovery applications.

Significance. If the claims hold, this is a substantial contribution: it provides evidence that a specialized, modest-compute GNN pipeline can outperform general-purpose LLMs on structured materials-table extraction, and it demonstrates a path toward automated large-scale knowledge-base construction from the literature. The work is strengthened by extensive ablations, expert-annotated evaluation sets, algorithmic appendices that describe the annotation and post-processing logic, and public code/data links. The main significance risk is that the headline claims about the constructed database and its novelty rest on extrapolation from small test sets to a large, unvalidated corpus.

major comments (5)
  1. [Abstract and Section 2.3] The headline numbers are internally inconsistent. The abstract states property-extraction F1 of 89.33 and a speed advantage of 6–496x, while the body (Section 2.3.1, Section 2.3.6, and Table C.1) reports 88.68 and 19–496x. The abstract also reports 66,267 tables from more than 45,500 publications yielding 509,281 entries, whereas Section 2.4 and Table E.1 report 68,933 tables from 47,242 articles yielding 535,643 entries. The authors should reconcile the abstract with the body, since these are precisely the results a reader will quote.
  2. [Section 2.4 and Appendix A.1] The database-scale claims are not supported by the evaluation presented. The reported F1, precision, and recall figures come from 368 annotated property tables and 737 annotated composition tables (Appendix A.1), but no evidence is given that these tables are representative of the 68,933 tables in the deployed corpus, and no corpus-level audit, stratified sample, or confidence interval is provided. The abstract itself concedes that the database is "pending manual validation." Consequently, statements that the pipeline constructs a "comprehensive database" and that the reported accuracies transfer to the full corpus are extrapolations, not demonstrated results.
  3. [Section 2.4 and Appendix E.3] The "104,000 compositions absent from existing databases" claim is much weaker than stated. Novelty is defined only as absence from INTERGLAD and SciGlass, both glass-specific databases, whereas the corpus deliberately includes journals such as Ceramics International, Thin Solid Films, Materials Letters, and others covering non-glass systems (Appendix E.1). Finding compositions absent from two glass databases is therefore expected for a large fraction of the corpus. The claim would need comparison against broader materials databases (e.g., ICSD, Materials Project, or a similarly general repository) or, at minimum, a re-framing as "absent from the two glass databases used here." No detail is given about composition canonicalization, duplicate handling, or matching thresholds for this comparison.
  4. [Section 2.3.2 and Table A.2] The composition-extraction recall of 62.97% overall, and 38.93% on Partial-Information tables, is a load-bearing limitation for the large-scale database claim. PI tables constitute a non-negligible fraction of the test and dev splits (111/737 test tables, 109/738 dev tables in Table A.2), and the deployed corpus is processed by the same models. With recall below 40% on this table type, the resulting database will substantially undercount compositions that require contextual inference. The paper should quantify how PI-table recall affects completeness of the 535k-entry knowledge base and should temper the "comprehensive" wording accordingly.
  5. [Section 2.3.4 and Appendix C] The provenance of the pair-level precision figure of 78.08% is unclear. Section 2.3.4 describes an evaluation of linked composition-property entities but does not specify the sample size, annotation protocol, or whether the evaluation is on the test set; Appendix C then states that this precision applies "across our comprehensive knowledge base of over 535,000 entries." If 78.08% is a test-set number, it should not be presented as a database-level estimate without a corpus audit. If it is intended as a database-level number, the current text provides no evidence for it.
minor comments (5)
  1. [Equation (1)] The citation to MatSciBERT appears as "MatSciBERT [?]" in Equation (1); the reference marker should be resolved.
  2. [Throughout] The framework name is inconsistently capitalized as "MatSKRAFT" in some places and "Matskraft" in others; the acronym is also expanded only in the Introduction, not at first use in the Abstract.
  3. [Table C.2] The Poisson ratio unit accuracy is reported as "—" without explanation; a footnote stating that this property is dimensionless would avoid confusion.
  4. [Appendix B.1.2] The text refers to removing the "Annotation Algorithm" in one ablation label while the corresponding row in Table B.1 is labeled "w/o Annotation algorithms"; the nomenclature should be made consistent.
  5. [Section 2.5 and Figure 6] The temporal analysis is derived from MatSciBERT analysis of abstracts but no evaluation or validation of that analysis is provided; a brief statement about the source or precision of this application would improve clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central extraction F1 claims are evaluated against independent expert annotations and external LLM baselines, while the large-scale KB/database claims are extrapolations with an explicit 'pending manual validation' caveat rather than consequences of the model's own fitted values.

full rationale

The claimed property and composition F1 scores (88.68 and 71.35) are computed on manually annotated dev/test sets (368 property and 737 composition tables; App. A.1, Sec. 2.3) held out from the automated training pipeline, so the extraction results are not defined by the training data generation rules. LLM comparisons use external models (Gemini, GPT-4o, Claude, DeepSeek) under optimized prompts, providing an independent benchmark. Distant supervision from INTERGLAD supplies training labels, but the test labels are human annotations, so the performance numbers do not reduce to the distant-supervision source. The composition module reuses the authors' prior DiSCoMaT architecture and MatSciBERT embeddings, but this is architectural reuse and is not the evidence for the claimed improvements; the enhancements are again scored against manual ground truth. The database-scale claims (535,643 entries, 104,000 compositions absent from INTERGLAD/SciGlass) are obtained by applying the trained models to a larger corpus and comparing against external databases; the abstract explicitly says the database is 'pending manual validation,' which is a stated limitation rather than a circular derivation. The concern that test-set accuracy may not transfer to the 68,933-table corpus is a validity/extrapolation risk, not a circularity of the kind where an input is renamed as a prediction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest on a modest number of fitted hyperparameters (augmentation exponents, constraint weight, threshold) and several domain assumptions about label quality, test-set representativeness, and database-novelty interpretation. The paper does not introduce a new physical entity or force; the only 'invented' resource is the KB itself, which is released for inspection.

free parameters (5)
  • Power-law augmentation exponents (a, alpha) = a=10, alpha=0.65
    Tuned in Algorithm 3 to align target and observed property frequency distributions; directly sets how many synthetic training rows/columns are generated for rare properties.
  • Constraint loss weight lambda = 50
    Eq. (13) in Section 4.3.1; hand-chosen balance between cross-entropy and constraint regularization.
  • Composition confidence threshold = 0.7
    Appendix B.2.1; softmax cutoff below which compositional predictions are reclassified as non-compositional, trading recall for precision.
  • Property-specific validation thresholds t_p = Not specified; tunable
    Algorithm 2; each property has a calibrated score threshold for accepting header annotations, but the values are not reported.
  • Augmentation Gaussian noise floor = sigma=0.05
    Section 4.2.3; injected when the source column has zero variance so that synthetic values remain non-degenerate.
assumptions (5)
  • domain assumption Numeric table entries that match INTERGLAD values within tolerance correspond to the same property or composition.
    Section 4.2.1; this alignment is the basis of distant supervision and could inject systematic label noise if INTERGLAD records are incomplete or mismatched.
  • domain assumption The manually annotated dev/test sets are representative of the ~69,000 deployed tables.
    Section 2.3 and App A.1; all reported accuracy and all KB-quality claims extrapolate from 368/737 tables to the full corpus without a distribution-shift check.
  • domain assumption Absence from INTERGLAD and SciGlass indicates a composition is novel or underexplored.
    Section 2.4; both databases focus on glass, while the corpus includes ceramics, thin films, and solid-state materials, so absence may reflect scope rather than novelty.
  • domain assumption Gaussian sampling from observed property values preserves realistic table statistics for augmentation.
    Section 4.2.3; synthetic values are drawn from Normal(mu, sigma) with +/-3sigma clipping, assuming unimodal stable distributions per property.
  • domain assumption Exact-match entity-level scoring is an appropriate accuracy measure.
    Section 4.6; partial matches count as failures, which will depress absolute numbers for both MatSKRAFT and LLMs, but relative comparisons remain valid.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MatSKRAFT: A framework for large-scale materials knowledge extraction from scientific tables." pith.science (2026). https://pith.science/paper/3D7JGU4E

@misc{pith2026250910448,
  author       = {Pith},
  title        = {Pith review of: MatSKRAFT: A framework for large-scale materials knowledge extraction from scientific tables},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3D7JGU4E}},
  note         = {Machine review of arXiv:2509.10448}
}
read the original abstract

Scientific progress increasingly depends on synthesizing knowledge across vast literature, yet most experimental data remains trapped in semi-structured formats that resist systematic extraction and analysis. Here, we present MatSKRAFT, a computational framework that automatically extracts and integrates materials science knowledge from tabular data at unprecedented scale. Our approach transforms tables into graph-based representations processed by constraint-driven GNNs that encode scientific principles directly into model architecture. MatSKRAFT significantly outperforms contemporary frontier large language models, achieving F1 scores of 89.33 for property extraction and 71.35 for composition extraction, while processing data 6-496 times faster compared to the fastest and the slowest models respectively, with modest hardware requirements. Applied to 66,267 tables from more than 45,500 research publications, we construct a comprehensive database containing 509,281 entries, including 104,000 compositions that expand coverage beyond major existing databases. This systematic approach reveals previously overlooked materials with distinct property combinations and enables data-driven discovery of composition-property relationships forming the cornerstone of materials and scientific discovery.

Figures

Figures reproduced from arXiv: 2509.10448 by the authors.

Figure 1
Figure 1. MatSKRAFT framework for automated knowledge extraction from scientific tables. The pipeline comprises: (1) Dataset preparation involving systematic processing of scientific articles and automated training data generation (2) Property extraction using graph neural network (GNN1) with domain-based post-processing; (3) Composition extraction employing specialized GNNs for different table structures; (4) Knowledge-base … view at source ↗
Figure 2
Figure 2. Performance comparison of MaTSKRAFT with leading LLMs for information ex￾traction from tables. (A) F1 scores for property and composition extraction across different approaches, showing Matskraft’s superior performance. (B) Radar chart visualizing F1 scores across individual properties grouped by category (optical, electrical, physical, and mechanical), demonstrating consistent performance advantages across diverse … view at source ↗
Figure 3
Figure 3. Statistical analysis of extracted scientific knowledge. (A) Periodic table visualization of elemental frequency in extracted compositions, with color intensity representing occurrence frequency. (B) Distribution of extracted properties showing coverage across 18 distinct material properties on logarithmic scale. 0 2 4 6 MatSKRAFT Activation energy (eV) 0 2 4 6 INTERGLAD 0 2 4 6 SciGlass No data 0 500 1000 1500 2000 … view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparison of property value distributions across materials databases. Violin plots comparing range and distribution of 18 key properties in Matskraft (top row), Interglad (middle row), and SciGlass (bottom row). Matskraft demonstrates substantially broader coverage ac…
Figure 5
Figure 5. Figure 5: Applications for materials discovery and screening. (A–C): Systematic analysis reveals distinct material families achieving remarkable property combinations across thermal and mechanical prop￾erty spaces through compositional interdependencies. (D–F): Multi-property sc…
Figure 6
Figure 6. Figure 6: Temporal analysis of materials research evolution (1999–2022). Publication trends across application domains derived from analysis of abstracts, revealing strategic shifts in research focus with explosive growth in Energy, Coatings/Barriers, and Optical/Photonics after…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Generative and multimodal AI for materials prediction and design: Progress, challenges, and perspectives

    cond-mat.mtrl-sci 2026-07 conditional novelty 6.0 of 10

    Generative and multimodal AI can propose compositionally new materials, but a three-level novelty taxonomy shows current evidence supports only structural novelty, not physical or deployment novelty.

Reference graph

Works this paper leans on

95 extracted references · 67 canonical work pages · cited by 1 Pith paper

  1. [1]

    Data-driven materials research enabled by natural language processing and information extraction.Applied Physics Reviews, 7(4), 2020

    Elsa A Olivetti, Jacqueline M Cole, Edward Kim, Olga Kononova, Gerbrand Ceder, Thomas Yong-Jin Han, and Anna M Hiszpanski. Data-driven materials research enabled by natural language processing and information extraction.Applied Physics Reviews, 7(4), 2020

    2020

  2. [2]

    Science of science.Science, 359(6379):eaao0185, 2018

    Santo Fortunato, Carl T Bergstrom, Katy Börner, James A Evans, Dirk Helbing, Staša Milojević, Alexander M Petersen, Filippo Radicchi, Roberta Sinatra, Brian Uzzi, et al. Science of science.Science, 359(6379):eaao0185, 2018

    2018

  3. [3]

    Lutz Bornmann and Rüdiger Mutz. Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references.Journal of the association for information science and technology, 66(11):2215–2222, 2015

    2015

  4. [4]

    Columbia university press, 1963

    Derek J De Solla Price.Little science, big science. Columbia university press, 1963

    1963

  5. [5]

    The science of science: From the perspective of complex systems.Physics reports, 714:1–73, 2017

    An Zeng, Zhesi Shen, Jianlin Zhou, Jinshan Wu, Ying Fan, Yougui Wang, and H Eugene Stanley. The science of science: From the perspective of complex systems.Physics reports, 714:1–73, 2017

    2017

  6. [6]

    Looking through glass: Knowledge discovery from materials science literature using natural language processing.Patterns, 2(7), 2021

    Vineeth Venugopal, Sourav Sahoo, Mohd Zaki, Manish Agarwal, Nitya Nand Gosvami, and NM Anoop Krishnan. Looking through glass: Knowledge discovery from materials science literature using natural language processing.Patterns, 2(7), 2021

    2021

  7. [7]

    Reconstructing the materials tetrahedron: challenges in materials information extraction.Digital Discovery, 3(5):1021–1037, 2024

    Kausik Hira, Mohd Zaki, Dhruvil Sheth, NM Anoop Krishnan, et al. Reconstructing the materials tetrahedron: challenges in materials information extraction.Digital Discovery, 3(5):1021–1037, 2024

    2024

  8. [8]

    Zach Jensen, Edward Kim, Soonhyoung Kwon, Terry Z. H. Gani, Yuriy Román-Leshkov, Manuel Moliner, Avelino Corma, and Elsa Olivetti. A machine learning approach to zeolite synthesis enabled by automatic literature data extraction.ACS Central Science, 5(5):892–899, 2019

    2019

Show all 95 references
  1. [9]

    Chemdataextractor: a toolkit for automated extraction of chemical information from the scientific literature.Journal of chemical information and modeling, 56(10):1894–1904, 2016

    Matthew C Swain and Jacqueline M Cole. Chemdataextractor: a toolkit for automated extraction of chemical information from the scientific literature.Journal of chemical information and modeling, 56(10):1894–1904, 2016

    1904

  2. [10]

    Matscibert: A materials domain language model for text mining and information extraction.npj Computational Materials, 8(1):102, 2022

    Tanishq Gupta, Mohd Zaki, NM Anoop Krishnan, and Mausam. Matscibert: A materials domain language model for text mining and information extraction.npj Computational Materials, 8(1):102, 2022

    2022

  3. [11]

    Opticalbert and opticaltable-sqa: Text-and table- based language models for the optical-materials domain.Journal of Chemical Information and Modeling, 63(7):1961–1981, 2023

    Jiuyang Zhao, Shu Huang, and Jacqueline M Cole. Opticalbert and opticaltable-sqa: Text-and table- based language models for the optical-materials domain.Journal of Chemical Information and Modeling, 63(7):1961–1981, 2023

    1961

  4. [12]

    Matsci-nlp: Evaluating scientific language models on materials science language tasks using text-to-schema modeling.arXiv preprint arXiv:2305.08264, 2023

    Yu Song, Santiago Miret, and Bang Liu. Matsci-nlp: Evaluating scientific language models on materials science language tasks using text-to-schema modeling.arXiv preprint arXiv:2305.08264, 2023

    2023 arXiv

  5. [13]

    DiSCoMaT: Distantly supervised composition extraction from tables in materials science articles

    Tanishq Gupta, Mohd Zaki, Devanshi Khatsuriya, Kausik Hira, N M Anoop Krishnan, and Mausam . DiSCoMaT: Distantly supervised composition extraction from tables in materials science articles. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguist...

    2023

  6. [14]

    Leveraging large language models for predictive chemistry.Nature Machine Intelligence, pages 1–9, 2024

    Kevin Maik Jablonka, Philippe Schwaller, Andres Ortega-Guerrero, and Berend Smit. Leveraging large language models for predictive chemistry.Nature Machine Intelligence, pages 1–9, 2024

    2024

  7. [15]

    14 examples of how llms can transform materials science and chemistry: a reflection on a large language model hackathon

    Kevin Maik Jablonka, Qianxiang Ai, Alexander Al-Feghali, Shruti Badhwar, Joshua D Bocarsly, Andres M Bran, Stefan Bringuier, L Catherine Brinson, Kamal Choudhary, Defne Circi, et al. 14 examples of how llms can transform materials science and chemistry: a reflection on a large...

    2023

  8. [16]

    Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller

    Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller. Augmenting large language models with chemistry tools.Nature Machine Intelligence, 6(5):525–535, 2024

    2024

  9. [17]

    Structured information extraction from scientific text with large language models.Nature Communications, 15(1):1418, 2024

    John Dagdelen, Alexander Dunn, Sanghoon Lee, Nicholas Walker, Andrew S Rosen, Gerbrand Ceder, Kristin A Persson, and Anubhav Jain. Structured information extraction from scientific text with large language models.Nature Communications, 15(1):1418, 2024

    2024

  10. [18]

    Matablegpt: Gpt-based table data extractor from materials science literature.arXiv preprint arXiv:2406.05431, 2024

    Gyeong Hoon Yi, Jiwoo Choi, Hyeongyun Song, Olivia Miano, Jaewoong Choi, Kihoon Bang, Byungju Lee, Seok Su Sohn, David Buttler, Anna Hiszpanski, et al. Matablegpt: Gpt-based table data extractor from materials science literature.arXiv preprint arXiv:2406.05431, 2024

    2024 arXiv

  11. [19]

    HoneyBee: Progressive instruction finetuning of large language models for materials science

    Yu Song, Santiago Miret, Huan Zhang, and Bang Liu. HoneyBee: Progressive instruction finetuning of large language models for materials science. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5...

    2023

  12. [20]

    From text to insight: large language models for materials science data extraction.arXiv preprint arXiv:2407.16867, 2024

    Mara Schilling-Wilhelmi, Martiño Ríos-García, Sherjeel Shabih, María Victoria Gil, Santiago Miret, Christoph T Koch, José A Márquez, and Kevin Maik Jablonka. From text to insight: large language models for materials science data extraction.arXiv preprint arXiv:2407.16867, 2024

    2024 arXiv

  13. [21]

    Enabling large language models for real-world materials discovery.Nature Machine Intelligence, pages 1–8, 2025

    Santiago Miret and NM Anoop Krishnan. Enabling large language models for real-world materials discovery.Nature Machine Intelligence, pages 1–8, 2025

    2025

  14. [22]

    How well do large language models understand tables in materials science?Integrating Materials and Manufacturing Innovation, pages 1–19, 2024

    Defne Circi, Ghazal Khalighinejad, Anlan Chen, Bhuwan Dhingra, and L Catherine Brinson. How well do large language models understand tables in materials science?Integrating Materials and Manufacturing Innovation, pages 1–19, 2024

    2024

  15. [23]

    Mascqa: Investigating materials science knowledge of large language models.Digital Discovery, 2024

    Mohd Zaki, J Jayadeva, Mausam Mausam, and NM Anoop Krishnan. Mascqa: Investigating materials science knowledge of large language models.Digital Discovery, 2024

    2024

  16. [24]

    Extracting materials science data from scientific tables

    Defne Circi, Ghazal Khalighinejad, Anlan Chen, Bhuwan Dhingra, and L Brinson. Extracting materials science data from scientific tables. InACL 2024 Workshop Language+ Molecules, 2024

    2024

  17. [25]

    Are large language models superhuman chemists?arXiv preprint arXiv:2404.01475, 2024

    Adrian Mirza, Nawaf Alampara, Sreekanth Kunchapu, Benedict Emoekabu, Aswanth Krishnan, Mara Wilhelmi, Macjonathan Okereke, Juliane Eberhardt, Amir Mohammad Elahi, Maximilian Greiner, et al. Are large language models superhuman chemists?arXiv preprint arXiv:2404.01475, 2024

    2024 arXiv

  18. [26]

    Mattext: Do language models need more than text & scale for materials modeling? InAI for Accelerated Materials Design-Vienna 2024, 2024

    Nawaf Alampara, Santiago Miret, and Kevin Maik Jablonka. Mattext: Do language models need more than text & scale for materials modeling? InAI for Accelerated Materials Design-Vienna 2024, 2024

    2024

  19. [27]

    Materials synthesis insights from scientific literature via text extraction and machine learning.Chemistry of Materials, 29(21):9436–9444, 2017

    Edward Kim, Kevin Huang, Adam Saunders, Andrew McCallum, Gerbrand Ceder, and Elsa Olivetti. Materials synthesis insights from scientific literature via text extraction and machine learning.Chemistry of Materials, 29(21):9436–9444, 2017

    2017

  20. [28]

    Springer, 2024

    NM Anoop Krishnan, Hariprasad Kodamana, and Ravinder Bhattoo.Machine learning for materials discovery: numerical recipes and practical applications. Springer, 2024

    2024

  21. [29]

    Matkg: An autonomously generated knowledge graph in material science.Scientific Data, 11(1):217, 2024

    Vineeth Venugopal and Elsa Olivetti. Matkg: An autonomously generated knowledge graph in material science.Scientific Data, 11(1):217, 2024

    2024

  22. [30]

    The impact of domain-specific pre-training on named entity recognition tasks in materials science.Available at SSRN 3950755, 2021

    Nicholas Walker, Amalie Trewartha, Haoyan Huo, Sanghoon Lee, Kevin Cruse, John Dagdelen, Alexander Dunn, Kristin Persson, Gerbrand Ceder, and Anubhav Jain. The impact of domain-specific pre-training on named entity recognition tasks in materials science.Available at SSRN 3950755, 2021

    2021

  23. [31]

    Markus J Buehler. Mechgpt, a language-based strategy for mechanics and materials modeling that connects knowledge across scales, disciplines, and modalities.Applied Mechanics Reviews, 76(2):021001, 2024

    2024

  24. [32]

    Chemllm: A chemical large language model.arXiv preprint arXiv:2402.06852, 2024

    Di Zhang, Wei Liu, Qian Tan, Jingdan Chen, Hang Yan, Yuliang Yan, Jiatong Li, Weiran Huang, Xiangyu Yue, Dongzhan Zhou, et al. Chemllm: A chemical large language model.arXiv preprint arXiv:2402.06852, 2024

    2024 arXiv

  25. [33]

    Mohd Zaki, NM Anoop Krishnan, et al. Extracting processing and testing parameters from materials science literature for improved property prediction of glasses.Chemical Engineering and Processing- Process Intensification, 180:108607, 2022

    2022

  26. [34]

    Natural language processing-guided meta-analysis and structure factor database extraction from glass literature.Journal of Non-Crystalline Solids: X, 15:100103, 2022

    Mohd Zaki, Sahith Reddy Namireddy, Tanu Pittie, Vaibhav Bihani, Shweta Rani Keshri, Vineeth Venugopal, Nitya Nand Gosvami, NM Anoop Krishnan, et al. Natural language processing-guided meta-analysis and structure factor database extraction from glass literature.Journal of Non-C...

    2022

  27. [35]

    A survey on data collection for machine learning: a big data-ai integration perspective.IEEE Transactions on Knowledge and Data Engineering, 33(4):1328–1347, 2019

    Yuji Roh, Geon Heo, and Steven Euijong Whang. A survey on data collection for machine learning: a big data-ai integration perspective.IEEE Transactions on Knowledge and Data Engineering, 33(4):1328–1347, 2019

    2019

  28. [36]

    Data collection and labeling techniques for machine learning.arXiv preprint arXiv:2407.12793, 2024

    Qianyu Huang and Tongfang Zhao. Data collection and labeling techniques for machine learning.arXiv preprint arXiv:2407.12793, 2024

    2024 arXiv

  29. [37]

    Learning from noisy labels with deep neural networks: A survey.IEEE transactions on neural networks and learning systems, 34(11):8135–8153, 2022

    Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with deep neural networks: A survey.IEEE transactions on neural networks and learning systems, 34(11):8135–8153, 2022

    2022

  30. [38]

    Distant supervision for relation extraction without labeled data

    Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. Distant supervision for relation extraction without labeled data. InProceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the...

    2009

  31. [39]

    International glass database system, March 2019

    Japan NGF. International glass database system, March 2019

    2019

  32. [40]

    Gemini 1.5 pro.https://deepmind.google/technologies/gemini/#gemini-1-5,

    Google DeepMind. Gemini 1.5 pro.https://deepmind.google/technologies/gemini/#gemini-1-5,

  33. [41]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  34. [42]

    Claude 3.5 sonnet.https://www.anthropic.com/news/claude-3-5, 2024

    Anthropic. Claude 3.5 sonnet.https://www.anthropic.com/news/claude-3-5, 2024. Accessed De- cember 2024

    2024

  35. [43]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  36. [44]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  37. [45]

    Publisher: EPAM Systems

    epam/SciGlass, April 2020. Publisher: EPAM Systems

    2020

  38. [46]

    Executive Office of the President, National Science and Technology Council, 2011

    National Science and Technology Council (US).Materials genome initiative for global competitiveness. Executive Office of the President, National Science and Technology Council, 2011

    2011

  39. [47]

    Elsevier, 2013

    Arun K Varshneya.Fundamentals of inorganic glasses. Elsevier, 2013

    2013

  40. [48]

    Ravinder Bhattoo, Suresh Bishnoi, Mohd Zaki, and NM Anoop Krishnan. Understanding the com- positional control on electrical, mechanical, optical, and physical properties of inorganic glasses with interpretable machine learning.Acta materialia, 242:118439, 2023

    2023

  41. [49]

    A new transferable interatomic potential for molecular dynamics simulations of borosilicate glasses.Journal of Non-Crystalline Solids, 498:294–304, 2018

    Mengyi Wang, NM Anoop Krishnan, Bu Wang, Morten M Smedskjaer, John C Mauro, and Mathieu Bauchy. A new transferable interatomic potential for molecular dynamics simulations of borosilicate glasses.Journal of Non-Crystalline Solids, 498:294–304, 2018

    2018

  42. [50]

    Fu Wang, Yuanlin Wang, Daiyu Zhang, Yan Hao, Qilong Liao, Hanzhen Zhu, Junjie Zhou, and Yongchang Zhu. Effects of moo3 and nd2o3 on the structural features, thermal stability and properties of iron-boron-phosphate based glasses and composites.Journal of Nuclear Materials, 560:...

    2022

  43. [51]

    Synthesis and characterization of clinopyroxene based glasses and glass-ceramics along diopside (camgsi2o6)–jadeite (naalsi2o6) join

    Rinkel Jindal, R Jayaganthan, Indra Vir Singh, and Reinhard Conradt. Synthesis and characterization of clinopyroxene based glasses and glass-ceramics along diopside (camgsi2o6)–jadeite (naalsi2o6) join. Ceramics International, 37(3):741–748, 2011

    2011

  44. [52]

    Preparation and characterization of enstatite-leucite glass-ceramics for dental restoration.Journal of Non-Crystalline Solids, 563:120810, 2021

    GA Khater and EM Safwat. Preparation and characterization of enstatite-leucite glass-ceramics for dental restoration.Journal of Non-Crystalline Solids, 563:120810, 2021

    2021

  45. [53]

    Strain-rate sensitivity of glasses.Journal of Non-Crystalline Solids, 404:124–134, 2014

    Rene Limbach, Bruno P Rodrigues, and Lothar Wondraczek. Strain-rate sensitivity of glasses.Journal of Non-Crystalline Solids, 404:124–134, 2014

    2014

  46. [54]

    Lateral hardness and the scratch resistance of glasses in the na2o-cao-sio2 system.Journal of Non-Crystalline Solids, 492:94–101, 2018

    Guilherme NBM de Macedo, Shigeki Sawamura, and Lothar Wondraczek. Lateral hardness and the scratch resistance of glasses in the na2o-cao-sio2 system.Journal of Non-Crystalline Solids, 492:94–101, 2018

    2018

  47. [55]

    Hardness and refractive index of ca–si–o–n glasses

    Ali Sharafat, Jekabs Grins, and Saeid Esmaeilzadeh. Hardness and refractive index of ca–si–o–n glasses. Journal of non-crystalline solids, 355(4-5):301–304, 2009. 19

    2009

  48. [56]

    Two centuries of glass research: historical trends, current status, and grand challenges for the future.International Journal of Applied Glass Science, 5(3):313–327, 2014

    John C Mauro and Edgar D Zanotto. Two centuries of glass research: historical trends, current status, and grand challenges for the future.International Journal of Applied Glass Science, 5(3):313–327, 2014

    2014

  49. [57]

    High-pressure synthesis of high-performance submicron-sized polycrystallineβ-si3n4 bulk without additives.Ceramics International, 46(8):12449–12457, 2020

    Zhiqiang Hou, Haikuo Wang, Ya-nan Yang, Xudong Song, Shuaipeng Chen, Shun Wan, Xiaomiao Zhao, Mengya Shang, and Bin Chen. High-pressure synthesis of high-performance submicron-sized polycrystallineβ-si3n4 bulk without additives.Ceramics International, 46(8):12449–12457, 2020

    2020

  50. [58]

    Investigation on glass-forming ability, flexural strength and microwave dielectric properties of al2o3-doped lmzbs glasses

    Tianying Qin, Chaowei Zhong, Hongcheng Yang, Yang Qin, and Shuren Zhang. Investigation on glass-forming ability, flexural strength and microwave dielectric properties of al2o3-doped lmzbs glasses. Ceramics International, 45(8):10899–10906, 2019

    2019

  51. [59]

    Role of tri-capped triangular prism (ttp) polyhedra in formation and destabilization of fe–y–b glassy alloys.Journal of Non-Crystalline Solids, 425:67–73, 2015

    HJ Ma, KC Shen, SP Pan, J Zhao, JY Qin, KB Kim, and WM Wang. Role of tri-capped triangular prism (ttp) polyhedra in formation and destabilization of fe–y–b glassy alloys.Journal of Non-Crystalline Solids, 425:67–73, 2015

    2015

  52. [60]

    Fiber drawing ability and loss optimization of niobium rich borophosphate optical glass fibers.Optical Materials, 131:112628, 2022

    Georges El Dib, Ronan Lebullenger, Laura Loi, Thierry Pain, Frédéric Adamietz, Lionel Canioni, Thierry Cardinal, Sébastien Chenu, and Sylvain Danto. Fiber drawing ability and loss optimization of niobium rich borophosphate optical glass fibers.Optical Materials, 131:112628, 2022

    2022

  53. [61]

    Synthesis of sm3+-doped strontium barium niobate crystals in glass by samarium atom heat processing

    Nakorn Chayapiwut, Tsuyoshi Honma, Yasuhiko Benino, Takumi Fujiwara, and Takayuki Komatsu. Synthesis of sm3+-doped strontium barium niobate crystals in glass by samarium atom heat processing. Journal of Solid State Chemistry, 178(11):3507–3513, 2005

    2005

  54. [62]

    Heat capacity measurements of mg–y–si–al–o–n glasses

    J Rocherullé, M Matecki, and Y Delugeard. Heat capacity measurements of mg–y–si–al–o–n glasses. Journal of non-crystalline solids, 238(1-2):51–56, 1998

    1998

  55. [63]

    Femtosecond-laser-writing in various glasses

    D Ehrt, T Kittel, M Will, S Nolte, and A Tünnermann. Femtosecond-laser-writing in various glasses. Journal of non-crystalline solids, 345:332–337, 2004

    2004

  56. [64]

    Development of a ceramic-based composite for direct bonded copper substrate.Ceramics International, 43(6):5236–5246, 2017

    SS Akhtar, LT Kareem, AFM Arif, MU Siddiqui, and AS Hakeem. Development of a ceramic-based composite for direct bonded copper substrate.Ceramics International, 43(6):5236–5246, 2017

    2017

  57. [65]

    Sintering of glasses in the system ro–al2o3–bao– sio2 (r= ca, mg, zn) studied by hot-stage microscopy.Solid State Ionics, 170(3-4):201–208, 2004

    C Lara, M Jesús Pascual, MO Prado, and Alicia Durán. Sintering of glasses in the system ro–al2o3–bao– sio2 (r= ca, mg, zn) studied by hot-stage microscopy.Solid State Ionics, 170(3-4):201–208, 2004

    2004

  58. [66]

    Stability, electronic and mechanical properties of superhard materials formed by 4+ 6+ 8 membered rings of carbon.Journal of Solid State Chemistry, 277:454–465, 2019

    Junru Kou, Aihua Cao, Zuo-Hua Liu, and Li-Hua Gan. Stability, electronic and mechanical properties of superhard materials formed by 4+ 6+ 8 membered rings of carbon.Journal of Solid State Chemistry, 277:454–465, 2019

    2019

  59. [67]

    Synthesis, characterization and in vitro bioactivity of magnesium- doped sol–gel glass and glass-ceramics.Ceramics International, 37(5):1637–1644, 2011

    J Ma, CZ Chen, DG Wang, and JH Hu. Synthesis, characterization and in vitro bioactivity of magnesium- doped sol–gel glass and glass-ceramics.Ceramics International, 37(5):1637–1644, 2011

    2011

  60. [68]

    20 Supplementary Materials Contents A Hierarchical Training Data Generation and Performance Analysis 2 A.1 Dataset Preparation and Annotation Strategy

    Elsevier Developer Portal. 20 Supplementary Materials Contents A Hierarchical Training Data Generation and Performance Analysis 2 A.1 Dataset Preparation and Annotation Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 A.2 Performance Validation and Component An...

    2009

  61. [70]

    MatSKRAFT_P - F1 = 88.68 :The complete framework represents our optimal configuration, incorporating all architectural and data preparation components in a synergistic manner. This baseline demonstrates excellent precision (90.35) and a strong recall (87.07), indicating effect...

  62. [71]

    MatSKRAFT_P w/o Constrained Learning - F1 = 88.38 :Removing the constraint-driven learning component results in the smallest performance degradation of only 0.30 F1 points. This configuration does achieve the second-highest recall in the entire ablation studies (88.72, +1.65 p...

  63. [72]

    MatSKRAFT_P w/o Caption Information - F1 = 86.94 :The removal of caption information leads to a moderate F1 decrease of 1.74 points, driven primarily by precision reduction (85.94, -4.41 points) while maintaining comparable recall (87.95, +0.88 points). Caption information ser...

  64. [73]

    s” or “n

    MatSKRAFT_P w / o post-processing - F1 = 79.30 :The elimination of post-processing produces highest F1 degradation of 9.38 points, making it the most vital architectural component for property extraction. This performance loss manifests through severe precision reduction (76.6...

  65. [74]

    MatSKRAFT_P w/o Data Augmentation - F1 = 87.50 :Removing the data augmentation component results in a moderate F1 decrease of 1.18 points, characterized by precision reduction (88.37, -1.98 points) and slight recall degradation (86.65, -0.42 points). Our power-law guided augme...

  66. [75]

    This performance shift reveals the critical role of annotation codes in maintaining extraction precision across diverse property types

    MatSKRAFT_P w/o Annotation Algorithm - F1 = 79.66 :Removing the domain-specific annotation algorithms results in the most notable F1 decrease of 9.02 points among data preparation components, characterized by severe precision degradation (72.34, -18.01 points) while maintainin...

  67. [76]

    MatSKRAFT_P w/o Distant Supervision - F1 = 86.62 :The removal of distant supervision reveals a distinctive and highly significant performance profile — achieving the highest precision (93.04, +2.69 10 points) among all the configurations while experiencing considerable recall ...

  68. [77]

    MatSKRAFT_P w/o (Distant Supervision + Data Augmentation) - F1 = 86.20 :This combined ablation retains only annotation codes for training data generation — removing both distant supervision and data augmentation components, resulting in a moderate F1 decrease of 2.48 points th...

  69. [78]

    MatSKRAFT_P w/o (Annotation Codes + Data Augmentation) - F1 = 84.31 :This configuration shows F1 degradation of 4.37 points while achieving the highest recall (89.01, +1.94 points) but significant reduction in precision (80.09, -10.26 points). The removal of both components le...

  70. [79]

    MatSKRAFT_C - F1 = 71.35 :The complete composition extraction framework achieves optimal performance across all metrics — F1 score (71.35), precision (82.31), and recall (62.97). This baseline represents our enhanced DiSCoMaT architecture with comprehensive improvements includ...

  71. [80]

    MatSKRAFT_CC w/o Thresholding - F1 = 68.73 :Removing the thresholding component decreases F1 score by 2.62 points, characterized by precision reduction (78.19, -4.12 points) and slight recall degradation (61.32, -1.65 points). The thresholding mechanism implements a confidence...

  72. [81]

    This balanced performance loss indicates that constrained learning provides structural guidance essential for composition extraction accuracy across diverse table formats

    MatSKRAFT_C w/o Constrained Learning - F1 = 64.83 :The elimination of constrained learning triggers a notable 6.52-point F1 decline, with precision dropping to 76.86 (-5.45 points) while recall simultaneously deteriorates to 56.05 (-6.92 points). This balanced performance loss...

  73. [82]

    Na2O-CaO-SiO2

    MatSKRAFT_C w/o Annotation Codes - F1 = 62.42:Annotation codes emerge as one of the most critical components for composition extraction, with their removal resulting in 8.93-point F1 collapse through severe precision (73.57, -8.74 points) and significant recall deterioration (...

  74. [83]

    This severe loss highlights the critical dependence of composition extraction on contextual information

    MatSKRAFT_C w/o Caption Information - F1 = 61.64 :Caption information removal produces the most severe impact on composition extraction with a 9.71-point F1 decrease, driven by both precision (73.12, -9.19 points) and recall reduction (53.27, -9.70 points). This severe loss hi...

  75. [84]

    Accuracy across extraction tasks.Matskraftachieves exceptional F1 scores of 88.68 for property extraction and 71.35 for composition extraction, establishing clear superiority over all large language model baselines by margins exceeding 15 percentage points. For property extrac...

  76. [85]

    Computational efficiency enabling knowledge extraction at large scale.Beyond accuracy advantages, Matskraftdemonstrates the computational efficiency that redefines the economics of scientific knowledge extraction. By processing tables at 0.22 seconds for property extraction an...

  77. [86]

    Domain specialization advantages over generalist architectures.The systematic performance superiority across all baselines validates our hypothesis that domain-specialized graph neural network architectures outperform generalist large language models for scientific knowledge e...

  78. [87]

    Democratization of advanced materials knowledge extraction.By achieving superior performance with reduced computational requirements, Matskraftdemocratizes access to advanced scientific knowledge extraction capabilities previously available only to organizations with extensive...

  79. [88]

    Systemic few-shot learning with domain-specific examples.We developed sophisticated prompting strategies incorporating carefully curated few-shot examples that demonstrate the full complexity of materials science table structures. For property extraction, our prompts included ...

  80. [89]

    This taxonomic approach enables LLMs to understand structural variations in materials science reporting conventions, providing advantages over generic table processing capabilities

    containing no relevant compositional data. This taxonomic approach enables LLMs to understand structural variations in materials science reporting conventions, providing advantages over generic table processing capabilities

  81. [90]

    Extensive contextual information provision.Beyond standard table content, we provided LLMs with comprehensive contextual information to maximize extraction accuracy. For composition extraction tasks, every table was supplemented with complete research article abstracts—providi...

  82. [91]

    Comprehensive unit normalization and domain knowledge integration.Our prompts incorporated extensive domain expertise through detailed unit normalization instructions spanning all 18 target properties. We provided explicit conversion guidelines for density measurements (g/cm3,...

  83. [92]

    Sophisticated ID construction and structural understanding.We developed detailed identifier construction protocols that explicitly taught LLMs to understand table orientations and generate unique 16 identifiers following our framework’s conventions. Instructions differentiated...

  84. [93]

    We configured 10-attempt retry loops with intelligent error classification, distinguishing between rate limiting (HTTP 429), connection timeouts, and parsing failures

    Robust error handling and retry mechanisms.Our evaluation framework implemented com- prehensive error-handling mechanisms with automated retry protocols designed to maximize LLM success rates. We configured 10-attempt retry loops with intelligent error classification, distingu...

  85. [94]

    Iterative prompt engineering with validation-driven optimization.We established small validation sets comprising carefully selected examples representing the full spectrum of extraction challenges. Our iterative optimization process continued until LLMs achieved perfect perfor...

  86. [95]

    hardness

    Computational resource advantages for LLMs.Our evaluation framework maximized LLM performance through optimal configuration settings, including generous token limits (8,192 tokens) to avoid truncation, deterministic temperature settings, and comprehensive contextual informatio...

  87. [2024]

    Accessed January 2025

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.