Pith. sign in

REVIEW 4 major objections 4 minor 40 references

Conversational Lexicography: Querying Lexicographic Data on Knowledge Graphs with SPARQL through Natural Language

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that natural-language access to Wikidata lexicographic data works across tested models on familiar templates, but that only GPT-3.5-Turbo transfers to structurally new queries, because smaller models memorize SPARQL…

desk verdict New lexicographic text-to-SPARQL dataset with real utility, but the generalization claim is not demonstrated because ASK templates leak into training and the model comparison mismatches k. read the letter →

arxiv 2505.19971 v1 pith:HTLIAKWS submitted 2025-05-26 cs.CL

Kilian Sennrich , Sina Ahmadi This is my paper · ORCID
classification cs.CL
keywords naturallanguagetoSPARQLlexicographicdataWikidataknowledgegraphstemplate-baseddatasetlargemodelsgeneralizationpass@kevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper seeks to show that plain-English querying of dictionary-style data in Wikidata is feasible, and that the decisive challenge is not producing correct SPARQL for familiar questions but adapting to structurally new question shapes. It builds a four-dimension taxonomy of lexicographic query types and uses it to create a template-based dataset of over 1.2 million natural-language-to-SPARQL pairs, then trains GPT-2 from scratch, fine-tunes Phi-1.5, and evaluates GPT-3.5-Turbo in a few-shot setting. On queries that match the training templates, all three models perform well, with pass@1 scores between 0.86 and 0.90. On a generalization test that turns a SELECT question into an ASK question, the two smaller models drop to 0 pass@k, while GPT-3.5-Turbo reaches pass@3 of 0.57. A sympathetic reader would take away that robust natural-language interfaces to lexicographic knowledge graphs will require larger, diversely pretrained models or more sophisticated adaptation methods.

What carries the argument

The argument is carried by three linked components. First, a taxonomy with four dimensions—lexical property categories, single versus multi-lexeme output, mono- versus multilingual scope, and simple versus complex property count—organizes the space of lexicographic requests and yields 189 SPARQL templates. Second, those templates are populated with lexemes and properties drawn from an April-May 2024 Wikidata snapshot, producing more than 1.2 million utterance-query pairs, with GPT-4-generated paraphrases used to decouple wording from query syntax. Third, evaluation uses an execution-based pass@k metric in which generated queries are run against Wikidata and judged correct when their returned triples match the gold query's results, supplemented by granularity checks and BLEU scores. The decisive mechanism is the generalization probe, which transforms a SELECT question into an ASK question; this single structural change is what separates GPT-3.5-Turbo from the two smaller models.

What would settle it

Freeze the Wikidata snapshot and rerun the generalization tests with additional structural transformations beyond SELECT-to-ASK, such as reordering triple patterns, inserting FILTER or OPTIONAL clauses, or adding a second lexeme, and compare whether GPT-3.5-Turbo still generalizes while GPT-2 and Phi-1.5 stay at 0. If the smaller models also pass those transformations, or if GPT-3.5-Turbo fails them, the paper's claim that only large pretrained models generalize would be contradicted.

Watch

Extended reading notes

Core claim

The central discovery is a measurable asymmetry between memorization and generalization in text-to-SPARQL models. When the input utterances match the query structures seen in training, a 124M-parameter GPT-2 trained from scratch slightly beats a few-shot 1.3B fine-tuned Phi-1.5 and GPT-3.5-Turbo, achieving pass@1 of 0.90 versus 0.86 and 0.87, respectively. But when the evaluation changes a training-style SELECT question into an ASK question, GPT-2 and Phi-1.5 both score 0 on pass@k, whereas GPT-3.5-Turbo scores 0.41 at pass@1 and 0.57 at pass@3. The paper interprets this as evidence that the smaller models map utterances into concrete template skeletons without learning the underlying query-construction rules, while the larger pretrained model can adapt its output structure to a new request type.

Load-bearing premise

The central generalization result rests on a single type of structural change—converting SELECT questions into ASK questions—and assumes this one transformation captures the kind of generalization real users need; it also assumes that the live Wikidata endpoint returns stable answers for queries derived from an April-May 2024 snapshot.

Editorial extensions

If this is right

  • A template-based dataset in this scale can train small models to near-90 percent functional accuracy on familiar lexicographic query patterns, making lightweight deployed interfaces plausible for fixed query menus.
  • Performance on familiar patterns is not a reliable proxy for adaptability, since both small models collapse to 0 pass@k on the SELECT-to-ASK transformation despite their high template accuracy.
  • Few-shot prompting of a large pretrained model, without any fine-tuning, provides the only meaningful structural generalization observed in this paper, suggesting that diverse pretraining contributes capabilities that template training does not instill.
  • Using execution-based pass@k instead of string matching changes the measured ranking in the generalization setting, indicating that functional correctness and textual similarity diverge when models explore novel query forms.
  • Resource-constrained deployments should expect rapid degradation when users ask questions that fall outside the template distribution, and should plan for a fallback to larger models or more sophisticated fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The generalization finding rests on a single transformation, SELECT to ASK; additional structural perturbations such as reordering triple patterns, inserting FILTER or OPTIONAL clauses, or adding joins could either reinforce or narrow the paper's conclusion that only large pretrained models generalize.
  • A practical hybrid interface could route template-matching questions to a small fast model and reserve a larger model for low-confidence or structurally novel inputs; the paper's numbers suggest this would preserve accuracy on familiar patterns while buying generalization where it matters.
  • Because the evaluation relies on a live Wikidata endpoint, the reported pass@k figures may include small fluctuations from endpoint-side changes; freezing a snapshot would make the benchmark reproducible and could alter the exact scores without necessarily changing the ordering.
  • The GPT-4-based utterance paraphrasing means surface wording already varies within the training set, so the small models' memorization likely operates at the level of query skeleton rather than at the level of individual words.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces a four-dimensional taxonomy for Wikidata lexicographic SPARQL queries and uses it to generate a template-based dataset of over 1.2 million natural-language-to-SPARQL pairs across 189 templates. It fine-tunes Phi-1.5 and trains GPT-2 from scratch on this dataset, and compares them with few-shot GPT-3.5-Turbo using functional pass@k, a granularity ratio, and BLEU in two settings: a non-generalization split and a generalization test that transforms SELECT queries into ASK queries. The central claim is that all models perform well on familiar patterns (pass@1 0.86–0.90) but only GPT-3.5-Turbo generalizes meaningfully (pass@3 0.57 in the generalization scenario).

Significance. The dataset and taxonomy are a potentially useful resource for the text-to-SPARQL community, particularly for the under-explored lexicographic domain, and the evaluation framework based on functional correctness is a reasonable step beyond exact-match metrics. The paper also makes the falsifiable prediction that small fine-tuned models memorize template structures while larger pretrained models adapt to new structures. However, as detailed below, the empirical support for the central generalization claim is currently compromised by evaluation design issues; the resource contribution remains valuable regardless of the outcome of a corrected comparison.

major comments (4)
  1. [3.3 D, 4.1, 4.3] The generalization test is not a held-out query type. Section 3.3 D transforms a training SELECT question into a test ASK question, and the paper concludes that GPT-3.5-Turbo is the only model with meaningful generalization. However, Section 4.1 states that the Property Templates were 'further adapted to handle multi-lexeme outputs and ASK statements,' and Section 4.3 lists template ask_t9_P7243 with up to 29,922 instances. ASK queries are therefore part of the training distribution for the fine-tuned/trained models and may also be sampled as few-shot examples for GPT-3.5-Turbo. The observed GPT-3.5 advantage could reflect recognition of a familiar pattern rather than adaptation to a novel structure. The authors must either exclude all ASK templates from training and prompting, or use a query structure that is genuinely absent from the training data, for the generalization claim to stand.
  2. [Table 1, Eq. (1)] The comparison of generalization performance uses different k values across models. Table 1 reports GPT-3.5-Turbo at pass@3 (0.57) while Phi-1.5 and GPT-2 are reported at pass@1 (0). Because Eq. (1) defines pass@k as the proportion of correct responses among k samples, these numbers are not directly comparable; a model that occasionally succeeds at k=1 could still have pass@3 > 0, and the conclusion 'only GPT-3.5-Turbo demonstrates meaningful generalization' is not supported without a matched evaluation. At minimum, the authors should report pass@1 for GPT-3.5-Turbo in the same table row and either run k=3 for the smaller models or provide a bounded estimate (e.g., with confidence intervals) to justify the qualitative claim.
  3. [Appendix C, Section 3.3 C] The granularity checks are inconsistent with the dataset's own templates. Appendix C lists 'The response must not contain the keyword VALUES' as a required check, yet the gold queries shown in Figure 1, the q20 example in Section 4, and the few-shot prompts in Appendix A all use VALUES clauses. This means the granularity ratio R_granularity penalizes the reference queries themselves, making the reported R_granularity values (e.g., 0.94 for GPT-3.5, 0.84 for Phi-1.5/GPT-2) difficult to interpret as quality measures. The check should be removed or reformulated to distinguish legitimate VALUES usage from template misapplication.
  4. [3.3 D, Section 6] The generalization claim rests on a single structural transformation. Section 3.3 D only tests SELECT-to-ASK conversion, yet the abstract and Section 6 generalize to 'meaningful generalization capabilities' and 'adaptability in this domain.' Real-world structural changes include clause reordering, additional joins, aggregation, and negation, none of which are evaluated. The authors should either narrow the conclusion to 'generalization to ASK queries' or evaluate a broader set of held-out structural changes; otherwise the central claim overstates what the experiment can show.
minor comments (4)
  1. [Section 4.3, Figure 3] Dataset size discrepancy: Section 4.3 reports 1,270,113 data tuples, while the caption of Figure 3 states a total dataset size of 1,170,118 instances; the abstract says 'over 1.2 million.' Please align these numbers and explain the difference.
  2. [Eq. (1)] Eq. (1) defines pass@k as the ratio of correct responses out of k, which differs from the common pass@k metric in Chen et al. (2021) where a query counts as passed if at least one of k samples is correct. Please clarify the convention to readers.
  3. [Table 1 caption] The table caption 'few-shot fine-tuned GPT-3.5 Turbo' in Table 1 is misleading: GPT-3.5-Turbo is evaluated with few-shot prompting, not fine-tuned. Rephrase to avoid confusion.
  4. [Section 2] The related-work sentence 'Early work by Soru et al. (2018, 2017) establish...' has a subject-verb agreement error; also 'develope' appears later in the same section.

Circularity Check

1 steps flagged · score 6.0 of 10

The generalization claim is circular: the SELECT-to-ASK test uses a query type that is already present in the training templates, so 'meaningful generalization' reduces to performance on familiar ASK patterns.

  1. fitted input called prediction [Section 3.3 D (Generalization), with Sections 4.1 and 4.3]
    "Generalization assessing the model’s ability to generalize by altering input questions to trigger different query types. To do so, we transform a training question like “What is the gender of ‘Apfel’ in German?” (requiring a SELECT query) into a test question like “Is the gender of ‘Apfel’ in German feminine?” (requiring an ASK query), testing whether the model can adapt to this structural change."

    The transformation claims to test adaptation to a novel query type, but the ASK query type is already part of the training distribution. Section 4.1 states that the Property Templates 'are further adapted to handle multi-lexeme outputs and ASK statements,' and Section 4.3 reports a template named ask_t9_P7243 with up to 29,922 tuples in the dataset. Both fine-tuned/trained models are trained on this dataset (Section 3.2), and GPT-3.5-Turbo's prompts sample 'two random utterances and corresponding SPARQL queries from the training dataset' (Section 5.1). Thus the generalization test does not actually measure an unseen structure; it measures recognition of familiar ASK patterns.

full rationale

The only load-bearing circular step is the generalization evaluation. The paper defines generalization as 'adapt[ing] to this structural change' from SELECT to ASK (Section 3.3 D), yet the dataset used for training and few-shot prompting already contains ASK templates (Sections 4.1 and 4.3). As a result, the supposedly held-out query structure is in the training distribution, so the reported pass@k differences partly reflect template familiarity rather than generalization to novel structure. This is a fitted-input-called-prediction pattern: the model is exposed to ASK queries during training or prompting, and the 'prediction' that only GPT-3.5 generalizes is evaluated on that same query type. The non-generalization evaluation on the same 189 templates is not circular because the paper describes it as testing 'familiar patterns' rather than as a prediction; it is a sanity check, albeit with limited external validity. The self-citations (Ahmadi 2022; SPARQLify) are not load-bearing: they support background and tooling, not the generalization claim. The pass@3-vs-pass@1 comparison is an evaluation confound but not a definitional reduction. Overall, one central evaluation step reduces by construction to training-distribution content, giving a partial circularity score of 6.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central contribution is a dataset and taxonomy; no new physical or formal entities are introduced. The free parameters listed are evaluation and training design choices that the reported results depend on.

free parameters (3)
  • Few-shot example count = 2
    Section 5.1 samples two random utterance-query pairs from the training set for each template when prompting GPT-3.5-Turbo. The generalization pass@3 of 0.57 could change with more or fewer examples.
  • Training epochs = Phi-1.5: 1, GPT-2: 3
    Section 3.2 sets Phi-1.5 training to a single epoch (following Schimanski et al.) and GPT-2 to three epochs. These choices affect the memorization-generalization tradeoff and thus the model comparison.
  • Test-set cap per template = Max 20 instances
    Section 4.3 limits the evaluation set to at most 10% per template with a maximum of 20 data points. This directly shapes the difficulty of the non-generalization test.
assumptions (4)
  • domain assumption Wikidata's lexicographic data module is a sufficiently representative and complete source for lexicographic queries.
    The dataset is built entirely from Wikidata lexicographic properties (Appendix C.1). If Wikidata's coverage is sparse or skewed, the benchmark may not reflect real-world lexicographic information needs.
  • ad hoc to paper The SELECT-to-ASK transformation measures generalization.
    Section 3.3 D defines generalization solely through this transformation. There is no evidence that this correlates with other types of structural generalization.
  • domain assumption Semantic correctness of generated SPARQL can be judged by comparing result triples from the live endpoint.
    Section 3.3 B relies on executing queries against Wikidata Query Service and checking that retrieved triples match or include gold-standard triples. This assumes endpoint availability, data stability, and that triple inclusion is a valid correctness criterion.
  • domain assumption GPT-4 generated paraphrases preserve the meaning of the utterances.
    Section 4 uses GPT-4 to create alternative phrasings. If a paraphrase changes the intended query semantics, the training data will contain misaligned pairs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Conversational Lexicography: Querying Lexicographic Data on Knowledge Graphs with SPARQL through Natural Language." pith.science (2026). https://pith.science/paper/HTLIAKWS

@misc{pith2026250519971,
  author       = {Pith},
  title        = {Pith review of: Conversational Lexicography: Querying Lexicographic Data on Knowledge Graphs with SPARQL through Natural Language},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HTLIAKWS}},
  note         = {Machine review of arXiv:2505.19971}
}
read the original abstract

Knowledge graphs offer an excellent solution for representing the lexical-semantic structures of lexicographic data. However, working with the SPARQL query language represents a considerable hurdle for many non-expert users who could benefit from the advantages of this technology. This paper addresses the challenge of creating natural language interfaces for lexicographic data retrieval on knowledge graphs such as Wikidata. We develop a multidimensional taxonomy capturing the complexity of Wikidata's lexicographic data ontology module through four dimensions and create a template-based dataset with over 1.2 million mappings from natural language utterances to SPARQL queries. Our experiments with GPT-2 (124M), Phi-1.5 (1.3B), and GPT-3.5-Turbo reveal significant differences in model capabilities. While all models perform well on familiar patterns, only GPT-3.5-Turbo demonstrates meaningful generalization capabilities, suggesting that model size and diverse pre-training are crucial for adaptability in this domain. However, significant challenges remain in achieving robust generalization, handling diverse linguistic data, and developing scalable solutions that can accommodate the full complexity of lexicographic knowledge representation.

Figures

Figures reproduced from arXiv: 2505.19971 by the authors.

Figure 1
Figure 1. Conversational lexicography: enabling natural [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our approach to creating SPARQL templates based on a four-dimension taxonomy followed by dataset [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Distribution of the number of populated data [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 27 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Sina Ahmadi. 2022. Monolingual alignment of word senses and definitions in lexicographical resources. arXiv preprint arXiv:2209.02465

    work page Pith review arXiv 2022

  4. [4]

    Caio Viktor S Avila, V \^a nia MP Vidal, Wellington Franco, and Marco A Casanova. 2024. Experiments with text-to-SPARQL based on ChatGPT . In 2024 IEEE 18th International Conference on Semantic Computing (ICSC), pages 277--284. IEEE

    work page 2024

  5. [5]

    Debayan Banerjee, Pranav Ajit Nair, Jivat Neet Kaur, Ricardo Usbeck, and Chris Biemann. 2022. https://www.microsoft.com/en-us/research/publication/modern-baselines-for-sparql-semantic-parsing/ Modern baselines for SPARQL S emantic P arsing . In SIGIR

    work page 2022

  6. [6]

    Julia Bosque - Gil, Jorge Gracia, and Elena Montiel - Ponsoda. 2017. https://ceur-ws.org/Vol-1899/OntoLex\_2017\_paper\_5.pdf Towards a module for lexicography in ontolex . In Proceedings of the LDK 2017 Workshops: 1st Workshop on the OntoLex Model , volume 1899 of CEUR Workshop Proceedings , pages 74--84

    work page 2017

  7. [7]

    Felix Brei, Johannes Frey, and Lars-Peter Meyer. 2024. Leveraging small language models for Text2SPARQL tasks to improve the resilience of AI assistance. arXiv preprint arXiv:2405.17076

    arXiv 2024

  8. [8]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. http://arxiv.org/abs/2402.03216 BGE M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation . arXiv:2402.03216

    arXiv 2024

Show all 40 references
  1. [9]

    Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William H

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde, Jared Kaplan, Harrison Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder...

    2021 arXiv

  2. [10]

    Philipp Cimiano, Paul Buitelaar, John McCrae, and Michael Sintek. 2011. LexInfo : A declarative model for the lexicon-ontology interface. Web Semantics: Science, Services and Agents on the World Wide Web, 9(1):29--51

    2011

  3. [11]

    Bretonnel Cohen and Jin-Dong Kim

    K. Bretonnel Cohen and Jin-Dong Kim. 2013. https://aclanthology.org/W13-5202 Evaluation of SPARQL query generation from natural language questions . In Proceedings of the Joint Workshop on NLP & LOD and SWAIE : Semantic Web, Linked Open Data and Information Extraction , pages ...

    2013

  4. [12]

    Jacopo D ' Abramo, Andrea Zugarini, and Paolo Torroni. 2025. https://aclanthology.org/2025.knowledgenlp-1.5/ Investigating large language models for text-to- SPARQL generation . In Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Languag...

    2025

  5. [13]

    Mohnish Dubey, Debayan Banerjee, Abdelrahman Abdelkawi, and Jens Lehmann. 2019. LC-QuAD 2.0: a large dataset for complex question answering over Wikidata and DBpedia . In The Semantic Web -- ISWC 2019, pages 69--78, Cham. Springer International Publishing

    2019

  6. [14]

    Gil Francopoulo, Monte George, Nicoletta Calzolari, Monica Monachini, Nuria Bel, Mandy Pet, and Claudia Soria. 2006. Lexical markup framework ( LMF ). In International Conference on Language Resources and Evaluation-LREC 2006, page 5

    2006

  7. [15]

    Gregson, J.M

    J. Gregson, J.M. Brownlee, R. Playforth, and N. Bimbe. 2015. The Future of Knowledge Sharing in a Digital Age: Exploring Impacts and Policy Implications for Development. Number 125 in IDS Evidence Report. Brighton

    2015

  8. [16]

    Ann-Kathrin Hartmann, Edgard Marx, and Tommaso Soru. 2018. Generating a large dataset for neural question answering over the DBpedia knowledge base. In Workshop on Linked Data Management, co-located with the W3C WEBBR, volume 2018

    2018

  9. [17]

    Moshe Hazoom, Vibhor Malik, and Ben Bogin. 2021. https://doi.org/10.18653/v1/2021.nlp4prog-1.9 Text-to- SQL in the wild: A naturally-occurring dataset based on stack exchange data . In Proceedings of the 1st Workshop on Natural Language Processing for Programming (NLP4Prog 202...

    2021 doi

  10. [18]

    Sen Hu, Lei Zou, Jeffrey Xu Yu, Haixun Wang, and Dongyan Zhao. 2018. https://doi.org/10.1109/TKDE.2017.2766634 Answering natural language questions by subgraph matching over knowledge graphs . IEEE Trans. Knowl. Data Eng. , 30(5):824--837

    2018

  11. [19]

    Catherine Kosten, Philippe Cudr \'e -Mauroux, and Kurt Stockinger. 2023. Spider4SPARQL: a complex benchmark for evaluating knowledge graph question answering systems . In 2023 IEEE International Conference on Big Data, pages 5272--5281. IEEE

    2023

  12. [20]

    Yuanzhi Li, S \'e bastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. 2023. https://www.microsoft.com/en-us/research/publication/textbooks-are-all-you-need-ii-phi-1-5-technical-report/ Textbooks are all you need II : Phi-1.5 technical report . arX...

    2023 arXiv

  13. [21]

    Jia - Huei Lin and Eric Jui - Lin Lu. 2022. https://doi.org/10.13052/jwe1540-9589.2155 SPARQL generation with an NMT -based approach . J. Web Eng., 21(5)

    2022

  14. [22]

    McCrae, Julio Bosque-Gil, Jorge Gracia, Paul Buitelaar, and Philipp Cimiano

    John P. McCrae, Julio Bosque-Gil, Jorge Gracia, Paul Buitelaar, and Philipp Cimiano. 2017. The Ontolex-Lemon model: Development and applications. In Proceedings of eLex 2017 Conference, pages 19--21, Leiden, Netherlands. Lexical Computing

    2017

  15. [23]

    Elena Montiel-Ponsoda, Guadalupe Aguado De Cea, Asunci \'o n G \'o mez-P \'e rez, and Wim Peters. 2008. Modelling multilinguality in ontologies. COLING 2008: Companion volume: Posters , pages 67--70

    2008

  16. [24]

    Axel-Cyrille Ngonga Ngomo, Lorenz B \"u hmann, Christina Unger, Jens Lehmann, and Daniel Gerber. 2013. Sorry, I don't speak SPARQL: translating SPARQL queries into natural language . In Proceedings of the 22nd international conference on World Wide Web, pages 977--988

    2013

  17. [25]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311--318

    2002

  18. [26]

    Aleksandr Perevalov and Andreas Both. 2024. Towards LLM -driven natural language generation based on SPARQL queries and RDF knowledge graphs. 3rd international workshop on knowledge graph generation from text (Text2KG) at ESWC

    2024

  19. [27]

    Dmitrii Pliukhin, Daniil Radyush, Liubov Kovriguina, and Dmitry Mouromtsev. 2023. Improving subgraph extraction algorihtms for one-shot SPARQL query generation with large language models. In QALD/SemREC@ ISWC

    2023

  20. [28]

    Matt Post. 2018. https://www.aclweb.org/anthology/W18-6319 A call for clarity in reporting BLEU scores . In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186--191, Belgium, Brussels. Association for Computational Linguistics

    2018

  21. [29]

    Jiexing Qi, Chang Su, Zhixin Guo, Lyuwen Wu, Zanwei Shen, Luoyi Fu, Xinbing Wang, and Chenghu Zhou. 2024. https://doi.org/10.3390/app14041521 Enhancing SPARQL query generation for knowledge base question answering systems by learning to correct triplets . Applied Sciences, 14(4)

    2024 doi

  22. [30]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  23. [31]

    Md Rashad Al Hasan Rony, Uttam Kumar, Roman Teucher, Liubov Kovriguina, and Jens Lehmann. 2022. SGPT: A generative approach for SPARQL query generation from natural language questions . IEEE access, 10:70712--70723

    2022

  24. [32]

    Tobias Schimanski, Jingwei Ni, Mathias Kraus, Elliott Ash, and Markus Leippold. 2024. https://aclanthology.org/2024.acl-long.105 Towards faithful and robust LLM specialists for evidence-based question-answering . In Proceedings of the 62nd Annual Meeting of the Association for...

    2024

  25. [33]

    Gilles S \' e rasset. 2012. http://www.lrec-conf.org/proceedings/lrec2012/summaries/387.html Dbnary: Wiktionary as a LMF based multilingual RDF network . In Proceedings of the Eighth International Conference on Language Resources and Evaluation, LREC 2012, Istanbul, Turkey, Ma...

    2012

  26. [34]

    Tommaso Soru, Edgard Marx, Diego Moussallem, Gustavo Publio, Andre Valdestilhas, Diego Esteves, and Ciro Baron Neto. 2017. http://ceur-ws.org/Vol-2044/paper14/ SPARQL as a foreign language . In Proceedings of the Posters and Demos Track of the 13th International Conference on ...

    2017

  27. [35]

    Tommaso Soru, Edgard Marx, Andr \' e Valdestilhas, Diego Esteves, Diego Moussallem, and Gustavo Publio. 2018. http://arxiv.org/abs/1806.10478 Neural machine translation for query construction and composition . CoRR, abs/1806.10478

    2018 arXiv

  28. [36]

    Ricardo Usbeck, Xi Yan, Aleksandr Perevalov, Longquan Jiang, Julius Schulz, Angelie Kraft, Cedric M \"o ller, Junbo Huang, Jan Reineke, Axel-Cyrille Ngonga Ngomo, Muhammad Saleem, and Andreas Both. 2023. https://doi.org/10.3233/SW-233471 Qald-10 -- the 10th challenge on questi...

    2023 doi

  29. [37]

    Ruijie Wang, Luca Rossetto, Michael Cochez, and Abraham Bernstein. 2024. QAGCN : Answering multi-relation questions via single-step implicit reasoning over knowledge graphs. In The Semantic Web, pages 41--58, Cham. Springer Nature Switzerland

    2024

  30. [38]

    Paul Warren and Paul Mulholland. 2020. A comparison of the cognitive difficulties posed by SPARQL query constructs. In International Conference on Knowledge Engineering and Knowledge Management, pages 3--19. Springer

    2020

  31. [39]

    Silei Xu, Shicheng Liu, Theo Culhane, Elizaveta Pertseva, Meng-Hsi Wu, Sina Semnani, and Monica Lam. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.353 Fine-tuned LLM s know more, hallucinate less with few-shot sequence-to-sequence semantic parsing over W ikidata . In Proce...

    2023 doi

  32. [40]

    Jianyun Zou, Min Yang, Lichao Zhang, Yechen Xu, Qifan Pan, Fengqing Jiang, Ran Qin, Shushu Wang, Yifan He, Songfang Huang, et al. 2021. A Chinese multi-type complex questions answering dataset over Wikidata . arXiv preprint arXiv:2111.06086

    2021 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.