Pith. sign in

REVIEW 3 major objections 4 minor 24 references

LOGICPO: Efficient Translation of NL-based Logical Problems to FOL using LLMs and Preference Optimization

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning open LLMs on a bootstrapped preference dataset turns them into first-order logic translators that beat GPT-4 on logical correctness and syntax errors.

desk verdict A useful dataset and preference-optimization recipe for whole-problem NL-to-FOL translation, but the GPT-4 comparison is not controlled and the label-match metric is partly self-referential. read the letter →

arxiv 2506.18383 v1 pith:XT3THRUL submitted 2025-06-23 cs.LG cs.AI

Koushik Viswanadha , Deepanway Ghosal , Somak Aditya This is my paper · ORCID
classification cs.LGcs.AI
keywords NL-to-FOLtranslationpreferenceoptimizationDPOKTOlogicalreasoningneurosymbolicProver9LLMfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to make open-source large language models reliable translators of natural-language reasoning problems into first-order logic (FOL), so that neurosymbolic pipelines can offload the translation step to a cheap fine-tuned model instead of an expensive few-shot API. To do this, the authors bootstrap a 26k-example dataset, LOGICPO, from the expert-written FOLIO seed set: they generate many candidate FOL renderings of each story, keep those whose Prover9-computed logical conclusion matches the story's ground-truth label as positive examples, and use the mismatched ones as negative examples. They then fine-tune several open LLMs with supervised learning followed by direct preference optimization (DPO) or Kahneman-Tversky optimization (KTO). Their central finding is that this recipe produces translators that are both more logically correct and far less prone to syntax errors than sentence-wise baselines and GPT-3.5, with the best Qwen-2.5-14B model outperforming GPT-4's 8-shot results by 6% in logical correctness while making 8% fewer syntax errors on the FOLIO validation set.

What carries the argument

The central machinery is the LOGICPO bootstrapping pipeline: candidate whole-problem FOL stories are generated by LLMs, filtered by label agreement with Prover9 to form a supervised set (D_sft) and a preference set (D_pref) of chosen/rejected pairs, and then a two-stage fine-tune applies SFT followed by DPO or KTO. What it does is teach the model not just what a correct translation looks like but what a near-miss incorrect translation looks like, and the whole-problem framing enforces predicate-level consistency across sentences, which sentence-wise parsers miss.

What would settle it

A human-annotation study on a random sample of FOLIO validation outputs would settle it: if the translations judged 'logically correct' by label agreement are found to be semantically wrong (e.g., wrong predicates, missing quantifiers) in a large fraction of cases, and the LOGICPO-trained models show no better semantic fidelity than the baselines, then the reported gains are an artifact of the metric rather than real translation quality. Concretely, the manual analysis in §5.1 already suggests 'either-or' sentences are a known failure point; a larger annotated set that counts such semantic errors across all label-correct outputs would test the proxy directly.

Watch

Extended reading notes

Core claim

The central claim is that the main bottleneck in neurosymbolic reasoning — converting a natural-language problem into a consistent first-order logic program — can be dramatically improved by training open-source LLMs on a preference dataset constructed from whole-problem translations rather than sentence-by-sentence parses. Starting from FOLIO's expert-written stories and labels, the authors use several LLMs to generate multiple FOL versions of each story, then accept a version as a 'chosen' example if its Prover9-evaluated conclusion label matches the ground truth, and as a 'rejected' example otherwise. Supervised fine-tuning on the accepted set followed by DPO or KTO on the chosen/rejected pairs yields models that produce internally consistent predicate use, far fewer syntax errors, and better logical accuracy than the sentence-wise LogicLLaMA baseline and GPT-3.5 few-shot variants. The strongest model, Qwen-2.5-14B fine-tuned with KTO, reaches 70.20% logical correctness and 4.56% syntax errors on FOLIO validation, surpassing GPT-4's 8-shot LINC results (64.01% correctness, 12.91% syntax errors); the same training transfers to ProofWriter, PrOntoQA, and ProverQA, where the models also outperform or match the baselines.

Load-bearing premise

The whole approach rests on the assumption that matching the final logical label with Prover9 is a reliable sign of a good translation, so that label matches can be used both to pick training examples and to measure success.

Editorial extensions

If this is right

  • Neurosymbolic pipelines can replace expensive GPT-4 few-shot translation with a fine-tuned 14B open model that is both more accurate and cheaper per call, lowering the cost of end-to-end logical reasoning systems.
  • The bootstrapping recipe (generate, filter by solver label, build chosen/rejected pairs, apply SFT plus preference optimization) transfers across datasets and is a template for other low-resource formal languages such as SAT, SMT, or proof assistants.
  • KTO consistently improves over SFT alone across model families, whereas DPO does not; this suggests the preference signal from 'rejected' samples is valuable and that KTO's non-paired objective fits this data distribution better.
  • Whole-problem translation fixes predicate-level inconsistencies that sentence-wise parsers exhibit, because the model is trained on full stories rather than isolated sentences.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The label-matching filter may make the dataset self-reinforcing: examples that Prover9 can already decide are over-represented, so the reported gains could partly reflect training toward easy-to-prove fragments rather than general FOL competence; a distribution analysis of which stories survive the filter would test this.
  • Because the evaluation metric is the same Prover9 label match used to build the data, there is a risk of circularity: the models could be learning to satisfy the label criterion rather than to translate faithfully; an independent semantic metric (e.g., back-translation similarity or human-rated equivalence) on held-out data would separate the two.
  • The same recipe could be applied to other formal targets, but the authors only demonstrate English-to-FOL; the method's generality could be tested by porting the pipeline to, say, Python or SQL, where the label-matching oracle would be a different execution engine.
  • The paper does not report inference cost per problem; a practical extension would measure tokens and wall-clock time for the tuned models versus GPT-4 few-shot to quantify the 'efficient' claim in real deployment terms.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper introduces LOGICPO, a procedure for bootstrapping a supervised and preference-optimization dataset for whole-problem natural-language-to-first-order-logic (NL-to-FOL) translation. Starting from the FOLIO training split, the authors use several LLMs with varying numbers of in-context demonstrations and temperatures to generate candidate FOL stories, keep those whose Prover9-computed conclusion label matches the FOLIO label in D_sft (and as chosen examples in D_pref), and use label-mismatching or error-producing stories as rejected examples. They then fine-tune Llama-3-8B, Gemma-2-2B, Phi-3.5-Mini, and Qwen-2.5-14B with SFT followed by DPO or KTO, and evaluate on FOLIO validation, ProofWriter, PrOntoQA, and ProverQA. The reported results show large reductions in syntax errors and improvements in the Prover9 label-match rate relative to few-shot baselines and the LINC GPT-3.5/GPT-4 numbers, with the best full-text-abstract result attributed to Qwen-2.5-14B SFT+KTO (70.20% 'logically correct' on FOLIO vs. 64.01% for GPT-4-LINC). The paper also includes manual semantic-error analysis, context-length ablations, and back-translation-based semantic similarity checks.

Significance. If the central claim is taken at face value, the paper makes a useful practical contribution: it shows that a bootstrapped preference dataset, generated without human gold FOL annotation beyond FOLIO's existing labels, can be used to fine-tune open-weight models that produce syntactically executable FOL with label-match rates close to or above GPT-4-LINC on FOLIO, while generalizing to other reasoning benchmarks. The release of the 26k dataset and code, the breadth of base models, and the explicit error-category analysis (Tables 5 and 6) are genuine strengths. The significance is moderated, however, by the fact that the main evaluation metric is the same Prover9 label-match criterion used to filter the training data, and by the uncontrolled comparison with the LINC baselines. The paper's contribution is therefore better characterized as improving a specific proxy for translation quality rather than establishing semantic equivalence; whether the gains reflect genuine translation quality needs additional evidence.

major comments (3)
  1. [§4.1, Tables 3–4] The comparison with GPT-4-LINC is not controlled. The paper explicitly deviates from LINC's k-majority voting by averaging 10 per-story outcomes, but the GPT-4-LINC numbers are taken from Olausson et al. (2023), which use k-majority voting. These two aggregation schemes measure different quantities: a model that produces the correct label in 6 of 10 samples is counted as 60% under the paper's metric but as fully correct under LINC's voting, and the discrepancy can go in either direction. The headline '6% more logically correct and 8% less syntax errors' in the full-text abstract should be recomputed under a single protocol, either by applying the paper's per-sample averaging to GPT-4-LINC outputs (if the raw outputs are available) or by applying majority voting to the fine-tuned models. Without this, the comparison is not valid evidence for 'consistently outperforms GPT-4'.
  2. [§3.2, §4.1, Table 2, §5.1, §B.3] The Prover9 label-match criterion is used both to select the training examples that enter D_sft and D_pref and to define the primary 'Logically Correct' metric. A generated FOL story is treated as correct if its Prover9-computed label equals the NL label, but, as the Limitations section concedes, a correct label does not guarantee a correct program. The scale of this concern is concrete: D_sft contains 8,792 Uncertain samples (Table 2), and Uncertain can be obtained by omitting premises, by using inconsistent predicates that prevent Prover9 from proving or refuting, or by other failures of faithful translation. The manual analysis in §5.1 already finds either-or mistakes in label-correct outputs (6/48 for Phi-KTO and 5/35 for Phi-SFT). The back-translation evaluation in §B.3 compares generated-FOL paraphrases only for LLAMA3 and GPT3.5 and has no gold-FOL baseline, so it does not establish that the fine-tuned models are semantically closer to the reference FOL. The claim that the models produce 'logically correct' translations should be qualified as 'Prover9-label-correct' until an independent evaluation (e.g., human semantic-equivalence judgments or back-translation against gold FOL) is reported for all compared models.
  3. [§4.3 and Tables 3, 4, 9, 10, 11] The reported numeric differences between methods are averages over 5–10 generations without variance or significance testing. For instance, the FOLIO gain of SFT+KTO over SFT for Phi-3.5 is 3.09 points (61.52 vs 58.43) and for Qwen-2.5 is 12.36 points (70.20 vs 57.84), but adjacent cells differ by as little as 0.49 points (e.g., Qwen SFT vs 2-shot on ProofWriter in Table 4). Without error bars or confidence intervals, a claim such as 'SFT + KTO always leads to improvement compared to SFT' is not supportable as a general statement. I recommend reporting standard deviations or confidence intervals, especially for the headline comparison against GPT-4-LINC.
minor comments (4)
  1. [Abstract] The abstract presented at the top of the paper names Phi-3.5 and GPT-3.5-turbo as the best model and baseline, while the full-text abstract and §4.3 attribute the best results to Qwen-2.5-14B against GPT-4-LINC. These should be reconciled, since the current text leaves the reader unsure which claim is being made.
  2. [§3.2, Table 2] Please clarify whether the preferred samples in D_pref are a subset of D_sft, since the text says samples with matching labels are included in both sets. If so, state this explicitly, because the two-stage training then reuses the same preferred examples, which is a design choice worth reporting.
  3. [§5.1, Table 5] The manual semantic analysis is performed only on label-correct True examples from three models. It would be more informative to also include False and Uncertain labels, since the automated metric's main weakness is for Uncertain, as acknowledged in §5.2 S3.
  4. [Throughout] There are numerous typos and grammatical issues, including 'it's role' in the abstract, 'utilzing', 'synactic', 'exectutability', and 'recieve' in the appendix prompts. A careful proofreading pass is needed.

Circularity Check

1 steps flagged · score 6.0 of 10

The 'logically correct' metric is the same Prover9 label-match criterion used to filter the training data, so the headline gains partly measure fit to the training filter rather than independent translation quality.

  1. fitted input called prediction [§3.2 Automated Data Generation and §4.1 Datasets/Metrics]
    "If the computed logical outcome matches with the original groundtruth label, we include the NL-FOL pair ( S, ˜FS) as part of our supervised finetuning (Ds f t) dataset. ... For correctness, we report the average number of matches to the original label."

    The same Prover9 label-match criterion is used twice: it selects which LLM-generated FOL stories enter D_sft/D_pref (the training signal), and it defines the 'Logically Correct' metric in Tables 3-4 and the headline 6% improvement over GPT-4. A model trained to produce FOL stories that satisfy this filter is then scored by the same filter, so the metric is not an independent check of semantic equivalence. The paper's Limitations concede 'it is not guaranteed that if the logical label is correct, the program will also be correct,' and §5.1's manual analysis already finds either-or mistakes among label-correct outputs. The circularity is partial: evaluation labels come from external benchmarks and untrained baselines are compared, so the ranking is not entirely forced.

full rationale

The paper's central empirical claim is that preference-optimized models produce more 'logically correct' FOL translations. The measure of logical correctness is the Prover9 label-match criterion, and the same criterion is the admission rule for the bootstrapped training data: generated FOL stories whose Prover9 label equals the FOLIO gold label become D_sft preferred samples, and rejected samples are those whose label differs or errors. Thus the metric and the training filter are identical by construction. This does not make the experiments vacuous—the models must still generalize to unseen FOLIO validation, ProofWriter, PrOntoQA, and ProverQA instances, and untrained GPT-3.5/GPT-4 baselines are scored with the same metric—but it means the reported 'logically correct' numbers are not an independent validation of translation quality. The paper itself acknowledges the gap between label correctness and program correctness in the Limitations and in the §5.1 manual analysis, where label-correct outputs still contain semantic errors such as exclusive-or rendered as inclusive-or. No load-bearing self-citation chain or imported uniqueness theorem is present; the only self-citation (Sharma et al. 2015) is unrelated to the core argument. Overall, the headline claim is partially circular because the evaluation metric reduces to the training-data selection criterion, though external benchmarks and manual checks give the work some independent content.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the Prover9 label-matching proxy, on the correctness of FOLIO gold annotations, and on the assumption that FOLIO-trained translators generalize to other datasets. The free parameters listed are hand-chosen data-generation and evaluation settings the paper does not optimize over. No new physical, logical, or ontological entities are introduced; formal terms are drawn from the source stories.

free parameters (4)
  • Number of in-context demonstrations n = 2, 4, and 8
    Hand-chosen diversity settings for generating candidate FOL stories; not justified by a selection criterion.
  • Generation temperature = 0.25 and 0.6
    Hand-chosen values to increase output diversity during dataset bootstrap.
  • Number of candidates per story = 30
    Hand-chosen cap on generated FOL stories per input before subsampling into Dsft and Dpref.
  • Evaluation sample count = 10 (5 for few-shot)
    Hand-chosen number of generation attempts; affects variance of reported averages and differs from k-majority voting used by LINC baselines.
assumptions (3)
  • domain assumption Prover9 label matching is a sufficient proxy for FOL translation correctness.
    Used to filter Dsft and Dpref in §3.2 and to compute the main 'logically correct' metric in §4.1; the paper's Limitations section concedes that label correctness does not imply program correctness.
  • standard math FOLIO gold FOL annotations are ground truth.
    FOLIO is treated as expert-written ground truth; any errors in FOLIO annotations propagate to training and evaluation.
  • domain assumption Finetuning on FOLIO-derived data generalizes to ProofWriter, ProntoQA, and ProverQA distributions.
    The models are evaluated out-of-distribution; the paper assumes the translation skill transfers to synthetically generated datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LOGICPO: Efficient Translation of NL-based Logical Problems to FOL using LLMs and Preference Optimization." pith.science (2026). https://pith.science/paper/XT3THRUL

@misc{pith2026250618383,
  author       = {Pith},
  title        = {Pith review of: LOGICPO: Efficient Translation of NL-based Logical Problems to FOL using LLMs and Preference Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XT3THRUL}},
  note         = {Machine review of arXiv:2506.18383}
}
read the original abstract

Logical reasoning is a key task for artificial intelligence due to it's role in major downstream tasks such as Question Answering, Summarization. Recent methods in improving the reasoning ability of LLMs fall short in correctly converting a natural language reasoning problem to an equivalent logical formulation, which hinders the framework's overall ability to reason. Towards this, we propose to use finetuning on a preference optimization dataset to learn to parse and represent a natural language problem as a whole to a consistent logical program by 1) introducing a new supervised and preference optimization dataset LogicPO, and 2) adopting popular techniques such as Direct Preference Optimization (DPO), Kahneman-Tversky optimization (KTO) to finetune open-source LLMs. Our best model with Phi-3.5 consistently outperforms GPT-3.5-turbo's (8-shot) by producing 10% more logically correct and with 14% less syntax errors. Through the framework and our improved evaluation metrics, we offer a promising direction in improving the logical reasoning of LLMs by better representing them in their logical formulations.

Figures

Figures reproduced from arXiv: 2506.18383 by the authors.

Figure 1
Figure 1. The Data creation pipeline: We use natural language stories from FOLIO, use [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 8 canonical work pages

  1. [5]

    Ruifang Ge and Raymond J

    URL https://arxiv.org/abs/ 2402.01306. Ruifang Ge and Raymond J. Mooney. A statistical semantic parser that integrates syntax and semantics. In Ido Dagan and Daniel Gildea (eds.),Proceedings of the Ninth Conference on Computational Natural Language Learning, CoNLL 2005, Ann Arbor, Michigan, USA, June 29-30, 2005, pp. 9–16. ACL,

    arXiv 2005

  2. [6]

    Yu Gu, Vardaan Pahuja, Gong Cheng, and Yu Su

    URL https://arxiv.org/ abs/2407.21783. Yu Gu, Vardaan Pahuja, Gong Cheng, and Yu Su. Knowledge base question answer- ing: A semantic parsing perspective.ArXiv, abs/2209.04994,

    arXiv

  3. [7]

    URL https: //api.semanticscholar.org/CorpusID:252200059. Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alex Wardle-Solano, Hannah Szabo, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kas...

    arXiv 2024

  4. [8]

    URL https://doi.org/10

    doi: 10.48550/ARXIV .2310.06825. URL https://doi.org/10. 48550/arXiv.2310.06825. Hyeonji Kim, Byeong-Hoon So, Wook-Shin Han, and Hongrae Lee. Natural language to SQL: where are we today?Proc. VLDB Endow., 13(10):1737–1750,

    arXiv doi:10.48550/arxiv.2310.06825

  5. [10]

    doi: 10.48550/ARXIV .2408. 05109. URLhttps://doi.org/10.48550/arXiv.2408.05109. Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi, and Iryna Gurevych. Are emergent abilities in large language models just in-context learning? In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.),Proceedings of the 62nd Annual Meeting of the Associati...

    doi:10.48550/arxiv.2408.05109

  6. [11]

    URL https://doi.org/10

    doi: 10.18653/V1/2024.ACL-LONG.279. URL https://doi.org/10. 18653/v1/2024.acl-long.279. Xuantao Lu, Jingping Liu, Zhouhong Gu, Hanwen Tong, Chenhao Xie, Junyang Huang, Yanghua Xiao, and Wenguang Wang. Parsing natural language into propositional and first-order logic with dual reinforcement learning. In Nicoletta Calzolari, Chu- Ren Huang, Hansaem Kim, Jam...

    doi:10.18653/v1/2024.acl-long.279 2024

  7. [12]

    URL https://aclanthology.org/2022.coling-1.481

    International Committee on Computational Linguistics. URL https://aclanthology.org/2022.coling-1.481. W. McCune. Prover9 and mace4.http://www.cs.unm.edu/˜mccune/prover9/, 2005–2010. Yaniv Nikankin, Anja Reusch, Aaron Mueller, and Yonatan Belinkov. Arithmetic without algorithms: Language models solve math with a bag of heuristics,

    work page 2022

  8. [13]

    Theo Olausson, Alex Gu, Benjamin Lipkin, Cedegao E

    URL https: //arxiv.org/abs/2410.21272. Theo Olausson, Alex Gu, Benjamin Lipkin, Cedegao E. Zhang, Armando Solar-Lezama, Joshua B. Tenenbaum, and Roger Levy. LINC: A neurosymbolic approach for logical reasoning by combining language models with first-order logic provers. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.),Proceedings of the 2023 Conference...

    arXiv

Show all 24 references
  1. [14]

    EMNLP-MAIN.313

    doi: 10.18653/V1/2023. EMNLP-MAIN.313. URLhttps://doi.org/10.18653/v1/2023.emnlp-main.313. Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning. In Houda Bouamor, Juan Pi...

    2023 doi

  2. [15]

    URL https://doi.org/10.18653/v1/2023.findings-emnlp.248

    doi: 10.18653/V1/2023.FINDINGS-EMNLP .248. URL https://doi.org/10.18653/v1/2023.findings-emnlp.248. 12 Chengwen Qi, Ren Ma, Bowen Li, He Du, Binyuan Hui, Jinwang Wu, Yuanjun Laili, and Conghui He. Large language models meet symbolic provers for logical reasoning evaluation. In...

    2023 doi

  3. [16]

    Abulhair Saparov and He He

    URLhttps://arxiv.org/abs/2305.18290. Abulhair Saparov and He He. Language models are greedy reasoners: A systematic formal analysis of chain-of-thought. InThe Eleventh International Conference on Learning Representations,

    arXiv

  4. [17]

    Arpit Sharma, Nguyen Ha Vo, Somak Aditya, and Chitta Baral

    URLhttps://openreview.net/forum?id=qFVVBzXxR2V. Arpit Sharma, Nguyen Ha Vo, Somak Aditya, and Chitta Baral. Towards addressing the winograd schema challenge - building and using a semantic parser and a knowledge hunting module. In Qiang Yang and Michael J. Wooldridge (eds.),Pr...

    2015

  5. [19]

    Ramya Keerthy Thatikonda, Jiuzhou Han, Wray Buntine, and Ehsan Shareghi

    URLhttps://arxiv.org/abs/2408.00118. Ramya Keerthy Thatikonda, Jiuzhou Han, Wray Buntine, and Ehsan Shareghi. Strategies for improving nl-to-fol translation with llms: Data generation, incremental fine-tuning, and verification.arXiv preprint arXiv:2409.16461,

    arXiv

  6. [20]

    Dick, and Anton van den Hengel

    Qi Wu, Peng Wang, Chunhua Shen, Anthony R. Dick, and Anton van den Hengel. Ask me anything: Free-form visual question answering based on knowledge from external sources. In2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV , USA, June 27-...

    2016

  7. [22]

    doi: 10.18653/v1/2024.acl-long.375

    Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.375. URLhttps://aclanthology.org/2024.acl-long.375. Xi Ye, Qiaochu Chen, Isil Dillig, and Greg Durrett. Satlm: Satisfiability-aided language models using declarative prompting. In Alice Oh, Tristan Nauma...

    2024 doi

  8. [23]

    A Appendix A.1 Few-shot Baseline prompts For our baseline models, we evaluate off-the-shelf models in a few-shot setting

    URL http://papers.nips.cc/paper files/paper/2023/hash/ 8e9c7d4a48bdac81a58f983a64aaf42b-Abstract-Conference.html. A Appendix A.1 Few-shot Baseline prompts For our baseline models, we evaluate off-the-shelf models in a few-shot setting. As explained in section 4.2, we use FOLIO...

    2023

  9. [24]

    As the process of ProverQA creation was quite different, the high improvement of the preference optimized models underlines the generalizability of our LOGICPO dataset

    We achieve the highest syntactic correctness again with the Phi3.5 models. As the process of ProverQA creation was quite different, the high improvement of the preference optimized models underlines the generalizability of our LOGICPO dataset. Model Setting Easy Set Medium Set...

  10. [2014]

    doi: 10.3115/v1/P14-1114

    Association for Computational Linguistics. doi: 10.3115/v1/P14-1114. URLhttps://aclanthology.org/P14-1114. David Chanin. Open-source frame semantic parsing.arXiv preprint arXiv:2303.12788,

    arXiv

  11. [2016]

    URLhttps://doi.org/10.1109/CVPR.2016.500

    doi: 10.1109/CVPR.2016.500. URLhttps://doi.org/10.1109/CVPR.2016.500. Yuan Yang, Siheng Xiong, Ali Payani, Ehsan Shareghi, and Faramarz Fekri. Harnessing the power of large language models for natural language to first-order logic translation. In Lun-Wei Ku, Andre Martins, and...

    2016 doi

  12. [2020]

    URLhttp://www.vldb.org/pvldb/vol13/p1737-kim.pdf

    doi: 10.14778/ 3401960.3401970. URLhttp://www.vldb.org/pvldb/vol13/p1737-kim.pdf. Xinyu Liu, Shuyu Shen, Boyan Li, Peixian Ma, Runzhi Jiang, Yuyu Luo, Yuxin Zhang, Ju Fan, Guoliang Li, and Nan Tang. A survey of NL2SQL with large language models: Where are we, and where are we ...

  13. [2021]

    URL https: //arxiv.org/abs/2012.13048. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L´eonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram ´e, Johan Ferret, Peter Liu, and et al. Pouya Tafti. Gemma 2: Improving ...

    2012 arXiv

  14. [2022]

    URLhttps://doi.org/10.24963/ijcai.2022/568

    doi: 10.24963/IJCAI.2022/568. URLhttps://doi.org/10.24963/ijcai.2022/568. Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization,

    2022 doi

  15. [2023]

    In- terpretable amr-based question decomposition for multi-hop question answering

    Zhenyun Deng, Yonghua Zhu, Yang Chen, Michael Witbrock, and Patricia Riddle. In- terpretable amr-based question decomposition for multi-hop question answering. In Luc De Raedt (ed.),Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCA...

    2022

  16. [2024]

    Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider

    URLhttps://arxiv.org/abs/2404.14219. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. Abstract Meaning Representation for sembanking. In Antonio Pareja-Lora, Maria Liaka...

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.