Pith. sign in

REVIEW 3 major objections 5 minor 196 references

Credit Cards, Confusion, Computation, and Consequences: What Can We Uncover About Language Model Reasoning?

T0 review · 3 major / 5 minor · reviewed 2026-07-30 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read On real credit-card agreements, language models fail less at arithmetic than at rules, conditions, and contractual exceptions.

desk verdict Solid new consumer-finance literacy benchmark with clean PoT/CoT results; the contractual-error story is directionally right but over-claimed from a 50-example single-model audit. read the letter →

arxiv 2607.26952 v1 pith:VJMZUZAV submitted 2026-07-29 cs.CL

Arnav Hiray , Agam Shah , Caleb Lu , Meghaj Tarte , Harsit Mittal , Sudheer Chava This is my paper
classification cs.CL
keywords financialliteracynumericalreasoningcreditcardagreementsProgram-of-ThoughtChain-of-ThoughterroranalysisconditionallogicLLMevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces CreditCardQA, a 1,800-question benchmark built from real credit-card agreements so that models must reason about fees, interest, minimum payments, and penalties the way consumers actually encounter them. The authors evaluate open- and closed-source language and reasoning models with both step-by-step verbal reasoning and program-style reasoning. Program-style prompting improves accuracy for every model, helps weaker models most, and shrinks the gap between open and closed systems. Error audits and a difficulty regression show that wrong answers usually come from applying the wrong formula, skipping an if-condition, or misreading agreement language—not from botching the arithmetic once the right inputs are chosen. Comparisons, branching logic, and monetary constraints are the hardest structural features, and mistakes concentrate in edge cases such as late fees and small balances that hit financially vulnerable users hardest. The practical claim is that personal-finance literacy is a high-stakes reasoning setting where contractual interpretation, not calculator skill, is the bottleneck.

What carries the argument

CreditCardQA: 1,800 grounded question–answer pairs from real credit-card agreements, with operator-program annotations, first- versus third-person variants, and a logistic-regression difficulty model over structural operators (comparisons, branches, money inputs) plus agreement-level effects.

What would settle it

Re-run the same error taxonomy and difficulty regression on a multi-model sample of incorrect answers with independent annotators; if arithmetic or retrieval errors dominate, or if comparisons and conditions no longer predict failure, the central failure story does not hold.

Watch

Extended reading notes

Core claim

On CreditCardQA, model failures arise less from arithmetic than from misapplied financial rules, missed conditions, and misunderstandings of contractual terms. Program-of-Thought consistently beats Chain-of-Thought (gains of roughly 0.9 to 6.2 points at a 5% error tolerance), narrows open- versus closed-source gaps, and helps weaker baseline models most; comparisons, conditional logic, and monetary constraints are the structural features that most reduce correctness.

Load-bearing premise

The detailed map of why models fail rests mainly on a manual audit of fifty wrong answers from one strong open model, plus a regression fit on that same model’s outputs.

Editorial extensions

If this is right

  • Program-style prompting should be preferred over pure verbal chain-of-thought for consumer finance Q&A.
  • Benchmarks and product evals should stress comparison, branching, and fee/penalty edge cases rather than only multi-step arithmetic length.
  • First-person consumer phrasing can raise accuracy relative to textbook-style third-person questions.
  • Simple retrieval over agreements can drop accuracy when chunks omit cross-referenced conditions, so full-document or structure-aware context still matters.
  • Errors concentrated on late fees and small balances imply higher downside risk for lower-income users of AI finance helpers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same rule-and-exception brittleness likely appears in other consumer contracts (mortgages, insurance, leases) that mix caps, floors, and nested ifs.
  • Training or test-time methods that force explicit enumeration of triggering conditions before any arithmetic may close more of the gap than larger base models alone.
  • Agreement-level difficulty clusters suggest issuers could rewrite disclosures for machine readability as a consumer-protection lever.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces CreditCardQA (also called CREDITQA/FINLITQA), a 1,800-question numerical-reasoning benchmark built from real CFPB and related credit-card agreements, with third- and first-person variants and operator-program annotations. It evaluates 11 open- and closed-source LLMs/LRMs under CoT and PoT, reporting consistent PoT gains (about +0.9 to +6.2 points at ±5% tolerance in Table 2), competitive open-weight performance (notably GPT-OSS-120B), and a narrowing of open/closed and LRM/LLM gaps under PoT. Error analysis on 50 incorrect GPT-OSS-120B answers (Table 3) attributes failures mainly to formula/rule substitution and missed conditions rather than arithmetic; a logistic regression on structural, linguistic, and input features (§6, Eq. 1, Appendix I) identifies comparisons, branching, monetary inputs, and third-person framing as difficulty drivers, with discussion of equity-relevant edge cases.

Significance. If the measured results hold, the paper fills a clear gap: prior financial QA benchmarks emphasize professional/market tasks, whereas this one targets consumer financial literacy grounded in real contractual text. Strengths include a carefully constructed, agreement-sourced dataset with DSL solution programs, a multi-model CoT/PoT comparison with two numeric tolerances, and a difficulty analysis that goes beyond step count. The finding that PoT helps weaker models and that comparison/conditional structure hurts accuracy is useful for both evaluation design and practical prompting. The equity-facing interpretation is suggestive rather than fully established, but the benchmark itself is a concrete, reusable contribution for high-stakes personal-finance reasoning.

major comments (3)
  1. [§5 Error Analysis (RQ2), Table 3] §5 and Table 3: The central qualitative claim that failures arise less from arithmetic than from misapplied rules and missed conditions rests on a manual audit of 50 incorrect GPT-OSS-120B answers (non-mutually exclusive labels, no multi-annotator agreement reported). This is too thin to support abstract/conclusion-level generalization across the 11-model suite or closed models. Either expand the audit (multiple models, agreement stats) or clearly scope the claim to this focused open-weight audit and treat cross-model prevalence as a hypothesis.
  2. [§6–7, Abstract] §6–7 and the abstract: The equity narrative (errors concentrated in late-payment/small-balance cases affecting lower-income users) is only weakly tied to the regression and the 50-example audit. Agreement-level γ effects and hardship indicators are discussed, but there is no systematic stratification of error rates by card type (subprime/secured vs premium) or by penalty/small-balance question subsets with statistical tests. Either add that analysis or tone down causal/social claims to match the evidence.
  3. [§2, Table 1, Dataset Release] Naming and construction consistency: the manuscript alternates CreditCardQA, CREDITQA, CREDITCARDQA, and FINLITQA (§2 opening vs title/abstract/tables). This is load-bearing for a benchmark paper because release, leaderboard, and citation identity depend on a single canonical name and clear dev/test protocol (800/1000). Unify naming and state exactly what is released vs held out.
minor comments (5)
  1. [§4] Table 2 header typo: "desmonstrates" in the CoT vs PoT discussion; also fix "formodels" spacing in the "Who benefits most" paragraph.
  2. [Appendix B] Appendix B is titled "Additional Related Works on LLM Reasoning" but contains the question-generation pipeline; renumber/retitle to avoid collision with Appendix A.
  3. [Appendix J / §4] RAG evaluation (Appendix J) uses only 140 questions and one model; briefly caveat external validity in the main text when citing it, or move the negative RAG result fully to appendix without over-claiming.
  4. [§6.1, Eq. (1)] Eq. (1) writes the logit in a hard-to-parse stacked form; present a standard logit equation and define γ_c(i) and third-person coding more explicitly near the equation.
  5. [Table 1, §2.3] Flesch–Kincaid: text says "low" scores corresponding to college level while reporting Min/Avg 9.2/11.3; clarify directionality (higher grade level = harder) for non-specialist readers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical benchmark evaluation against human gold labels, not a self-encoding derivation.

full rationale

CreditCardQA is an external evaluation paper. Questions and operator programs are human-authored from real CFPB agreements; model accuracy under CoT/PoT is scored against those held-out gold answers (Table 2), not against quantities fitted from the same model outputs. The error taxonomy (Table 3) is a post-hoc manual audit of 50 failures, and the logistic regression (Eq. 1, Appendix I) relates annotated structural features to observed correctness—standard association analysis, not a prediction forced by construction. Self-citations (e.g., Hiray et al., Shah & Chava) supply related prior work and do not underwrite uniqueness theorems or force the main claims. Methodology is anchored in external precedents (ConvFinQA, DocMath-Eval). No self-definitional loop, fitted-input-as-prediction, or load-bearing self-citation chain appears in the derivation of the reported results.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

Empirical NLP benchmark paper. Load-bearing background is standard LLM eval practice plus domain assumptions that CFPB-style agreements and annotator-written operator programs are a valid proxy for financial-literacy numerical reasoning. No new physical entities; free choices are eval thresholds, sampling for error/RAG studies, and regression feature design.

free parameters (4)
  • relative_error_tolerances = 0.2% and 5%
    Correctness uses ±0.2% and ±5% relative error bands chosen by the authors; rankings are said to be stable but the headline ~80% figures depend on the looser band.
  • error_audit_sample_size_and_model = n=50, GPT-OSS-120B
    Failure-mode percentages in Table 3 come from 50 incorrect GPT-OSS-120B items only; taxonomy prevalence is sample-dependent.
  • RAG_retrieval_hyperparameters = 900/160/30+30/0.55/8
    Chunk size 900, overlap 160, top-30 BM25 + top-30 dense, fuse weight 0.55, keep top-8; these hand choices drive the RAG ablation.
  • logistic_regression_feature_set
    Difficulty conclusions depend on the chosen structural, linguistic, and input indicators and agreement fixed effects in Eq. 1 / Table 5.
assumptions (5)
  • domain assumption Operator-program gold solutions in the ConvFinQA-style DSL uniquely determine the intended numerical/categorical answer for each question.
    §2.2 and Appendix B require annotators to supply DSL programs; evaluation treats those outputs as ground truth.
  • domain assumption Markdown conversions of PDF agreements preserve all contractually material numeric terms and conditions needed for the questions.
    §2.1: Marker conversion plus manual verification; questions are generated only from Markdown.
  • ad hoc to paper Relative numeric tolerance matching plus limited manual review of categorical outputs is a valid accuracy metric for consumer financial answers.
    §3 and Appendix G define extraction, ±0.2%/±5% bands, and manual overrides for unit/category mismatches.
  • domain assumption Standard CoT and PoT prompting with temperature 0 adequately represents model reasoning skill on this task.
    §3/Appendix F follow prior finance-reasoning eval practice (Tang et al.; Zhao et al.).
  • standard math Logistic regression with agreement fixed effects identifies causal-ish difficulty factors from observational question features.
    §6 Eq. 1; coefficients interpreted as odds-ratio effects holding other features fixed.
invented entities (1)
  • CreditCardQA / CREDITQA dataset
    purpose: Provide 1,800 grounded financial-literacy numerical QA pairs with operator annotations, person framing, and train/leaderboard split.
    Core contribution; existence is evidenced by construction pipeline and promised HF/GitHub release, but independent reuse is not yet shown in this preprint.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Credit Cards, Confusion, Computation, and Consequences: What Can We Uncover About Language Model Reasoning?." pith.science (2026). https://pith.science/paper/VJMZUZAV

@misc{pith2026260726952,
  author       = {Pith},
  title        = {Pith review of: Credit Cards, Confusion, Computation, and Consequences: What Can We Uncover About Language Model Reasoning?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VJMZUZAV}},
  note         = {Machine review of arXiv:2607.26952}
}
read the original abstract

We introduce CreditCardQA, the first financial literacy benchmark for numerical reasoning derived from real credit card agreements. The dataset contains 1,800 questions, including first-person variants that reflect how consumers naturally ask about fees, interest, and payments. We evaluate a range of large language and reasoning models under Chain-of-Thought (CoT) and Program-of-Thought (PoT) prompting. Overall, PoT yields consistent performance gains, particularly for models with weaker baseline reasoning, and narrows gaps between open- and closed-source systems. Through error analysis, we show that failures arise less from arithmetic and more from misapplied financial rules, missed conditions, and misunderstandings of contractual terms. We further analyze question difficulty and find that comparisons, conditional logic, and monetary constraints are especially challenging. We also find that errors often arise in edge cases such as late-payment penalties or small-balance scenarios that are more likely to affect lower-income or financially vulnerable individuals.

Figures

Figures reproduced from arXiv: 2607.26952 by the authors.

Figure 1
Figure 1. CREDITCARDQA construction pipeline: CFPB credit card agreements → 280 extracted terms → 1,800 generated questions → LLM evaluation with CoT/PoT prompting. 2.1 Credit Card Agreements A credit card agreement is a legally binding document that specifies the terms and conditions governing a credit card account, including interest rates, fees, payment obligations, penalties, and consumer rights. These agreements are ofte… view at source ↗
Figure 2
Figure 2. Simplified process for answering CreditCardQA questions from credit card agree [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Chain-of-Thought (COT) prompt template. • What are the inputs required to compute P(A|T, B, Qn), and how do previous questions Q0, Q1, ..., Qn−1 factor into this computation? • What is the difference between Type I and Type II conversations? Testing Question Samples: • Use the ”train.json” provided as apart of the dataset and choose two conversations to test using GPT: one Type I and one Type II. Ensure that the use… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Program-of-Thought (PoT) prompt template. [PITH_FULL_IMAGE:figures/full_fig_p023_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

196 extracted references · 9 canonical work pages

  1. [1]

    Numerical Claim Detection in Finance: A New Financial Dataset, Weak-Supervision Model, and Market Analysis

    Shah, Agam and Hiray, Arnav and Shah, Pratvi and Banerjee, Arkaprabha and Singh, Anushka and Eidnani, Dheeraj Deepak and Chava, Sahasra and Chaudhury, Bhaskar and Chava, Sudheer. Numerical Claim Detection in Finance: A New Financial Dataset, Weak-Supervision Model, and Market Analysis. Proceedings of the Seventh Fact Extraction and VERification Workshop (...

    work page doi:10.18653/v1/2024.fever-1.21 2024

  2. [2]

    2024 , eprint=

    Alignment faking in large language models , author=. 2024 , eprint=

    2024

  3. [3]

    2025 , url=

    Kaixuan Huang and Jiacheng Guo and Zihao Li and Xiang Ji and Jiawei Ge and Wenzhe Li and Yingqing Guo and Tianle Cai and Hui Yuan and Runzhe Wang and Yue Wu and Ming Yin and Shange Tang and Yangsibo Huang and Chi Jin and Xinyun Chen and Chiyuan Zhang and Mengdi Wang , booktitle=. 2025 , url=

    2025

  4. [4]

    2025 , eprint=

    Benchmarking LLMs' Mathematical Reasoning with Unseen Random Variables Questions , author=. 2025 , eprint=

    2025

  5. [5]

    Lauter and Swarat Chaudhuri and Dawn Song , booktitle=

    Kaiyu Yang and Gabriel Poesia and Jingxuan He and Wenda Li and Kristin E. Lauter and Swarat Chaudhuri and Dawn Song , booktitle=. Position: Formal Mathematical Reasoning. 2025 , url=

    2025

  6. [6]

    Transactions on Machine Learning Research , issn=

    Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks , author=. Transactions on Machine Learning Research , issn=. 2023 , url=

    2023

  7. [7]

    2025 , eprint=

    Large Language Models Often Know When They Are Being Evaluated , author=. 2025 , eprint=

    2025

  8. [10]

    Nguyen and Quang Pham and Nghi D

    Dung Manh Nguyen and Thang Chau Phan and Nam Le Hai and Tien-Thong Doan and Nam V. Nguyen and Quang Pham and Nghi D. Q. Bui , booktitle=. Code. 2025 , url=

    2025

Show all 196 references
  1. [11]

    The 4th Workshop on Mathematical Reasoning and AI at NeurIPS'24 , year=

    DafnyBench: A Benchmark for Formal Software Verification , author=. The 4th Workshop on Mathematical Reasoning and AI at NeurIPS'24 , year=

  2. [12]

    Aryan Gulati and Brando Miranda and Eric Chen and Emily Xia and Kai Fronsdal and Bruno de Moraes Dumont and Sanmi Koyejo , booktitle=. Putnam-. 2024 , url=

    2024

  3. [13]

    F in QA : A Dataset of Numerical Reasoning over Financial Data

    Chen, Zhiyu and Chen, Wenhu and Smiley, Charese and Shah, Sameena and Borova, Iana and Langdon, Dylan and Moussa, Reema and Beane, Matt and Huang, Ting-Hao and Routledge, Bryan and Wang, William Yang. F in QA : A Dataset of Numerical Reasoning over Financial Data. Proceedings ...

    2021 doi

  4. [16]

    2023 , eprint=

    Zero is Not Hero Yet: Benchmarking Zero-Shot Performance of LLMs for Financial Tasks , author=. 2023 , eprint=

    2023

  5. [17]

    http://www.nber.org/papers/w17108

    Lusardi, Annamaria and Mitchell, Olivia S. Financial Literacy and Retirement Planning in the United States. 2011. doi:10.3386/w17108 , URL = "http://www.nber.org/papers/w17108", abstract =

    2011 doi

  6. [18]

    http://www.nber.org/papers/w17103

    Lusardi, Annamaria. Americans' Financial Capability. 2011. doi:10.3386/w17103 , URL = "http://www.nber.org/papers/w17103", abstract =

    2011 doi

  7. [19]

    The Review of Financial Studies , volume =

    Meta Brown and John Grigsby and Wilbert van der Klaauw and Jaya Wen and Basit Zafar , title =. The Review of Financial Studies , volume =. 2016 , doi =

    2016

  8. [20]

    2025 , month =

    Quarterly Report on Household Debt and Credit , institution =. 2025 , month =

    2025

  9. [21]

    Credit Card Agreements Database , year =

  10. [22]

    Flesch–Kincaid readability tests --- Wikipedia , The Free Encyclopedia

    Wikipedia contributors. Flesch–Kincaid readability tests --- Wikipedia , The Free Encyclopedia. 2025

    2025

  11. [23]

    2016 , month =

    Daniel Schorn , title =. 2016 , month =

    2016

  12. [24]

    and Korkou, Eleni D

    Giannikos, Christos L. and Korkou, Eleni D. , title =. International Journal of Financial Studies , volume =. 2025 , publisher =. doi:10.3390/ijfs13010022 , url =

    2025 doi

  13. [25]

    2024 , month =

    Action needed to address gaps in financial literacy among students , howpublished =. 2024 , month =

    2024

  14. [26]

    Can AI help with your personal finances? , url =

    Oudom Hean and Utsha Saha and Binita Saha and , doi =. Can AI help with your personal finances? , url =. 2025 , bdsk-url-1 =. https://doi.org/10.1080/00036846.2025.2450384 , journal =

    2025

  15. [27]

    Optimized Financial Planning: Integrating Individual and Cooperative Budgeting Models with LLM Recommendations , url =

    de Zarz. Optimized Financial Planning: Integrating Individual and Cooperative Budgeting Models with LLM Recommendations , url =. AI , number =. 2024 , bdsk-url-1 =. doi:10.3390/ai5010006 , issn =

    2024 doi

  16. [29]

    2023 , eprint=

    FinanceBench: A New Benchmark for Financial Question Answering , author=. 2023 , eprint=

    2023

  17. [30]

    TAT - QA : A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance

    Zhu, Fengbin and Lei, Wenqiang and Huang, Youcheng and Wang, Chao and Zhang, Shuo and Lv, Jiancheng and Feng, Fuli and Chua, Tat-Seng. TAT - QA : A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance. Proceedings of the 59th Annual Meeting of the...

    2021 doi

  18. [32]

    2024 , organization =

    The Personal Finance Index (P-Fin Index) , author =. 2024 , organization =

    2024

  19. [33]

    2024 , month = apr, organization =

    Can you answer these 3 questions about your finances? The majority of US adults cannot , author =. 2024 , month = apr, organization =

    2024

  20. [34]

    Market Share by Credit Card Issuer , year =

  21. [35]

    D oc M ath-Eval: Evaluating Math Reasoning Capabilities of LLM s in Understanding Long and Specialized Documents

    Zhao, Yilun and Long, Yitao and Liu, Hongjun and Kamoi, Ryo and Nan, Linyong and Chen, Lyuhao and Liu, Yixin and Tang, Xiangru and Zhang, Rui and Cohan, Arman. D oc M ath-Eval: Evaluating Math Reasoning Capabilities of LLM s in Understanding Long and Specialized Documents. Pro...

    2024 doi

  22. [36]

    2022 , organization =

    Do financial literacy interventions actually work better than I think they do? (and thoughts about meta-analyses) , author =. 2022 , organization =

    2022

  23. [37]

    2024 , month = nov, journal =

    Gen Z, Millennials Are Using AI for Personal Finance Advice, Report Finds , author =. 2024 , month = nov, journal =

    2024

  24. [38]

    2025 , month = may, organization =

    When Does Financial Education Work Best? Evidence-Based Insights , author =. 2025 , month = may, organization =

    2025

  25. [39]

    2025 , month = nov, journal =

    More than Half of People Use AI as a `Financial Adviser' , author =. 2025 , month = nov, journal =

    2025

  26. [40]

    http://www.nber.org/papers/w34255

    Chatterji, Aaron and Cunningham, Thomas and Deming, David J and Hitzig, Zoe and Ong, Christopher and Shan, Carl Yan and Wadman, Kevin. How People Use ChatGPT. 2025. doi:10.3386/w34255 , URL = "http://www.nber.org/papers/w34255", abstract =

    2025 doi

  27. [41]

    Redesigning Financial Education to Engage and Entertain Audiences Is Delivering Results , year =

  28. [42]

    http://www.nber.org/papers/w27057

    Kaiser, Tim and Lusardi, Annamaria and Menkhoff, Lukas and Urban, Carly J. Financial Education Affects Financial Knowledge and Downstream Behaviors. 2020. doi:10.3386/w27057 , URL = "http://www.nber.org/papers/w27057", abstract =

    2020 doi

  29. [43]

    Kilts Center at Chicago Booth Marketing Data Center Paper , year=

    Gender-Based Sorting in the Credit Card Market , author=. Kilts Center at Chicago Booth Marketing Data Center Paper , year=

  30. [44]

    2025 , eprint=

    MultiFinBen: A Multilingual, Multimodal, and Difficulty-Aware Benchmark for Financial LLM Evaluation , author=. 2025 , eprint=

    2025

  31. [45]

    F in DV er: Explainable Claim Verification over Long and Hybrid-content Financial Documents

    Zhao, Yilun and Long, Yitao and Jiang, Tintin and Wang, Chengye and Chen, Weiyuan and Liu, Hongjun and Tang, Xiangru and Zhang, Yiming and Zhao, Chen and Cohan, Arman. F in DV er: Explainable Claim Verification over Long and Hybrid-content Financial Documents. Proceedings of t...

    2024 doi

  32. [47]

    Financial Language Model Evaluation ( FL a ME )

    Matlin, Glenn and Okamoto, Mika and Pardawala, Huzaifa and Yang, Yang and Chava, Sudheer. Financial Language Model Evaluation ( FL a ME ). Findings of the Association for Computational Linguistics: ACL 2025. 2025. doi:10.18653/v1/2025.findings-acl.1164

    2025 doi

  33. [48]

    F inance R easoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging

    Tang, Zichen and E, Haihong and Ma, Ziyan and He, Haoyang and Liu, Jiacheng and Yang, Zhongjun and Rong, Zihua and Li, Rongjin and Ji, Kun and Huang, Qing and Hu, Xinyang and Liu, Yang and Zheng, Qianhe. F inance R easoning: Benchmarking Financial Numerical Reasoning More Cred...

    2025 doi

  34. [49]

    Wang and Kaylie Hausknecht and Jonah Brenner and Danxian Liu and Nianli Peng and Corey Wang and Michael Brenner , booktitle=

    Jingxuan Fan and Sarah Martinson and Erik Y. Wang and Kaylie Hausknecht and Jonah Brenner and Danxian Liu and Nianli Peng and Corey Wang and Michael Brenner , booktitle=. 2024 , url=

    2024

  35. [50]

    FOLIO : Natural Language Reasoning with First-Order Logic

    Han, Simeng and Schoelkopf, Hailey and Zhao, Yilun and Qi, Zhenting and Riddell, Martin and Zhou, Wenfei and Coady, James and Peng, David and Qiao, Yujie and Benson, Luke and Sun, Lucy and Wardle-Solano, Alexander and Szab \'o , Hannah and Zubova, Ekaterina and Burtell, Matthe...

    2024

  36. [51]

    The Thirteenth International Conference on Learning Representations , year=

    Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation , author=. The Thirteenth International Conference on Learning Representations , year=

  37. [52]

    The 4th Workshop on Mathematical Reasoning and AI at NeurIPS'24 , year=

    How Transformers Reason: A Case Study on a Synthetic Propositional Logic Problem , author=. The 4th Workshop on Mathematical Reasoning and AI at NeurIPS'24 , year=

  38. [53]

    We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?

    Qiao, Runqi and Tan, Qiuna and Dong, Guanting and MinhuiWu, MinhuiWu and Sun, Chong and Song, Xiaoshuai and Wang, Jiapeng and GongQue, Zhuoma and Lei, Shanglin and Zhang, YiFan and Wei, Zhe and Zhang, Miaoxuan and Qiao, Runfeng and Zong, Xiao and Xu, Yida and Yang, Peiqing and...

    2025

  39. [54]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  40. [55]

    Publications Manual , year = "1983", publisher =

    1983

  41. [56]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    1981

  42. [57]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  43. [58]

    Dan Gusfield , title =. 1997

    1997

  44. [59]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  45. [60]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  46. [61]

    2024 , eprint=

    Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correction , author=. 2024 , eprint=

    2024

  47. [62]

    2025 , eprint=

    FLAMES: Improving LLM Math Reasoning via a Fine-Grained Analysis of the Data Synthesis Pipeline , author=. 2025 , eprint=

    2025

  48. [63]

    2025 , eprint=

    A Toolbox, Not a Hammer -- Multi-TAG: Scaling Math Reasoning with Multi-Tool Aggregation , author=. 2025 , eprint=

    2025

  49. [64]

    2025 , eprint=

    Constructive Circuit Amplification: Improving Math Reasoning in LLMs via Targeted Sub-Network Updates , author=. 2025 , eprint=

    2025

  50. [65]

    2025 , eprint=

    Structured Reasoning with Tree-of-Thoughts for Bengali Math Word Problems , author=. 2025 , eprint=

    2025

  51. [66]

    2025 , eprint=

    DivLogicEval: A Framework for Benchmarking Logical Reasoning Evaluation in Large Language Models , author=. 2025 , eprint=

    2025

  52. [67]

    2025 , eprint=

    Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Study , author=. 2025 , eprint=

    2025

  53. [68]

    2025 , url=

    Anonymous , booktitle=. 2025 , url=

    2025

  54. [69]

    2025 , eprint=

    TabDSR: Decompose, Sanitize, and Reason for Complex Numerical Reasoning in Tabular Data , author=. 2025 , eprint=

    2025

  55. [70]

    2025 , eprint=

    NUMINA: A Natural Understanding Benchmark for Multi-dimensional Intelligence and Numerical Reasoning Abilities , author=. 2025 , eprint=

    2025

  56. [71]

    2025 , eprint=

    MDToC: Metacognitive Dynamic Tree of Concepts for Boosting Mathematical Problem-Solving of Large Language Models , author=. 2025 , eprint=

    2025

  57. [72]

    2025 , eprint=

    Can Large Reasoning Models Improve Accuracy on Mathematical Tasks Using Flawed Thinking? , author=. 2025 , eprint=

    2025

  58. [73]

    2025 , eprint=

    DAG-Math: Graph-Guided Mathematical Reasoning in LLMs , author=. 2025 , eprint=

    2025

  59. [74]

    2025 , eprint=

    Ax-Prover: A Deep Reasoning Agentic Framework for Theorem Proving in Mathematics and Quantum Physics , author=. 2025 , eprint=

    2025

  60. [75]

    2025 , eprint=

    Population-Evolve: a Parallel Sampling and Evolutionary Method for LLM Math Reasoning , author=. 2025 , eprint=

    2025

  61. [76]

    2025 , eprint=

    Do Math Reasoning LLMs Help Predict the Impact of Public Transit Events? , author=. 2025 , eprint=

    2025

  62. [77]

    2025 , eprint=

    Rectify Evaluation Preference: Improving LLMs' Critique on Math Reasoning via Perplexity-aware Reinforcement Learning , author=. 2025 , eprint=

    2025

  63. [78]

    ArXiv , year=

    Fill in the Blank: Exploring and Enhancing LLM Capabilities for Backward Reasoning in Math Word Problems , author=. ArXiv , year=

  64. [79]

    ArXiv , year=

    MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems , author=. ArXiv , year=

  65. [80]

    Conference on Empirical Methods in Natural Language Processing , year=

    Fewer is More: Boosting Math Reasoning with Reinforced Context Pruning , author=. Conference on Empirical Methods in Natural Language Processing , year=

  66. [81]

    ArXiv , year=

    Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification , author=. ArXiv , year=

  67. [82]

    ArXiv , year=

    Scheherazade: Evaluating Chain-of-Thought Math Reasoning in LLMs with Chain-of-Problems , author=. ArXiv , year=

  68. [83]

    ArXiv , year=

    Arithmetic Reasoning with LLM: Prolog Generation & Permutation , author=. ArXiv , year=

  69. [84]

    ArXiv , year=

    Confucius3-Math: A Lightweight High-Performance Reasoning LLM for Chinese K-12 Mathematics Learning , author=. ArXiv , year=

  70. [85]

    ArXiv , year=

    AgenticMath: Enhancing LLM Reasoning via Agentic-based Math Data Generation , author=. ArXiv , year=

  71. [86]

    ArXiv , year=

    LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts , author=. ArXiv , year=

  72. [87]

    ArXiv , year=

    From Ambiguity to Verdict: A Semiotic-Grounded Multi-Perspective Agent for LLM Logical Reasoning , author=. ArXiv , year=

  73. [88]

    Conference on Empirical Methods in Natural Language Processing , year=

    Verifiable, Debuggable, and Repairable Commonsense Logical Reasoning via LLM-based Theory Resolution , author=. Conference on Empirical Methods in Natural Language Processing , year=

  74. [89]

    ArXiv , year=

    Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning , author=. ArXiv , year=

  75. [90]

    IEEE Transactions on Automation Science and Engineering , year=

    LogiCode: An LLM-Driven Framework for Logical Anomaly Detection , author=. IEEE Transactions on Automation Science and Engineering , year=

  76. [91]

    Electronics , year=

    CRP-RAG: A Retrieval-Augmented Generation Framework for Supporting Complex Logical Reasoning and Knowledge Planning , author=. Electronics , year=

  77. [92]

    ArXiv , year=

    Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning , author=. ArXiv , year=

  78. [93]

    ArXiv , year=

    Complex Logical Reasoning over Knowledge Graphs using Large Language Models , author=. ArXiv , year=

  79. [94]

    2024 IEEE International Conference on Big Data and Smart Computing (BigComp) , year=

    Enhancing Numerical Reasoning Performance by Augmenting Distractor Numerical Values , author=. 2024 IEEE International Conference on Big Data and Smart Computing (BigComp) , year=

    2024

  80. [95]

    Annual Meeting of the Association for Computational Linguistics , year=

    MathPrompter: Mathematical Reasoning using Large Language Models , author=. Annual Meeting of the Association for Computational Linguistics , year=

  81. [96]

    The 3rd Workshop on Mathematical Reasoning and AI at NeurIPS'23 , year=

    Augmenting Large Language Models with Symbolic Rule Learning for Robust Numerical Reasoning , author=. The 3rd Workshop on Mathematical Reasoning and AI at NeurIPS'23 , year=

  82. [97]

    North American Chapter of the Association for Computational Linguistics , year=

    QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering , author=. North American Chapter of the Association for Computational Linguistics , year=

  83. [98]

    ArXiv , year=

    From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations , author=. ArXiv , year=

  84. [99]

    2025 , eprint=

    Adaptive Selection of Symbolic Languages for Improving LLM Logical Reasoning , author=. 2025 , eprint=

    2025

  85. [100]

    2024 , eprint=

    DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy , author=. 2024 , eprint=

    2024

  86. [101]

    Knowledge and Information Systems , year=

    Targeted training for numerical reasoning with large language models , author=. Knowledge and Information Systems , year=

  87. [102]

    ArXiv , year=

    Large Language Models in Numberland: A Quick Test of Their Numerical Reasoning Abilities , author=. ArXiv , year=

  88. [103]

    ArXiv , year=

    Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with Minimalist Rule-Based RL , author=. ArXiv , year=

  89. [104]

    Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security , year=

    Oedipus: LLM-enchanced Reasoning CAPTCHA Solver , author=. Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security , year=

    2025

  90. [105]

    ArXiv , year=

    JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capability , author=. ArXiv , year=

  91. [106]

    ArXiv , year=

    Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operations , author=. ArXiv , year=

  92. [107]

    2025 , eprint=

    Multi-LLM Collaboration for Medication Recommendation , author=. 2025 , eprint=

    2025

  93. [108]

    ArXiv , year=

    A Fragile Number Sense: Probing the Elemental Limits of Numerical Reasoning in LLMs , author=. ArXiv , year=

  94. [109]

    North American Chapter of the Association for Computational Linguistics , year=

    TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning , author=. North American Chapter of the Association for Computational Linguistics , year=

  95. [110]

    2025 , eprint=

    Gemini: A Family of Highly Capable Multimodal Models , author=. 2025 , eprint=

    2025

  96. [111]

    2025 , eprint=

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning , author=. 2025 , eprint=

    2025

  97. [112]

    QwQ-32B: Embracing the Power of Reinforcement Learning , url =

    Qwen Team , month =. QwQ-32B: Embracing the Power of Reinforcement Learning , url =

  98. [115]

    2024 , eprint=

    The Llama 3 Herd of Models , author=. 2024 , eprint=

    2024

  99. [117]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  100. [118]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    s1: Simple test-time scaling , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  101. [120]

    gpt-oss-120b & gpt-oss-20b model card

    Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925, 2025

    2025 arXiv

  102. [121]

    Taylor, Frank H

    Benjamin Breen, Marco Del Tredici, Jacob McCarran, Javier Aspuru Mijares, Weichen Winston Yin, Kfir Sulimany, Jacob M. Taylor, Frank H. L. Koppens, and Dirk Englund. Ax-prover: A deep reasoning agentic framework for theorem proving in mathematics and quantum physics, 2025. URL...

    2025 arXiv

  103. [122]

    Financial education and the debt behavior of the young

    Meta Brown, John Grigsby, Wilbert van der Klaauw, Jaya Wen, and Basit Zafar. Financial education and the debt behavior of the young. The Review of Financial Studies, 29 0 (9): 0 2490--2522, 2016. ISSN 0893-9454. doi:10.1093/rfs/hhw006. URL https://doi.org/10.1093/rfs/hhw006

    2016 doi

  104. [123]

    How people use chatgpt

    Aaron Chatterji, Thomas Cunningham, David J Deming, Zoe Hitzig, Christopher Ong, Carl Yan Shan, and Kevin Wadman. How people use chatgpt. Working Paper 34255, National Bureau of Economic Research, September 2025. URL http://www.nber.org/papers/w34255

    2025

  105. [124]

    Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=YfZ4ZPt8zd

    2023

  106. [125]

    F in QA : A dataset of numerical reasoning over financial data

    Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. F in QA : A dataset of numerical reasoning over financial data. In Marie-Francine Moens, Xuanjing Huang, Lucia Sp...

    2021

  107. [126]

    C onv F in QA : Exploring the chain of numerical reasoning in conversational finance question answering

    Zhiyu Chen, Shiyang Li, Charese Smiley, Zhiqiang Ma, Sameena Shah, and William Yang Wang. C onv F in QA : Exploring the chain of numerical reasoning in conversational finance question answering. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022...

    2022 doi

  108. [127]

    Nurendra Choudhary and Chandan K. Reddy. Complex logical reasoning over knowledge graphs using large language models. ArXiv, abs/2305.01157, 2023. URL https://api.semanticscholar.org/CorpusID:258436828

    2023 arXiv

  109. [128]

    Credit card agreements database, 2024

    Consumer Financial Protection Bureau . Credit card agreements database, 2024. URL https://www.consumerfinance.gov/credit-cards/agreements/

    2024

  110. [129]

    Selection-inference: Exploiting large language models for interpretable logical reasoning

    Antonia Creswell, Murray Shanahan, and Irina Higgins. Selection-inference: Exploiting large language models for interpretable logical reasoning. ArXiv, abs/2205.09712, 2022. URL https://api.semanticscholar.org/CorpusID:248887351

    2022 arXiv

  111. [130]

    Fill in the blank: Exploring and enhancing llm capabilities for backward reasoning in math word problems

    Aniruddha Deb, Neeva Oza, Sarthak Singla, Dinesh Khandelwal, Dinesh Garg, and Parag Singla. Fill in the blank: Exploring and enhancing llm capabilities for backward reasoning in math word problems. ArXiv, abs/2310.01991, 2023. URL https://api.semanticscholar.org/CorpusID:263609337

    2023 arXiv

  112. [131]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948

    2025 arXiv

  113. [132]

    Oedipus: Llm-enchanced reasoning captcha solver

    Gelei Deng, Haoran Ou, Yi Liu, Jie Zhang, Tianwei Zhang, and Yang Liu. Oedipus: Llm-enchanced reasoning captcha solver. Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, 2024. URL https://api.semanticscholar.org/CorpusID:269758101

    2025

  114. [133]

    Wang, Kaylie Hausknecht, Jonah Brenner, Danxian Liu, Nianli Peng, Corey Wang, and Michael Brenner

    Jingxuan Fan, Sarah Martinson, Erik Y. Wang, Kaylie Hausknecht, Jonah Brenner, Danxian Liu, Nianli Peng, Corey Wang, and Michael Brenner. HARDMATH : A benchmark dataset for challenging problems in applied mathematics. In The 4th Workshop on Mathematical Reasoning and AI at Neu...

    2024

  115. [134]

    Gender-based sorting in the credit card market

    Rohan Ganduri, Nikhil Paradkar, and Yafei Zhang. Gender-based sorting in the credit card market. Kilts Center at Chicago Booth Marketing Data Center Paper, 2023

    2023

  116. [135]

    Gemini: A family of highly capable multimodal models, 2025

    Gemini Team et al. Gemini: A family of highly capable multimodal models, 2025. URL https://arxiv.org/abs/2312.11805

    2025 arXiv

  117. [136]

    The personal finance index (p-fin index), 2024

    Global Financial Literacy Excellence Center (GFLEC) . The personal finance index (p-fin index), 2024. URL https://gflec.org/initiatives/personal-finance-index/

    2024

  118. [137]

    Bowman, and Evan Hubinger

    Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, Akbir Khan, Julian Michael, Sören Mindermann, Ethan Perez, Linda Petrini, Jonathan Uesato, Jared Kaplan, Buck Shlegeris, Sam...

    2024 arXiv

  119. [138]

    Putnam- AXIOM : A functional and static benchmark for measuring higher level mathematical reasoning

    Aryan Gulati, Brando Miranda, Eric Chen, Emily Xia, Kai Fronsdal, Bruno de Moraes Dumont, and Sanmi Koyejo. Putnam- AXIOM : A functional and static benchmark for measuring higher level mathematical reasoning. In The 4th Workshop on Mathematical Reasoning and AI at NeurIPS'24, ...

    2024

  120. [139]

    FOLIO : Natural language reasoning with first-order logic

    Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szab \'o , Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, A...

    2024

  121. [140]

    C o C o HD : Congress committee hearing dataset

    Arnav Hiray, Yunsong Liu, Mingxiao Song, Agam Shah, and Sudheer Chava. C o C o HD : Congress committee hearing dataset. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp.\ 15529--15542, Miami,...

    2024 doi

  122. [141]

    Benchmarking llms' mathematical reasoning with unseen random variables questions, 2025

    Zijin Hong, Hao Wu, Su Dong, Junnan Dong, Yilin Xiao, Yujing Zhang, Zhu Wang, Feiran Huang, Linyi Li, Hongxia Yang, and Xiao Huang. Benchmarking llms' mathematical reasoning with unseen random variables questions, 2025. URL https://arxiv.org/abs/2501.11790

    2025 arXiv

  123. [142]

    More than half of people use ai as a `financial adviser', November 2025

    Patrick Hosking. More than half of people use ai as a `financial adviser', November 2025. URL https://www.thetimes.com/business/technology/article/more-than-half-of-people-use-ai-as-financial-adviser-k68htgkn8. Financial Editor reporting on a Lloyds Banking Group–commissioned study

    2025

  124. [143]

    Towards reasoning in large language models: A survey

    Jie Huang and Kevin Chen-Chuan Chang. Towards reasoning in large language models: A survey. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp.\ 1049--1065, Toronto, Canada, July 2023. Associat...

    2023 doi

  125. [144]

    MATH -perturb: Benchmarking LLM s' math reasoning abilities against hard perturbations

    Kaixuan Huang, Jiacheng Guo, Zihao Li, Xiang Ji, Jiawei Ge, Wenzhe Li, Yingqing Guo, Tianle Cai, Hui Yuan, Runzhe Wang, Yue Wu, Ming Yin, Shange Tang, Yangsibo Huang, Chi Jin, Xinyun Chen, Chiyuan Zhang, and Mengdi Wang. MATH -perturb: Benchmarking LLM s' math reasoning abilit...

    2025

  126. [145]

    Fewer is more: Boosting math reasoning with reinforced context pruning

    Xijie Huang, Li Lyna Zhang, Kwang-Ting Cheng, Fan Yang, and Mao Yang. Fewer is more: Boosting math reasoning with reinforced context pruning. In Conference on Empirical Methods in Natural Language Processing, 2024. URL https://api.semanticscholar.org/CorpusID:273901230

    2024

  127. [146]

    Enhancing numerical reasoning performance by augmenting distractor numerical values

    Yechan Hwang, Jinsu Lim, Young-Jun Lee, and Ho-Jin Choi. Enhancing numerical reasoning performance by augmenting distractor numerical values. 2024 IEEE International Conference on Big Data and Smart Computing (BigComp), pp.\ 242--245, 2024. URL https://api.semanticscholar.org/...

    2024

  128. [147]

    Shrivastava

    Shima Imani, Liang Du, and H. Shrivastava. Mathprompter: Mathematical reasoning using large language models. In Annual Meeting of the Association for Computational Linguistics, 2023. URL https://api.semanticscholar.org/CorpusID:257427208

    2023

  129. [148]

    Financebench: A new benchmark for financial question answering, 2023

    Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. Financebench: A new benchmark for financial question answering, 2023

    2023

  130. [149]

    Livecodebench: Holistic and contamination free evaluation of large language models for code

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024

    2024 arXiv

  131. [150]

    Financial education affects financial knowledge and downstream behaviors

    Tim Kaiser, Annamaria Lusardi, Lukas Menkhoff, and Carly J Urban. Financial education affects financial knowledge and downstream behaviors. Working Paper 27057, National Bureau of Economic Research, April 2020. URL http://www.nber.org/papers/w27057

    2020

  132. [151]

    B iz B ench: A quantitative reasoning benchmark for business and finance

    Michael Krumdick, Rik Koncel-Kedziorski, Viet Dac Lai, Varshini Reddy, Charles Lovering, and Chris Tanner. B iz B ench: A quantitative reasoning benchmark for business and finance. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting ...

    2024 doi

  133. [152]

    Beyond chemical qa: Evaluating llm's chemical reasoning with modular chemical operations

    Hao Li, He Cao, Bin Feng, Yanjun Shao, Xiangru Tang, Zhiyuan Yan, Li Yuan, Yonghong Tian, and Yu Li. Beyond chemical qa: Evaluating llm's chemical reasoning with modular chemical operations. ArXiv, abs/2505.21318, 2025. URL https://api.semanticscholar.org/CorpusID:278911005

    2025

  134. [153]

    Targeted training for numerical reasoning with large language models

    Xiao Li, Sichen Liu, Yin Zhu, and Gong Cheng. Targeted training for numerical reasoning with large language models. Knowledge and Information Systems, 67: 0 197 -- 221, 2024. URL https://api.semanticscholar.org/CorpusID:272453829

    2024

  135. [154]

    Improving llm reasoning through scaling inference computation with collaborative verification

    Zhenwen Liang, Ye Liu, Tong Niu, Xiangliang Zhang, Yingbo Zhou, and Semih Yavuz. Improving llm reasoning through scaling inference computation with collaborative verification. ArXiv, abs/2410.05318, 2024. URL https://api.semanticscholar.org/CorpusID:273229503

    2024 arXiv

  136. [155]

    Beyond distillation: Pushing the limits of medical llm reasoning with minimalist rule-based rl

    Che Liu, Haozhe Wang, Jiazhen Pan, Zhongwei Wan, Yong Dai, Fangzhen Lin, Wenjia Bai, Daniel Rueckert, and Rossella Arcucci. Beyond distillation: Pushing the limits of medical llm reasoning with minimalist rule-based rl. ArXiv, abs/2505.17952, 2025 a . URL https://api.semantics...

    2025 arXiv

  137. [156]

    Agenticmath: Enhancing llm reasoning via agentic-based math data generation

    Xianyang Liu, Yilin Liu, Shuai Wang, Hao Cheng, Andrew Estornell, Yuzhi Zhao, and Jiaheng Wei. Agenticmath: Enhancing llm reasoning via agentic-based math data generation. ArXiv, abs/2510.19361, 2025 b . URL https://api.semanticscholar.org/CorpusID:282272499

    2025

  138. [157]

    Dafnybench: A benchmark for formal software verification

    Chloe R Loughridge, Qinyi Sun, Seth Ahrenbach, Federico Cassano, Chuyue Sun, Ying Sheng, Anish Mudide, Md Rakib Hossain Misu, Nada Amin, and Max Tegmark. Dafnybench: A benchmark for formal software verification. In The 4th Workshop on Mathematical Reasoning and AI at NeurIPS'2...

    2024

  139. [158]

    Tart: An open-source tool-augmented framework for explainable table-based reasoning

    Xinyuan Lu, Liangming Pan, Yubo Ma, Preslav Nakov, and Min-Yen Kan. Tart: An open-source tool-augmented framework for explainable table-based reasoning. In North American Chapter of the Association for Computational Linguistics, 2024. URL https://api.semanticscholar.org/Corpus...

    2024

  140. [159]

    Americans' financial capability

    Annamaria Lusardi. Americans' financial capability. Working Paper 17103, National Bureau of Economic Research, June 2011. URL http://www.nber.org/papers/w17103

    2011

  141. [160]

    Financial literacy and retirement planning in the united states

    Annamaria Lusardi and Olivia S Mitchell. Financial literacy and retirement planning in the united states. Working Paper 17108, National Bureau of Economic Research, June 2011. URL http://www.nber.org/papers/w17108

    2011

  142. [161]

    Mathdial: A dialogue tutoring dataset with rich pedagogical properties grounded in math reasoning problems

    Jakub Macina, Nico Daheim, Sankalan Pal Chowdhury, Tanmay Sinha, Manu Kapur, Iryna Gurevych, and Mrinmaya Sachan. Mathdial: A dialogue tutoring dataset with rich pedagogical properties grounded in math reasoning problems. ArXiv, abs/2305.14536, 2023. URL https://api.semanticsc...

    2023 arXiv

  143. [162]

    Do financial literacy interventions actually work better than i think they do? (and thoughts about meta-analyses), 2022

    David McKenzie. Do financial literacy interventions actually work better than i think they do? (and thoughts about meta-analyses), 2022. URL https://blogs.worldbank.org/en/impactevaluations/do-financial-literacy-interventions-actually-work-better-i-think-they-do-and. World Bank Blogs

    2022

  144. [163]

    Can you answer these 3 questions about your finances? the majority of us adults cannot, April 2024

    Michelle Meineke. Can you answer these 3 questions about your finances? the majority of us adults cannot, April 2024. URL https://www.weforum.org/stories/2024/04/financial-literacy-money-education/

    2024

  145. [164]

    The llama 3 herd of models, 2024

    Meta et al. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783

    2024 arXiv

  146. [165]

    Scheherazade: Evaluating chain-of-thought math reasoning in llms with chain-of-problems

    Stephen Miner, Yoshiki Takashima, Simeng Han, Ferhat Erata, Timos Antonopoulos, Ruzica Piskac, and Scott J Shapiro. Scheherazade: Evaluating chain-of-thought math reasoning in llms with chain-of-problems. ArXiv, abs/2410.00151, 2024. URL https://api.semanticscholar.org/CorpusI...

    2024 arXiv

  147. [166]

    Mistral Small 3

    Mistral AI Team . Mistral Small 3 . https://mistral.ai/news/mistral-small-3/, Jan 2025. 24B-parameter open-weight language model, Apache 2.0 license

    2025

  148. [167]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand \`e s, and Tatsunori B Hashimoto. s1: Simple test-time scaling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Langu...

    2025

  149. [168]

    Large language models often know when they are being evaluated, 2025

    Joe Needham, Giles Edkins, Govind Pimpale, Henning Bartsch, and Marius Hobbhahn. Large language models often know when they are being evaluated, 2025. URL https://arxiv.org/abs/2505.23836

    2025 arXiv

  150. [169]

    Nguyen, Quang Pham, and Nghi D

    Dung Manh Nguyen, Thang Chau Phan, Nam Le Hai, Tien-Thong Doan, Nam V. Nguyen, Quang Pham, and Nghi D. Q. Bui. Code MMLU : A multi-task benchmark for assessing code understanding & reasoning capabilities of code LLM s. In The Thirteenth International Conference on Learning Rep...

    2025

  151. [170]

    GPT-5 System Card

    OpenAI . GPT-5 System Card . https://cdn.openai.com/gpt-5-system-card.pdf, Aug 2025. [Large language model documentation]

    2025

  152. [171]

    Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning

    Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning. ArXiv, abs/2305.12295, 2023. URL https://api.semanticscholar.org/CorpusID:258833332

    2023 arXiv

  153. [172]

    Multifinben: A multilingual, multimodal, and difficulty-aware benchmark for financial llm evaluation, 2025

    Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, ...

    2025

  154. [173]

    Large language models meet symbolic provers for logical reasoning evaluation

    Chengwen Qi, Ren Ma, Bowen Li, He Du, Binyuan Hui, Jinwang Wu, Yuanjun Laili, and Conghui He. Large language models meet symbolic provers for logical reasoning evaluation. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/...

    2025

  155. [174]

    Runqi Qiao, Qiuna Tan, Guanting Dong, MinhuiWu MinhuiWu, Chong Sun, Xiaoshuai Song, Jiapeng Wang, Zhuoma GongQue, Shanglin Lei, YiFan Zhang, Zhe Wei, Miaoxuan Zhang, Runfeng Qiao, Xiao Zong, Yida Xu, Peiqing Yang, Zhimin Bao, Muxi Diao, Chen Li, and Honggang Zhang. We-math: Do...

    2025

  156. [175]

    Large language models in numberland: A quick test of their numerical reasoning abilities

    Roussel Rahman. Large language models in numberland: A quick test of their numerical reasoning abilities. ArXiv, abs/2504.00226, 2025. URL https://api.semanticscholar.org/CorpusID:277467774

    2025 arXiv

  157. [176]

    A fragile number sense: Probing the elemental limits of numerical reasoning in llms

    Roussel Rahman and Aashwin Ananda Mishra. A fragile number sense: Probing the elemental limits of numerical reasoning in llms. ArXiv, abs/2509.06332, 2025. URL https://api.semanticscholar.org/CorpusID:281203640

    2025 arXiv

  158. [177]

    Multi-llm collaboration for medication recommendation, 2025

    Huascar Sanchez, Briland Hitaj, Jules Bergmann, and Linda Briesemeister. Multi-llm collaboration for medication recommendation, 2025. URL https://arxiv.org/abs/2512.05066

    2025

  159. [178]

    Can't understand your credit card agreement? you're not alone, sep 2016

    Daniel Schorn. Can't understand your credit card agreement? you're not alone, sep 2016. URL https://www.cbsnews.com/news/cant-understand-your-credit-card-agreement-youre-not-alone/. Accessed: 2025-09-24

    2016

  160. [179]

    Zero is not hero yet: Benchmarking zero-shot performance of llms for financial tasks, 2023

    Agam Shah and Sudheer Chava. Zero is not hero yet: Benchmarking zero-shot performance of llms for financial tasks, 2023. URL https://arxiv.org/abs/2305.16633

    2023 arXiv

  161. [180]

    Numerical claim detection in finance: A new financial dataset, weak-supervision model, and market analysis

    Agam Shah, Arnav Hiray, Pratvi Shah, Arkaprabha Banerjee, Anushka Singh, Dheeraj Deepak Eidnani, Sahasra Chava, Bhaskar Chaudhury, and Sudheer Chava. Numerical claim detection in finance: A new financial dataset, weak-supervision model, and market analysis. In Michael Schlicht...

    2024

  162. [181]

    Gen z, millennials are using ai for personal finance advice, report finds, November 2024

    Ana Teresa Solá. Gen z, millennials are using ai for personal finance advice, report finds, November 2024. URL https://www.cnbc.com/2024/11/04/how-to-use-artificial-intelligence-for-personal-finance.html. Reports survey data from Experian on AI use for personal finance

    2024

  163. [182]

    Determlr: Augmenting llm-based logical reasoning from indeterminacy to determinacy, 2024

    Hongda Sun, Weikai Xu, Wei Liu, Jian Luan, Bin Wang, Shuo Shang, Ji-Rong Wen, and Rui Yan. Determlr: Augmenting llm-based logical reasoning from indeterminacy to determinacy, 2024. URL https://arxiv.org/abs/2310.18659

    2024 arXiv

  164. [183]

    F inance R easoning: Benchmarking financial numerical reasoning more credible, comprehensive and challenging

    Zichen Tang, Haihong E, Ziyan Ma, Haoyang He, Jiacheng Liu, Zhongjun Yang, Zihua Rong, Rongjin Li, Kun Ji, Qing Huang, Xinyang Hu, Yang Liu, and Qianhe Zheng. F inance R easoning: Benchmarking financial numerical reasoning more credible, comprehensive and challenging. In Wanxi...

    2025

  165. [184]

    Kimi k2: Open agentic intelligence

    Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534, 2025

    2025 arXiv

  166. [185]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https://qwenlm.github.io/blog/qwq-32b/

    2025

  167. [186]

    Rectify evaluation preference: Improving llms' critique on math reasoning via perplexity-aware reinforcement learning, 2025

    Changyuan Tian, Zhicong Lu, Shuang Qian, Nayu Liu, Peiguang Li, Li Jin, Leiyi Hu, Zhizhao Zeng, Sirui Wang, Ke Zeng, and Zhi Guo. Rectify evaluation preference: Improving llms' critique on math reasoning via perplexity-aware reinforcement learning, 2025. URL https://arxiv.org/...

    2025

  168. [187]

    Market share by credit card issuer

    WalletHub . Market share by credit card issuer. https://wallethub.com/edu/cc/market-share-by-credit-card-issuer/25530, 2024

    2024

  169. [188]

    From scores to steps: Diagnosing and improving llm performance in evidence-based medical calculations

    Benlu Wang, Iris Xia, Yifan Zhang, Junda Wang, Feiyun Ouyang, Shuo Han, Arman Cohan, Hong Yu, and Zonghai Yao. From scores to steps: Diagnosing and improving llm performance in evidence-based medical calculations. ArXiv, abs/2509.16584, 2025 a . URL https://api.semanticscholar...

    2025

  170. [189]

    Jmlr: Joint medical llm and retrieval training for enhancing reasoning and professional question answering capability

    Junda Wang, Zhichao Yang, Zonghai Yao, and Hong Yu. Jmlr: Joint medical llm and retrieval training for enhancing reasoning and professional question answering capability. ArXiv, abs/2402.17887, 2024. URL https://api.semanticscholar.org/CorpusID:268041575

    2024 arXiv

  171. [190]

    Adaptive selection of symbolic languages for improving llm logical reasoning, 2025 b

    Xiangyu Wang, Haocheng Yang, Fengxiang Cheng, and Fenrong Liu. Adaptive selection of symbolic languages for improving llm logical reasoning, 2025 b . URL https://arxiv.org/abs/2510.10703

    2025

  172. [191]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  173. [192]

    Flesch–Kincaid readability tests --- Wikipedia , The Free Encyclopedia

    Wikipedia contributors . Flesch–Kincaid readability tests --- Wikipedia , The Free Encyclopedia . https://en.wikipedia.org/wiki/Flesch [Online; accessed 25-September-2025]

    2025

  174. [193]

    Redesigning financial education to engage and entertain audiences is delivering results

    World Bank . Redesigning financial education to engage and entertain audiences is delivering results. https://www.worldbank.org/en/news/feature/2017/08/02/redesigning-financial-education-to-engage-audiences-is-delivering-results worldbank.org , August 2017. World Bank Feature Story

    2017

  175. [194]

    Confucius3-math: A lightweight high-performance reasoning llm for chinese k-12 mathematics learning

    Lixin Wu, Na Cai, Qiao Cheng, Jiachen Wang, and Yitao Duan. Confucius3-math: A lightweight high-performance reasoning llm for chinese k-12 mathematics learning. ArXiv, abs/2506.18330, 2025. URL https://api.semanticscholar.org/CorpusID:280000222

    2025 arXiv

  176. [195]

    Crp-rag: A retrieval-augmented generation framework for supporting complex logical reasoning and knowledge planning

    Kehan Xu, Kun Zhang, Jingyuan Li, Wei Huang, and Yuanzhuo Wang. Crp-rag: A retrieval-augmented generation framework for supporting complex logical reasoning and knowledge planning. Electronics, 2024. URL https://api.semanticscholar.org/CorpusID:275103348

    2024

  177. [196]

    Lauter, Swarat Chaudhuri, and Dawn Song

    Kaiyu Yang, Gabriel Poesia, Jingxuan He, Wenda Li, Kristin E. Lauter, Swarat Chaudhuri, and Dawn Song. Position: Formal mathematical reasoning a new frontier in AI . In Forty-second International Conference on Machine Learning Position Paper Track, 2025. URL https://openreview...

    2025

  178. [197]

    Arithmetic reasoning with llm: Prolog generation & permutation

    Xiaocheng Yang, Bingsen Chen, and Yik-Cheung Tam. Arithmetic reasoning with llm: Prolog generation & permutation. ArXiv, abs/2405.17893, 2024. URL https://api.semanticscholar.org/CorpusID:270067866

    2024 arXiv

  179. [198]

    Qa-gnn: Reasoning with language models and knowledge graphs for question answering

    Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. Qa-gnn: Reasoning with language models and knowledge graphs for question answering. In North American Chapter of the Association for Computational Linguistics, 2021. URL https://api.semanticschol...

    2021

  180. [199]

    STA - C o T : Structured target-centric agentic chain-of-thought for consistent multi-image geological reasoning

    Beibei Yu, Tao Shen, and Ling Chen. STA - C o T : Structured target-centric agentic chain-of-thought for consistent multi-image geological reasoning. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Findings of the Association for Compu...

    2025 doi

  181. [200]

    Glm-4.5: Agentic, reasoning, and coding (arc) foundation models

    Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, et al. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models. arXiv preprint arXiv:2508.06471, 2025 a

    2025 arXiv

  182. [201]

    Numina: A natural understanding benchmark for multi-dimensional intelligence and numerical reasoning abilities, 2025 b

    Changyu Zeng, Yifan Wang, Zimu Wang, Wei Wang, Zhengni Yang, Muyi Bao, Jiming Xiao, Anh Nguyen, and Yutao Yue. Numina: A natural understanding benchmark for multi-dimensional intelligence and numerical reasoning abilities, 2025 b . URL https://arxiv.org/abs/2509.16656

    2025

  183. [202]

    Population-evolve: a parallel sampling and evolutionary method for llm math reasoning, 2025

    Yanzhi Zhang, Yitong Duan, Zhaoxi Zhang, Jiyan He, and Shuxin Zheng. Population-evolve: a parallel sampling and evolutionary method for llm math reasoning, 2025. URL https://arxiv.org/abs/2512.19081

    2025

  184. [203]

    Logicode: An llm-driven framework for logical anomaly detection

    Yiheng Zhang, Yunkang Cao, Xiaohao Xu, and Weiming Shen. Logicode: An llm-driven framework for logical anomaly detection. IEEE Transactions on Automation Science and Engineering, 22: 0 7712--7723, 2024. URL https://api.semanticscholar.org/CorpusID:270357822

    2024

  185. [204]

    T a PERA : Enhancing faithfulness and interpretability in long-form table QA by content planning and execution-based reasoning

    Yilun Zhao, Lyuhao Chen, Arman Cohan, and Chen Zhao. T a PERA : Enhancing faithfulness and interpretability in long-form table QA by content planning and execution-based reasoning. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting ...

    2024 doi

  186. [205]

    Financemath: Knowledge-intensive math reasoning in finance domains

    Yilun Zhao, Hongjun Liu, Yitao Long, Rui Zhang, Chen Zhao, and Arman Cohan. Financemath: Knowledge-intensive math reasoning in finance domains. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational ...

    2024 doi

  187. [206]

    D oc M ath-eval: Evaluating math reasoning capabilities of LLM s in understanding long and specialized documents

    Yilun Zhao, Yitao Long, Hongjun Liu, Ryo Kamoi, Linyong Nan, Lyuhao Chen, Yixin Liu, Xiangru Tang, Rui Zhang, and Arman Cohan. D oc M ath-eval: Evaluating math reasoning capabilities of LLM s in understanding long and specialized documents. In Lun-Wei Ku, Andre Martins, and Vi...

    2024

  188. [207]

    TAT - QA : A question answering benchmark on a hybrid of tabular and textual content in finance

    Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. TAT - QA : A question answering benchmark on a hybrid of tabular and textual content in finance. In Proceedings of the 59th Annual Meeting of the Association for Compu...

    2021 doi

Pith tools

Reviewed July 30, 2026 · model on record in the stance chip above.