Pith. sign in

REVIEW 5 major objections 6 minor 30 references

Towards Automated Regulatory Compliance Verification in Financial Auditing with Large Language Models

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that an open-source 70-billion-parameter model, Llama-2-70b, identifies non-compliant passages in English IFRS reports more reliably than proprietary GPT models, while GPT-4 remains the strongest overall model across…

desk verdict Clean incremental evaluation with a useful prompt study, but the headline Llama-2 'No'-class claim does not survive arithmetic: Table IV conflicts with Table I and an always-'No' baseline on the same distribution scores higher. read the letter →

arxiv 2507.16642 v1 pith:PIJXVB34 submitted 2025-07-22 cs.CL cs.AIcs.LG

Armin Berger , Lars Hillebrand , David Leonhard , Tobias Deu{ss}er , Thiago Bell Felix de Oliveira , Tim Dilmaghani , Mohamed Khaled , Bernd Kliem
show 3 more authors
Rüdiger Loitz Christian Bauckhage Rafet Sifa
This is my paper
classification cs.CLcs.AIcs.LG
keywords largelanguagemodelsfinancialauditingregulatorycomplianceIFRSHGBpromptengineeringLlama-2GPT-4
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether off-the-shelf large language models can take the next step in automated auditing: not just retrieving relevant text passages for a legal requirement, but judging whether those passages actually comply with it. On two auditor-annotated datasets, 100 English IFRS requirement-passage pairs and 120 German HGB pairs, the authors ran six LLMs under eight prompt designs. They find that the open-source Llama-2-70b model detects non-compliance ('No' verdicts) on the IFRS data better than any proprietary model, with a class-F1 of 87.50%, while GPT-4 shows the most consistent overall performance across datasets and prompts. The result matters because a competitive open-source compliance checker could let audit firms process sensitive financial data locally, although the authors caution that these off-the-shelf models are not yet reliable enough for real deployment.

What carries the argument

The load-bearing component is a compliance-check prompt attached to an existing retrieval pipeline: given a legal requirement and the top five text passages returned by a recommender, the LLM must return a verdict ('yes', 'no', 'unclear', or 'not applicable'). The authors test three open-source Llama-2 sizes (7b, 13b, 70b) and three proprietary GPT variants under eight hand-written prompts that vary task phrasing (direct answer, chain-of-thought, tree-of-thought, one-shot examples with either compliant or non-compliant exemplars) and response format (closed JSON vs open-ended explanation). The decisive quantity is per-class F1, with the 'No' class treated as the operational target because a false 'yes' would wrongly certify a non-compliant passage as compliant.

What would settle it

Conduct a pre-registered replication: fix Prompt VI in advance, run Llama-2-70b and GPT-4 on a fresh or held-out set of IFRS requirement-passage pairs with auditor ground truth, and compare 'No'-class F1; if GPT-4 matches or beats 87.50% while Llama-2 falls below, the open-source advantage fails. Independently re-annotate the 100 IFRS pairs by a second team of auditors; substantial disagreement on the 'No' labels would make the reported F1 unstable.

Watch

Extended reading notes

Core claim

The paper's central claim is that an open-weights 70-billion-parameter model can outperform closed commercial models on the auditing subtask that practitioners care most about: correctly spotting text passages that do NOT comply with a regulatory requirement. On the IFRS dataset, Llama-2-70b combined with a tree-of-thought plus one-shot prompt reached an F1 score of 87.50% for the 'No' class (precision 80.21%, recall 96.25%), ahead of every proprietary model the authors tested. The paper also reports that GPT-4 is the strongest model overall on micro F1 across both datasets, and that all models, especially Llama-2, perform poorly on German HGB data, which the authors attribute to English-dominated training corpora. The authors conclude that out-of-the-box LLMs cannot yet be trusted for reliable compliance assessment, despite the strong true-negative result.

Load-bearing premise

The claim that Llama-2-70b beats all proprietary models at detecting non-compliance rests on choosing the best of eight prompts after looking at the same 100 IFRS test samples, with no held-out validation; if the prompt had been fixed in advance, or a different prompt chosen, the ranking of models on the 'No' class could change.

Editorial extensions

If this is right

  • A capable open-source model for 'No'-class detection could act as a first-pass filter in English IFRS audits, flagging suspicious passages for human review and shrinking the workload.
  • Client confidentiality becomes a practical reason to choose open-source models, since financial reports would not need to be sent to a commercial API.
  • Prompt design is model-specific: the best prompt differs across models and datasets, so a deployed system must fix and validate its prompt per model rather than assume one universal template.
  • Advanced prompting techniques such as chain-of-thought and tree-of-thought did not consistently beat simple brief prompts with one-shot examples, so extra inference cost is not automatically justified.
  • German HGB compliance checking with off-the-shelf LLMs is not reliable enough for automation, particularly for the Llama-2 family.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline F1 is the maximum over eight prompts evaluated on the same 100-sample test set, so a deployment with a single fixed prompt can expect lower performance; pre-registering the prompt or using a separate validation set before the test set would give a less optimistic estimate.
  • With 82 'No' labels out of 100 IFRS samples, a model that leans toward 'No' can achieve high recall on that class; the reported 80.21% precision reduces but does not eliminate the class-imbalance concern.
  • The paper's own caution suggests the natural next experiment is fine-tuning Llama-2-70b on audited compliance pairs and checking whether the true-negative advantage is preserved on a held-out set.
  • The same eight-prompt protocol could be applied to other reporting standards, such as US GAAP or the EU's CSRD, to test whether the open-source advantage generalizes beyond English IFRS text.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper proposes and evaluates an automated regulatory compliance verification step for financial auditing, in which LLMs judge whether text passages retrieved by the prior ALI/ZeroShotALI pipeline comply with IFRS or HGB requirements. The authors compare six LLMs under eight prompt templates on two PwC-annotated datasets (100 IFRS items, 120 HGB items), using per-class precision/recall/F1 and micro/macro averages. They report that GPT-4 generally achieves the best micro F1, that the open-source Llama-2-70b shows 'outstanding' performance on the IFRS 'No' class (F1=87.50%, prompt VI), and that all models degrade markedly on German HGB.

Significance. The practical question is timely: a compliance-checking model that runs locally on open-weight models could reduce cost and data-privacy exposure in auditing. The paper's strengths are the domain-specific datasets with auditor supervision, the breadth of model/prompt configurations, and the explicit discussion of the open-vs-proprietary trade-off. However, the central 'outstanding performance' claim is not supported by the reported numbers: Table IV cannot be reconciled with Table I's class distribution, the reported F1 is below an always-'No' baseline on the same distribution, and the headline value is a best-of-eight prompt selected on the same 100-item test set. The paper's own conclusion (Section V) disclaims reliability. With corrected metrics, added baselines, and proper validation, the comparison could be a useful empirical contribution; in its current form it does not establish the headline.

major comments (5)
  1. [IV-E3, Tables I and IV] Table IV reports for Llama-2-70b on IFRS class 'No' a precision of 80.21%, recall of 96.25%, and F1 of 87.50%. Table I gives the IFRS ground truth as Yes=17, No=82, Unclear=1 (total 100). With 82 actual No items, recall=96.25% implies about 79 true positives, but 79/82=96.34% and 78/82=95.12%, so no integer TP count yields 96.25%. If TP=79, precision 80.21% would require about 19.5 false positives, exceeding the 18 non-No samples available; if TP=78, it requires about 19.25 false positives, also impossible. The reported precision/recall/F1 triple cannot be produced from the stated class distribution under the metrics defined in Section IV-B, and the paper nowhere describes excluding invalid responses from the 'No'-class denominator. The headline claim in Section IV-E3 therefore rests on internally inconsistent numbers.
  2. [IV-C and IV-E3] Even if the numbers were consistent, the reported 'No'-class performance is not shown to exceed a trivial baseline. On the Table I distribution, a classifier that always answers 'No' achieves precision 82/100=82%, recall 100%, and F1 about 90.1%, which dominates the reported 80.21/96.25/87.50 on all three metrics. The paper's baseline section (IV-C) includes only LLM baselines, not an always-majority-class baseline. Because the 'No' class constitutes 82% of the IFRS data, the 'outstanding true-negative detection' claim requires reporting this baseline and, ideally, additional imbalance-robust measures such as Matthews correlation coefficient. Section IV-E3 also states that this performance 'beats all proprietary counterparts' but provides no numeric per-model 'No'-class comparison for the proprietary models; Table IV reports only Llama-2-70b, and Figures 2-3 do not supply the per-prompt values needed to verify the selected maximum.
  3. [Table III vs Table V] Table III, which lists the best-performing prompt per model and dataset based on micro F1, is inconsistent with Table V, the table of micro F1 scores it is supposed to summarize. For example, on IFRS the best Table V value for Llama-2-70b is prompt II (70.04), but Table III lists prompt III (50.69); for Llama-2-13b IFRS the best is VI (65.58), but Table III lists III (22.59); and for GPT-3.5-Turbo-16k IFRS the best is VI (77.58), but Table III lists I (77.21). Similar mismatches occur in the HGB rows. Because Section IV-E2's prompt-consistency conclusion and the 'best prompt' values in Table II depend on Table III, these errors propagate into the paper's central comparison and need to be corrected before the experimental results can be relied upon.
  4. [IV-A, IV-D, IV-E] The reported 87.50 F1 is the best of eight prompt-specific scores for one class on the same 100-item test set, but Section IV-D states that the eight prompts were selected through 'trial and error and qualitative assessment' on that set. There is no held-out validation, no confidence interval, and no multiple-testing correction. The statement in Section IV-A that 'no splitting of the data was required' is therefore not defensible for prompt selection: choosing the best prompt on the test set is a form of model selection. The paper should either fix the prompt configuration before evaluation, use a validation split for prompt selection, or report the results for all prompts with appropriate correction and caveats.
  5. [V and Abstract] The conclusion states that 'we are currently not confident in the ability of out-of-the-box Language Models to reliably assess the compliance of legal requirements' (Section V). This caution directly contradicts the Abstract's claim of 'outstanding performance' by Llama-2-70b in detecting non-compliance, and it is consistent with the numerical problems identified above. The manuscript should either provide corrected, validated evidence for the strong claim or reframe the contribution around the more modest and defensible finding that prompt choice and language strongly affect out-of-the-box LLM compliance checking.
minor comments (6)
  1. [II] There is a typo in 'points in tine' in the related-work discussion; it should be 'points in time'.
  2. [IV-D and Appendix VII-B] The roman numeral VIII is listed in Section IV-D as 'In-Out-One-Shot-No-Template', but the corresponding appendix prompt is titled 'In-Out-Tot-One-Shot-Template' for both IFRS and HGB; the appendix prompt VIII contains no tree-of-thought instruction, so the title and the content disagree.
  3. [IV-E1 and Table II] The sentence stating that Llama-2-70b performed worse overall than Llama-2-7b across both datasets is contradicted by Table II for IFRS (average micro F1 53.02 vs 47.56); the conclusion should be qualified by dataset.
  4. [Prompts V and VI] The English templates contain a duplicated 'is is' in the tree-of-thought instruction, and the German templates contain typos such as 'Expereten' and 'Denkprocess'.
  5. [IV-E3] The definitions of false positive and false negative appear inverted: with 'complying' treated as positive, a false negative is a complying passage predicted as non-complying, not one 'falsely being classified as complying'.
  6. [IV-A footnote] The dataset and code are not released; at minimum, the class-level confusion matrices or per-item counts needed to verify Table IV should be provided in the appendix, since the aggregate metrics are not reproducible from the paper alone.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline 'No'-class result is the selected maximum over eight prompts on the same test set, making it a fitted input presented as a validated performance claim.

  1. fitted input called prediction [Section IV-D and Section IV-E3 (Tables III and IV)]
    "Through a process of trial and error and qualitative assessment, we have selected a total of eight prompts aimed at solving the above-stated task. ... Since this paper assesses the ability of publicly available LLMs and does not train any models on domain-specific data, no splitting of the data was required. ... In our evaluations, we found that the best model for the ‘No’ class in terms of F 1-Score is the open-source Llama-2-70b with a Precision of 80.21%, Recall of 96.25% and an F 1-Score of 87.50% on IFRS data (see Table IV)."

    The reported 87.50% F1 is not an independent estimate of Llama-2-70b's deployable No-class performance; it is the maximum of eight prompt-specific F1 scores in Table IV (prompt VI), all computed on the same 100-sample IFRS test set used for prompt selection. The paper explicitly says no data split was required because no model is trained, but prompt selection is also a form of fitting to the test set, and the reported F1 is the value of the selection objective. The claim 'best model for the No class' is therefore true only by construction as 'largest F1 in the evaluated grid'; presenting it as evidence that Llama-2-70b beats all proprietary counterparts converts a selected maximum into a validated prediction.

full rationale

This paper contains no mathematical derivation chain, so classical self-definitional circularity is absent. The only load-bearing reduction-by-construction step is the prompt-selection procedure: eight prompts were chosen by trial and error and all were scored on the same 100-sample IFRS test set, with no held-out split because no splitting of the data was required. The headline No-class F1 of 87.50% for Llama-2-70b is the maximum over those eight prompt configurations (prompt VI in Table IV), so the claimed best model for the No class is the argmax of the evaluation grid by construction. That makes the reported deployable-performance claim a selected maximum rather than an independent estimate; the abstract's outstanding performance beating all proprietary counterparts is not supported by any corresponding per-model comparison table. The self-citations to ALI/ZeroShotALI are contextual and not load-bearing for the new compliance-check result. Separate arithmetic inconsistencies between Table I and Table IV and the absence of an always-No baseline are correctness risks, not circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities, forces, or conserved quantities; it evaluates existing models on a new task. The only invented 'things' are the prompt set and the best-prompt selection, covered under free parameters.

free parameters (2)
  • Best prompt per model = Prompt VI for Llama-2-70b on IFRS ('In-Out-Tot-One-Shot'), F1 87.50 on 'No' class
    The best-performing prompt per model and dataset is selected from the same eight prompts evaluated on the test data (Tables III, IV). This selection is a free choice that directly inflates the reported performance of each model, especially the Llama-2-70b headline.
  • Prompt set (8 prompts) = Prompts I-VIII
    The eight prompts were chosen through 'trial and error and qualitative assessment' (Section IV-D). The composition of this set is a hand-selected hyperparameter; different prompts would likely change rankings.
assumptions (4)
  • domain assumption PwC auditor annotations of yes/no/unclear/not-applicable are correct and consistent.
    The evaluation treats the auditor labels as gold standard (Section IV-A). If labels are noisy or inconsistent, all F1 scores are unreliable.
  • domain assumption The randomly sampled 100 IFRS and 120 HGB requirement-passage pairs are representative of the full compliance-checking task.
    The paper samples a small subset of a larger corpus (Section IV-A) but does not analyze how the sample reflects requirement difficulty or distribution.
  • domain assumption A heuristic that parses LLM outputs into the four answer classes is a valid measurement of model performance.
    The paper notes that some Llama-2 outputs were unparseable and scored as 0 (Table V note). This assumes the parsing heuristic does not systematically disadvantage certain models.
  • standard math Precision, recall, and F1 as defined in Section IV-B are the appropriate metrics for the auditing use case.
    Standard evaluation metrics; the choice is conventional, though the paper does not justify the unweighted macro/micro choice for the class imbalance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Automated Regulatory Compliance Verification in Financial Auditing with Large Language Models." pith.science (2026). https://pith.science/paper/PIJXVB34

@misc{pith2026250716642,
  author       = {Pith},
  title        = {Pith review of: Towards Automated Regulatory Compliance Verification in Financial Auditing with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PIJXVB34}},
  note         = {Machine review of arXiv:2507.16642}
}
read the original abstract

The auditing of financial documents, historically a labor-intensive process, stands on the precipice of transformation. AI-driven solutions have made inroads into streamlining this process by recommending pertinent text passages from financial reports to align with the legal requirements of accounting standards. However, a glaring limitation remains: these systems commonly fall short in verifying if the recommended excerpts indeed comply with the specific legal mandates. Hence, in this paper, we probe the efficiency of publicly available Large Language Models (LLMs) in the realm of regulatory compliance across different model configurations. We place particular emphasis on comparing cutting-edge open-source LLMs, such as Llama-2, with their proprietary counterparts like OpenAI's GPT models. This comparative analysis leverages two custom datasets provided by our partner PricewaterhouseCoopers (PwC) Germany. We find that the open-source Llama-2 70 billion model demonstrates outstanding performance in detecting non-compliance or true negative occurrences, beating all their proprietary counterparts. Nevertheless, proprietary models such as GPT-4 perform the best in a broad variety of scenarios, particularly in non-English contexts.

Figures

Figures reproduced from arXiv: 2507.16642 by the authors.

Figure 1
Figure 1. Schematic visualization of the complete auditing pipeline combing [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Grouped bar plot of F1-Scores by model and answer choices on IFRS data [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Grouped bar plot of F1-Scores by model and answer choices on HGB data. Note: Due to poor model performance in the German language, some LLMs were incapable of generating any machine-readable consistent outputs that are interpretable with a heuristic for some prompt formats, leading to some F1-Scores being 0. TABLE IV RESULTS FOR LLAMA-2-70B IN % IFRS DATA - CLASS ‘NO’. No Precision Recall F1 I 75.00 41.25 53.23 II 8… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 21 canonical work pages

  1. [1]

    Automatic consistency checking of table and text in financial documents,

    S. M. Ali, T. Deußer, S. Houben, L. Hillebrand, T. Metzler, and R. Sifa, “Automatic consistency checking of table and text in financial documents,” in Proc. NLDL, 2023

    work page 2023

  2. [2]

    Finbert: Financial sentiment analysis with pre-trained language models,

    D. Araci, “Finbert: Financial sentiment analysis with pre-trained language models,” arXiv:1908.10063, 2019

    arXiv 1908

  3. [3]

    A comparison of pre-trained language models for multi-class text classification in the financial domain,

    Y . Arslan, K. Allix, L. Veiber, et al. , “A comparison of pre-trained language models for multi-class text classification in the financial domain,” in Proc. WWW, 2021

    work page 2021

  4. [4]

    Zero-shot text match- ing for automated auditing using sentence transformers,

    D. Biesner, M. Pielka, R. Ramamurthy, et al., “Zero-shot text match- ing for automated auditing using sentence transformers,” in Proc. ICML-A, 2022

    work page 2022

  5. [5]

    Can gpt models be financial analysts? an evaluation of chatgpt and gpt-4 on mock cfa exams,

    E. Callanan, A. Mbakwe, A. Papadimitriou, et al., “Can gpt models be financial analysts? an evaluation of chatgpt and gpt-4 on mock cfa exams,” arXiv:2310.08678, 2023

    arXiv 2023

  6. [6]

    Bridging the gap–the impact of chatgpt on fi- nancial research,

    Y . Cao and J. Zhai, “Bridging the gap–the impact of chatgpt on fi- nancial research,” Journal of Chinese Economic and Business Studies, 2023

    work page 2023

  7. [7]

    Towards automatic numerical cross-checking: Extracting formulas from text,

    Y . Cao, H. Li, P. Luo, and J. Yao, “Towards automatic numerical cross-checking: Extracting formulas from text,” in Proc. WWW, 2018

    work page 2018

  8. [8]

    KPI-EDGAR: A novel dataset and accompanying metric for relation extraction from financial documents,

    T. Deußer, S. M. Ali, L. Hillebrand, et al. , “KPI-EDGAR: A novel dataset and accompanying metric for relation extraction from financial documents,” in Proc. ICMLA, 2022

    work page 2022

Show all 30 references
  1. [9]

    Contradiction detection in financial reports,

    T. Deußer, M. Pielka, L. Pucknat, et al., “Contradiction detection in financial reports,” in Proc. NLDL, 2023

    2023

  2. [10]

    BERT: Pre- training of deep bidirectional transformers for language understand- ing,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- training of deep bidirectional transformers for language understand- ing,” in Proc. NAACL, 2019

    2019

  3. [11]

    Improving zero-shot text matching for financial auditing with large language models,

    L. Hillebrand, A. Berger, T. Deußer, et al., “Improving zero-shot text matching for financial auditing with large language models,” in Proc. DocEng, 2023, pp. 1–4

    2023

  4. [12]

    Kpi-bert: A joint named entity recognition and relation extraction model for financial reports,

    L. Hillebrand, T. Deußer, T. Dilmaghani, et al. , “Kpi-bert: A joint named entity recognition and relation extraction model for financial reports,” in Proc. ICPR, 2022

    2022

  5. [13]

    Towards automating numerical consistency checks in financial reports,

    L. Hillebrand, T. Deußer, T. Dilmaghani, et al., “Towards automating numerical consistency checks in financial reports,” in Proc. BigData, 2022

    2022

  6. [14]

    Finbert: A large language model for extracting information from financial text,

    A. H. Huang, H. Wang, and Y . Yang, “Finbert: A large language model for extracting information from financial text,” Contemporary Accounting Research, 2022

    2022

  7. [15]

    Hulbert, Tree of knowledge: Tok aka tree of knowledge dataset for large language models llm , https://github.com/dave1010/tree- of- thought-prompting, 2023

    D. Hulbert, Tree of knowledge: Tok aka tree of knowledge dataset for large language models llm , https://github.com/dave1010/tree- of- thought-prompting, 2023

    2023

  8. [16]

    Sentiment spin: Attacking financial sentiment with gpt- 3,

    M. Leippold, “Sentiment spin: Attacking financial sentiment with gpt- 3,” Finance Research Letters, 2023

    2023

  9. [17]

    Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,

    P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,” ACM Computing Surveys , pp. 1–35, 2023

    2023

  10. [18]

    Finbert: A pre- trained financial language representation model for financial text mining,

    Z. Liu, D. Huang, K. Huang, Z. Li, and J. Zhao, “Finbert: A pre- trained financial language representation model for financial text mining,” in Proc. IJCAI, 2021

    2021

  11. [19]

    Large language model guided tree-of-thought,

    J. Long, “Large language model guided tree-of-thought,” arXiv:2305.08291, 2023

    2023 arXiv

  12. [20]

    Artificial intelligence authoring financial recommenda- tions: Comparative australian evidence,

    B. Neilson, “Artificial intelligence authoring financial recommenda- tions: Comparative australian evidence,” Journal of Financial Regu- lation, 2023

    2023

  13. [21]

    Gpt-4 technical report,

    OpenAI, “Gpt-4 technical report,” 2023

    2023

  14. [22]

    Alibert: Improved automated list inspection (ali) with bert,

    R. Ramamurthy, M. Pielka, R. Stenzel, et al. , “Alibert: Improved automated list inspection (ali) with bert,” in Proc. DocEng, 2021

    2021

  15. [23]

    Towards automated auditing with machine learning,

    R. Sifa, A. Ladi, M. Pielka, et al., “Towards automated auditing with machine learning,” in Proc. DocEng, 2019

    2019

  16. [24]

    Financial fraudulent statements detection through a deep dense artificial neural network,

    G. S. Temponeras, S.-A. N. Alexandropoulos, S. B. Kotsiantis, and M. N. Vrahatis, “Financial fraudulent statements detection through a deep dense artificial neural network,” in Proc. IISA, 2019

    2019

  17. [25]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv:2307.09288, 2023

    2023 arXiv

  18. [26]

    Bloomberggpt: A large language model for finance,

    S. Wu, O. Irsoy, S. Lu, et al., “Bloomberggpt: A large language model for finance,” arXiv:2303.17564, 2023

    2023 arXiv

  19. [27]

    Finbert: A pretrained language model for financial communications,

    Y . Yang, M. C. S. Uy, and A. Huang, “Finbert: A pretrained language model for financial communications,” arXiv:2006.08097, 2020

    2006 arXiv

  20. [28]

    Tree of thoughts: Deliberate problem solving with large language models,

    S. Yao, D. Yu, J. Zhao, et al., “Tree of thoughts: Deliberate problem solving with large language models,” arXiv:2305.10601, 2023

    2023 arXiv

  21. [29]

    Leveraging llms for kpis retrieval from hybrid long-document: A comprehensive framework and dataset,

    C. Yue, X. Xu, X. Ma, et al., “Leveraging llms for kpis retrieval from hybrid long-document: A comprehensive framework and dataset,” arXiv:2305.16344, 2023

    2023 arXiv

  22. [30]

    A novel cost-sensitive capsule network for audit fraud detection,

    F. Zhu, D. Ning, Y . Wang, and S. Liu, “A novel cost-sensitive capsule network for audit fraud detection,” in Proc. IUCC, 2021. VII. A PPENDIX A. Prompt evaluation Table V shows the detailed evaluation of all prompt config- urations per dataset and model based on the micro F 1...

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.