Enhancing Financial Domain Adaptation of Language Models via Model Augmentation

Hiroki Sakaji; Itsuki Noda; Kazuki Matoya; Kentaro Imajo; Kota Tanabe; Masanori Hirano

REVIEW 5 major objections 6 minor 25 references

Enhancing Financial Domain Adaptation of Language Models via Model Augmentation

T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By connecting a general Japanese chat model to a finance-specialized model with one trainable cross-attention layer, this paper reports financial dialogue scores of 0.711, beating LoRA fine-tuning (0.521) and both original models, with a…

desk verdict A useful CALM application study whose headline result is undercut by a single-run, self-authored benchmark with near-zero scores in half the tasks. read the letter →

arxiv 2411.09249 v1 pith:X2OX7LH5 submitted 2024-11-14 cs.CL cs.AI

Kota Tanabe , Masanori Hirano , Kazuki Matoya , Kentaro Imajo , Hiroki Sakaji , Itsuki Noda This is my paper · ORCID

classification cs.CLcs.AI

keywords CALMcross-attentionfinancialdomainadaptationJapaneselargelanguagemodelsmodelcompositionLoRAdialoguebenchmark

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that composing two frozen Japanese LLMs with CALM—a trainable cross-attention layer that lets a general instruction-tuned model selectively pull knowledge from a finance-specialized model—adapts the general model to financial dialogue better than LoRA fine-tuning. This matters because domain adaptation normally requires expensive continued pre-training or fine-tuning, whereas CALM trains only a small set of connection parameters and works even when the connection-training data are not the same as the finance model's proprietary corpus. On a 360-dialogue Japanese financial benchmark, the connected model averaged 0.711, compared with 0.521 for LoRA, 0.188 for the finance model alone, and 0.043 for the general model alone. A single connection at the middle layer (layer 20) beat connections at the head (layer 0) and tail (layer 38), which the paper interprets as the sweet spot for integrating abstract knowledge.

What carries the argument

The mechanism is Composition to Augment Language Models (CALM), which inserts a cross-attention layer between a chosen layer of the augmenting model and a chosen layer of the anchor model. Query comes from the anchor's representation, keys and values come from the augmenting model; the cross-attention output is added residually to the anchor's representation and passed to the next anchor layer, so the anchor decides at each step how much financial detail to draw in. Because the two same-architecture models share the same hidden dimension, no projection layer is needed; the learned parameters are the 128-head cross-attention weight matrices. The connection was trained on the Japanese Financial Instruction Dataset with the base models frozen, and the paper also varies the connection depth (every four layers, plus single head/middle/tail connections) to ask where the knowledge transfer works best.

What would settle it

Run the same CALM setup on an independent financial benchmark not written by the augmenting model's creators, and include a control with a randomly re-initialized or unrelated augmenting model; if the gain over LoRA disappears or the random control scores equally well, the claimed knowledge transfer from the financial specialist is not the cause.

Watch

Extended reading notes

Core claim

The central discovery is that one cross-attention bridge between two same-architecture models transfers financial capability to the anchor without changing either model's weights, and the transfer does not depend on the connector being trained on the same corpus that created the financial specialist. The paper demonstrates this with a general instruction-tuned Japanese model as anchor and a finance-specialized Japanese model as augmenting, training only the cross-attention on the Japanese Financial Instruction Dataset. Evaluated by an external judge on a 360-dialogue Japanese financial benchmark, the composed model reaches an average score of 0.711, with stronger second-turn (0.794) than first-turn (0.628) responses; LoRA, the strongest baseline, reaches 0.521. The paper's interpretation is that CALM performs contextual, token-by-token retrieval of financial knowledge from the augmenting model rather than storing limited facts in new adapter weights, which is why it answers more comprehensively in the qualitative examples.

Load-bearing premise

The evaluation benchmark is an unbiased measure of financial dialogue quality; since it was built by the same group that created the augmenting model on a proprietary dataset, the reported advantage could be inflated if the benchmark happens to reward that model's strengths.

Editorial extensions

If this is right

CALM offers a parameter-efficient route to financial domain adaptation: only cross-attention weights are trained, and both base models keep their original behavior, so the general model's broad capabilities are not overwritten.
Using a training dataset that differs from the augmenting model's proprietary corpus means the method is usable by external groups who cannot access the specialist model's training data.
A single middle-layer connection beats head and tail connections and does so consistently across task categories, suggesting no per-task search over connection depth is needed for financial dialogue.
The composed model's higher second-turn scores suggest the benefit is especially strong in multi-turn dialogue, where the model must build on earlier context.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the benchmark was authored by the same team that built the augmenting model on a non-public corpus, the reported gap over LoRA should be re-checked on independently written financial questions; this is a testable prediction, not a paper claim.
A control with a randomly initialized or unrelated augmenting model would isolate whether the gain comes from financial knowledge in that model's weights or from the cross-attention structure itself.
The middle-layer result hints at a general design rule for model composition—connect near the middle where representations are abstract but not yet committed to surface form—that could be probed across model families and tasks beyond finance.
The frequent end-of-text generation failure in the base models means a portion of CALM's advantage is repairing unusable outputs rather than purely adding knowledge; separating these two effects would sharpen the interpretation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a solid, clearly written application paper. It doesn't invent a new mechanism—CALM is Bansal et al.'s composition method—but it shows CALM trained on a public financial instruction set (JAFIN) can compose a Japanese generalist anchor with a finance-specialized augmenting model when the training data is intentionally different from the augmenting model's proprietary corpus. The mid-layer connection result is a nice empirical datapoint. The authors also keep LoRA parameter-matched and are upfront that many tasks fail with endoftext.

The soft spot is exactly where the reader's report puts it. The quantitative claim, CALM 0.711 vs LoRA 0.521, comes from one run with no confidence interval or significance test, and the benchmark is pfmt-bench-fin-ja, authored by two of this paper's co-authors (also the augmenting model's creators), with no external audit. The augmenting model's training data is proprietary, so you can't rule out that the benchmark is implicitly aligned with its strengths. On top of that, GPT-4 is the judge and CALM's outputs are visibly longer and more structured, so part of the gap could be style. The absolute scores are also tiny—0.711 out of 10—so even the headline number is not practically impressive. Half the tasks are at or near zero, which the paper acknowledges.

I don't think this is misconduct or sloppiness; it's a common evaluation-limitation pattern in applied LLM work. But because the entire advantage claim rests on this benchmark, the limitation is load-bearing, not cosmetic. An external benchmark or at least a second judge and repeated runs with error bars would be needed before I'd put weight on the LoRA comparison.

Who is this for? Anyone working on Japanese financial dialogue models or on compositional adaptation of LLMs. It's a legitimate extension that deserves a serious referee—the method is reproducible in principle, the writing is clear, and the authors disclose the main failure mode. I'd send it to peer review, with the requirement that the evaluation be strengthened. I would not cite it in my own work until the result is independently confirmed.

Referee Report

5 major / 6 minor

Summary. The paper applies Composition to Augment Language Models (CALM) to adapt a Japanese general-purpose instruction-tuned model to the financial domain. The anchor model is nekomata-14b-instruction and the augmenting model is nekomata-14b-pfn-qfin, a financial-specialized model; cross-attention connections between the two are trained on the JAFIN instruction dataset, which is distinct from the proprietary corpus used to build the augmenting model. On the pfmt-bench-fin-ja benchmark, the CALM model achieves an average score of 0.711, compared with 0.521 for a LoRA-tuned anchor, 0.188 for the augmenting model alone, and 0.043 for the anchor alone. A single cross-attention connection at the middle layer (layer 20) outperforms connections at the head (layer 0) and tail (layer 38), leading the authors to conclude that CALM is an effective method for financial domain adaptation and that middle-layer connections are most effective.

Significance. If the quantitative results are reliable, the paper provides evidence that model composition via cross-attention can transfer financial knowledge without modifying the base model's weights, including when the training data differ from the augmenting model's pretraining corpus. The use of a distinct training dataset and a parameter-matched LoRA baseline are good experimental design choices, and the connection-point ablation is informative. However, the strength of the evidence is limited by the benchmark's overlap with the augmenting model's authors, the absence of repeated runs or significance tests, and the paper's own admission of near-zero scores on several task categories. These issues must be addressed before the headline claims can be accepted.

major comments (5)

[Section IV-D and V-A, Table I] The benchmark pfmt-bench-fin-ja [26] is released by the same two researchers (Hirano and Imajo) who built the augmenting model [3] and are co-authors of this paper, and the augmenting model's training corpus is proprietary. This overlap means the benchmark may be implicitly aligned with knowledge contained in the augmenting model, so the reported advantage of CALM over LoRA (0.711 vs 0.521) may be inflated. The paper neither discloses this overlap nor provides any independent evaluation. Please add results on a public, independently authored benchmark (or a human evaluation with external raters) and discuss the potential for benchmark alignment.
[Section V-A, Table I] All quantitative results are point estimates from a single run. With 360 two-turn dialogues, the 0.19-point gap between CALM and LoRA could be within run-to-run noise, especially given the large variability implied by near-zero scores in several categories. Please provide multiple training runs, bootstrap confidence intervals over dialogue-level scores, and a paired significance test for the CALM versus LoRA comparison.
[Section V-A, Fig. 1] The paper reports that scores are zero or near zero in half of the remaining tasks because the models emit an early <endoftext> token, yet the abstract and conclusion state a general result of 'superior responses with higher scores.' The average score aggregates tasks on which CALM succeeds and tasks on which it produces no output, and the paper does not provide a per-task numerical table or an explicit list of the failed categories. Please include per-task scores and qualify the headline claim accordingly; if CALM fails completely on several categories, the contribution should be described as selective rather than general financial-domain adaptation.
[Section IV-C] The LoRA baseline is described only as having a rank set to match the parameter count of CALM, with no statement of which linear modules were adapted, the LoRA alpha, dropout, or training details. This makes the LoRA result (0.521) difficult to reproduce and leaves open the possibility that the comparison is not parameter-for-parameter fair. Please provide the full LoRA configuration and, if possible, an additional LoRA baseline with a larger rank or different target modules to show that the conclusion is robust.
[Section V-A, Table II] The conclusion that middle-layer connections are most effective rests on a single comparison of one connection at layer 0, 20, or 38, with no variance estimates. Moreover, the paper does not explain how this single-connection result relates to the main CALM model that inserts connections every four layers, which reaches 0.711 while the best single connection reaches only 0.244. Please provide confidence intervals for the connection-point comparison and discuss the relationship between the single-connection finding and the multi-connection setting.

minor comments (6)

[Section II, Reference [21]] The citation for DARE is incorrect: reference [21] is a paper on safety alignment, not the DARE method for model merging. Please cite the original DARE paper (e.g., Yu et al., 'Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch').
[Section V-B] The qualitative examples are presented as representative, but the augmenting model's zero <endoftext> responses in the examples contradict its nonzero average score of 0.1875 in Table I; please acknowledge that these examples are selected and provide a more systematic summary of response lengths and success rates across the benchmark.
[Section IV-B] The description of the JAFIN dataset says it was primarily collected from government websites and Wikipedia, but the paper does not discuss whether these sources overlap with the proprietary corpus used to build the augmenting model; a brief analysis of potential overlap would strengthen the claim that the training data are genuinely different.
[Section IV-C] Please state the exact number of additional parameters introduced by CALM and by the LoRA baseline; the claim that the ranks are matched is not verifiable without these numbers.
[Section IV-D] The GPT-4 evaluation protocol is underspecified: please report the exact prompt, decoding parameters, and whether the judge was run more than once per dialogue to assess stability.
[Section V-A, Fig. 1] The phrase 'half of the remaining tasks' is ambiguous; please specify exactly which task categories have near-zero scores and which tasks contribute to the improvement over LoRA.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the empirical claims are not derived from the evaluation or from self-authored artifacts by construction.

full rationale

The paper reports an empirical comparison, not a derivation. The CALM parameters are trained on JAFIN [24] (public, distinct from the augmenting model's proprietary corpus) and then evaluated on pfmt-bench-fin-ja [26]. There is no equation or fitted value in the paper that makes the benchmark scores equal to the training objective or to the augmenting model's pretraining objective; no parameter is fitted to the benchmark and then reported as a prediction. The benchmark and the augmenting model share two authors, which is a legitimate evaluation-independence concern and should be checked by running the public benchmark and by auditing the proprietary corpus, but overlapping authorship alone is not circularity. The benchmark is a fixed, publicly available artifact with a GPT-4 scoring protocol, so the headline comparison is externally reproducible rather than forced by the paper's own construction. The paper's own admission that many scores are near zero due to early end-of-text is a robustness caveat, not evidence that the reported advantages reduce to the inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim relies on the CALM composition architecture from prior work, the validity of a self-authored benchmark, GPT-4 as judge, and several hand-set hyperparameters (128 heads, every-4-layer interval, LoRA rank matching). The free parameters are training and hyperparameter choices; no new entities are postulated.

free parameters (4)

Number of cross-attention heads = 128
Set in Section IV-C without ablation; the capacity of the connection depends on this choice.
Connection interval for main CALM = every 4 layers
Chosen in Section IV-C; the single-connection comparison later varies the layer, so the interval is a hand-selected setting.
LoRA rank = matched to CALM parameter count
Set 'to ensure comparability' in Section IV-C, but no parameter count is reported, so the fairness of the comparison cannot be verified.
Training epochs and checkpoint selection = 20 epochs; best validation loss
Section IV-C selects the model by lowest validation loss, a model-selection procedure that can favor the validation set; no independent test set is described.

assumptions (5)

domain assumption Both models share the same architecture and hidden dimension D, so the projection fproj is omitted.
Section III says since architectures are identical and dimension D is the same, fproj is not introduced; if false, the CALM connection math changes.
domain assumption The pfmt-bench-fin-ja benchmark is a valid and unbiased measure of Japanese financial dialogue quality.
Section IV-D uses this benchmark by Hirano and Imajo [26], the same authors as the augmenting model [3], and no independent validation of the benchmark is given.
domain assumption GPT-4 scoring on a 10-point scale reliably ranks model responses.
Section IV-D states evaluation is conducted on a 10-point scale by GPT-4; no human agreement or calibration is reported.
domain assumption The JAFIN dataset is representative of financial domain adaptation needs.
Section IV-B describes 1,490 QA samples from Japanese government and Wikipedia sources; no coverage or overlap analysis with the benchmark is provided.
domain assumption CALM's cross-attention composition works as described in Bansal et al. [4].
The paper relies on the original CALM formulation without re-deriving or independently validating its behavior.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enhancing Financial Domain Adaptation of Language Models via Model Augmentation." pith.science (2026). https://pith.science/paper/X2OX7LH5

@misc{pith2026241109249,
  author       = {Pith},
  title        = {Pith review of: Enhancing Financial Domain Adaptation of Language Models via Model Augmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X2OX7LH5}},
  note         = {Machine review of arXiv:2411.09249}
}

read the original abstract

The domain adaptation of language models, including large language models (LLMs), has become increasingly important as the use of such models continues to expand. This study demonstrates the effectiveness of Composition to Augment Language Models (CALM) in adapting to the financial domain. CALM is a model to extend the capabilities of existing models by introducing cross-attention between two LLMs with different functions. In our experiments, we developed a CALM to enhance the financial performance of an LLM with strong response capabilities by leveraging a financial-specialized LLM. Notably, the CALM was trained using a financial dataset different from the one used to train the financial-specialized LLM, confirming CALM's ability to adapt to various datasets. The models were evaluated through quantitative Japanese financial benchmarks and qualitative response comparisons, demonstrating that CALM enables superior responses with higher scores than the original models and baselines. Additionally, comparative experiments on connection points revealed that connecting the middle layers of the models is most effective in facilitating adaptation to the financial domain. These findings confirm that CALM is a practical approach for adapting LLMs to the financial domain.

Figures

Figures reproduced from arXiv: 2411.09249 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 12 canonical work pages

[26]

pfmt-bench-fin-ja: Preferred Multi-turn Benchmark for Finance in Japanese,

M. Hirano and K. Imajo, “pfmt-bench-fin-ja: Preferred Multi-turn Benchmark for Finance in Japanese,” 2024. [Online]. Available: https://github.com/pfnet-research/pfmt-bench-fin-ja APPENDIX Response Example 1 以下の金融商品の特徴を比較し、長期的な資産形成に最も適していると考えられるものを選択し、その理由を説明してください。 a) 株式:高いリターンが期待できるが、価格変動リスクが高い。 b) 債券: 安定した利子収入が得られるが、金利変動リスクがある。 c) 投資信託: 分散投資により、リス...

work page 2024
[3]

Construction of Domain-specified Japanese Large Language Model for Finance through Continual Pre-training

M. Hirano and K. Imajo, “Construction of domain-specified japanese large language model for finance through continual pre-training,” arXiv preprint arXiv:2404.10555, 2024

work page Pith review arXiv 2024
[2]

FinGPT: Open-Source Financial Large Language Models,

H. Yang, X.-Y . Liu, and C. D. Wang, “FinGPT: Open-Source Financial Large Language Models,” FinLLM Symposium at IJCAI 2023, 2023

work page 2023
[4]

Llm augmented llms: Expanding capabilities through composition,

R. Bansal, B. Samanta, S. Dalmia, N. Gupta, S. Vashishth, S. Ganapathy, A. Bapna, P. Jain, and P. Talukdar, “Llm augmented llms: Expanding capabilities through composition,” arXiv preprint arXiv:2401.02412 , 2024

arXiv 2024
[5]

Universal language model fine-tuning for text classification,

J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Association for Computational Linguistics, 2018, pp. 328–339

work page 2018
[6]

Scibert: A pretrained language model for scientific text,

I. Beltagy, K. Lo, and A. Cohan, “Scibert: A pretrained language model for scientific text,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , 2019, pp. 3615–3620

work page 2019
[7]

Med-bert: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction,

L. Rasmy, Y . Xiang, Z. Xie, C. Tao, and D. Zhi, “Med-bert: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction,” NPJ digital medicine, vol. 4, no. 1, p. 86, 2021

work page 2021
[8]

Finbert: A large language model for extracting information from financial text,

A. H. Huang, H. Wang, and Y . Yang, “Finbert: A large language model for extracting information from financial text,” Contemporary Accounting Research, vol. 40, no. 2, pp. 806–841, 2023

2023

Show all 25 references

[9]

Finbert: A pre-trained financial language representation model for financial text mining,

Z. Liu, D. Huang, K. Huang, Z. Li, and J. Zhao, “Finbert: A pre-trained financial language representation model for financial text mining,” in Proceedings of the twenty-ninth international conference on interna- tional joint conferences on artificial intelligence , 2021, pp. 4513–4519

2021
[10]

Construction and Validation of a Pre-Training and Additional Pre-Training Financial Language Model [in Japanese],

M. SUZUKI, H. SAKAJI, M. HIRANO, and K. IZUMI, “Construction and Validation of a Pre-Training and Additional Pre-Training Financial Language Model [in Japanese],” in The 28th meeting of Special Interest Group on Financial Informatics of Japanese Society for Artificial Intellig...

2022
[11]

Don’t stop pretraining: Adapt language models to domains and tasks,

S. Gururangan, A. Marasovi ´c, S. Swayamdipta, K. Lo, I. Beltagy, D. Downey, and N. A. Smith, “Don’t stop pretraining: Adapt language models to domains and tasks,” arXiv preprint arXiv:2004.10964 , 2020

2004 arXiv
[12]

Constructing and Analyzing Domain-Specific Language Model for Financial Text Mining,

M. SUZUKI, H. SAKAJI, M. HIRANO, and K. IZUMI, “Constructing and Analyzing Domain-Specific Language Model for Financial Text Mining,” p. e103194, 2023

2023
[13]

BloombergGPT: A Large Language Model for Finance,

S. Wu, O. Irsoy, S. Lu, V . Dabravolski, M. Dredze, S. Gehrmann, P. Kambadur, D. Rosenberg, and G. Mann, “BloombergGPT: A Large Language Model for Finance,” arXiv, 2023, https://arxiv.org/abs/2303.17564v2

2023 arXiv
[14]

Fin-LLAMA: Efficient Finetuning of Quantized LLMs for Finance,

P. B. William Todt, Ramtin Babaei, “Fin-LLAMA: Efficient Finetuning of Quantized LLMs for Finance,” 2023, https://github.com/Bavest/fin- llama

2023
[15]

LLaMA: Open and Efficient Foundation Language Models,

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “LLaMA: Open and Efficient Foundation Language Models,” arXiv, 2023, https://arxiv.org/abs/2302.13971

2023 arXiv
[16]

Instruct-FinGPT: Financial Sen- timent Analysis by Instruction Tuning of General-Purpose Large Lan- guage Models,

B. Zhang, H. Yang, and X.-Y . Liu, “Instruct-FinGPT: Financial Sen- timent Analysis by Instruction Tuning of General-Purpose Large Lan- guage Models,” arXiv, 2023, https://arxiv.org/abs/2306.12659

2023 arXiv
[17]

Llama-2-econ: Enhancing title generation, abstract classification, and academic q&a in economic research,

O. Keles and O. T. Bayraklı, “Llama-2-econ: Enhancing title generation, abstract classification, and academic q&a in economic research,” in Proceedings of the Joint Workshop of the 7th Financial Technology and Natural Language Processing, the 5th Knowledge Discovery from Unstr...

2024
[18]

Editing models with task arithmetic,

G. Ilharco, M. T. Ribeiro, M. Wortsman, S. Gururangan, L. Schmidt, H. Hajishirzi, and A. Farhadi, “Editing models with task arithmetic,” arXiv preprint arXiv:2212.04089 , 2022

2022 arXiv
[19]

Scaling monosemantic- ity: Extracting interpretable features from claude 3 sonnet,

A. Templeton, T. Conerly, J. Marcus, J. Lindsey, T. Bricken, B. Chen, A. Pearce, C. Citro, E. Ameisen, A. Jones et al., “Scaling monosemantic- ity: Extracting interpretable features from claude 3 sonnet,” Transformer Circuits Thread, 2024

2024
[20]

Ties- merging: Resolving interference when merging models,

P. Yadav, D. Tam, L. Choshen, C. A. Raffel, and M. Bansal, “Ties- merging: Resolving interference when merging models,” Advances in Neural Information Processing Systems , vol. 36, 2024

2024
[21]

Towards comprehensive and efficient post safety alignment of large language models via safety patching,

W. Zhao, Y . Hu, Z. Li, Y . Deng, Y . Zhao, B. Qin, and T.-S. Chua, “Towards comprehensive and efficient post safety alignment of large language models via safety patching,” arXiv preprint arXiv:2405.13820, 2024

2024 arXiv
[22]

Evolutionary opti- mization of model merging recipes,

T. Akiba, M. Shing, Y . Tang, Q. Sun, and D. Ha, “Evolutionary opti- mization of model merging recipes,” arXiv preprint arXiv:2403.13187 , 2024

2024 arXiv
[23]

Qwen technical report,

J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huang, B. Hui, L. Ji, M. Li, J. Lin, R. Lin, D. Liu, G. Liu, C. Lu, K. Lu, J. Ma, R. Men, X. Ren, X. Ren, C. Tan, S. Tan, J. Tu, P. Wang, S. Wang, W. Wang, S. Wu, B. Xu, J. Xu, A. Yang, H. Yang, J. ...

2023 arXiv
[24]

Jafin: Japanese financial instruction dataset,

K. Tanabe, M. Suzuki, H. Sakaji, and I. Noda, “Jafin: Japanese financial instruction dataset,” arXiv preprint arXiv:2404.09260 , 2024

2024 arXiv
[25]

Lora: Low-rank adaptation of large language models,

E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685 , 2021

2021 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [26]

pfmt-bench-fin-ja: Preferred Multi-turn Benchmark for Finance in Japanese,

M. Hirano and K. Imajo, “pfmt-bench-fin-ja: Preferred Multi-turn Benchmark for Finance in Japanese,” 2024. [Online]. Available: https://github.com/pfnet-research/pfmt-bench-fin-ja APPENDIX Response Example 1 以下の金融商品の特徴を比較し、長期的な資産形成に最も適していると考えられるものを選択し、その理由を説明してください。 a) 株式:高いリターンが期待できるが、価格変動リスクが高い。 b) 債券: 安定した利子収入が得られるが、金利変動リスクがある。 c) 投資信託: 分散投資により、リス...

work page 2024

[2] [3]

Construction of Domain-specified Japanese Large Language Model for Finance through Continual Pre-training

M. Hirano and K. Imajo, “Construction of domain-specified japanese large language model for finance through continual pre-training,” arXiv preprint arXiv:2404.10555, 2024

work page Pith review arXiv 2024

[3] [2]

FinGPT: Open-Source Financial Large Language Models,

H. Yang, X.-Y . Liu, and C. D. Wang, “FinGPT: Open-Source Financial Large Language Models,” FinLLM Symposium at IJCAI 2023, 2023

work page 2023

[4] [4]

Llm augmented llms: Expanding capabilities through composition,

R. Bansal, B. Samanta, S. Dalmia, N. Gupta, S. Vashishth, S. Ganapathy, A. Bapna, P. Jain, and P. Talukdar, “Llm augmented llms: Expanding capabilities through composition,” arXiv preprint arXiv:2401.02412 , 2024

arXiv 2024

[5] [5]

Universal language model fine-tuning for text classification,

J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Association for Computational Linguistics, 2018, pp. 328–339

work page 2018

[6] [6]

Scibert: A pretrained language model for scientific text,

I. Beltagy, K. Lo, and A. Cohan, “Scibert: A pretrained language model for scientific text,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , 2019, pp. 3615–3620

work page 2019

[7] [7]

Med-bert: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction,

L. Rasmy, Y . Xiang, Z. Xie, C. Tao, and D. Zhi, “Med-bert: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction,” NPJ digital medicine, vol. 4, no. 1, p. 86, 2021

work page 2021

[8] [8]

Finbert: A large language model for extracting information from financial text,

A. H. Huang, H. Wang, and Y . Yang, “Finbert: A large language model for extracting information from financial text,” Contemporary Accounting Research, vol. 40, no. 2, pp. 806–841, 2023

2023