REVIEW 5 cited by
CFBenchmark: Chinese Financial Assistant Benchmark for Large Language Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have demonstrated great potential in the financial domain. Thus, it becomes important to assess the performance of LLMs in the financial tasks. In this work, we introduce CFBenchmark, to evaluate the performance of LLMs for Chinese financial assistant. The basic version of CFBenchmark is designed to evaluate the basic ability in Chinese financial text processing from three aspects~(\emph{i.e.} recognition, classification, and generation) including eight tasks, and includes financial texts ranging in length from 50 to over 1,800 characters. We conduct experiments on several LLMs available in the literature with CFBenchmark-Basic, and the experimental results indicate that while some LLMs show outstanding performance in specific tasks, overall, there is still significant room for improvement in basic tasks of financial text processing with existing models. In the future, we plan to explore the advanced version of CFBenchmark, aiming to further explore the extensive capabilities of language models in more profound dimensions as a financial assistant in Chinese. Our codes are released at https://github.com/TongjiFinLab/CFBenchmark.
Forward citations
Cited by 5 Pith papers
-
FinProBench: Evaluating Financial AI Agents with Role-Grounded Rubrics Derived from Professional Deliverables
Rubrics derived from authentic financial deliverables beat prompt-only rubrics on role-specialized tasks (99.1% vs 78.0% coverage) while matching them on conventional tasks.
-
Beyond Knowledge to Agency: Evaluating Expertise, Autonomy, and Integrity in Finance with CNFinBench
CNFinBench finds LLMs lose about 15 points from single modules to full agentic execution chains, and their financial-compliance violations surge roughly 160-170% by the second round of multi-turn adversarial attacks.
-
FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation
FinMME is a new 11,099-sample financial chart benchmark where top AI models average around 50% and FinScore adds penalties for guessing.
-
Enhancing LLMs' Reasoning-Intensive Multimedia Search Capabilities through Fine-Tuning and Reinforcement Learning
A two-stage fine-tuning and reinforcement-learning method makes LLMs generate token-efficient natural-language search plans, reporting strong accuracy gains on financial and news search benchmarks.
-
A Survey of Automatic Evaluation Methods on Text, Visual and Speech Generations
A unified taxonomy and comparative meta-evaluation of automatic evaluation methods across text, vision, and speech generation, concluding that LLM-based evaluators dominate current practice.
Discussion (0). Continue with ORCID to comment.