Pith. sign in

REVIEW 4 major objections 6 minor 75 references

FinSAgent argues that SEC filing QA fails because model priors misalign with how filings are written, and corrects both query generation and evidence selection with corpus-side conditioning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 15:59 UTC pith:YYZBHUSS

load-bearing objection A well-built SEC-filing QA system with a plausible but not yet isolated mechanism claim, and an abstract that overstates an online experiment the body never describes. the 4 major comments →

arxiv 2607.18102 v2 pith:YYZBHUSS submitted 2026-07-20 cs.IR cs.CLcs.MA

FinSAgent: Corpus-Aligned Multi-Agent RAG Framework for Evidence-Grounded SEC Filing Question Answering

Jijun Chi , Zhenghan Tai , Hanwei Wu , Tung Sum Thomas Kwok , Hailin He , Zixing Liao , Bohuai Xiao , Chaolong Jiang
show 16 more authors
Jianliang Lei Jerry Huang Peng Lu Muzhi Li Liheng Ma Yihong Wu Sicheng Lyu Jingrui Tian Yihan Li Yanzhang Ma Sizhe Guan Dingtao Hu Yufei Cui Ling Zhou Lei Ding Xinyu Wang
This is my paper
classification cs.IR cs.CLcs.MA
keywords SEC filingsretrieval-augmented generationmulti-agent systemsquestion answeringevidence groundingquery decompositionrerankingprior–corpus misalignment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that question answering over SEC filings fails for a specific reason: the language models that generate search queries and rank retrieved passages rely on generic knowledge of what a financial answer should look like, while valid evidence is written and organized according to the filings' own conventions. It calls this mismatch prior–corpus misalignment and claims it corrupts both ends of retrieval: query generation misses corpus-specific evidence, and semantic reranking promotes boilerplate that resembles the question but is not valid evidence for that company. The proposed system, FinSAgent, injects corpus-side conditioning at both ends — role-specialized agents tied to the 10-K item structure, sub-queries generated with a lightweight view of the local filing database, and a learned feature gate that demotes confident false positives. Across five benchmarks and a randomized user study, the system reports better retrieval coverage and answer correctness than strong baselines, and it beats full-document long-context prompting on the benchmark most favorable to that alternative.

Core claim

The paper's central discovery is that the two observed failure modes in SEC filing QA — missed evidence and false-positive boilerplate — have one cause: prior–corpus misalignment. It claims that conditioning retrieval planning on a lightweight view of the local corpus and gating reranking by non-semantic validity features corrects both ends. The evidence includes component ablations showing each addition improves recall on at least some corpora, matched-budget comparisons showing the gains are not from extra retrieval perspectives, and a double-dissociation showing the two mechanisms help different corpora for different reasons. The authors state the conclusion directly: SEC filing QA is bes

What carries the argument

The load-bearing mechanism is the feature-gated reranker: a semantic cross-encoder score is multiplied by a penalty (1 − λ(1 − p̂)), where p̂ is a gradient-boosted tree's estimate of evidential validity from 31 non-semantic features such as path provenance, path scores, lexical overlap, and chunk metadata. Because the penalty scales with the reranker's own confidence, high-scoring boilerplate is demoted more than low-scoring chunks. The second mechanism is database-aware query decomposition: before generating sub-queries, each agent retrieves a small set of section-level summaries of the local corpus and conditions its sub-queries on that view. Together they instantiate the paper's principle

Load-bearing premise

The gate's training labels come from the system's own pipeline — an LLM writes synthetic questions from random chunks, the retrieval pipeline returns candidates, and chunks matching the seed chunk are called positive — so the gate may be learning to recognize the synthetic question style rather than true evidential validity.

What would settle it

Train the feature gate on human-annotated evidence-validity labels, or replace it with an independent claim-entailment verifier, holding everything else fixed. If retrieval coverage and answer correctness do not improve over the ungated reranker, the central mechanism is an artifact of self-labeling; if the gains persist, the mechanism genuinely separates validity from semantics.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, the same two-end correction should transfer to any standardized, redundancy-heavy corpus, such as patents, clinical trial reports, or regulatory disclosures, where boilerplate can outrank company- or case-specific evidence.
  • Reranking pipelines that rely purely on semantic similarity may be leaving cheap, non-semantic signals on the table; a learned gate over retrieval-path and lexical features can separate evidential validity from topical similarity.
  • Planned, corpus-aware retrieval can outperform handing an entire document to a long-context model, even on a single-document benchmark designed to favor the long-context approach.
  • The error analysis implies that once retrieval and comprehension failures are suppressed, the remaining bottleneck shifts to fine-grained numerical precision, pointing to a clear next target for financial QA systems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the prior–corpus misalignment framing suggests a direct transfer test — applying the same database-aware decomposition and feature gate to another standardized corpus should reproduce the recall and correctness gains if the cause is truly structural rather than SEC-specific.
  • Editorial inference: the paper's own future-work comparison against an NLI-style claim verifier would clarify whether the statistical gate learns evidential validity or merely the style of the synthetic training questions; a human-labeled validation set would settle the same question.
  • Editorial inference: the online user study and the five benchmarks measure different question mixes; breaking down gains by question type (quantitative, legal, market, company) could show which role specializations earn their inference cost and which are optional.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes FinSAgent, a multi-agent retrieval-augmented framework for evidence-grounded question answering over SEC filings. Its central claim is that SEC filing QA is hampered by 'prior–corpus misalignment': model priors cause query decomposition to miss corpus-specific evidence and semantic reranking to favor topically similar but evidentially invalid chunks. FinSAgent instantiates three mechanisms: role-specialized agents anchored to the 10-K item structure, database-aware query decomposition conditioned on a lightweight summary-level corpus view, and multi-path retrieval with a learned feature-gated reranker that combines semantic scores with non-semantic validity features. The paper evaluates on five benchmarks (Lotus, Zeekr, FinanceBench, FinDER, and a 100-question SECQUE subset), reporting improvements in retrieval Macro-Recall and answer correctness over single-agent and multi-agent baselines, plus ablations, a matched-budget fairness study, a blind human validation, a full-document long-context comparison, RAGAS noise sensitivity, overhead measurements, and an error analysis.

Significance. The paper addresses a practical and timely problem: answering complex financial questions over long, standardized, and redundant SEC filings. The proposed 'corpus-side conditioning' principle is a coherent and potentially useful design lens, and the system-level results are broad: five datasets, multiple backbone models, matched retrieval budgets, human validation on 150 answers, and a long-context baseline. The strengths are the breadth of evaluation, the matched-budget study (Appendix B), the blind human check (Table 4), and the honest Limitations section, which explicitly identifies the missing separated baselines. If the mechanism-level claims are confirmed with direct ablations and the evaluation is tightened statistically, the work would be a solid systems contribution. As it stands, the core attribution of gains to the two named mechanisms is not yet isolated, and the gate's training labels derive from the very pipeline it is supposed to regulate, so the headline claims require further support.

major comments (4)
  1. [§3.4.2, Appendix C.2] The feature-gate training labels are generated by FinSAgent's own pipeline: an LLM synthesizes questions from random chunks, the full retrieval/reranking pipeline produces top-k candidates, and chunks matching the seed chunk are labeled positive. This is an in-distribution, pipeline-specific labeling scheme, not an external gold standard. The gate may therefore learn to recognize artifacts of the synthetic-question style or retrieval behavior rather than true evidential validity. This is load-bearing for the back-end claim (e.g., Table 3 Lotus recall 41→50 and Table 5 noise-sensitivity attribution). Please add a validation set with human/expert-annotated evidential-validity labels (or at least an independently sourced label set) and report gate performance on it, or otherwise show that the self-labeled training data does not drive the reported gains.
  2. [§4.3, Limitations (end of main text)] Table 3 is a cumulative ablation in a fixed order; it does not include either mechanism alone or leave-one-out configurations. The authors explicitly concede in the Limitations paragraph: 'Our evidence for the two mechanisms currently rests on full-system ablations and fixed-pool comparisons.' As a result, the gains attributed to database-aware decomposition and feature-gated reranking could in principle come from generic added context, extra retrieval paths, or other confounds. The FinanceBench row is instructive: the full system drops relative to '+ DB-Aware Decomp.' (76 vs 80), which the text explains as an order/interference effect; this further shows cumulative contributions do not isolate stable mechanism-level effects. The planned separated baselines (static corpus-overview prompt vs. database-aware decomposition; NLI/claim verifier vs. feature-gated reranker) are exactly what is
  3. [§1, §2, Abstract; §4.2] The abstract claims 'a three-arm randomized online experiment with 1,000 anonymous user ratings,' but the body never describes such an experiment and §1 instead refers to '~1,400 anonymous user ratings.' The discrepancy is not merely numerical: no protocol, randomization details, rating instrument, or statistical analysis for this experiment is provided anywhere in the paper. Since the abstract's headline claim rests on this experiment, and the offline head-to-head results in Table 2 are reported without error bars, confidence intervals, or significance tests, the evaluation is not yet statistically grounded. Please unify the numbers, present the online-experiment protocol and results, and add variance estimates/significance tests (or clearly label the offline comparisons as descriptive).
  4. [§4.3, Table 3; §4.2, Table 2] The matched-budget study (Appendix B; Table 7) is a valuable control, but it gives the baselines FinSAgent's budgets with the two mechanisms simply disabled. This does not directly test the mechanisms: the baselines also lack the role-specific prompts, the orchestrator, and other pipeline components. Consequently the phrase 'the advantage comes from database-aware decomposition and feature-gated reranking, not from having more retrieval perspectives' (Appendix B) overstates what the design can establish. Please add a budget-matched baseline that includes a static corpus-overview prompt (or an equivalent non-database-aware decomposition) with the same role structure, and a baseline that replaces the feature gate with an NLI/claim verifier, so the two mechanisms are isolated.
minor comments (6)
  1. [Table 2] The LC (long-context) rows are shown as zeros on Lotus, SECQUE, FinDER, and Zeekr; it would be clearer to mark these as 'not applicable' or explain why long-context is infeasible there.
  2. [§4.2, Table 2] FinGPT is described as a 'single-agent tool-augmented reasoner' but is given web/MCP access (§A.3). The comparison is reasonable, but the caption should note that FinGPT is not strictly a document-retrieval-only baseline.
  3. [§4.5] RAGAS is mentioned without a reference; please add a citation for the RAGAS framework and for the noise-sensitivity metric.
  4. [Abstract and §4.1] The abstract mentions 'five offline financial QA benchmarks' but the SECQUE subset is only 100 of 565 questions; please state this sampling explicitly in the abstract or keep the 'five benchmarks' phrasing but note the subset size in the main text (already in §4.1 and Appendix A.2).
  5. [§4.3, Figure 3] The co-retrieval heatmap is useful, but the caption and text could clarify whether the counts are per-query averaged or raw totals across the full evaluated set.
  6. [General] Some sentences are overly programmatic (e.g., 'we read this as evidence that the remaining bottleneck has moved downstream' in §4.8). A more measured phrasing would better match the results presented.

Circularity Check

1 steps flagged

Partial circularity in the feature-gate 'validity' labels; core benchmark results remain independent.

specific steps
  1. fitted input called prediction [§3.4.2 Feature-Gated Evidence Reranking; Appendix C.2 Gating Model Training]
    "Using targeted hard-negative mining, an LLM generates questions from random chunks; these are run through the full retrieval and reranking pipeline; within the top-𝑘 results, chunks matching the chosen ground truth are labeled positive and the rest serve as competitive hard negatives..."

    The gate's 'validity' label is not an external evidence standard: positive means the seed chunk that survives FinSAgent's own full retrieval/reranking pipeline for an LLM-written question, and negative means the pipeline's other top-k chunks. Thus the LightGBM output p̂=M(x), later called a 'validity estimate' that 'separates evidential validity from semantic similarity,' is by construction an estimate of seed-chunk identity under the same pipeline being corrected. The validity signal is partly the pipeline's own echo. External Macro-Recall benchmarks provide an independent check, so this is partial rather than total circularity.

full rationale

The load-bearing evaluation is external: FinanceBench, FinDER, SECQUE, Lotus, and Zeekr use annotated relevance labels, and a blind human study and matched-budget comparison add independent checks. So the overall claim that FinSAgent improves retrieval coverage and answer correctness is not forced by construction. The one genuinely circular element is the feature gate's training objective: positive labels are generated by the pipeline itself, then the fitted model is presented as a predictor of evidential validity. The Limitations section admits 'Our evidence for the two mechanisms currently rests on full-system ablations and fixed-pool comparisons' and plans direct separated baselines; that is a causal-attribution gap, not a circularity, and I weigh it as such rather than raising the score further. Self-citations (FinSage, VeritasFi, R2R) are descriptive related-work or baselines and are not load-bearing; no uniqueness theorem or ansatz is imported from the authors' prior work. Cumulative ablations make mechanism attribution uncertain but do not make the derivation circular.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

No new physical or natural-kind entities are posited; the 'agents' and 'feature gate' are software components, not entities with independent falsifiable handles. The central claim rests on domain assumptions about SEC-filing structure and on the internal construction of the gate's training labels, which are generated without external gold labels.

free parameters (6)
  • Per-agent gating strength lambda_a = 0.1–0.3 per agent (validation-tuned; case study uses 0.30)
    Tuned on a validation split (§3.4.2, Appendix C.3); ablation shows robustness, but it is a fitted hyperparameter of the headline mechanism.
  • Retrieval budgets (k0, k, ke; final context ~35 chunks) = k0 summaries; top-k per path; ke final; ~35 chunks
    Hand-chosen in §3.3–§3.4; baselines in the matched-budget study use per-role retrieve_top_k=10, rerank_top_k=5 (Appendix B); the 35-chunk budget is fixed to isolate recall quality.
  • Near-duplicate removal threshold tau_sim = BGE-M3 cosine threshold (not numerically specified)
    Set during corpus preprocessing (Appendix A.1); affects which boilerplate is deduplicated before retrieval.
  • Gate training positive rate = ≈20%
    Downsampling rate for hard negatives in gate training (Appendix C.2); chosen by hand.
  • Agent roles / 10-K item mapping = 5 roles (general, quantitative, market, legal, company)
    Hand-designed and expert-validated (Table 1, §3.2); a design choice that encodes domain knowledge into the system.
  • 31-feature set for the gate = 31 features across 5 groups
    Hand-specified (Table 8); the choice determines what validity signal the gate can learn.
axioms (5)
  • domain assumption The mandated 10-K item structure is a stable, useful decomposition of evidence for filing QA.
    Agent roles in Table 1 are anchored to 10-K items (Item 1, 1A, 3, 7, 7A, 8); if filings diverge from these categories, role-specialized retrieval loses coverage.
  • domain assumption A query-dependent, summary-level view of the corpus (top-k0 FAISS-retrieved section summaries) is sufficient signal to align sub-query generation.
    Introduced in §3.3; no analysis is given beyond downstream recall improvements to show the summaries contain the needed terminology.
  • domain assumption Non-semantic features (retrieval paths, lexical overlap, metadata) carry signal about evidential validity beyond the cross-encoder.
    Core premise of §3.4.2 and Appendix C; supported only by SHAP on the gate's own training distribution, not by an external validity benchmark.
  • domain assumption LLM-as-judge scores (Likert 1–5) are valid and comparable across systems after blinding and order randomization.
    Used for all metrics; human validation (§4.5) covers only 150 answers on one benchmark, so judge validity on the other four benchmarks is assumed.
  • domain assumption Averaging 1–5 Likert scores yields an interval-scale correctness metric.
    Correctness and Likert dimensions are averaged across questions in Table 2, treating ordinal scales as interval; standard practice in Fin-RATE but an unproven measurement assumption.

pith-pipeline@v1.3.0-alltime-deepseek · 24036 in / 16104 out tokens · 157091 ms · 2026-08-01T15:59:04.503798+00:00 · methodology

0 comments
read the original abstract

Financial question answering over U.S. Securities and Exchange Commission (SEC) filings requires retrieving and synthesizing heterogeneous evidence dispersed across long, standardized, and highly redundant disclosures. Existing retrieval-augmented and multi-agent systems typically derive retrieval queries directly from the user's question and rank candidates by semantic similarity. Together, these choices create prior-corpus misalignment: a mismatch between model priors and the target filings' structure, terminology, and evidence standards. As a result, query generation misses corpus-specific evidence, while semantic reranking favors topically similar but evidentially invalid false-positive chunks. We propose FinSAgent, an evidence-grounded multi-agent framework that reframes SEC filing QA as corpus-aligned retrieval planning and corrects both ends with a single principle: inject corpus-side conditioning wherever model priors would otherwise dominate. FinSAgent combines (1) role-specialized agents anchored to the mandated 10-K item structure, (2) database-aware query decomposition that conditions each agent's sub-queries on a lightweight, summary-level view of the local corpus, and (3) multi-path retrieval with a learned feature-gated reranker that separates evidential validity from semantic similarity. Across five offline financial QA benchmarks, FinSAgent improves retrieval coverage and answer correctness over strong single-agent and multi-agent baselines; in a three-arm randomized online experiment with 1,000 anonymous user ratings, it also receives higher scores than baselines.

Figures

Figures reproduced from arXiv: 2607.18102 by Bohuai Xiao, Chaolong Jiang, Dingtao Hu, Hailin He, Hanwei Wu, Jerry Huang, Jianliang Lei, Jijun Chi, Jingrui Tian, Lei Ding, Liheng Ma, Ling Zhou, Muzhi Li, Peng Lu, Sicheng Lyu, Sizhe Guan, Tung Sum Thomas Kwok, Xinyu Wang, Yanzhang Ma, Yihan Li, Yihong Wu, Yufei Cui, Zhenghan Tai, Zixing Liao.

Figure 1
Figure 1. Figure 1: Answer correctness across five financial QA bench [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overall framework of FinSAgent. Role-specialized agents first construct complementary analytical views. Database￾aware decomposition then conditions each agent’s sub-queries on a lightweight summary-level view of the local filing database (correcting the front-end query gap), and multi-path retrieval with a feature-gated reranker selects reliable evidence (correcting the back-end validity gap), before evid… view at source ↗
Figure 3
Figure 3. Figure 3: Agent activation and contribution. Every special [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: System overhead. (a) Total token usage split [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Global SHAP feature importance. The gate relies [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Error subtype composition on SECQUE. FinSAgent achieves the lowest error count with a qualitatively shifted profile: C1 emerges while D1 is largely suppressed [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Error subtype composition on ZEEKR. The multi￾entity setting amplifies B4 in retrieval-only baselines; Fin￾SAgent reduces both B4 and D1. serves only as a contextual prior: final answers remain grounded in newly retrieved filing evidence. Architecture. The MemoryManager uses a URI-abstracted storage layer for horizontal scalability, an asynchronous queue-based dual￾write mechanism for peak shaving, a multi… view at source ↗
Figure 8
Figure 8. Figure 8: Multi-turn performance. The memory module im [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Orchestrator prompts, covering agent activation and answer synthesis. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: The instruction prompts for the Quant Analysis Agent, outlining the role description, query rewriting, and answering [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: The instruction prompts for the Company Researcher Agent, outlining the role description, query rewriting, and [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: The instruction prompts for the General Agent, outlining the role description, query rewriting, and answering [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: The instruction prompts for the Legal & Risk Agent, outlining the role description, query rewriting, and answering [PITH_FULL_IMAGE:figures/full_fig_p019_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: The instruction prompts for the Market Researcher Agent, outlining the role description, query rewriting, and [PITH_FULL_IMAGE:figures/full_fig_p020_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

75 extracted references · 14 linked inside Pith

  1. [1]

    Gagan Bhatia, El Moatez Billah Nagoudi, Hasan Cavusoglu, and Muhammad Abdul-Mageed. 2024. FinTral: A Family of GPT-4 Level Multimodal Finan- cial Large Language Models. InFindings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 1...

    doi:10.18653/v1/2024.findings-acl.774 2024

  2. [2]

    Tianshi Cai, Guanxu Li, Nijia Han, Ce Huang, Zimu Wang, Changyu Zeng, Yuqi Wang, Jingshi Zhou, Haiyang Zhang, Qi Chen, et al . 2025. FinDebate: Multi- Agent Collaborative Intelligence for Financial Analysis. InProceedings of The 10th Workshop on Financial Technology and Natural Language Processing. 268–282

    2025

  3. [3]

    Xinran Chen, Xuanang Chen, Ben He, Tengfei Wen, and Le Sun. 2024. Ana- lyze, Generate and Refine: Query Expansion with LLMs for Zero-Shot Open- Domain QA. InFindings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 11908–11922. do...

    doi:10.18653/v1/ 2024

  4. [4]

    Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. 2021. FinQA: A Dataset of Numerical Reasoning over Financial Data. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Marie-Francine Moens, Xuanjing Huang...

    2021

  5. [5]

    Zhiyu Chen, Shiyang Li, Charese Smiley, Zhiqiang Ma, Sameena Shah, and William Yang Wang. 2022. Convfinqa: Exploring the chain of numerical rea- soning in conversational finance question answering. InProceedings of the 2022 conference on empirical methods in natural language processing. 6279–6292

    2022

  6. [6]

    Chanyeol Choi, Jihoon Kwon, Jaeseon Ha, Hojun Choi, Chaewoon Kim, Yong- jae Lee, Jy yong Sohn, and Alejandro Lopez-Lira. 2025. FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Genera- tion. arXiv:2504.15800 [cs.IR] https://arxiv.org/abs/2504.15800

    Pith/arXiv arXiv 2025

  7. [7]

    Abhishek Darji, Fenil Kheni, Dhruvil Chodvadia, Parth Goel, Dweepna Garg, and Bankim Patel. 2024. Enhancing Financial Risk Analysis using RAG-based Large Language Models. In2024 3rd International Conference on Automation, Computing and Renewable Systems (ICACRS). 754–760. doi:10.1109/ICACRS62842. 2024.10841711

    arXiv 2024

  8. [8]

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2025. The faiss library.IEEE Transactions on Big Data(2025)

    2025

  9. [9]

    Lavanya Gupta, Saket Sharma, and Yiyun Zhao. 2024. Systematic evaluation of long-context LLMs on financial concepts. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track. 1163–1175

    2024

  10. [10]

    Siwei Han, Peng Xia, Ruiyi Zhang, Tong Sun, Yun Li, Hongtu Zhu, and Huaxiu Yao. 2025. Mdocagent: A multi-modal multi-agent framework for document understanding.arXiv preprint arXiv:2503.13964(2025)

    Pith/arXiv arXiv 2025

  11. [11]

    Suchow, Xiao-Yang Liu, Benyou Wang, Alejandro Lopez-Lira, Qianqian Xie, Sophia Ananiadou, and Junichi Tsujii

    Jimin Huang, Mengxi Xiao, Dong Li, Zihao Jiang, Yuzhe Yang, Yifei Zhang, Lingfei Qian, Yan Wang, Xueqing Peng, Yang Ren, Ruoyu Xiang, Zhengyu Chen, Xiao Zhang, Yueru He, Weiguang Han, Shunian Chen, Lihang Shen, Daniel Kim, Yangyang Yu, Yupeng Cao, Zhiyang Deng, Haohang Li, Duanyu Feng, Yongfu Dai, VijayaSai Somasundaram, Peng Lu, Guojun Xiong, Zhiwei Liu,...

  12. [13]

    Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. 2023. Financebench: A new benchmark for financial question answering.arXiv preprint arXiv:2311.11944(2023)

    Pith/arXiv arXiv 2023

  13. [14]

    Yidong Jiang, Junrong Chen, Eftychia Makri, Jialin Chen, Peiwen Li, Ali Maatouk, Leandros Tassiulas, Eliot Brenner, Bing Xiang, and Rex Ying. 2026. Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filings.arXiv preprint arXiv:2602.07294(2026)

    Pith/arXiv arXiv 2026

  14. [15]

    Jiajie Jin, Yuyao Zhang, Yimeng Xu, Hongjin Qian, Yutao Zhu, and Zhicheng Dou. 2025. Finsight: Towards real-world financial deep research.arXiv preprint arXiv:2510.16844(2025)

    arXiv 2025

  15. [16]

    Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. LightGBM: A Highly Efficient Gradient Boosting Decision Tree. InAdvances in Neural Information Processing Systems (NeurIPS), Vol. 30

    2017

  16. [17]

    Electronics Engineers, author

    Tung Sum Thomas Kwok, Chi-Hua Wang, Guang Cheng, Institute of Electrical, and issuing body. Electronics Engineers, author. 2025.GReaTER: Generate Realistic Tabular data after data Enhancement and Reduction. IEEE, [Place of publication not identified] :

    2025

  17. [18]

    Tung Sum Thomas Kwok, Xinyu Wang, Hengzhi He, Xiaofeng Lin, Peng Lu, Liheng Ma, Chunhe Wang, Ying Nian Wu, Lei Ding, and Guang Cheng

  18. [19]

    Bennett, and Marti A

    Philippe Laban, Tobias Schnabel, Paul N. Bennett, and Marti A. Hearst. 2022. SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summa- rization.Transactions of the Association for Computational Linguistics (TACL)10 (2022), 163–177

    2022

  19. [20]

    Changlun Li, Yao SHI, Chen Wang, Qiqi Duan, Runke RUAN, Weijie Huang, Hao- nan Long, Lijun Huang, Nan Tang, and Yuyu Luo. 2025. Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track. https://openreview.net/fo...

    2025

  20. [21]

    Rensis Likert. 1932. A technique for the measurement of attitudes.Archives of psychology(1932)

    1932

  21. [22]

    Aditya Prakash

    Haoxin Liu, Chenghao Liu, and B. Aditya Prakash. 2025. A Picture is Worth A Thousand Numbers: Enabling LLMs Reason about Time Series via Visualization. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Luis Chiruzzo, Alan Ritter...

    2025

  22. [23]

    Xiao-Yang Liu, Guoxuan Wang, Hongyang Yang, and Daochen Zha. 2023. Fingpt: Democratizing internet-scale data for financial large language models.arXiv preprint arXiv:2307.10485(2023)

    Pith/arXiv arXiv 2023

  23. [24]

    Yue Liu, Zhongying Ru, Shimin Di, Jipeng Zhang, Ruiyuan Zhang, and Xiaofang Zhou. 2026. Faithful in Steps: Improving Generalization and Citation in RAG via Query Decomposition. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 35671–35679

    2026

  24. [25]

    Qi Luo, Xiaonan Li, Yuxin Wang, Tingshuo Fan, Yuan Li, Xinchi Chen, and Xipeng Qiu. 2025. MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval.arXiv preprint arXiv:2510.27569(2025)

    arXiv 2025

  25. [27]

    Lebede Ngartera, Saralees Nadarajah, and Rodoumta Koina. 2026. Bayesian RAG: uncertainty-aware retrieval for reliable financial question answering. (2026)

    2026

  26. [28]

    Thang Nguyen, Peter Chin, and Yu-Wing Tai. 2025. Ma-rag: Multi-agent retrieval- augmented generation via collaborative chain-of-thought reasoning.arXiv preprint arXiv:2505.20096(2025)

    arXiv 2025

  27. [29]

    Min Pan, Yu Liu, Jinguang Chen, Ellen Anne Huang, and Jimmy X. Huang

  28. [30]

    Roxana Petcu, Kenton Murray, Daniel Khashabi, Evangelos Kanoulas, Maarten de Rijke, Dawn Lawrie, and Kevin Duh. 2025. Query Decomposition for RAG: Balancing Exploration-Exploitation.arXiv preprint arXiv:2510.18633(2025)

    arXiv 2025

  29. [31]

    Varshini Reddy, Rik Koncel-Kedziorski, Viet Dac Lai, Michael Krumdick, Charles Lovering, and Chris Tanner. 2024. Docfinqa: A long-context financial reason- ing dataset. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 445–458

    2024

  30. [32]

    Zhenghan Tai, Hanwei Wu, Qingchen Hu, Jijun Chi, Hailin He, Lei Ding, Tung Sum Thomas Kwok, Bohuai Xiao, Yuchen Hua, Suyuchen Wang, et al . 2025. VeritasFi: An Adaptable, Multi-tiered RAG Framework for Multi-modal Financial Question Answering.arXiv preprint arXiv:2510.10828(2025)

    arXiv 2025

  31. [33]

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal

  32. [34]

    Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, et al. 2024. Mineru: An open-source solution for precise document content extraction.arXiv preprint arXiv:2409.18839 (2024)

    Pith/arXiv arXiv 2024

  33. [35]

    Feng Wang, Yiding Sun, Jiaxin Mao, Xue Wei, and Danqing Xu. 2025. FinS- Pilot: A Benchmark for Online Financial RAG System. InProceedings of the 34th ACM International Conference on Information and Knowledge Management(Seoul, Republic of Korea)(CIKM ’25). Association for Computing Machinery, New York, NY, USA, 6544–6548. doi:10.1145/3746252.3761643

    arXiv 2025

  34. [36]

    Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. 2024. Mixture-of-agents enhances large language model capabilities.arXiv preprint arXiv:2406.04692(2024)

    Pith/arXiv arXiv 2024

  35. [37]

    Xinyu Wang, Jijun Chi, Zhenghan Tai, Tung Sum Thomas Kwok, Hailin He, Zhuhong Li, Yuchen Hua, Muzhi Li, Peng Lu, Suyucheng Wang, et al . 2025. Finsage: A multi-aspect rag system for financial filings question answering. InProceedings of the 34th ACM International Conference on Information and Knowledge Management. 6144–6152

    2025

  36. [38]

    Ziao Wang, Yuhang Li, Junda Wu, Jaehyeon Soon, and Xiaofeng Zhang. 2023. FinVis-GPT: A Multimodal Large Language Model for Financial Chart Analysis. arXiv:2308.01430 [cs.CL] https://arxiv.org/abs/2308.01430

    Pith/arXiv arXiv 2023

  37. [39]

    2026.R 2R: A Post-training Framework for Multi-domain Decoder-Only Rerankers

    Hanwei Wu, Qingchen Hu, Zhenghan Tai, Jingrui Tian, Lei Ding, Jijun Chi, Hailin He, Tung Sum Thomas Kwok, Yufei Cui, Sicheng Lyu, Muzhi Li, Mingze Li, Xinyue Yu, Ling Zhou, Peng Lu, and Xinyu Wang. 2026.R 2R: A Post-training Framework for Multi-domain Decoder-Only Rerankers. InAdvances in Knowledge Discovery and Data Mining, Raymond Chi-Wing Wong, Hanghan...

    2026

  38. [40]

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. 2024. Autogen: Enabling next-gen LLM applications via multi-agent conversations. InFirst conference on language modeling

    2024

  39. [41]

    Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebas- tian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann

  40. [42]

    Yihong Wu, Liheng Ma, Muzhi Li, Jiaming Zhou, Lei Ding, Jianye Hao, Ho-fung Leung, Irwin King, Yingxue Zhang, and Jian-Yun Nie. 2025. Advancing Multi- Agent RAG Systems with Minimalist Reinforcement Learning.arXiv preprint arXiv:2505.17086(2025)

    Pith/arXiv arXiv 2025

  41. [43]

    Yingqian Wu, Qiushi Wang, Zefei Long, Rong Ye, Zhongtian Lu, Xianyin Zhang, Bingxuan Li, Wei Chen, Liwen Zhang, and Zhongyu Wei. 2025. FinTeam: A Multi- agent Collaborative Intelligence System for Comprehensive Financial Scenarios. InCCF International Conference on Natural Language Processing and Chinese Computing. Springer, 443–455

    2025

  42. [44]

    Xiaobo Xing, Wei Yuan, Tong Chen, Quoc Viet Hung Nguyen, Xiangliang Zhang, and Hongzhi Yin. 2025. TableDART: Dynamic Adaptive Multi-Modal Routing for Table Understanding.arXiv preprint arXiv:2509.14671(2025)

    arXiv 2025

  43. [45]

    Shi-Qi Yan, Ya Li, Quan Liu, and Zhen-Hua Ling. 2026. Learn to be Honest: Mitigate LLMs’ Overconfidence for Improving Hallucination Detection with Self-Hesitation Activation. https://openreview.net/forum?id=FRtKUpgEZ9

    2026

  44. [46]

    Antonio Jimeno Yepes, Yao You, Jan Milczek, Sebastian Laverde, and Renyu Li

  45. [47]

    Noga Ben Yoash, Meni Brief, Oded Ovadia, Gil Shenderovitz, Moshik Mishaeli, Rachel Lemberg, and Eitam Sheetrit. 2025. SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities.arXiv preprint arXiv:2504.04596 (2025)

    Pith/arXiv arXiv 2025

  46. [48]

    Le Zhang, Yihong Wu, Qian Yang, and Jian-Yun Nie. 2024. Exploring the Best Practices of Query Expansion with Large Language Models. InFindings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 1872–1883. doi:10.18653/v1/202...

    doi:10.18653/v1/2024.findings-emnlp.103 2024

  47. [49]

    Xuanyu Zhang and Qing Yang. 2023. Xuanyuan 2.0: A large chinese financial chat model with hundreds of billions parameters. InProceedings of the 32nd ACM international conference on information and knowledge management. 4435–4439

    2023

  48. [50]

    Suifeng Zhao, Zhuoran Jin, Sujian Li, and Jun Gao. 2025. FinRAGBench-V: A Benchmark for Multimodal RAG with Visual Citation in the Financial Domain. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational L...

    doi:10.18653/v1/2025.emnlp-main.211 2025

  49. [51]

    Andy Zhu and Yingjun Du. 2025. A Role-Aware Multi-Agent Framework for Financial Education QA. InProceedings of the 6th ACM International Conference on AI in Finance. 483–491

    2025

  50. [52]

    arXiv preprint arXiv:2402.05131(2024)

    Financial report chunking for effective retrieval augmented generation. arXiv preprint arXiv:2402.05131(2024)

    Pith/arXiv arXiv 2024

  51. [53]

    Fengbin Zhu, Ziyang Liu, Fuli Feng, Chao Wang, Moxin Li, and Tat Seng Chua

  52. [59]

    InProceedings of the 5th ACM International Conference on AI in Finance

    Tat-llm: A specialized language model for discrete reasoning over financial tabular and textual data. InProceedings of the 5th ACM International Conference on AI in Finance. 310–318

  53. [61]

    selected_agents

    TAT-LLM: A Specialized Language Model for Discrete Reasoning over Financial Tabular and Textual Data. InProceedings of the 5th ACM International Conference on AI in Finance(Brooklyn, NY, USA)(ICAIF ’24). Association for Com- puting Machinery, New York, NY, USA, 310–318. doi:10.1145/3677052.3698685 A Datasets, Preprocessing, and Baselines A.1 Corpus Constr...

    arXiv

  54. [62]

    (Do silently.)

    For open-ended question, first infer the user’s intent. (Do silently.)

  55. [63]

    Every sub-question must stay tightly anchored to the original intent

  56. [64]

    If the original question is already atomic and data-seeking, include it as one of the sub-questions

  57. [65]

    Not found in provided data

    If the question asks for quater data, then it refers to the three-month period, not the cumulative data, unless explicitly stated. Return only a JSON array of strings. User question: {question} History: {history} Phase 2: Answer Generation You are a Quant Analysis Specialist.Rules: - No speculation. If data is missing, say "Not found in provided data". - ...

  58. [66]

    Key Findings (bullet points with period + number)

  59. [67]

    Supporting Evidence (table refs / snippets)

  60. [68]

    Computations (show formula + steps)

  61. [69]

    Accounting Policy Notes (ONLY if the question asks policy or the evidence explicitly affects interpretation)

  62. [70]

    Not found in provided evidence

    Missing Data (only items blocking the answer) Figure 10: The instruction prompts for the Quant Analysis Agent, outlining the role description, query rewriting, and answering phases. FinSAgent: Corpus-Aligned Multi-Agent RAG Framework for Evidence-Grounded SEC Filing Question Answering Prompt for Company Agent Role Description Summary:Focuses on company hi...

  63. [71]

    Company & Industry Overview - Company growth: - Founder/management background:

  64. [72]

    History & Ownership Structure - Timeline (chronological bullets with dates): - Equity/Control events: - Special arrangements:

  65. [73]

    Not evidenced in provided documents

    Uncertainties Figure 11: The instruction prompts for the Company Researcher Agent, outlining the role description, query rewriting, and answering phases. Chi et al. Prompt for General Agent Role Description Summary:Lightweight responder for simple or low-risk questions; keeps answers minimal. Responsibilities: - Rewrite or split the user query into at mos...

  66. [74]

    Extracted Evidence (short quoted snippets + where found)

  67. [75]

    Risk Assessment (risk→mechanism→severity→confidence)

  68. [76]

    Compliance Checklist (items found vs not found)

  69. [77]

    Deal Terms Summary (if deal context)

  70. [78]

    Not found in provided evidence

    Missing Documents / Questions for Counsel Figure 13: The instruction prompts for the Legal & Risk Agent, outlining the role description, query rewriting, and answering phases. Chi et al. Prompt for Market Agent Role Description Summary:Covers industry and market context for the user question. Responsibilities: - Industry/market size and growth - Company m...

  71. [2018]

    In Proceedings of NAACL-HLT

    FEVER: a Large-scale Dataset for Fact Extraction and VERification. In Proceedings of NAACL-HLT

  72. [2023]

    Bloomberggpt: A large language model for finance.arXiv preprint arXiv:2303.17564(2023)

    Pith/arXiv arXiv 2023

  73. [2024]

    doi:10.1038/s41598- 024-82871-0

    A multi-dimensional semantic pseudo-relevance feedback framework for information retrieval.Scientific Reports14, 1 (2024), 31806. doi:10.1038/s41598- 024-82871-0

    doi:10.1038/s41598- 2024

  74. [2025]

    arXiv:2408.11878 [cs.CL] https://arxiv.org/abs/2408.11878

    Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications. arXiv:2408.11878 [cs.CL] https://arxiv.org/abs/2408.11878

    Pith/arXiv arXiv

  75. [2026]

    Chi et al

    Enhancing TableQA through Verifiable Reasoning Trace Reward. Chi et al. arXiv:2601.22530 [cs.AI] https://arxiv.org/abs/2601.22530

    Pith/arXiv arXiv