{"id":"0f99f794-13c7-497c-be84-40a2139417d2","arxiv_id":"2412.12827","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"TabSniper reports improved table detection and structure recognition on bank statements by fine-tuning DETR with CIoU loss, long-table split-merge, and padding variations, evaluated on a private dataset and two public benchmarks.","lead":"TabSniper is a machine-learning pipeline that finds tables in bank statements, identifies their rows and columns, and converts them into structured transaction records. It is built by fine-tuning DETR object-detection models on a new private dataset of annotated bank statements, and it reports gains over several table-extraction baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed end-to-end transaction extraction is never measured: Section 4.4 discards transactions lacking Date/Amount, and no checksum or transaction-level result is reported anywhere.","rationale":"The reader's weakest assumption identifies the same gap: the end-to-end accuracy claim assumes every page is axis-aligned and every transaction has a detectable Date/Amount, and the paper provides no evidence of how often these assumptions hold. I agree. The internal TSR ablations are plausible and the external comparisons on PubTables-1M and FinTabNet provide some evidence that the TSR module generalizes, but they do not measure the end-to-end transaction extraction that the abstract and introduction promise. Because even a perfect table detector can fail the downstream task if text-cell matching drops rows or discards transactions, the absence of any transaction-level metric (including the checksum result that the authors themselves define) is a correctness risk, not merely a missing nicety. I would keep the REJECT verdict: the current manuscript's central claim is unsupported. If the authors release data and code and the proposed test shows high transaction F1 and low discard rates, a conditional acceptance would be warranted. No credibility attack is intended; the concern is about the evidence-to-claim match, and it is resolvable by a concrete, reportable experiment.","tokens_in":12642,"tokens_out":3148,"duration_ms":27704,"concrete_test":"Run the full TabSniper pipeline on the BankTabNet test page set (or a public substitute annotated with transaction rows) and report: (i) transaction-level precision, recall, and F1, matching ground-truth transactions on date, amount, description, and debit/credit category; (ii) the fraction of ground-truth transactions discarded because Date or Amount was not detected by the pipeline (Section 4.4); (iii) the per-statement checksum pass rate, checking OpenBal − Σ(debits) + Σ(credits) − EndBal = 0, with a stated tolerance for rounding; and (iv) the same transaction metrics restricted to tilted or scanned pages. If transaction-level F1 is materially below the claimed 'high-quality extraction' or the discard rate is non-negligible, the paper's central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that TabSniper accurately extracts transaction information from bank statements into a structured format—is not supported by any transaction-level evaluation. All quantitative results (Tables 4–6) are object-detection AP/AR scores for table/cell bounding boxes. Section 4.4 states 'For every extracted transaction we make sure that its “Date” and “Amount” exists otherwise it is discarded,' yet the paper never reports how many transactions are discarded in practice. The checksum defined in Section 4.4 is the only proposed end-to-end validation, but no checksum pass rate or transaction F1 appears in the results. Section 6.2.2 admits that tilted scanned pages cause predicted cell boundaries to cut text, leading to partial OCR capture. Consequently, high box-level AP/AR can coexist with many missing or garbled transactions; the headline claim depends on the unverified assumption that the discard rate is negligible and that rows missed by the detector are rare. The private dataset and code are not released, so these numbers cannot be independently checked. This is the load-bearing gap between the claimed end-to-end accuracy and the evidence actually presented.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents TabSniper, an end-to-end pipeline for detecting and categorizing tables in bank statements and recognizing their structure, with the stated goal of extracting transaction-level information into a structured format. It introduces a private annotated dataset, BankTabNet, for table detection/categorization (TDC) and table structure recognition (TSR). The TDC stage uses a DETR-based vision model plus a text-based Naive Bayes classifier; the TSR stage uses DETR with split-merge handling for long tables, padding augmentations, and a CIoU loss. A post-processing module maps detected cells to OCR text and assembles transactions, with an optional checksum validation. The paper reports AP/AR scores for TDC and TSR, ablation studies, comparisons against baselines, and results on external public TSR datasets.","tokens_in":12870,"tokens_out":4873,"duration_ms":43721,"significance":"If the claimed end-to-end accuracy held, TabSniper would be practically valuable for automating bank statement processing, and the BankTabNet dataset could support further research. The paper has several concrete strengths: inter-annotator agreement is reported for both annotation tasks; the TDC and TSR comparisons include multiple baselines; the ablations in Table 5 correspond to clearly described design choices; and the external-dataset results in Table 6 provide some evidence of generalizability. However, the distinctive claim in the abstract and Section 1—that TabSniper extracts transaction information accurately—is not directly measured anywhere in the paper. All quantitative results are object-detection AP/AR for table regions and cells. The checksum defined in Section 4.4 is never reported, and the transaction rows discarded by the Section 4.4 rule are never counted. Until transaction-level metrics are provided, the significance of the paper relative to its stated goal remains unverified, though the TSR improvements alone are a modest but reasonable contribution.","major_comments":[{"comment":"The headline claim of \"high-quality extraction of transaction information\" is not supported by any transaction-level evaluation. Tables 4-6 report only AP/AR for table regions, rows, columns, and cells. No transaction precision/recall/F1, no checksum pass rate, and no end-to-end output accuracy is reported. Please either add these metrics on the BankTabNet test set or revise the claims to be about table detection and structure recognition only.","section":"Abstract, Section 7, Section 6.1"},{"comment":"The rule \"For every extracted transaction we make sure that its 'Date' and 'Amount' exists otherwise it is discarded\" is a silent failure mode. The paper never reports how many transactions are discarded in practice, and the Section 6.2.2 discussion of tilted scans causing partial OCR capture suggests this discard rule could remove a substantial fraction of rows. Please quantify the discard rate on the test set and its effect on the checksum and transaction extraction accuracy.","section":"Section 4.4"},{"comment":"BankTabNet is private and not released, so the central TSR results in Table 5 cannot be independently reproduced or compared against by other researchers. Please release the dataset and annotations, or provide a reproducible end-to-end evaluation on a public benchmark and report detailed dataset statistics. Without this, the generalizability and comparative claims rest entirely on an unreleased resource.","section":"Section 3.2, Section 5.2"},{"comment":"The ablation gains, e.g., the +2.9 AP improvement from Complete IoU Loss (83.1 vs 80.2), are reported on a single test split without variance or significance testing. Since the ablations are a stated contribution, please report results across multiple seeds or provide confidence intervals so the reader can judge whether the differences are stable.","section":"Table 5, Section 5.2"}],"minor_comments":[{"comment":"The text says the TSR model uses \"five object classes\" but later refers to \"a seventh implicit class, table grid cell,\" and the class list in Section 3.2 has six categories. Please make the class count consistent.","section":"Section 4.3"},{"comment":"The sentence \"DETR is pre-trained on the PubTables-1M\" is imprecise: DETR is an architecture, and PubTables-1M is a dataset. Please clarify what weights were initialized from (e.g., a Table Transformer pretrained on PubTables-1M or a DETR pretrained on COCO).","section":"Section 4.1"},{"comment":"The inference input size is given as 1200×1200 pixels while the training description says aspect ratio is kept intact. Please state explicitly how aspect ratio is preserved at inference time.","section":"Section 5.1.2"},{"comment":"The \"CPU Inference\" column uses checkmarks and dashes, but inference times are given only for some rows. Please provide times for all models or explain why they are not applicable.","section":"Table 4"},{"comment":"The reference to \"sample5 of Figure 7\" is not self-contained because the figure panels are not individually labeled in the text. Please label the panels and describe the failure mode more concretely.","section":"Section 6.2.2"}],"recommendation":"major_revision","confidential_remarks":"The missing transaction-level evaluation is the core issue. If the authors can supply transaction-level precision/recall, checksum pass rates, and discard rates on BankTabNet, the paper could become a solid applied contribution. If they cannot, the claims should be narrowed to table detection and structure recognition, which would substantially reduce the paper's novelty. The private dataset is also a concern for a venue that values reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid engineering paper that overclaims at the edges. The TSR ablations (split-merge, padding, CIoU loss) show real gains on their internal BankTabNet test set, and the generalization check on PubTables-1M and FinTabNet is a reasonable add. But the abstract and conclusion promise accurate transactional extraction, and no transaction-level metric appears anywhere. All quantitative results are object-detection AP/AR for boxes. That gap is the one that matters.\n\nWhat's actually new: BankTabNet, a private dataset of bank statement tables with page- and transaction-level annotations, is a legitimate contribution even if not public. The table categorization via DETR plus Naive Bayes refinement from captions/headers is a practical idea. The long-table split-merge and padding variations are sensible engineering, and Table 5 shows they help on the internal benchmark. The external results (Table 6) suggest the TSR model generalizes decently to other table types when trained on them.\n\nSoft spots, in proportion: the core problem is that the paper's central claim is unsupported as stated. Section 4.4 discards any transaction without a Date and Amount, but never quantifies the discard rate. Section 6.2.2 admits tilted mobile captures cut text and lose OCR words. High box-level AP/AR can coexist with a large fraction of missing or garbled transactions, so the reader cannot trust the headline. Also: no error bars or variance estimates, data and code are not released, and the checksum rule from Section 4.4 is defined but never measured. The comparison on external datasets is selective (two datasets, trained separately), which is fine but modest.\n\nBottom line: the narrow claim—that these TSR modifications improve box-level detection on bank statement tables—is supported by the internal evidence. The broad claim of accurate end-to-end transaction extraction is not measured and therefore not supported. This is a load-bearing gap, but it is fixable with more evaluation, not a fundamental flaw in the approach. The paper deserves a serious peer review, not a desk reject, but the referee should push hard for transaction-level metrics (checksum pass rate, transaction F1, discard rate) and ideally a release of data/code. If the authors cannot provide those, the claims should be narrowed to table structure recognition performance, not transaction extraction.","headline":"Competent DETR-based table structure recognition for bank statements, but the headline claim of transaction-level extraction is never measured—only box-level AP/AR is reported.","tokens_in":13437,"tokens_out":1969,"would_cite":false,"duration_ms":20500,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TabSniper extracts every transaction from bank statement tables","keywords":["table detection","table structure recognition","bank statement processing","DETR","transaction extraction","document AI","BankTabNet","CIoU loss"],"falsifier":"Take a sample of bank statements that includes tilted or skew-corrected scans and mobile captures, run TabSniper end-to-end, and compare transaction-level precision and recall (not just table-geometry AP/AR) against the image-level annotations in BankTabNet; if transaction recall on such pages is materially below the table-geometry metrics, the end-to-end claim overstates real-world accuracy.","tokens_in":12399,"feed_emoji":"💳","tokens_out":3850,"duration_ms":30696,"temperature":0.7,"pith_summary":"This paper tries to establish that transactional information can be extracted from bank statement tables end-to-end: detecting each table, categorizing it by type (credit, debit, check, balance, etc.), recognizing its rows and columns, and converting the cells into a standardized transaction record. If correct, it would make bank-statement processing for credit underwriting largely automatic across many bank templates, including long multi-page tables and densely packed multi-line rows, which existing table-structure systems handle poorly. The authors back the claim with a new annotated dataset (BankTabNet), a two-stage DETR-based pipeline, and a balance-checksum post-processing step that verifies extraction completeness.","feed_headline":"TabSniper extracts every transaction from bank statement tables","feed_subtitle":"A DETR-based pipeline detects, categorizes, and parses tables across bank layouts, then balance-checks the output.","key_machinery":"The pipeline is built on DETR (a transformer object detector with set-based loss) fine-tuned with BankTabNet. The TDC model detects tables in ten categories, refined by a multinomial Naive Bayes classifier over caption/header text. The TSR model detects five structural object classes (table, row, column, column header, spanning row) whose intersections implicitly define grid cells; key enhancements are padding variations around table images, splitting long tables (>20 rows) horizontally and merging predictions, and replacing GIoU with Complete IoU (CIoU) loss to improve row alignment. A checksum over opening balance, extracted debits/credits, and closing balance serves as a deterministic completeness check.","core_discovery":"TabSniper is presented as the first end-to-end system for bank statement spreading: a table detection and categorization (TDC) model, a table structure recognition (TSR) model built on DETR, and a post-processing module that assembles transactions and verifies them against opening and closing balances. The central quantitative claim is that TabSniper outperforms strong baselines—Faster/Mask/Cascade R-CNN, DiT, and Dynamic Head for detection; Table Transformer, LGPMA, and TabStructNet for structure—on the BankTabNet test set, with the TSR module reaching AP50 94.8 and AR 90.6, and that it also generalizes to public datasets PubTables-1M and FinTabNet with better AP/AR than Table Transformer. The qualitative claim is that the system captures complete transaction descriptions in multi-line rows where baselines cut text.","pith_inferences":["The checksum verification suggests a natural active-learning loop: statements whose checksum does not balance could be routed for human review, and those corrections could retrain the models—an extension the paper does not spell out.","Because the pipeline already masks personally identifiable information before processing, it could be paired with differential privacy or on-device OCR to make bank-statement analysis more privacy-preserving at scale.","A likely next test is whether the Naive Bayes category refinement could be replaced by a small language model on caption/header text, which might improve generalization to unseen bank templates without new vision training data.","The reported CPU inference time (1.91 minutes per 20-page statement) suggests the method is deployable in batch, but real-time interactive use may still require GPU acceleration; the paper does not quantify GPU latency."],"forward_implications":["Financial underwriting can ingest raw bank statement PDFs and receive structured, categorized transaction lists without manual data entry, with a built-in balance check flagging incomplete extractions.","The same TSR backbone transfers to other financial documents (e.g., FinTabNet) and to scientific tables (PubTables-1M), so the method is not limited to bank layouts.","The category refinement step (vision table detection plus text-based Naive Bayes on captions and headers) lets one detection model handle visually similar credit and debit tables.","The split-merge handling of long tables and the CIoU loss directly target known failure modes of existing TSR systems on long, dense tables."],"supporting_citations":[{"why":"DETR is the base object-detection architecture for both the TDC and TSR models, providing the set-based loss and query mechanism.","marker":"[3]"},{"why":"PubTables-1M supplies pre-training data for DETR and defines the Table Transformer baseline that TabSniper must beat in TSR.","marker":"[18]"},{"why":"LGPMA is a strong TSR baseline included in the comparison to show TabSniper's advantage on long, complex bank tables.","marker":"[14]"},{"why":"TabStructNet is another end-to-end TSR baseline that TabSniper compares against on structure recognition.","marker":"[15]"},{"why":"FinTabNet is a public financial-document TSR dataset used to benchmark TabSniper's generalizability outside bank statements.","marker":"[23]"},{"why":"Dynamic Head is a high-accuracy detection baseline that TabSniper compares against on table detection, showing competitive accuracy at lower inference cost.","marker":"[4]"}],"fun_headline_variants":["TabSniper beats baselines on bank table parsing","TabSniper extracts full transactions from complex bank tables","TabSniper: end-to-end table detection and structure for banks","TabSniper captures complete transaction rows baselines miss"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim assumes that every bank statement page is roughly upright and axis-aligned, and that every transaction row has a detectable Date and Amount; the paper's own limitation section notes that tilted mobile captures cause predicted boxes to cut text, and the post-processing discards any row missing a Date or Amount.","fun_headline_variants_meta":{"raw":{"variants":["TabSniper beats baselines on bank table parsing","TabSniper extracts full transactions from complex bank tables","TabSniper: end-to-end table detection and structure for banks","TabSniper captures complete transaction rows baselines miss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000215,"raw_usage":{"total_tokens":1428,"prompt_tokens":942,"completion_tokens":486,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":416}},"tokens_in":558,"tokens_out":486,"duration_ms":4985,"temperature":1.0,"reasoning_tokens":416,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:41:30.202660+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a sample of bank statements that includes tilted or skew-corrected scans and mobile captures, run TabSniper end-to-end, and compare transaction-level precision and recall (not just table-geometry AP/AR) against the image-level annotations in BankTabNet; if transaction recall on such pages is materially below the table-geometry metrics, the end-to-end claim overstates real-world accuracy.","supporting_citations":[{"cited_title":"PubTables-1M: Towards comprehensive table extraction from unstructured documents","cited_arxiv_id":"2110.00061","evidence_quote":"PubTables-1M supplies pre-training data for DETR and defines the Table Transformer baseline that TabSniper must beat in TSR."},{"cited_title":"LGPMA: Complicated Table Structure Recognition with Local and Global Pyramid Mask Alignment","cited_arxiv_id":"2105.06224","evidence_quote":"LGPMA is a strong TSR baseline included in the comparison to show TabSniper's advantage on long, complex bank tables."},{"cited_title":"Table Structure Recognition using Top-Down and Bottom-Up Cues","cited_arxiv_id":"2010.04565","evidence_quote":"TabStructNet is another end-to-end TSR baseline that TabSniper compares against on structure recognition."},{"cited_title":"In Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"FinTabNet is a public financial-document TSR dataset used to benchmark TabSniper's generalizability outside bank statements."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Dynamic Head is a high-accuracy detection baseline that TabSniper compares against on table detection, showing competitive accuracy at lower inference cost."}],"review_version":1}