Multi-Stream Temporal Fusion for Financial Fraud Detection

Mohammadamin Dashti Moghaddam; Nick Sciarrilli

arxiv: 2606.25007 · v1 · pith:U4FQSQYRnew · submitted 2026-06-23 · 💻 cs.LG · q-fin.ST

Multi-Stream Temporal Fusion for Financial Fraud Detection

Mohammadamin Dashti Moghaddam , Nick Sciarrilli This is my paper

Pith reviewed 2026-06-25 23:46 UTC · model grok-4.3

classification 💻 cs.LG q-fin.ST

keywords financial fraud detectionmulti-stream transformertime-aware positional encodingevent stream fusiontransformer encodersAUROCdigital bankinggated fusion

0 comments

The pith

A multi-stream transformer fuses separate encodings of banking event streams with time-aware positional encoding to reach 0.9961 AUROC in fraud detection.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that financial fraud often hides across multiple heterogeneous event streams that appear normal when viewed alone. It proposes the Multi-Stream Fraud Transformer, which runs an independent transformer encoder on each stream before fusing the outputs through one of several mechanisms. The key result is that time-aware positional encoding among the fusion options delivers the highest discrimination at 0.9961 AUROC on a 10-million-user dataset with a 1.5 percent fraud rate. This approach also produces an 18-point AUROC gap over a single-stream transformer baseline and a 22 percent relative gain over an XGBoost model on proprietary production data. Readers would care because the work shows a concrete architecture that turns per-stream inductive bias into measurable detection gains.

Core claim

The Multi-Stream Fraud Transformer encodes each event stream—transactions, login sessions, risk signals—with its own transformer encoder and fuses the representations using one of five strategies. On the 10M-user dataset the time-aware positional encoding strategy reaches 0.9961 AUROC, gated fusion reaches 0.989 precision, per-stream encoding is required to avoid an 18-point drop, and the full method improves more than 22 percent over the XGBoost baseline on production data.

What carries the argument

The Multi-Stream Fraud Transformer, which applies independent transformer encoders to each input stream before applying one of five configurable fusion mechanisms.

If this is right

Sequence models operating on raw streams outperform gradient-boosted trees on aggregated features by a wide margin.
Per-stream encoding is required; collapsing everything into one stream drops performance from 0.99 to 0.82 AUROC.
Time-aware positional encoding produces the highest discrimination among the five fusion strategies tested.
Gated fusion delivers the highest precision of 0.989, making it the most suitable for production thresholds.
The risk event stream supplies the strongest single-stream signal contribution.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same per-stream encoding plus fusion pattern could be tested on other problems that combine asynchronous event streams, such as user sessions in e-commerce or sensor logs in industrial monitoring.
Production teams might choose gated fusion over time-aware encoding when precision at low false-positive rates matters more than raw AUROC.
An experiment that varies the number and types of input streams while holding model size fixed would show how robust the fusion rankings remain.

Load-bearing premise

The reported AUROC and precision figures reflect genuine detection of new fraud patterns rather than fitting to the specific dataset or the way labels were defined.

What would settle it

Evaluating the same trained models on a fresh dataset collected from a different digital bank that uses different event definitions and fraud labeling rules, then checking whether the 0.9961 AUROC is reproduced.

Figures

Figures reproduced from arXiv: 2606.25007 by Mohammadamin Dashti Moghaddam, Nick Sciarrilli.

**Figure 2.** Figure 2: Five fusion mechanisms compared in our ablation. (a) Concat: simple concatenation. (b) Gated: learned per-stream [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Figure 3: Main results across models on the 10M user test set. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

**Figure 4.** Figure 4: Stream ablation showing the contribution of each [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

read the original abstract

Financial fraud detection in digital banking requires reasoning over multiple heterogeneous event streams -- transactions, login sessions, risk signals -- that individually appear benign but collectively reveal fraudulent patterns. We propose the Multi-Stream Fraud Transformer (MSFT), a unified architecture that encodes each event stream with independent Transformer encoders and fuses their representations through configurable mechanisms. We conduct a systematic ablation study comparing five fusion strategies: concatenation, gated fusion, time-aware positional encoding, cross-stream attention, and a full combination. On a large-scale dataset (10M users, 1.5% fraud rate) with 85M parameter models, we demonstrate that (1) sequence models significantly outperform gradient-boosted trees operating on aggregated features (0.74 vs. 0.99 AUROC), (2) per-stream encoding is essential -- a single-stream Transformer baseline with matched parameter budget reaches only 0.82 AUROC, an 18-point gap that confirms the multi-stream inductive bias is necessary, (3) time-aware positional encoding achieves the highest discrimination (0.9961 AUROC), (4) gated fusion yields the best precision (0.989) suitable for production deployment, and (5) the risk event stream provides the strongest individual signal contribution. We further validate on proprietary production data from a digital banking platform, showing over 22% relative AUROC improvement over the XGBoost baseline.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Solid ablation on fusion strategies for multi-stream fraud data, but the 0.9961 AUROC rests on an unverified temporal split that needs checking before the numbers can be trusted.

read the letter

The main thing to know is that this paper runs a clean ablation on five ways to fuse per-stream Transformer outputs for fraud detection and finds that time-aware positional encoding works best on their 10M-user banking dataset while gated fusion wins on precision. The multi-stream setup beats both a single-stream baseline (0.82 vs 0.99 AUROC) and an XGBoost model on aggregated features (0.74), which is the kind of practical result that matters for people who actually run these systems.

What the work does well is keep the comparison focused: each stream gets its own encoder, then the five fusion variants are tested head-to-head with matched parameter budgets. The risk-event stream coming out strongest also lines up with domain intuition. That systematic check is more useful than another generic transformer paper.

The soft spot is the evaluation protocol. The abstract and stress-test note both flag that we do not see the exact train-test split procedure or how timestamps were used to enforce causality. In time-series fraud data that is the load-bearing assumption behind the 22% lift and the 0.9961 number. If the split allows any future leakage, the gap between single-stream and multi-stream shrinks in importance. Hyperparameter search details and label-noise handling are also thin, though those are common in applied work.

This is for applied ML groups or fraud teams who want to see concrete fusion comparisons rather than new theory. It is coherent on its own terms and shows honest engagement with the task, so it deserves a serious referee who can verify the split and the baseline tuning. I would send it to review.

Referee Report

2 major / 1 minor

Summary. The paper proposes the Multi-Stream Fraud Transformer (MSFT), which encodes heterogeneous event streams (transactions, logins, risk signals) using independent Transformer encoders and fuses their representations via five strategies (concatenation, gated fusion, time-aware positional encoding, cross-stream attention, full combination). On a 10M-user dataset (1.5% fraud rate) with 85M-parameter models, it reports that time-aware positional encoding yields the highest AUROC of 0.9961, per-stream encoding is essential (single-stream baseline at 0.82 AUROC), sequence models outperform XGBoost on aggregated features (0.99 vs 0.74 AUROC), gated fusion achieves best precision (0.989), and the approach delivers 22% relative AUROC improvement over XGBoost on proprietary production data.

Significance. If the temporal evaluation is sound, the systematic ablation of fusion mechanisms and the quantified gap between multi-stream and single-stream models would provide useful evidence for the value of per-stream inductive biases in fraud detection. The large-scale dataset and production validation are positive aspects of the empirical design.

major comments (2)

[Experimental Setup / Dataset description] The manuscript provides no description of the train-test split procedure (e.g., whether timestamps enforce a strictly future-held-out test set with no cross-stream event leakage). This detail is load-bearing for the central claims of 0.9961 AUROC for time-aware positional encoding and the 18-point gap to the single-stream baseline, as any non-causal split would render the reported discrimination unreliable in time-series fraud data.
[Ablation study / Model comparison] No information is given on the hyperparameter tuning budget or search procedure applied to the 85M-parameter MSFT variants versus the XGBoost baseline. This directly affects the validity of the performance gap reported in the abstract (0.74 vs 0.99 AUROC) and the ablation conclusions.

minor comments (1)

[Abstract] The abstract states a 22% relative AUROC improvement on production data but does not report the absolute AUROC values for either MSFT or the XGBoost baseline on that dataset.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address each major comment below and commit to revisions that strengthen the experimental reporting without altering the core claims.

read point-by-point responses

Referee: [Experimental Setup / Dataset description] The manuscript provides no description of the train-test split procedure (e.g., whether timestamps enforce a strictly future-held-out test set with no cross-stream event leakage). This detail is load-bearing for the central claims of 0.9961 AUROC for time-aware positional encoding and the 18-point gap to the single-stream baseline, as any non-causal split would render the reported discrimination unreliable in time-series fraud data.

Authors: We agree that an explicit description of the split procedure is necessary to support the temporal validity of the results. We will add a subsection in the Experimental Setup detailing that all experiments, including the main results and production validation, used a strict temporal split based on event timestamps to enforce a future-held-out test set. Per-stream encoders operate independently on timestamp-aligned events, which precludes cross-stream leakage. Due to the proprietary nature of the financial dataset we cannot publish exact cutoff dates, but the procedure guarantees no future-to-past leakage. revision: yes
Referee: [Ablation study / Model comparison] No information is given on the hyperparameter tuning budget or search procedure applied to the 85M-parameter MSFT variants versus the XGBoost baseline. This directly affects the validity of the performance gap reported in the abstract (0.74 vs. 0.99 AUROC) and the ablation conclusions.

Authors: We acknowledge the omission of tuning details. We will revise the Experimental Setup section to describe the hyperparameter search procedure applied to both the MSFT variants and the XGBoost baseline, including the search method, ranges explored, and selection criterion based on validation performance. This addition will clarify that the reported gaps reflect models optimized under comparable procedures. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical ablation study with external baselines

full rationale

The paper reports experimental AUROC/precision results from a multi-stream Transformer on a 10M-user dataset, with ablations across fusion strategies and a direct comparison to an external XGBoost baseline on aggregated features. No equations, derivations, or 'predictions' are claimed; performance numbers are presented as measured outcomes rather than quantities defined or fitted inside the paper itself. No self-citations are invoked as load-bearing premises, and the central claims rest on reported metrics from held-out evaluation rather than any self-referential reduction.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central performance claims rest on the assumption that the 1.5% fraud rate labels are accurate and that the 10M-user dataset distribution matches future production traffic; no new physical entities are postulated. The five fusion strategies are design choices rather than fitted parameters.

free parameters (1)

fusion strategy hyperparameters
Choice of which of the five fusion methods to deploy and their internal gating or attention dimensions are selected after seeing validation performance.

axioms (1)

domain assumption Event streams are independent enough that separate encoders capture distinct signals without destructive interference
Invoked when the abstract states that per-stream encoding is essential and produces an 18-point AUROC gap.

pith-pipeline@v0.9.1-grok · 5776 in / 1464 out tokens · 20780 ms · 2026-06-25T23:46:24.896945+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

13 extracted references · 1 canonical work pages · 1 internal anchor

[1]

XGBoost: A scalable tree boosting system,

T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” inProc. KDD, 2016, pp. 785–794

2016
[2]

LightGBM: A highly efficient gradient boosting decision tree,

G. Ke et al., “LightGBM: A highly efficient gradient boosting decision tree,” inProc. NeurIPS, 2017, pp. 3146–3154

2017
[3]

Attention is all you need,

A. Vaswani et al., “Attention is all you need,” inProc. NeurIPS, 2017, pp. 5998–6008

2017
[4]

Sequence classification for credit-card fraud detec- tion,

J. Jurgovsky et al., “Sequence classification for credit-card fraud detec- tion,”Expert Systems with Applications, vol. 100, pp. 234–245, 2018

2018
[5]

Learning transferable visual models from natural language supervision,

A. Radford et al., “Learning transferable visual models from natural language supervision,” inProc. ICML, 2021, pp. 8748–8763

2021
[6]

Transformer Hawkes process,

S. Zuo et al., “Transformer Hawkes process,” inProc. ICML, 2020, pp. 11692–11702

2020
[7]

Transaction fraud detection based on total order relation and behavior diversity,

Z. Zhang et al., “Transaction fraud detection based on total order relation and behavior diversity,”IEEE Trans. Computational Social Systems, vol. 8, no. 6, pp. 1367–1377, 2021

2021
[8]

Graph neural network for fraud detection via spatial- temporal attention,

D. Cheng et al., “Graph neural network for fraud detection via spatial- temporal attention,”IEEE Trans. Knowledge and Data Engineering, vol. 34, no. 8, pp. 3800–3813, 2020

2020
[9]

Pick and choose: A GNN-based imbalanced learning approach for fraud detection,

Z. Liu et al., “Pick and choose: A GNN-based imbalanced learning approach for fraud detection,” inProc. WWW, 2021, pp. 3168–3177

2021
[10]

Session-based fraud detection in online e-commerce transactions using recurrent neural networks,

S. Wang et al., “Session-based fraud detection in online e-commerce transactions using recurrent neural networks,” inProc. ECML-PKDD, 2019, pp. 241–257

2019
[11]

Efficient transformers: A survey,

Y . Tay et al., “Efficient transformers: A survey,”ACM Computing Surveys, vol. 55, no. 6, pp. 1–28, 2022

2022
[12]

Scaling Laws for Neural Language Models

J. Kaplan et al., “Scaling laws for neural language models,”arXiv preprint arXiv:2001.08361, 2020

work page internal anchor Pith review Pith/arXiv arXiv 2001
[13]

Training compute-optimal large language models,

J. Hoffmann et al., “Training compute-optimal large language models,” inProc. NeurIPS, 2022

2022

[1] [1]

XGBoost: A scalable tree boosting system,

T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” inProc. KDD, 2016, pp. 785–794

2016

[2] [2]

LightGBM: A highly efficient gradient boosting decision tree,

G. Ke et al., “LightGBM: A highly efficient gradient boosting decision tree,” inProc. NeurIPS, 2017, pp. 3146–3154

2017

[3] [3]

Attention is all you need,

A. Vaswani et al., “Attention is all you need,” inProc. NeurIPS, 2017, pp. 5998–6008

2017

[4] [4]

Sequence classification for credit-card fraud detec- tion,

J. Jurgovsky et al., “Sequence classification for credit-card fraud detec- tion,”Expert Systems with Applications, vol. 100, pp. 234–245, 2018

2018

[5] [5]

Learning transferable visual models from natural language supervision,

A. Radford et al., “Learning transferable visual models from natural language supervision,” inProc. ICML, 2021, pp. 8748–8763

2021

[6] [6]

Transformer Hawkes process,

S. Zuo et al., “Transformer Hawkes process,” inProc. ICML, 2020, pp. 11692–11702

2020

[7] [7]

Transaction fraud detection based on total order relation and behavior diversity,

Z. Zhang et al., “Transaction fraud detection based on total order relation and behavior diversity,”IEEE Trans. Computational Social Systems, vol. 8, no. 6, pp. 1367–1377, 2021

2021

[8] [8]

Graph neural network for fraud detection via spatial- temporal attention,

D. Cheng et al., “Graph neural network for fraud detection via spatial- temporal attention,”IEEE Trans. Knowledge and Data Engineering, vol. 34, no. 8, pp. 3800–3813, 2020

2020

[9] [9]

Pick and choose: A GNN-based imbalanced learning approach for fraud detection,

Z. Liu et al., “Pick and choose: A GNN-based imbalanced learning approach for fraud detection,” inProc. WWW, 2021, pp. 3168–3177

2021

[10] [10]

Session-based fraud detection in online e-commerce transactions using recurrent neural networks,

S. Wang et al., “Session-based fraud detection in online e-commerce transactions using recurrent neural networks,” inProc. ECML-PKDD, 2019, pp. 241–257

2019

[11] [11]

Efficient transformers: A survey,

Y . Tay et al., “Efficient transformers: A survey,”ACM Computing Surveys, vol. 55, no. 6, pp. 1–28, 2022

2022

[12] [12]

Scaling Laws for Neural Language Models

J. Kaplan et al., “Scaling laws for neural language models,”arXiv preprint arXiv:2001.08361, 2020

work page internal anchor Pith review Pith/arXiv arXiv 2001

[13] [13]

Training compute-optimal large language models,

J. Hoffmann et al., “Training compute-optimal large language models,” inProc. NeurIPS, 2022

2022