arxiv: 2605.12287 · v1 · submitted 2026-05-12 · 📡 eess.AS · cs.SD

Recognition: 2 theorem links

· Lean Theorem

The SMC Blind Spot: A Failure Mode Analysis of State-of-the-Art Beat Tracking

Jaehoon Ahn, Moon-Ryul Jung, Tae Gum Hwang

Pith reviewed 2026-05-13 02:55 UTC · model grok-4.3

classification 📡 eess.AS cs.SD

keywords beat trackingfailure mode analysisSMC datasetdynamic Bayesian networktempo estimationdeep neural networksmusic information retrieval

0 comments

The pith

State-of-the-art beat tracking models produce octave errors, continuity breaks, and total failures on the SMC dataset because their post-processing assumes a minimum tempo of 55 BPM.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests current deep neural network beat trackers track-by-track on the SMC dataset and isolates three repeatable failure patterns that explain the stubbornly low aggregate scores. Models output confident beat activations even when they are systematically wrong, and the standard dynamic Bayesian network decoder cannot recover the correct slow tempi because its default floor of 55 BPM forces it to report double time on 21 percent of the tracks. The authors argue that these are not random errors but predictable consequences of training data that over-represents fast, percussive music and of tempo inference that never entertains multiple hypotheses.

Core claim

By examining per-track metrics, the authors show that state-of-the-art beat trackers exhibit three distinct failure modes on SMC material: octave errors in which the model locks onto double or half tempo, continuity errors in which tracking collapses midway through a piece, and complete tracking failure where every metric falls below 0.3. They further demonstrate that the default minimum-tempo constraint of 55 BPM in the widely used dynamic Bayesian network post-processor prevents correct inference on 21 percent of SMC tracks and compels double-tempo output on slow music. Models also generate high-confidence activations for these incorrect predictions.

What carries the argument

Per-track failure-mode dissection of SMC recordings together with the tempo-floor constraint inside the standard dynamic Bayesian network decoder.

If this is right

Diversifying training data with more slow-tempo and non-percussive examples would reduce octave and continuity errors.
Replacing the single-hypothesis DBN decoder with a multi-hypothesis version would allow recovery of the correct tempo on the 21 percent of tracks currently forced into double time.
Separate diagnostic metrics for octave and continuity errors would let developers target each failure mode with targeted fixes.
Uncertainty-aware output heads could flag the confident-but-wrong activations before they reach the decoder.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same three failure modes are likely to appear in any dataset containing slow or rhythmically ambiguous music, so aggregate F-measure alone is insufficient for diagnosing progress.
Immediate performance gains on SMC could be obtained simply by lowering the DBN tempo floor or swapping the decoder, without retraining the neural network.
Future beat-tracking benchmarks should report the distribution of octave and continuity errors rather than a single summary score.

Load-bearing premise

That the failure patterns seen on SMC tracks are caused by fundamental model limitations rather than by quirks of this particular dataset or by untested decoder settings.

What would settle it

Replace the 55 BPM floor with a lower bound or a multi-hypothesis tempo estimator, retrain or fine-tune on a broader range of tempi and styles, and check whether the fraction of SMC tracks with all metrics below 0.3 drops substantially.

read the original abstract

Over the past two decades, the task of musical beat tracking has transitioned from heuristic onset detection algorithms to highly capable deep neural networks (DNN). Although DNN-based beat tracking models achieve near-perfect performance on mainstream, percussive datasets, the SMC dataset has stubbornly yielded low F-measure scores. By testing how well state-of-the-art models detect beats on individual tracks in the SMC dataset, we identify three distinct failure modes: octave errors, continuity errors, and complete tracking failure where all metrics fall below 0.3. We reveal that state-of-the-art models tend to generate "confident-but-wrong" activations. Furthermore, we show that the standard DBN's default minimum tempo of 55 BPM prevents it from inferring the correct tempo for 21\% of SMC tracks, forcing double-tempo predictions on slow music. By exposing such fundamental oversights, we provide concrete directions for improving beat and downbeat detection, specifically emphasizing training data diversification and multi-hypothesis tempo estimation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper catalogs three failure modes on SMC tracks with a specific DBN tempo claim, but that claim rests on an untested assumption about the prior.

read the letter

The punchline is that this paper gives a track-by-track breakdown of why SOTA beat trackers fail on the SMC dataset, splitting the problems into octave errors, continuity errors, and complete failures where metrics drop below 0.3. It also flags that the standard DBN minimum tempo of 55 BPM forces double-tempo predictions on 21% of the tracks. That taxonomy and the percentage are the concrete new pieces here. The work does well by moving past aggregate F-measures to show that models often output confident but incorrect activations, which points to practical debugging steps like diversifying training data and trying multi-hypothesis tempo search. The empirical setup is direct and uses existing public models on a fixed dataset, so there is no circularity or invented parameters. The results feel grounded in the reported metrics. The soft spot is the causal link on the DBN default. Without any re-runs using a lower tempo bound or alternative priors, it remains possible that the upstream DNN activations are the main driver rather than the fixed 55 BPM floor. The stress-test note is on target there. Minor gaps include clearer reporting on track selection criteria and exact model versions to support full reproducibility. This is aimed at MIR researchers who work on beat and downbeat detection, especially those who already know SMC is hard and want granular diagnostics. A reader focused on evaluation practices or improving trackers for non-percussive music will find the observations useful. It deserves peer review because the per-track analysis and failure categories are solid enough to warrant referee input on tightening the tempo claim and adding controls.

Referee Report

2 major / 2 minor

Summary. The manuscript empirically analyzes state-of-the-art DNN-based beat tracking models on individual tracks from the SMC dataset. It identifies three failure modes—octave errors, continuity errors, and complete tracking failure (all metrics below 0.3)—and reports that models produce confident-but-wrong activations. The paper further claims that the standard DBN's default minimum tempo of 55 BPM prevents correct tempo inference for 21% of SMC tracks, forcing double-tempo predictions on slow music, and recommends training data diversification and multi-hypothesis tempo estimation as remedies.

Significance. If the empirical findings hold, the work is significant for exposing concrete limitations of current beat tracking systems on challenging, non-percussive music and for offering actionable directions such as diversified training data and multi-hypothesis tempo search. The direct per-track evaluation on a fixed dataset provides a clear taxonomy of failures that could guide future model development, though the absence of controls on key parameters reduces the strength of causal attributions.

major comments (2)

[Results section on DBN tempo analysis] The central claim that the standard DBN's default minimum tempo of 55 BPM forces double-tempo predictions on 21% of SMC tracks (abstract and results discussion) is load-bearing for the 'fundamental oversight' narrative but lacks any ablation: no re-runs of the DBN (or upstream DNN) with a lowered bound such as 40 BPM or multi-hypothesis tempo search are reported. Without this control, the observed octave errors and low F-measures could originate in the DNN activations rather than the fixed DBN prior.
[Methods and abstract] Track selection and statistical controls are underspecified (abstract and methods): the paper states testing on 'individual tracks' but does not detail selection criteria, exact model implementations, or variance estimates across runs, weakening support for the three failure-mode taxonomy as representative of fundamental limitations.

minor comments (2)

[Abstract] The abstract uses informal phrasing ('stubbornly yielded') that could be replaced with more neutral language for a formal journal.
[Figures and tables] Figure captions and table legends should explicitly state the exact metrics (F-measure, etc.) and thresholds used for the 'complete tracking failure' category.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive feedback on our manuscript. We agree that the suggested additions will strengthen the empirical claims and improve reproducibility. We address each major comment below and will revise the manuscript accordingly.

read point-by-point responses

Referee: The central claim that the standard DBN's default minimum tempo of 55 BPM forces double-tempo predictions on 21% of SMC tracks (abstract and results discussion) is load-bearing for the 'fundamental oversight' narrative but lacks any ablation: no re-runs of the DBN (or upstream DNN) with a lowered bound such as 40 BPM or multi-hypothesis tempo search are reported. Without this control, the observed octave errors and low F-measures could originate in the DNN activations rather than the fixed DBN prior.

Authors: We acknowledge that the manuscript presents the 21% figure from tempo annotation analysis and the known 55 BPM hard minimum in standard DBN implementations (e.g., madmom) but does not include an explicit ablation re-running the pipeline with a lowered bound. The claim follows directly from the DBN's tempo prior preventing selection of slower hypotheses, which mathematically produces octave errors on those tracks. To strengthen the causal link, we will add the requested ablation: re-running the DBN with a 40 BPM minimum on the slow tracks and reporting updated F-measures and error types. We will also include a brief discussion of multi-hypothesis tempo search as a remedy. revision: yes
Referee: Track selection and statistical controls are underspecified (abstract and methods): the paper states testing on 'individual tracks' but does not detail selection criteria, exact model implementations, or variance estimates across runs, weakening support for the three failure-mode taxonomy as representative of fundamental limitations.

Authors: We agree the Methods section requires expansion for clarity and reproducibility. The evaluation used every track in the full SMC dataset (217 tracks) with no additional selection criteria. We employed the publicly released pre-trained weights and inference code from the original model papers. Because inference is deterministic, run-to-run variance is zero; we will state this explicitly. In revision we will add precise model versions, repository links, and a statement confirming the complete dataset was used, thereby supporting the failure-mode taxonomy as representative. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical failure-mode analysis

full rationale

The paper conducts direct evaluations of existing beat-tracking models on the fixed SMC dataset to catalog observed failure modes (octave errors, continuity errors, complete failure). No derivations, equations, fitted parameters, or predictions are presented; the 21% tempo claim is an empirical count from running standard DBN defaults, not a self-referential fit or renamed input. No self-citations are load-bearing for any central claim, and the analysis does not reduce any result to its own inputs by construction. This is a standard observational study whose claims remain externally falsifiable.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The paper relies on standard domain assumptions in music information retrieval without introducing new free parameters or invented entities.

axioms (2)

domain assumption F-measure and related metrics are appropriate for quantifying beat tracking performance
Used to define failure modes and report scores throughout the analysis.
domain assumption The SMC dataset constitutes a representative set of challenging cases for current beat trackers
Forms the basis for identifying the reported failure modes.

pith-pipeline@v0.9.0 · 5479 in / 1346 out tokens · 130947 ms · 2026-05-13T02:55:33.663234+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Foundation/DimensionForcing.lean reality_from_one_distinction unclear
standard DBN's default minimum tempo of 55 BPM prevents it from inferring the correct tempo for 21% of SMC tracks, forcing double-tempo predictions
IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear
three distinct failure modes: octave errors, continuity errors, and complete tracking failure

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages · 1 internal anchor

[1]

The SMC Blind Spot: A Failure Mode Analysis of State-of-the-Art Beat Tracking

INTRODUCTION Beat tracking is a longstanding challenge in Music Infor- mation Retrieval (MIR) that involves estimating the tem- poral locations of musical beats in an audio signal. Mod- ern approaches have progressed from onset detection func- tions [1–6] and recurrent neural networks [7–10] to tem- poral convolutional networks [11–13] and Transformer- ba...

work page internal anchor Pith review Pith/arXiv arXiv 2026
[2]

The activation function is then provided to a Dynamic Bayesian Network (DBN), which can then be used to determine the most likely beat and downbeat sequence

BACKGROUND 2.1 Dynamic Bayesian Networks Most beat tracking models do not directly produce a fi- nal sequence of beats and downbeats but rather produce a temporal activation function, in which the values at each time index refer to the probability that a beat is present at that location. The activation function is then provided to a Dynamic Bayesian Netwo...

work page
[3]

The audio is formatted as mono W A V files sampled at 44.1 kHz

REVISITING THE SMC DA TASET The SMC dataset [23] is a beat tracking dataset containing 217 manually annotated 40-second Western music excerpts specifically compiled to evaluate beat tracking algorithms on rhythmically complex audio. The audio is formatted as mono W A V files sampled at 44.1 kHz. The excerpts were selected using a Query-by-Committee approa...

work page
[4]

We employ a rigorous 8-fold cross-validation setup for the Transformer models to ensure every track is evaluated by a checkpoint that held it out during training

EXPERIMENTAL SETUP To isolate the causes of beat tracking failures on the SMC dataset, we evaluate using three representative architec- tures: Beat This [19], a Transformer-based system that rep- resents the current state-of-the-art; Beat Transformer [15]; and madmom’sTCNBeatProcessor[11, 12, 22]. We employ a rigorous 8-fold cross-validation setup for the...

work page
[5]

tempo ceil- ing

RESULTS & ANALYSIS 5.1 SMC dataset analysis After normalizing spelling variants, singular/plural forms, and parenthesized duplicates in the per-track.tagfiles, the dataset contains 23 unique difficulty descriptors. We grouped these tags into four axes based on the type of chal- lenge each presents:weak beat cues, with absent or faint acoustic beat markers...

work page
[6]

DISCUSSION 6.1 Two performance ceilings Our experiments converge on two independent perfor- mance ceilings on SMC. The activation ceiling (∼F = 0.67) is the maximum F-measure achievable across all system and DBN combinations; it exists because approximately 100 tracks produce confident activation peaks at wrong positions that no DBN can override. The temp...

work page
[7]

Our analysis identifies three distinct failure modes and reveals that the dominant cause of low performance is confident-but-wrong activa- tion peaks

CONCLUSION We presented the first per-track diagnostic analysis of beat tracking on the SMC dataset [23]. Our analysis identifies three distinct failure modes and reveals that the dominant cause of low performance is confident-but-wrong activa- tion peaks. The per-track optimal threshold experiment places an upper bound ofF= 0.673on any decoder oper- atin...

work page
[8]

AI USAGE STA TEMENT We declare the following use of AI tools in the preparation of this manuscript. Claude (Anthropic) was used as a writ- ing assistant for drafting and revising manuscript prose, structuring the presentation of results, and reviewing the manuscript for numerical inconsistencies. Claude Code (Anthropic) was used to verify reported figures...

work page
[9]

Tempo and beat analysis of acoustic musical signals,

E. D. Scheirer, “Tempo and beat analysis of acoustic musical signals,”The Journal of the Acoustical Society of America, vol. 103, no. 1, pp. 588–601, 1998

work page 1998
[10]

Automatic extraction of tempo and beat from expressive performances,

S. Dixon, “Automatic extraction of tempo and beat from expressive performances,”Journal of New Music Research, vol. 30, no. 1, pp. 39–58, 2001

work page 2001
[11]

Beat tracking by dynamic programming,

D. P. Ellis, “Beat tracking by dynamic programming,” Journal of New Music Research, vol. 36, no. 1, pp. 51– 60, 2007

work page 2007
[12]

Context-dependent beat tracking of musical audio,

M. E. Davies and M. D. Plumbley, “Context-dependent beat tracking of musical audio,”IEEE Transactions on Audio, Speech, and Language Processing, vol. 15, no. 3, pp. 1009–1020, 2007

work page 2007
[13]

Evaluation of the audio beat tracking sys- tem beatroot,

S. Dixon, “Evaluation of the audio beat tracking sys- tem beatroot,”Journal of New Music Research, vol. 36, no. 1, pp. 39–50, 2007

work page 2007
[14]

Better beat tracking through robust onset aggregation,

B. McFee and D. P. Ellis, “Better beat tracking through robust onset aggregation,” in2014 IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP). IEEE, 2014, pp. 2154–2158

work page 2014
[15]

Enhanced beat tracking with context-aware neural networks,

S. Böck and M. Schedl, “Enhanced beat tracking with context-aware neural networks,” inProc. Int. Conf. Digital Audio Effects, 2011, pp. 135–139

work page 2011
[16]

A multi-model ap- proach to beat tracking considering heterogeneous mu- sic styles

S. Böck, F. Krebs, and G. Widmer, “A multi-model ap- proach to beat tracking considering heterogeneous mu- sic styles.” inISMIR, 2014, pp. 603–608

work page 2014
[17]

Joint beat and downbeat tracking with recurrent neural networks

——, “Joint beat and downbeat tracking with recurrent neural networks.” inISMIR. New York City, 2016, pp. 255–261

work page 2016
[18]

Down- beat tracking using beat synchronous features with re- current neural networks

F. Krebs, S. Böck, M. Dorfer, and G. Widmer, “Down- beat tracking using beat synchronous features with re- current neural networks.” inISMIR, 2016, pp. 129– 135

work page 2016
[19]

Temporal convolu- tional networks for musical audio beat tracking,

E. MatthewDavies and S. Böck, “Temporal convolu- tional networks for musical audio beat tracking,” in 2019 27th European Signal Processing Conference (EUSIPCO). IEEE, 2019, pp. 1–5

work page 2019
[20]

Deconstruct, analyse, re- construct: How to improve tempo, beat, and downbeat estimation

S. Böck and M. E. Davies, “Deconstruct, analyse, re- construct: How to improve tempo, beat, and downbeat estimation.” inISMIR, 2020, pp. 574–582

work page 2020
[21]

Wavebeat: End-to-end beat and downbeat tracking in the time domain,

C. J. Steinmetz and J. D. Reiss, “Wavebeat: End-to-end beat and downbeat tracking in the time domain,”arXiv preprint arXiv:2110.01436, 2021

work page arXiv 2021
[22]

Modeling beats and downbeats with a time- frequency transformer,

Y .-N. Hung, J.-C. Wang, X. Song, W.-T. Lu, and M. Won, “Modeling beats and downbeats with a time- frequency transformer,” inICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 401– 405

work page 2022
[23]

Beat transformer: Demixed beat and downbeat tracking with dilated self- attention,

J. Zhao, G. Xia, and Y . Wang, “Beat transformer: Demixed beat and downbeat tracking with dilated self- attention,”arXiv preprint arXiv:2209.07140, 2022

work page arXiv 2022
[24]

Transformer-based beat track- ing with low-resolution encoder and high-resolution decoder

T. Cheng and M. Goto, “Transformer-based beat track- ing with low-resolution encoder and high-resolution decoder.” inISMIR, 2023, pp. 466–473

work page 2023
[25]

All-in-one metrical and func- tional structure analysis with neighborhood attentions on demixed audio,

T. Kim and J. Nam, “All-in-one metrical and func- tional structure analysis with neighborhood attentions on demixed audio,” in2023 IEEE Workshop on Appli- cations of Signal Processing to Audio and Acoustics (WASPAA). IEEE, 2023, pp. 1–5

work page 2023
[26]

Beast: Online joint beat and downbeat tracking based on streaming transformer,

C.-C. Chang and L. Su, “Beast: Online joint beat and downbeat tracking based on streaming transformer,” inICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 396–400

work page 2024
[27]

Beat this! accurate beat tracking without dbn postprocessing,

F. Foscarin, J. Schlüter, and G. Widmer, “Beat this! accurate beat tracking without dbn postprocessing,” arXiv preprint arXiv:2407.21658, 2024

work page arXiv 2024
[28]

Bayesian modelling of temporal structure in musical audio

N. Whiteley, A. T. Cemgil, and S. J. Godsill, “Bayesian modelling of temporal structure in musical audio.” in ISMIR, 2006, pp. 29–34

work page 2006
[29]

An efficient state- space model for joint tempo and meter tracking

F. Krebs, S. Böck, and G. Widmer, “An efficient state- space model for joint tempo and meter tracking.” inIS- MIR, 2015, pp. 72–78

work page 2015
[30]

Madmom: A new python audio and mu- sic signal processing library,

S. Böck, F. Korzeniowski, J. Schlüter, F. Krebs, and G. Widmer, “Madmom: A new python audio and mu- sic signal processing library,” inProceedings of the 24th ACM international conference on Multimedia, 2016, pp. 1174–1178

work page 2016
[31]

Selective sampling for beat tracking evaluation,

A. Holzapfel, M. E. Davies, J. R. Zapata, J. L. Oliveira, and F. Gouyon, “Selective sampling for beat tracking evaluation,”IEEE Transactions on Audio, Speech, and Language Processing, vol. 20, no. 9, pp. 2539–2548, 2012

work page 2012
[32]

Toward postprocessing-free neural networks for joint beat and downbeat estima- tion

T.-P. Chen and L. Su, “Toward postprocessing-free neural networks for joint beat and downbeat estima- tion.” inISMIR, 2022, pp. 27–35

work page 2022
[33]

Beat tracking as object detec- tion,

J. Ahn and M.-R. Jung, “Beat tracking as object detec- tion,”arXiv preprint arXiv:2510.14391, 2025

work page arXiv 2025
[34]

Eval- uation methods for musical audio beat tracking algo- rithms,

M. E. Davies, N. Degara, and M. D. Plumbley, “Eval- uation methods for musical audio beat tracking algo- rithms,”Queen Mary University of London, Centre for Digital Music, Tech. Rep. C4DM-TR-09-06, 2009

work page 2009
[35]

Mir_eval: A transparent implementation of common mir metrics

C. Raffel, B. McFee, E. J. Humphrey, J. Salamon, O. Nieto, D. Liang, D. P. Ellis, and C. C. Raffel, “Mir_eval: A transparent implementation of common mir metrics.” inISMIR, vol. 10, 2014, p. 2014

work page 2014
[36]

Query by committee,

H. S. Seung, M. Opper, and H. Sompolinsky, “Query by committee,” inProceedings of the fifth annual work- shop on Computational learning theory, 1992, pp. 287–294

work page 1992
[37]

Particle fil- tering applied to musical tempo tracking,

S. W. Hainsworth and M. D. Macleod, “Particle fil- tering applied to musical tempo tracking,”EURASIP Journal on Advances in Signal Processing, vol. 2004, no. 15, p. 927847, 2004

work page 2004
[38]

An experimental compari- son of audio tempo induction algorithms,

F. Gouyon, A. Klapuri, S. Dixon, M. Alonso, G. Tzane- takis, C. Uhle, and P. Cano, “An experimental compari- son of audio tempo induction algorithms,”IEEE Trans- actions on Audio, Speech, and Language Processing, vol. 14, no. 5, pp. 1832–1844, 2006

work page 2006
[39]

Rhythmic pattern modeling for beat and downbeat tracking in musical audio

F. Krebs, S. Böck, and G. Widmer, “Rhythmic pattern modeling for beat and downbeat tracking in musical audio.” inIsmir, 2013, pp. 227–232

work page 2013
[40]

Swing ratio estimation,

U. Marchand and G. Peeters, “Swing ratio estimation,” inDigital Audio Effects 2015 (Dafx15), 2015

work page 2015