{"id":"980db1fd-4e59-4ebc-b956-9e6377c83c96","arxiv_id":"2509.06654","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A unified graph neural network trained on heterogeneous symbolic music datasets achieves competitive multi-task music analysis with better cross-dataset robustness than single-corpus models.","lead":"This paper trains a single graph neural network to perform many music-analysis tasks at once by merging three differently annotated datasets and filtering out non-chord notes at inference. It reports that the joint model stays competitive with single-task systems while transferring better across datasets, which matters for automating musicology on large corpora.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Domain-shift resilience claim is not tested: the joint model is trained and evaluated on the same corpora; no held-out corpus evaluation is performed.","rationale":"The reader's weakest assumption was that the unified label mapping is consistent enough for joint training to transfer rather than conflict, citing the DLC RN drop from 0.563 to 0.516 as evidence. I agree this drop signals a real multi-corpus tension. However, the more load-bearing issue is that the experimental protocol never tests the paper's headline claim of domain-shift resilience. The joint model is evaluated only on corpora it was trained on, so the reported 'resilience' is at best a statement about in-domain multi-corpus training. The single-corpus cross-evaluations (e.g., AugNet-only on DLC) establish that domain shift exists, but they do not show AnalysisGNN is robust to it. The proper test—training on a subset of corpora and testing on a held-out corpus—is absent. This missing experiment is decisive for the central claim, but it is a fixable gap rather than a fundamental flaw, so I keep the reader's CONDITIONAL verdict. I propose a concrete leave-one-corpus-out test to settle it. The reader's label-schema concern is related but secondary: even if the label mapping were perfect, the domain-shift claim would still lack support without a held-out evaluation.","tokens_in":9208,"tokens_out":9401,"duration_ms":86462,"concrete_test":"Run a leave-one-corpus-out evaluation: train AnalysisGNN on AugmentedNet + Cadence (excluding all DLC data) and evaluate on the DLC test set. Compare the RN CSR to the 0.516 reported for the full joint model and to the 0.301 RNBert cross-domain baseline. Repeat this protocol holding out AugmentedNet and Cadence in turn. If held-out performance is substantially lower than the in-domain joint numbers, the domain-shift resilience claim is unsupported as reported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The Abstract's central claim of 'increased resilience to domain shifts' is not actually evaluated. In Table 3, the 'All corpora (combined)' row reports results on the Cadence, AugmentedNet, and DLC test sets—all of which were included in the joint model's training set. These are in-domain evaluations, not domain-shift tests. The only cross-corpus numbers in Table 3 are for single-corpus baselines (e.g., AugNet-only evaluated on DLC RN=0.441; DLC-only evaluated on AugNet RN=0.503), which show that single-corpus models degrade on other corpora but do not demonstrate that AnalysisGNN generalizes to an unseen domain. Similarly, the RNBert comparison (trained on AugNet, evaluated on DLC, CSR=0.301) versus AnalysisGNN (CSR=0.516) is confounded because AnalysisGNN was trained on DLC itself. To support the domain-shift resilience claim, the authors must train on a subset of corpora and evaluate on a held-out corpus. Without such an experiment, the modest DLC RN drop (0.563→0.516) is better explained as a multi-corpus training trade-off, not as evidence of robustness to domain shift.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"AnalysisGNN proposes a unified graph-neural-network framework for multiple symbolic music analysis tasks, trained jointly on the Distant Listening Corpus (DLC), the AugmentedNet dataset, and three cadence-annotated corpora. The model combines a HybridGNN encoder with a data-shuffling training strategy, a learnable weighted multi-task cross-entropy loss, cross-task logit fusion via self-attention, and a Non-Chord-Tone (NCT) prediction head that gates out predicted non-chord-tones at inference. Experiments compare AnalysisGNN against single-corpus baselines and report configuration ablations. The paper claims performance comparable to specialist single-task systems and increased resilience to domain shifts and annotation inconsistencies across heterogeneous corpora.","tokens_in":9541,"tokens_out":4280,"duration_ms":45598,"significance":"If the empirical claims were fully supported, this would be a useful contribution to computational music analysis: the paper releases source code, assembles a large heterogeneous corpus collection, introduces new note-level analytical tasks, and proposes a plausible mechanism for joint training across inconsistent annotation standards. The architectural ideas—data shuffling, weighted multi-task loss, logit fusion, and NCT-based filtering—are reasonable and worth exploring. However, the central claim of domain-shift resilience is not actually tested, and several load-bearing comparisons rest on small, single-run performance differences without uncertainty estimates. The framework is promising, but the current evidence is insufficient for the strength of the abstract and conclusion claims.","major_comments":[{"comment":"The domain-shift resilience claim is not tested. In Table 3, the 'All corpora (combined)' row trains on all three corpora and evaluates on test splits of those same corpora; these are in-domain evaluations, not held-out domain-shift tests. The single-corpus rows show that single-corpus models degrade when evaluated on another corpus, but they do not show that AnalysisGNN generalizes to an unseen corpus. The RNBert comparison (RNBert trained on AugNet evaluated on DLC, CSR 0.301, versus AnalysisGNN CSR 0.516) is confounded because AnalysisGNN was trained on DLC itself. To support the claim, the authors need to train on a subset of corpora and evaluate on a held-out corpus, or explicitly reframe the claim as 'multi-corpus training trade-off' rather than domain-shift resilience.","section":"Section 5.1, Table 3"},{"comment":"All results are reported as single runs without error bars or statistical significance. Several differences that drive the narrative are small—for example, DLC cadence F1 changes from 0.556 (DLC-only) to 0.558 (all corpora), and phrase DLC F1 changes from 0.752 (without logit fusion) to 0.742 (full model). These differences may be within run-to-run noise. Since the paper's central claims about positive transfer, minor performance drops, and component contributions depend on numerical comparisons, multiple seeds with standard deviations (or significance tests) are necessary.","section":"Section 5.2, Tables 2-4"},{"comment":"The contribution of the NCT gating mechanism is not ablated. The paper claims that excluding predicted non-chord-tones at inference improves the consistency of label signals, but the configuration study only reports 'w/o Aux-Tasks', which removes auxiliary tasks as a group, not specifically the NCT head or the inference-time gate. A controlled comparison with and without the NCT gate at inference is needed to support the abstract's claim about the NCT module and the conclusion's claim about 'cleaner label signals and faster inference'.","section":"Section 3.2, Table 4"},{"comment":"The label unification step—Roman numerals simplified to root/quality, local keys, and inversions—is not validated. Table 3 shows that DLC-only Roman numeral CSR drops from 0.563 (DLC-only) to 0.516 (all corpora). If the unification is lossy, this drop may reflect conflicting annotation schemas being averaged rather than genuine robustness. The authors should quantify how often the simplified labels disagree across the two datasets, and discuss whether the observed drop is acceptable under their unification or is an artifact of schema tension.","section":"Section 4.1, Table 3"}],"minor_comments":[{"comment":"The text refers to 'Figure 3' for single-corpus versus joint-corpus comparisons, but no Figure 3 appears in the manuscript. This is either a missing figure or a broken cross-reference.","section":"Section 5.1"},{"comment":"Typo: 'DCL' should be 'DLC' in the first paragraph.","section":"Section 4.2"},{"comment":"Typo: 'comparativeaccuracy' should be 'comparative accuracy'.","section":"Section 6"},{"comment":"The 'labels*' column is defined only implicitly; clarify in the caption what 'original files before the expansion' means and why only DLC has this column.","section":"Table 1"},{"comment":"The paper states the model predicts 20 properties, but experimental results report only cadence, Roman numeral, phrase, pedal, metrical, and section. Consider providing a full breakdown in a supplementary table, or clearly noting that additional tasks are not evaluated.","section":"Section 4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a useful system and open-source code, but the headline claim of domain-shift resilience requires a held-out corpus experiment. The absence of uncertainty estimates across all tables is a recurring issue, especially for small differences in Tables 3 and 4. The NCT ablation gap is also notable given it is a stated contribution. These issues are fixable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the useful part: the paper assembles three heterogeneous corpora into a unified graph representation, adds a novel Non-Chord-Tone prediction head, and shows that a single multi-task model can hit competitive numbers on cadence, Roman numerals, phrase, and metrical tasks. The ablation table is genuinely informative—removing transposition hits hardest, and single-corpus models degrade sharply on other corpora (Table 3). Releasing code and a 1,700-piece combined dataset is a real service to the community.\n\nWhere I part ways with the abstract is the 'increased resilience to domain shifts' claim. The experiments never evaluate the joint model on a held-out corpus. Table 3's 'All corpora (combined)' row reports results on the DLC, AugNet, and Cadence test sets—all of which are in the training mixture. That's a multi-corpus training trade-off, not a domain-shift test. The RNBert comparison is similarly confounded: RNBert trained on AugNet and evaluated on DLC is being compared to AnalysisGNN that was trained on DLC itself. To support 'domain-shift resilience' you'd need to train on a subset of corpora and evaluate on a completely unseen one, or at least simulate a shift. The paper doesn't do that, so the claim overreaches.\n\nThe softer problems are real but secondary. All metrics are single-run, no error bars, and the DLC Roman-numeral drop from 0.563 to 0.516 is described as 'minor'—that's nearly a 10% relative drop, and it's more consistent with label-schema tension than robustness. The NCT module is not isolated in the ablations; removing 'aux-tasks' removes several things at once, so we don't know what the gating actually contributes. The simplification of Roman numerals into root/quality/etc. is reasonable but potentially lossy, and the paper doesn't quantify what gets thrown away.\n\nBottom line: this is a solid systems paper with a valuable dataset and a sensible architecture. The central claim about domain-shift resilience needs either a held-out corpus experiment or a rewritten abstract. As it stands, I'd send it to peer review—it's worth referee time—but with a clear request to fix the claim or add the missing experiment. For a reading group, it's a maybe, mainly because the dataset assembly and ablation are worth discussing. I wouldn't cite the domain-shift result as-is, but the corpus and code are citable.","headline":"Useful multi-task GNN with a valuable corpus combination, but the 'domain-shift resilience' headline is not backed by the experiments—the joint model is only ever tested on training corpora.","tokens_in":10002,"tokens_out":3037,"would_cite":false,"duration_ms":31355,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AnalysisGNN claims that a single graph neural network, trained by shuffling mini-batches across heterogeneous corpora and fusing task logits, can match single-corpus models while staying resilient to domain shift.","keywords":["music analysis","graph neural networks","multi-task learning","harmonic analysis","cadence detection","Roman numeral analysis","non-chord-tone prediction","domain shift"],"falsifier":"Train AnalysisGNN on the combined corpora but evaluate it on a held-out corpus whose Roman-numeral annotations use a fuller expressive vocabulary (including details the simplification drops); if performance collapses relative to a single-corpus model, the claimed resilience is an artifact of label simplification. A controlled ablation that turns off the non-chord-tone gate at inference while keeping every other component identical would also settle whether the gate contributes any of the cadence or phrase gains.","tokens_in":9107,"feed_emoji":"🎵","tokens_out":9375,"duration_ms":91884,"temperature":0.7,"pith_summary":"AnalysisGNN sets out to solve a fragmentation problem in computational music analysis: harmony, cadence, phrase, and meter are usually modeled by separate systems trained on separate datasets, each with its own annotation conventions. The paper argues a single graph neural network can be trained on all corpora at once, treating every score as a graph of notes and predicting twenty-odd note-level properties through shared features plus cross-task attention. Trained this way, the model is not the best on any single benchmark, but it is competitive, and it holds up when evaluated on a corpus it did not specialize for—where specialized models lose accuracy. The specific mechanism the paper adds is a non-chord-tone head that flags passing and non-functional notes and, at inference, lets only chord tones through to the task heads. If the claim holds, unified training could replace per-dataset pipelines and make cross-corpus analysis and benchmarking practical.","feed_headline":"Trained on three corpora, one model handles all music-analysis tasks","feed_subtitle":"AnalysisGNN learns harmony, cadence, phrase, and meter together, and it barely drops when the corpus changes.","key_machinery":"The load-bearing mechanism is logit fusion between task-specific classifiers, layered on a hybrid graph-convolution/GRU encoder. Each task head emits raw logits, projects them into a common space, and a transformer-style self-attention layer lets every head attend to the others before the final prediction; this turns separate analysis tasks into mutually corrective signals. The second part is the non-chord-tone prediction head, which at inference gates predictions to notes classified as chord tones. The third is the training schedule: mini-batches sampled across all tasks with an uncertainty-weighted loss, so no corpus dominates and label conflicts are balanced.","core_discovery":"The central claim is that heterogeneous annotation schemes can be reconciled not by harmonizing labels first, but by training a shared encoder with a multi-task objective that samples across all corpora each step, weights each task's loss by a learned uncertainty, and lets task-specific classifiers exchange information through an attention-based logit-fusion layer. On the combined corpus the model reaches 0.530 CSR on the smaller Roman-numeral set and 0.516 on the larger, internally consistent one, with cadence F1 of 0.558 on the larger corpus—compared with single-corpus baselines that are better on their home corpus but collapse elsewhere. The joint model's larger-corpus Roman-numeral score","pith_inferences":["Editorial inference: the paper's most striking result—positive cadence transfer on the larger corpus (F1 rising from 0.479 to 0.558)—suggests cadence detection may benefit more from shared harmonic context than from the cadence annotations themselves; testing cadence accuracy while ablating the harmonic heads would isolate that.","Editorial inference: the robustness to domain shift may partly trace to the simplified label vocabulary (root, quality, local key, inversion); if the unified mapping discards distinctions the original corpora make, then 'resilience' could be averaging over coarser labels. A test on corpora with full Roman-numeral vocabularies would settle how much robustness is real.","Editorial inference: the non-chord-tone gate is a test-time filtering mechanism that could be studied independently of multi-task training—for example, applying the trained gate to a frozen single-task model should quantify how much of the gain comes from filtering versus from shared representations.","Editorial inference: the same recipe—shuffle batches across corpora, weight losses by uncertainty, fuse task logits—is a generic template for any domain with overlapping but inconsistent annotation layers, such as multi-annotator medical labels or multilingual parsing; the graph-specific details are secondary."],"forward_implications":["A single AnalysisGNN model can be applied to any corpus with any subset of the supported annotations, including pieces with missing or partial labels, because invalid labels are masked rather than discarded.","Cross-task training transfers positively: knowing which notes are functional improves cadence detection, and harmonic context helps identify chord tones, so future systems can add auxiliary tasks as a form of self-supervision.","The resilience numbers imply that annotation-schema differences need not block joint training; benchmarks could be built by pooling existing corpora instead of waiting for one perfectly consistent dataset.","Non-chord-tone filtering at inference gives a cheap way to reduce false positives at cadence boundaries and focus phrase and section predictions on structurally relevant notes, which could carry over to higher-level form analysis.","Because the model predicts 20-plus note-level properties from one shared representation, analysis tools can present harmony, cadence, and phrasing as one coherent output rather than as separate pipelines."],"supporting_citations":[{"why":"Supplies the large Roman-numeral corpus, the multi-task annotation format, and the CSR metric used for evaluation.","marker":"[18]"},{"why":"Contributes the note-graph representation and onset-wise pooling for automatic Roman-numeral analysis that AnalysisGNN extends.","marker":"[9]"},{"why":"Provides the hybrid graph-convolution/GRU encoder architecture and the neighbor-sampling training procedure the model uses as its backbone.","marker":"[10]"},{"why":"Establishes cadence detection as note-level graph node classification, the formulation the cadence head adopts.","marker":"[8]"},{"why":"Supplies the uncertainty-weighted multi-task loss that the paper adapts and normalizes over the number of tasks.","marker":"[11]"},{"why":"Provides a strong Roman-numeral baseline whose sharp performance drop on the larger corpus illustrates the domain-shift problem the paper targets.","marker":"[20]"},{"why":"Supplies the Distant Listening Corpus with chord, phrase, cadence, and pedal annotations, the largest and most task-diverse corpus in the combined set.","marker":"[6]"},{"why":"One of the cadence-only corpora (Mozart string quartets) folded into the combined cadence dataset.","marker":"[1]"},{"why":"One of the cadence-only corpora (Bach's Well-Tempered Clavier) folded into the combined cadence dataset.","marker":"[4]"},{"why":"One of the cadence-only corpora (Haydn string quartets) folded into the combined cadence dataset.","marker":"[21]"}],"fun_headline_variants":["One model, all music tasks: AnalysisGNN beats domain shift","Multi-task GNN fuses annotations, robust across corpora","AnalysisGNN unifies harmony, cadence, phrase, meter in one","Shuffle-then-fuse: GNN solves music analysis across datasets","Single GNN for all music analysis, resilient to annotation chaos"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The paper's unified label mapping—Roman numerals simplified to root and quality, local keys, and inversion—is consistent enough across corpora that joint training transfers knowledge instead of averaging conflicting annotations; if that mapping loses distinctions real annotators make, the reported domain-shift resilience is partly an artifact of coarser labels.","fun_headline_variants_meta":{"raw":{"variants":["One model, all music tasks: AnalysisGNN beats domain shift","Multi-task GNN fuses annotations, robust across corpora","AnalysisGNN unifies harmony, cadence, phrase, meter in one","Shuffle-then-fuse: GNN solves music analysis across datasets","Single GNN for all music analysis, resilient to annotation chaos"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000238,"raw_usage":{"total_tokens":1300,"prompt_tokens":646,"completion_tokens":654,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":390,"completion_tokens_details":{"reasoning_tokens":572}},"tokens_in":390,"tokens_out":654,"duration_ms":6533,"temperature":1.0,"reasoning_tokens":572,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T23:18:58.289517+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train AnalysisGNN on the combined corpora but evaluate it on a held-out corpus whose Roman-numeral annotations use a fuller expressive vocabulary (including details the simplification drops); if performance collapses relative to a single-corpus model, the claimed resilience is an artifact of label simplification. A controlled ablation that turns off the non-chord-tone gate at inference while keeping every other component identical would also settle whether the gate contributes any of the cadence or phrase gains.","supporting_citations":[{"cited_title":"In: Proceedings of the 22nd International Society for Music Informa- tion Retrieval Conference, ISMIR","cited_arxiv_id":null,"evidence_quote":"Supplies the large Roman-numeral corpus, the multi-task annotation format, and the CSR metric used for evaluation."},{"cited_title":"In: Proceedings of the 23rd International Society for Music Information Retrieval Conference, ISMIR 2022, Bengaluru, India, December 4-8, 2022","cited_arxiv_id":null,"evidence_quote":"Establishes cadence detection as note-level graph node classification, the formulation the cadence head adopts."},{"cited_title":"In: Proceedings of the International Society for Music Information Retrieval Conference (ISMIR) (2024)","cited_arxiv_id":null,"evidence_quote":"Provides a strong Roman-numeral baseline whose sharp performance drop on the larger corpus illustrates the domain-shift problem the paper targets."},{"cited_title":"https://doi.org/10.5281/zenodo.13844105,https://doi.org/ 10.5281/zenodo.13844105","cited_arxiv_id":null,"evidence_quote":"Supplies the Distant Listening Corpus with chord, phrase, cadence, and pedal annotations, the largest and most task-diverse corpus in the combined set."},{"cited_title":"Transactions of the International Society for Music Information Retrieval (TISMIR)2(1), 82–96 (2019)","cited_arxiv_id":null,"evidence_quote":"One of the cadence-only corpora (Mozart string quartets) folded into the combined cadence dataset."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"One of the cadence-only corpora (Bach's Well-Tempered Clavier) folded into the combined cadence dataset."},{"cited_title":"Journal of New Music Research47(1), 29–52 (2018)","cited_arxiv_id":null,"evidence_quote":"One of the cadence-only corpora (Haydn string quartets) folded into the combined cadence dataset."}],"review_version":1}