{"id":"74d28c46-a6a5-4198-85f6-54e60905f275","arxiv_id":"2501.03038","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Decoding pretrained piano-roll encoder embeddings with three hierarchical language models improves onset-offset-velocity F1 by 0.010 to 0.022 over roll outputs on Maestro.","lead":"Piano transcription systems that generate note events with language models often beat frame-based piano rolls but need very long token sequences. This paper attaches a pretrained piano-roll encoder to three smaller language models that predict onset, pitch, velocity, and offset in stages, reporting small F1 gains over the roll output on Maestro.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains over 'Roll' baselines in Table I are not yet trustworthy because the roll threshold and post-processing are undisclosed; an untuned threshold could erase the 0.010/0.022 F1 advantage.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern I find: the roll baselines in Table I are not fully specified, and the threshold/post-processing choice can materially affect the reported F1 scores. The paper itself flags the threshold requirement in Section V but omits the value, and the claimed gains (0.010 and 0.022) are small enough that threshold variation could explain them. I considered whether end-to-end fine-tuning of the encoder is a more fundamental threat to the 'plug-in' claim; while fine-tuning means the encoder is changed, that is a framing/scope issue rather than a numerical correctness issue, and it does not undermine the internal comparison as directly as an untuned baseline. The proposed threshold sweep is a concrete, low-cost experiment that would settle the concern. Because the reader already assigned CONDITIONAL and this concern reinforces that verdict, no adjustment is needed.","tokens_in":8586,"tokens_out":6008,"duration_ms":56967,"concrete_test":"Sweep the binarization threshold for CRNN Roll and HPPNet Roll over a dense grid (e.g., 0.05 to 0.95 in 0.01 steps) using the same post-processing pipeline (note merging, minimum duration, velocity scaling, offset handling) that the authors would use for a fair roll baseline, and report the best On Off Vel F1 for each encoder. If the best-tuned roll F1 is within 0.005 of the corresponding Hierarchy F1, the claimed plug-in advantage is not robust; if the gap persists across a broad threshold plateau (e.g., >0.1 in width), the central claim survives this test. The paper should also report the exact threshold and post-processing used to produce Table I.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on the comparison in Table I: CRNN Hierarchy beats CRNN Roll by 0.010 in On Off Vel F1 (0.819 vs 0.809), and HPPNet Hierarchy beats HPPNet Roll by 0.022 (0.832 vs 0.810). Section V explicitly states that 'the Roll approach requires setting a threshold to gate notes as posterior information,' but the paper never reports the threshold value, the search procedure, or the post-processing pipeline (e.g., note merging, minimum duration, offset smoothing) for either roll baseline. The roll output is a continuous probability map, and note-level F1 is highly sensitive to the binarization threshold; a difference of 1-2 points is within the range that threshold choice alone can produce. Without this information, the reader cannot determine whether the hierarchy genuinely improves over a properly tuned roll decoder or merely over an arbitrarily thresholded one. The paper also reports no error bars or multiple-seed variance, so the magnitude of the claimed improvement is not statistically secured. The abstract's headline numbers (0.01 and 0.022) are exactly these differences, making this omission load-bearing for the 'performance-enhancing plug-in' claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a hybrid piano transcription system that couples a pretrained roll-based audio encoder (CRNN or HPPNet) with a decoder-only language model. The LM decoder predicts note events in three hierarchical stages: onset and pitch, then velocity, then offset, using task-specific query tokens. On the Maestro test set, the hierarchical decoder improves over the traditional roll output by 0.010 and 0.022 in onset-offset-velocity F1 for CRNN and HPPNet, respectively. The authors also report scaling experiments showing that larger LM decoders do not improve performance and argue that the hierarchical decoder can serve as a performance-enhancing plug-in for arbitrary roll-based transcription encoders.","tokens_in":8845,"tokens_out":3765,"duration_ms":33231,"significance":"If the reported gains are robust, the proposed hierarchical LM decoder is a simple and potentially impactful way to upgrade existing roll-based AMT systems: it removes manual thresholding and improves note-level F1 without retraining the encoder. The paper usefully separates encoder and decoder contributions and corroborates prior evidence that larger LM decoders overfit on the transcription task. However, the central quantitative claim rests on a roll-baseline comparison whose threshold and post-processing are not reported, and no uncertainty estimates are provided, so the magnitude of the claimed plug-in improvement is not yet fully supported.","major_comments":[{"comment":"The comparison between the hierarchical LM decoder and the Roll baselines is not reproducible because the thresholding and post-processing for CRNN Roll and HPPNet Roll are never described. The text states that 'the Roll approach requires setting a threshold to gate notes as posterior information,' but it does not report the threshold value, the search procedure (e.g., grid search on the validation set), or the post-processing steps such as note merging, minimum duration filtering, or offset smoothing. Since the reported gains are only 0.010 and 0.022 in onset-offset-velocity F1, an arbitrary or poorly tuned threshold could erase or inflate these differences. Please report the exact decoding pipeline for both roll baselines, including the threshold and any post-processing, or the central plug-in claim is not supported.","section":"Section V, Table I"},{"comment":"No error bars, confidence intervals, or multiple-seed results are reported for any of the conditions in Table I. The claimed improvements of 0.010 and 0.022 are small enough that they could fall within run-to-run variance, particularly given the 1e-5 learning rate and the maximum step count of 1 million without an explicit early-stopping criterion. Please report the mean and standard deviation over at least three independent training runs, or apply a paired significance test over the test set, to establish that the differences are not noise.","section":"Section V, Table I"},{"comment":"The flattened baseline collapses to onset-offset F1 values of 0.393 and 0.390 for CRNN and HPPNet, respectively, which is drastically below the corresponding roll baselines (0.825 and 0.822) and below Hawthorne et al. [14] (0.839 and 0.828). The text attributes this to the longer 10-second segment and the decoder-only architecture, but the magnitude of the drop is far larger than typical sequence-length effects, and no diagnostic evidence is provided. If the flattened baseline is impaired by a training or decoding artifact, its comparison to the hierarchy is uninformative for the claim that hierarchical prediction helps. Either fix the flattened baseline (for example, by evaluating it with shorter segments) or supply direct evidence, such as sequence-length statistics and loss curves, that the stated cause explains the collapse.","section":"Section V, Table I"},{"comment":"The formalization of the hierarchical model is internally inconsistent. Equation (8) writes the probability of a note as a product of three separate language models p_onset-pitch, p_velocity, and p_offset, and Section IV says 'we trained three models with the same model architecture.' In contrast, Eq. (4) describes a single decoder f_dec conditioned on a query token q, and the token dictionary includes 'task-specific query tokens' with vocabulary size three. Please clarify whether the three stages share parameters or are three independent models, and how the query token is used in each case. This also affects the interpretation of the parameter counts in Table I and the claimed threefold complexity reduction.","section":"Section III, Eqs. (4)-(8)"}],"minor_comments":[{"comment":"Typo: 'an feature' should be 'a feature'.","section":"Section II.A"},{"comment":"The complexity comparison states that the flattened single-model approach costs O((T + 3N)^2D), but a flattened sequence in Fig. 2(a) contains four tokens per note (onset, pitch, velocity, offset), so the complexity should be O((T + 4N)^2D). Please correct the formula or justify the N-count.","section":"Section III"},{"comment":"The caption says 'Two lines on o and p tokens denote that they are fixed from b1 during b2 and b3.' It should say 'The dashed lines' or 'The two dotted lines' to clarify what is being depicted.","section":"Figure 2 caption"},{"comment":"The statement 'We have also achieved new state-of-the-art result of the LM-based piano transcription model' is supported only by comparison to two Hawthorne et al. [14] models. Please specify the set of prior LM-based systems considered and, if possible, compare against additional recent LM-based transcription models to make the SOTA claim meaningful.","section":"Section V"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads like a strong workshop or conference submission; for a journal, the missing threshold and post-processing details plus the absence of uncertainty quantification are substantive. The flattened-baseline collapse also needs a convincing explanation or a corrected experiment. The novelty is incremental but the plug-in idea is worth publishing once the baseline details are disclosed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the hierarchical decoder idea is plausible and the scaling study is honest, but the main empirical claim is not yet secured because the roll-baseline threshold and post-processing are undisclosed, and there are no error bars.\n\nWhat's new: they take two pretrained roll-based piano transcription encoders (CRNN, HPPNet) and attach a decoder-only LLaMA-style LM that predicts notes hierarchically: onset+pitch first, then velocity, then offset. The hierarchical token decomposition keeps sequence lengths short, which is a real practical concern for LM-based AMT. The evaluation on the external Maestro test set means the headline improvement over the roll baselines (0.010 and 0.022 on On Off Vel F1) is not circular by construction. The scaling experiments are a genuine strength: they show larger LM decoders don't help and velocity overfits early, corroborating a known observation and pointing at encoder choice as the more important factor.\n\nSoft spots, in proportion: the load-bearing comparison is the hierarchy vs. the roll baseline, and the roll threshold is never reported. Section V explicitly says the Roll approach needs a threshold, but never says what threshold was used, how it was tuned, or what post-processing (note merging, minimum duration, offset smoothing) was applied. With a gap of 1-2 F1 points, threshold choice can easily erase the claimed gain. There are also no error bars or multiple seeds, so we cannot tell whether 0.010/0.022 is signal or noise. The paper also mixes two architectural descriptions: it says both 'task-specific query tokens' and 'three models' for the three hierarchies. These are different designs with different complexity profiles, and the complexity reduction claim depends on which one is true. The flattened baseline collapses to 0.39 On Off F1 without a demonstrated cause; the paper offers a hypothesis about segment length, but does not verify it. Finally, the state-of-the-art claim is limited to LM-based piano transcription and is compared against Hawthorne et al. trained on different segment lengths, so it is not a controlled comparison.\n\nBottom line: the core idea is worth pursuing and the paper is coherent on its own terms, but the central quantitative claim needs more evidence. A serious referee should ask for the threshold and post-processing details, error bars or significance tests, and a clarification of the architecture. I would send it to review, not desk-reject, but I would not rely on the reported gains until those gaps are closed.","headline":"Plausible plug-in architecture with an honest scaling study, but the headline gain over roll baselines is not yet trustworthy because the baseline threshold and post-processing are never disclosed.","tokens_in":9355,"tokens_out":2567,"would_cite":false,"duration_ms":25076,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A hierarchical language-model decoder lifts piano transcription note-level F1 by 0.010 to 0.022 over conventional piano-roll outputs.","keywords":["automatic music transcription","piano transcription","language model decoder","hierarchical prediction","piano-roll encoder","note-level F1","MAESTRO dataset","velocity prediction"],"falsifier":"Re-run the two roll-based systems with a systematic threshold sweep on a validation set (or publish the exact threshold and post-processing pipeline used), then compare the best tuned roll F1 with the hierarchical LM F1; if the tuned roll outputs match or exceed the hierarchical system's onset-offset-velocity F1, the claimed plug-in advantage collapses.","tokens_in":8401,"feed_emoji":"🎹","tokens_out":8113,"duration_ms":69369,"temperature":0.7,"pith_summary":"This paper proposes a hybrid piano transcription system: a pretrained frame-level piano-roll encoder is kept as the audio front end, but its final readout is replaced by a decoder-only language model that emits note events token by token. To make the token sequence tractable, the paper predicts onset and pitch first, then velocity, then offset, using three separate language models guided by query tokens instead of one flattened sequence. The authors report that, on two benchmark roll-based encoders, the hierarchical language-model decoder beats the traditional thresholded piano-roll output by 0.010 and 0.022 in onset-offset-velocity F1 on the MAESTRO dataset. The intended significance is a plug-in upgrade path: existing roll-based encoders could gain note-level accuracy and shed manual thresholding without being redesigned. The paper also claims that encoder choice matters more than language-model size, and that velocity prediction is the part most prone to overfitting.","feed_headline":"Hierarchical LM decoder beats piano-roll transcription by 0.022","feed_subtitle":"Attaching a three-stage token predictor to existing roll encoders raises note-level F1 and removes threshold tuning.","key_machinery":"The carrying mechanism is a hierarchical factorization of each note event into three conditionally dependent token streams: onset plus pitch, then velocity, then offset. Each stream is generated by its own decoder-only Transformer language model, conditioned on frame-level embeddings extracted from the pretrained roll encoder and steered by task-specific query tokens $q_p$, $q_v$, and $q_f$; the factorization is stated as $p(o_n,p_n|x_1,y_{<n},q_p) p(v_n|x_1,y_{<n},o_n,p_n,q_v) p(f_n|x_1,y_{<n},o_n,p_n,v_n,q_f)$. Splitting the flattened note-event sequence into three shorter streams changes the attention cost from $O((T+3N)^2D)$ to $O(3(T+N)^2D)$ and, according to the paper, is what stops the language-model decoder from degrading on long 10-second segments.","core_discovery":"The central claim is that attaching a hierarchical language-model decoder to a pretrained roll-based encoder yields more accurate note-level piano transcription than decoding the same encoder's piano roll with a threshold. In the paper's experiments, replacing the roll decoder with the hierarchical LM decoder improves onset-offset-velocity F1 by 0.010 with the CRNN encoder and by 0.022 with the HPPNet encoder, while matching or exceeding the roll system's onset and onset-offset scores and removing the need for threshold-based post-processing. The paper presents this as evidence that the hierarchical LM decoder can act as a performance-enhancing plug-in for arbitrary roll-based transcription encoders, and additionally finds that the choice of audio encoder affects the final score far more than the size of the language-model decoder.","pith_inferences":["Because the paper tests only two encoders on one dataset, the plug-in claim is a prediction for the broader family of roll-based encoders; the natural next experiment is to attach the same decoder to other pretrained roll systems and check whether the 0.010-0.022 gain is consistent.","The three-way split makes decoder overfitting visible: velocity loss rises early while onset-pitch and offset plateau, so a likely extension is to regularize velocity training specifically rather than enlarging the whole decoder.","The same query-token hierarchy could transfer beyond piano to any event stream with causally ordered attributes, such as multi-instrument note events, giving a testable way to see whether the factorization's benefit is general or piano-specific."],"forward_implications":["On the two tested encoders, swapping the piano-roll decoder for the hierarchical LM decoder raises onset-offset-velocity F1 by 0.010 (CRNN encoder) and 0.022 (HPPNet encoder).","The hierarchical system produces note outputs without thresholding or post-processing, unlike the roll baseline it is compared against.","Flattened note-token sequences degrade on 10-second segments, especially on offset prediction; the hierarchical split avoids that degradation.","Language-model decoder size has little effect on final F1, and velocity tokens overfit early during training, so encoder choice rather than decoder scale drives performance."],"supporting_citations":[{"why":"Supplies the CRNN roll-based encoder, the pretrained frame-level embedding that feeds the LM decoder, and the first roll-output baseline.","marker":"[10]"},{"why":"Supplies the HPPNet roll-based encoder, the second encoder and baseline the hierarchical LM decoder is evaluated against.","marker":"[11]"},{"why":"Defines the flattened note-token sequence-to-sequence baseline that the hierarchical approach is compared with, and reports the larger-model overfitting observation.","marker":"[14]"},{"why":"Provides the MAESTRO dataset with paired audio and MIDI recordings and the official split used for training and evaluation.","marker":"[27]"},{"why":"Supplies the decoder-only language-model architecture used for the three hierarchical predictors.","marker":"[25]"},{"why":"Supplies the rotary position embeddings used to encode the positions of note-event tokens in the decoder.","marker":"[26]"}],"fun_headline_variants":["LM decoder boosts piano transcription F1 by 0.022","Plug-in LM decoder improves note-level piano transcription","Three-stage LM decoder outperforms piano-roll thresholding","Pretrained roll encoders gain from hierarchical LM decoder"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains rest on how the two roll baselines were thresholded and post-processed, and the paper never states the threshold value or the search procedure used for them, so if those baselines were not tuned fairly, the 0.010 and 0.022 improvements could be inflated.","fun_headline_variants_meta":{"raw":{"variants":["LM decoder boosts piano transcription F1 by 0.022","Plug-in LM decoder improves note-level piano transcription","Three-stage LM decoder outperforms piano-roll thresholding","Pretrained roll encoders gain from hierarchical LM decoder"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000464,"raw_usage":{"total_tokens":2273,"prompt_tokens":857,"completion_tokens":1416,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":473,"completion_tokens_details":{"reasoning_tokens":1363}},"tokens_in":473,"tokens_out":1416,"duration_ms":10758,"temperature":1.0,"reasoning_tokens":1363,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:58:26.095797+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the two roll-based systems with a systematic threshold sweep on a validation set (or publish the exact threshold and post-processing pipeline used), then compare the best tuned roll F1 with the hierarchical LM F1; if the tuned roll outputs match or exceed the hierarchical system's onset-offset-velocity F1, the claimed plug-in advantage collapses.","supporting_citations":[{"cited_title":"Hppnet: Modeling the harmonic struc- ture and pitch invariance in piano transcription,","cited_arxiv_id":null,"evidence_quote":"Supplies the HPPNet roll-based encoder, the second encoder and baseline the hierarchical LM decoder is evaluated against."},{"cited_title":"Sequence-to-sequence piano transcription with transformers,","cited_arxiv_id":null,"evidence_quote":"Defines the flattened note-token sequence-to-sequence baseline that the hierarchical approach is compared with, and reports the larger-model overfitting observation."},{"cited_title":"Enabling factorized piano music modeling and generation with the maestro dataset,","cited_arxiv_id":null,"evidence_quote":"Provides the MAESTRO dataset with paired audio and MIDI recordings and the official split used for training and evaluation."},{"cited_title":"Roformer: Enhanced transformer with rotary position embedding,","cited_arxiv_id":null,"evidence_quote":"Supplies the rotary position embeddings used to encode the positions of note-event tokens in the decoder."}],"review_version":1}