Pith. sign in

REVIEW 3 major objections 4 minor 43 references

Equivariant Music Transformer

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Standard music transformers become progressively less equivariant as they scale or train longer; an auxiliary shift-matching loss restores equivariance and improves next-token prediction.

desk verdict Useful empirical finding and a clean regularizer, but the unresolved onset-encoding inconsistency means the equivariance numbers need verification before I'd trust them. read the letter →

arxiv 2608.03920 v1 pith:HOMQ5B3K submitted 2026-08-04 cs.SD cs.AI

Zixun Guo , Simon Dixon This is my paper · ORCID
classification cs.SDcs.AI
keywords equivariantmusictransformersymbolicgenerationequivarianceregularizationself-distillationnext-tokenpredictionpitchtranspositiontimeshiftrepresentationlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that ordinary next-token training does not give symbolic music transformers the ability to recognize a passage under pitch transposition or time shift, and that this ability gets worse, not better, as models grow and train longer. It proposes the Equivariant Music Transformer (EMT), which adds a self-distillation loss: a randomly shifted input is passed through the same transformer, and the model is trained to make the output distribution equal to the inverse-shifted distribution of the original input. On LakhMIDI data, this auxiliary loss lowers validation next-token loss and improves equivariance metrics compared with plain training, data augmentation, feature engineering, and two external baselines. The motivating claim is that transformer capacity is being spent memorizing absolute patterns, and that an explicit equivariance inductive bias makes that capacity available for musically meaningful structure.

What carries the argument

The mechanism is a self-distillation equivariance loss computed by a second forward pass through the same shared-weight transformer. For a random musical shift T (pitch transposition within ±12 semitones, time shift up to 5 seconds, or both), the shifted input yields distribution p^T; the loss KL(sg(p) || T^{-1}(p^T)) forces the inverse-shifted output to match the original prediction, with the stop-gradient preventing both branches from collapsing to a trivial distribution. The inverse shift T^{-1} is implemented as torch.roll over the logits with boundary masking, which presumes that a musical shift equals an additive shift in token indices. The architecture also pairs this model-level loss

What would settle it

Take a trained EMT and compare its output on a prompt to its output on a time-shifted version of the same prompt, then apply torch.roll to the shifted output and measure whether equality holds token-by-token, especially for tokens near chunk boundaries. If the rolled distribution diverges at those boundaries, the L_equiv metric is rewarding token-index alignment rather than musical equivariance.

Watch

Extended reading notes

Core claim

The central discovery is an empirical trend and a remedy for it. Across the published checkpoints of the Anticipatory Music Transformer, the measured equivariance loss L_equiv rises with both model size (0.292 for size S to 0.813 for size L at 800K steps) and training duration (0.397 for the medium model at 100K steps to 0.741 at 800K steps), meaning shifted inputs are mapped to increasingly uncorrelated output distributions. EMT instead optimizes L_total = L_NTP + lambda L_equiv with lambda = 0.001, where L_equiv is the average KL divergence between a stop-gradient copy of the unshifted prediction and the inverse-shifted prediction on a randomly shifted input. This reaches L_equiv = 0.049 a

Load-bearing premise

The central assumption is that a musical transposition or time shift corresponds exactly to adding a constant to the token values, so reversing the model's output shift is simply rolling the logits; if the tokenizer's chunk-relative onsets break that correspondence, the claimed equivariance gains may be partly an artifact of the evaluation.

Editorial extensions

If this is right

  • Standard symbolic-music transformers do not acquire equivariance from next-token prediction alone; explicit supervision is required.
  • Scaling and longer training currently make the problem worse, so larger music models need equivariance regularization rather than just more data or compute.
  • The equivariance loss is complementary to feature-level engineering: EMT with feature-level equivariance beats each form of regularization separately.
  • Model-level equivariance regularization generalizes to out-of-distribution shifts better than data augmentation exposed to the same transformations.
  • Adding this loss does not trade away generation quality; it lowers validation loss and maintains listening-test quality with a smaller model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the loss only requires a shift operation that acts additively on tokens, the recipe should transfer to other autoregressive token domains with translational symmetries, not just music.
  • The reported trend suggests that L_equiv could be used as a diagnostic during training: a rising equivariance loss may signal that capacity is being spent on absolute patterns, independent of validation NTP loss.
  • A natural stress test is to restrict evaluation shifts to those that do not cross chunk boundaries; if L_equiv improves mainly on within-chunk shifts, the method is partly learning tokenization artifacts rather than musical shifts.
  • The fixed loss weight lambda=0.001 was chosen because L_equiv is numerically larger than L_NTP; an adaptive or scale-invariant weighting could make the method robust across tokenizers and datasets.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes the Equivariant Music Transformer (EMT), an autoregressive symbolic-music model trained with a next-token prediction loss plus an auxiliary equivariance loss that aligns predictions on randomly shifted inputs with the inverse-shifted predictions of the original input. The central claim is that standard music transformers become progressively less equivariant as they scale in size or train longer, and that the proposed equivariance regularizer acts as a beneficial inductive bias, improving both next-token prediction and structural robustness to pitch and time shifts. The evaluation includes internal ablations (same architecture, same shift distribution, fixed seed for auxiliary models), external baselines (Anticipatory Music Transformer, MIDI-LLM), objective equivariance metrics, and a listening test.

Significance. If the central claim holds, the paper provides useful evidence that scaling alone does not induce translational equivariance in symbolic music transformers and that an explicit equivariance loss can help. The ablation design is thoughtful: the comparison with data augmentation uses identical transform distributions and a fixed seed for auxiliary models, and the authors state that code, weights, and demos are available. The main weakness is that the paper's token-level definition of musical shifts is internally inconsistent and underspecified; this affects both the training objective and the evaluation metric, so the central quantitative results rest on an assumption that is not yet established.

major comments (3)
  1. [Eq. (2), Sec. 3.1, Sec. 4.1] The definition of T and T^{-1} is internally inconsistent and is load-bearing. Sec. 4.1 says onset values are relativized for each segmented chunk following [15], while Sec. 3.1 says EMT predicts absolute onsets to make T easy to apply. If onsets are chunk-relative, a temporal shift does not act as a uniform additive index shift on the input tokens, and torch.roll over output logits cannot be the exact inverse of the data transformation. Since the same T^{-1} is used both in the training loss and in the reported L_equiv metric, the central equivariance results in Table 1 and Fig. 4 are at risk of measuring a token-alignment artifact rather than musical equivariance. Please give the exact tokenization and a precise, preprocessing-aware definition of T and T^{-1} for both pitch and time shifts, including boundary masking and the quantized shift grid.
  2. [Sec. 4.4, Table 1] The headline equivariance metric L_equiv is exactly the training objective in Eq. (2), so its value is partly minimized by construction. Testing on an out-of-distribution superset of shifts weakens this circularity but does not remove it: the superset still assumes the same token-level homomorphism. Please report at least one criterion that is not derived from the same inverse-shift KL, such as latent-representation cosine alignment between shifted and unshifted inputs, a transposition-detection probe, or a human-perceptual measure, to substantiate that 'equivariance level' is not simply a proxy for optimizing the training loss.
  3. [Sec. 5.2, Table 1] The scaling claim that standard transformers become less equivariant as they scale or train longer is based on external Anticipatory checkpoints with a different tokenizer and relative-onset representation. Applying a generic torch.roll-based T^{-1} to these models may not implement the inverse of the musical shift in their token space. The paper must specify how T and T^{-1} are instantiated for each external baseline (Anticipatory S/M/L, MIDI-LLM), or the comparison may not be measuring the same property across models.
minor comments (4)
  1. [Sec. 5.1 vs Table 1] The text says Model 3 (data augmentation) 'yielded noticeably weaker generalizability, indicated by higher L_equiv' compared to the regularized models. In Table 1, however, Model 3's L_equiv (0.069) is lower than Model 4's (0.077). If the intended comparison is Model 3 vs Model 5, state this explicitly and compare the same columns.
  2. [Sec. 3, Eq. (2)] Please clarify the KL argument order (target vs prediction) and the exact handling of boundary positions in T^{-1}; torch.roll by itself is circular, and 'necessary masking at logit boundaries' is described only in prose, not in a formula or pseudocode.
  3. [Sec. 4.4, Table 2] The listening test uses 12 participants, and only p-values are reported. Report effect sizes, confidence intervals, or per-item agreement; small-sample Wilcoxon tests are otherwise difficult to interpret.
  4. [Fig. 3, Fig. 4] The figure captions do not map line styles/colors to Models 1–5. Adding legends or textual identification would make the evolution plots easier to verify.

Circularity Check

1 steps flagged · score 2.0 of 10

No central circularity; the only overlap is that L_equiv serves as both training objective and headline metric, partially mitigated by OOD evaluation and independent metrics.

  1. fitted input called prediction [Section 3 Eq. (2); Section 4.4 'Objective and Subjective Evaluation Settings'; Table 1]
    "Lequiv = 1/t sum KL(sg(pi) || T^{-1}(pT_i)) ... We then report the equivariance level using: L_equiv (See Section 3), Top-1 matching accuracy, and Top-5 Jaccard similarity for both shifted and unshifted attributes."

    The headline metric for 'equivariance level' (Table 1) is the same KL divergence L_equiv that EMT and Models 4-5 are explicitly trained to minimize through L_total = L_NTP + lambda*L_equiv (Eq. 3). Reporting low L_equiv for models optimized on L_equiv partly measures optimization success rather than an independently discovered property, especially when compared to baselines that never saw this objective. This is only partial circularity: the evaluation uses an out-of-distribution superset of shifts, and the paper also reports Top-1 matching accuracy, Top-5 Jaccard similarity, and L_NTP, which are not optimized by the equivariance loss for the methodological variants compared.

full rationale

The central derivation chain is otherwise self-contained and not circular. The scaling finding (standard transformers become less equivariant with size/training) is measured on external Anticipatory and MIDI-LLM checkpoints that were not trained with L_equiv, so that trend is an independent empirical observation. The claimed generative benefit of EMT is supported by validation L_NTP, which is not the equivariance objective, and by a listening test with paired Wilcoxon tests; neither reduces to the training loss by construction. The self-citations to Moonbeam [9] and FME [7] are to prior implemented and released work used as architectural components, not to an unverified uniqueness theorem or ansatz imported solely via citation. A separate correctness risk (not circularity) is the unresolved tension between Sec. 3.1 ('we predict absolute rather than relative onsets') and Sec. 4.1 ('onset values are relativized for each segmented chunk, following the implementation of [15]'); if chunk-relative onsets are actually used, the torch.roll-based T^{-1} may not be the exact inverse of the transformation applied to the data, which would affect the validity of both L_equiv and the reported equivariance metrics. That issue concerns factual consistency rather than circular reasoning. Overall, the overlap between the training loss and one evaluation metric is real but partial, so a low score is appropriate.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the standard next-token objective (Eq. 1), the equivariance KL loss (Eq. 2) with a stop-gradient anchor, and the assumption that musical shifts act as clean additive token-index shifts compatible with the delay pattern and the chunk-relativized onset preprocessing. The main fitted knob is lambda = 0.001, chosen empirically because L_equiv is numerically larger than L_NTP; the training and evaluation shift-sampling distributions and the listening-test shift ranges are hand-chosen. No genuinely new theoretical entities are postulated; the model re-combines prior architecture components (FME, MRA from the authors' Moonbeam). The strongest load-bearing premise is the logit-roll compatibility of T^{-1} with the tokenization, described in the weakest_assumption field.

free parameters (5)
  • equivariance loss weight lambda = 0.001
    Chosen empirically in Sect. 4.2 because the KL-based L_equiv is numerically larger than L_NTP; no sweep is reported and one value is used for all regularized models.
  • training shift distribution (pitch range, time grid, mixing ratio) = pitch within [-12,12] semitones; time 0-5 s in 0.25 s steps; 50% single, 50% paired; 44 single + 480 combined transforms
    Hand-chosen in Sect. 4.2. This distribution defines what in-distribution equivariance means and shapes the headline L_equiv numbers, since the evaluation superset is built around it.
  • evaluation shift sampling fractions = 20% of single shifts and 1% of combined shifts, 79 shifts total
    Arbitrary sampling of the test superset in Sect. 4.4; a different sample would move the reported equivariance numbers.
  • dataset filtering thresholds = files >1 h or <5 s removed; IOI >60 s removed
    Preprocessing choices in Sect. 4.1 that determine the training corpus; not fitted to a target, but they define the data distribution.
  • listening-test shift ranges = pitch up to +/-6 semitones, onset shift up to 1 s
    Chosen in Sect. 4.4 to be mostly unnoticeable; the perceptual robustness claim is assessed under this narrow shift window, not under the full shift range used in objective evaluation.
assumptions (5)
  • domain assumption Pitch transposition and time shift are the musically relevant symmetries, and equivariance to them is a desirable inductive bias for generative music models.
    Introduction and Sect. 2. Rooted in the perceptual claim that humans recognize transposed/shifted melodies. Non-trivial because real music also has key/register and meter structure that exact translation equivariance ignores.
  • ad hoc to paper T acts as an additive index shift on the 5-way token sequence, and T^{-1} on output distributions is exactly torch.roll over logits with boundary masking.
    Eq. 2 and Sect. 3. The load-bearing compatibility assumption. It breaks if chunk-relativized onsets (Sect. 4.1) or the delay pattern destroy the homomorphism between musical shifts and token-index shifts.
  • domain assumption With stop-gradient on the anchor, KL(sg(p_i) || T^{-1}(p_i^T)) is a stable, non-collapsing training signal.
    Sect. 3, discussed after Eq. 2. The anchor is the same network and therefore a moving target; the paper shows that without explicit supervision the anchor branch becomes less equivariant over training, so the stability of this bootstrap is assumed, not demonstrated.
  • domain assumption Validation cross-entropy L_NTP is an adequate measure of generative capability.
    Eq. 1 and Sect. 4.4. Since L_NTP is the training objective, 'the auxiliary loss improves generation' is partly a statement about a smoother optimization landscape, not independent evidence of perceptual quality; the MOS test is the only external check.
  • domain assumption The LakhMIDI clean subset, with the stated filters and 5-way tokenization, is a representative proxy for symbolic music for the ablation conclusions.
    Sect. 4.1. All internal-model conclusions come from one dataset subset, one seed, and one architecture family; external baselines use different tokenizers and corpora.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Equivariant Music Transformer." pith.science (2026). https://pith.science/paper/HOMQ5B3K

@misc{pith2026260803920,
  author       = {Pith},
  title        = {Pith review of: Equivariant Music Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HOMQ5B3K}},
  note         = {Machine review of arXiv:2608.03920}
}
read the original abstract

Humans recognize a musical passage even when it is shifted in time or transposed in pitch, indicating a notion of equivariance in the representation space. Our analysis, however, shows that standard music transformers map such time-shifted or pitch-transposed inputs onto uncorrelated representations: these models become progressively less equivariant as they scale in size or train longer. This suggests that in standard music transformers, additional model capacity is allocated to memorizing absolute patterns rather than capturing shared musical structures. In this paper, we propose the Equivariant Music Transformer (EMT), which enforces equivariance through self-distillation by jointly optimizing a next-token-prediction and an auxiliary equivariance regularization loss. We find that the additional equivariance loss acts as a beneficial regularizer, simultaneously improving next-token prediction and producing equivariant latent representations. Through both objective and subjective evaluations, EMT demonstrates superior equivariance and generative capability compared to data augmentation, feature engineering, and state-of-the-art (SOTA) baselines. More broadly, our findings reveal that standard language modeling methods alone do not capture music's translational symmetries, and dedicated inductive biases are required to produce better music representations. The code, weights and demos are available online.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 37 canonical work pages

  1. [15]

    QMERC20.565.DSEECS25.027)

    ETHICS STA TEMENT The listening study in this paper was approved by Queen Mary University of London’s ethics review board (ref. QMERC20.565.DSEECS25.027). Participation was vol- untary and based on informed consent: participants were informed of the purpose, duration, and nature of the task before taking part, and received no compensation. No per- sonally...

    work page

  2. [1]

    Equivariant Music Transformer

    INTRODUCTION Humans effortlessly recognize a musical passage regard- less of whether it is transposed to a different key or shifted in time. This is because the perceived intervals remain unchanged even as the absolute pitches or onsets change. In computational terms, this corresponds to the notion of equivariance [1, 2]: a transformation of the input pro...

    work page Pith review arXiv 2026

  3. [2]

    The auxiliary loss not only pro- duces music representations that align with human perception of music, but also directly improves the model’s generative capabilities

    We propose the Equivariant Music Transformer (EMT), a transformer architecture jointly optimized with next-token prediction and equivariance regular- ization objectives. The auxiliary loss not only pro- duces music representations that align with human perception of music, but also directly improves the model’s generative capabilities

    work page

  4. [3]

    To the best of our knowledge, this provides the first empirical evidence that equivariance does not emerge from scaling

    We reveal that standard symbolic music transform- ers become progressively less equivariant as they scale or train for more iterations. To the best of our knowledge, this provides the first empirical evidence that equivariance does not emerge from scaling

    work page

  5. [4]

    Moreover, we confirm that EMT maintains consistent generative quality and smoothness under musically-shifted prompts, while the baseline exhibits more significant degradations

    Through a listening test, EMT achieves comparable unconditional music generation quality compared to the Anticipatory Music Transformer [15], despite being a smaller model. Moreover, we confirm that EMT maintains consistent generative quality and smoothness under musically-shifted prompts, while the baseline exhibits more significant degradations

    work page

  6. [5]

    EMT achieves substantially stronger equivariance and better generative capabilities compared to data augmentation, feature-level equivariance, and SOTA external baselines

    work page

  7. [6]

    CONCLUSION We proposed the Equivariant Music Transformer (EMT), jointly optimized with the next-token prediction and equiv- ariance regularization loss. Through both subjective and objective evaluations, we show that adding the equivari- ance inductive bias not only produces music representa- tions that align with human perception of music better, but als...

    work page

  8. [7]

    For example, transposing a melody by 2 semitones should shift the output pitch distri- bution by the same amount

    RELA TED WORK A model isequivariantto a transformation if the transfor- mation at the input results in a corresponding, predictable transformation at the output. For example, transposing a melody by 2 semitones should shift the output pitch distri- bution by the same amount. Formally, letTdenote a trans- formation in the input space andT ′ be its correspo...

    work page

Show all 43 references
  1. [8]

    The main branch to the left rep- resents a generative transformer, which is trained to au- toregressively predict the music tokenx t at the next stept

    METHOD Figure 1 outlines the architecture of our proposed Equiv- ariant Music Transformer. The main branch to the left rep- resents a generative transformer, which is trained to au- toregressively predict the music tokenx t at the next stept. More specifically, given a music s...

  2. [9]

    We also conduct a listening test to evaluate the robustness of symbolic music transformers under musically-shifted in- put prompts

    2 , with their combination yielding the best results. We also conduct a listening test to evaluate the robustness of symbolic music transformers under musically-shifted in- put prompts. Our results indicate that the baseline model exhibits a larger degradation in both smoothne...

  3. [10]

    We conduct our ablation studies using the clean subset (LMD clean) due to computation constraints

    EXPERIMENT SETTINGS 4.1 Dataset and Preprocessing Our models are trained using different subsets of the LakhMIDI dataset [24]. We conduct our ablation studies using the clean subset (LMD clean) due to computation constraints. We randomly hold out 10% of the clean subset as the...

  4. [11]

    Overall, models incorporating auxiliary signals (Model 3-

    RESULTS 5.1 Equivariance and Loss Evolution Next-token Prediction Loss:Throughout training, the validation loss across all models consistently decreases before consolidating into stable convergence (Figure 3). Overall, models incorporating auxiliary signals (Model 3-

  5. [12]

    This shows that explic- itly adding an equivariance regularizer not only enforces structural coherence but also directly improves the model’s generative capabilities

    achieve lower validation losses than the models trained only withL NTP (Models 1-2). This shows that explic- itly adding an equivariance regularizer not only enforces structural coherence but also directly improves the model’s generative capabilities. Among the regularized mod...

  6. [13]

    ACKNOWLEDGMENTS Zixun Guo is a PhD student at the UKRI Centre for Doc- toral Training (CDT) in Artificial Intelligence and Music (AIM), supported by UK Research and Innovation [grant number EP/S022694/1]

  7. [14]

    We also used Claude Code to assist in implementing the code- base, all of which was manually verified for correctness by the first author

    AI USAGE STA TEMENT We used generative AI assistants (Gemini and Claude) to improve the grammar, spelling, and phrasing of the au- thors’ original text, and to help format L ATEX tables. We also used Claude Code to assist in implementing the code- base, all of which was manual...

  8. [16]

    Group equivariant convolutional networks,

    T. Cohen and M. Welling, “Group equivariant convolutional networks,” inProceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19- 24, 2016, ser. JMLR Workshop and Conference Proceedings, M. Balcan and K. Q. Weinberger, Eds...

    2016

  9. [17]

    Scale and contour: Two components of a theory of memory for melodies,

    W. J. Dowling, “Scale and contour: Two components of a theory of memory for melodies,”Psychological Review, vol. 85, no. 4, pp. 341–354, 1978

    1978

  10. [18]

    PESTO: real-time pitch estimation with self-supervised transposition- equivariant objective,

    A. Riou, B. Torres, B. Hayes, S. Lattner, G. Hadjeres, G. Richard, and G. Peeters, “PESTO: real-time pitch estimation with self-supervised transposition- equivariant objective,”Trans. Int. Soc. Music. Inf. Retr., vol. 8, no. 1, pp. 334–352, 2025. [Online]. Available: https://d...

    2025 doi

  11. [19]

    Equivariant self-supervision for musical tempo estimation,

    E. Quinton, “Equivariant self-supervision for musical tempo estimation,” inProceedings of the 23rd International Society for Music Information Retrieval Conference, ISMIR 2022, Bengaluru, India, December 4-8, 2022, P. Rao, H. A. Murthy, A. Srinivasamurthy, R. M. Bittner, R. C....

    2022

  12. [20]

    Adapting pitch-based self supervised learning models for tempo estimation,

    A. Gagneré, S. Essid, and G. Peeters, “Adapting pitch-based self supervised learning models for tempo estimation,” inIEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2024, Seoul, Republic of Korea, April 14-19,

    2024

  13. [21]

    A predictive model for music based on learned interval representations,

    ——, “A predictive model for music based on learned interval representations,” inProceedings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France, September 23- 27, 2018, 2018, pp. 26–33

    2018

  14. [22]

    STONE: self- supervised tonality estimator,

    Y . Kong, V . Lostanlen, G. Meseguer-Brocal, S. Wong, M. Lagrange, and R. Hennequin, “STONE: self- supervised tonality estimator,” inProceedings of the 25th International Society for Music Information Retrieval Conference, ISMIR 2024, San Francisco, California, USA and Online,...

    2024 doi

  15. [23]

    A domain- knowledge-inspired music embedding space and a novel attention mechanism for symbolic mu- sic modeling,

    Z. Guo, J. Kang, and D. Herremans, “A domain- knowledge-inspired music embedding space and a novel attention mechanism for symbolic mu- sic modeling,” inThirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of ...

    2023

  16. [24]

    On the importance of time and pitch relativity for transformer- based symbolic music generation,

    T. Inaba, K. Yoshii, and E. Nakamura, “On the importance of time and pitch relativity for transformer- based symbolic music generation,” inAsia Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2024, Macau, December 3-6, 2024. IEEE,...

    2024

  17. [25]

    Moonbeam: A midi founda- tion model using both absolute and relative music at- tributes,

    Z. Guo and S. Dixon, “Moonbeam: A midi founda- tion model using both absolute and relative music at- tributes,”arXiv preprint arXiv:2505.15559, 2025

    2025 arXiv

  18. [26]

    F-stripe: Fast structure-informed positional encoding for sym- bolic music generation,

    M. Agarwal, C. Wang, and G. Richard, “F-stripe: Fast structure-informed positional encoding for sym- bolic music generation,” in2025 IEEE International Conference on Acoustics, Speech and Signal Process- ing, ICASSP 2025, Hyderabad, India, April 6-11,

    2025

  19. [28]

    Music transformer: Gen- erating music with long-term structure,

    C. A. Huang, A. Vaswani, J. Uszkoreit, I. Simon, C. Hawthorne, N. Shazeer, A. M. Dai, M. D. Hoffman, M. Dinculescu, and D. Eck, “Music transformer: Gen- erating music with long-term structure,” in7th Interna- tional Conference on Learning Representations, ICLR 2019, New Orlean...

    2019

  20. [29]

    Mustango: Toward con- trollable text-to-music generation,

    J. Melechovsky, Z. Guo, D. Ghosal, N. Majumder, D. Herremans, and S. Poria, “Mustango: Toward con- trollable text-to-music generation,” inProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technol...

    2024

  21. [30]

    Lakhnes: Improving multi- instrumental music generation with cross-domain pre-training,

    C. Donahue, H. H. Mao, Y . E. Li, G. W. Cottrell, and J. J. McAuley, “Lakhnes: Improving multi- instrumental music generation with cross-domain pre-training,” inProceedings of the 20th International Society for Music Information Retrieval Conference, ISMIR 2019, Delft, The Net...

    2019

  22. [31]

    Scaling self-supervised represen- tation learning for symbolic piano performance,

    L. Bradshaw, A. Spangher, H. Fan, S. Biderman, and S. Colton, “Scaling self-supervised represen- tation learning for symbolic piano performance,” inProceedings of the 26th International Soci- ety for Music Information Retrieval Conference, ISMIR 2025, Daejeon, South Korea, Sep...

    2025 doi

  23. [32]

    Anticipatory music transformer,

    J. Thickstun, D. L. W. Hall, C. Donahue, and P. Liang, “Anticipatory music transformer,”Trans. Mach. Learn. Res., vol. 2024, 2024

    2024

  24. [33]

    Leave- one-equivariant: Alleviating invariance-related in- formation loss in contrastive music representa- tions,

    J. Guinot, E. Quinton, and G. Fazekas, “Leave- one-equivariant: Alleviating invariance-related in- formation loss in contrastive music representa- tions,” in2025 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2025, Hyderabad, India, April 6-11,

    2025

  25. [34]

    IEEE, 2025, pp. 1–5. [Online]. Available: https://doi.org/10.1109/ICASSP49660.2025.10890270

    2025

  26. [35]

    Audio synthesizer inversion in symmetric parameter spaces with approximately equivariant flow matching,

    B. Hayes, C. Saitis, and G. Fazekas, “Audio synthesizer inversion in symmetric parameter spaces with approximately equivariant flow matching,” in Proceedings of the 26th International Society for Music Information Retrieval Conference, ISMIR 2025, Daejeon, South Korea, Septemb...

    2025

  27. [36]

    Position paper: Towards a unified representation evaluation framework beyond downstream tasks,

    C. Plachouras, J. Guinot, G. Fazekas, E. Quinton, E. Benetos, and J. Pauwels, “Position paper: Towards a unified representation evaluation framework beyond downstream tasks,” inInternational Joint Conference on Neural Networks, IJCNN 2025, Rome, Italy, June 30 - July 5, 2025. ...

    2025

  28. [37]

    Music102: And_{12}-equivariant trans- former for chord progression accompaniment,

    W. Luo, “Music102: And_{12}-equivariant trans- former for chord progression accompaniment,”arXiv preprint arXiv:2410.18151, 2024

    2024 arXiv

  29. [38]

    Learning transposition-invariant interval features from symbolic music and audio,

    S. Lattner, M. Grachten, and G. Widmer, “Learning transposition-invariant interval features from symbolic music and audio,” inProceedings of the 19th Inter- national Society for Music Information Retrieval Con- ference, ISMIR 2018, Paris, France, September 23-27, 2018, 2018, p...

    2018

  30. [40]

    Simple and controllable music generation,

    J. Copet, F. Kreuk, I. Gat, T. Remez, D. Kant, G. Synnaeve, Y . Adi, and A. Defossez, “Simple and controllable music generation,” inAdvances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, Eds., vol. 36. Curran Ass...

    2023

  31. [41]

    Time-shifted token scheduling for symbolic music generation,

    T. Wang, C. Tan, and Y . Yang, “Time-shifted token scheduling for symbolic music generation,” CoRR, vol. abs/2509.23749, 2025. [Online]. Available: https://doi.org/10.48550/arXiv.2509.23749

    2025 doi

  32. [42]

    Learning-based methods for comparing se- quences, with applications to audio-to-midi alignment and matching,

    C. Raffel, “Learning-based methods for comparing se- quences, with applications to audio-to-midi alignment and matching,” Ph.D. dissertation, Columbia Univer- sity, USA, 2016

    2016

  33. [43]

    The llama 3 herd of models,

    L. Team, “The llama 3 herd of models,”CoRR, vol. abs/2407.21783, 2024. [Online]. Available: https: //doi.org/10.48550/arXiv.2407.21783

    2024 doi arXiv

  34. [44]

    MIDI-LLM: Adapting large language models for text-to-MIDI mu- sic generation,

    S.-L. Wu, Y . Kim, and C.-Z. A. Huang, “MIDI-LLM: Adapting large language models for text-to-MIDI mu- sic generation,” inProc. NeurIPS AI4Music Workshop, 2025

    2025

  35. [2024]

    IEEE, 2024, pp. 956–960. [Online]. Available: https://doi.org/10.1109/ICASSP48485.2024.10447129

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.