{"id":"a3b14722-acf9-447f-8c57-bd3a74b4fbb1","arxiv_id":"2412.18955","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A contrastive music representation method, Leave-One-EquiVariant, keeps pitch and tempo information in separate embedding subspaces, improving key and tempo tasks without hurting tagging.","lead":"This paper introduces a way to train music-listening AI models that keeps information about pitch and tempo, instead of throwing it away during pretraining. In tests, the method recovers most of the lost accuracy on key, pitch, and tempo tasks while keeping automatic tagging performance unchanged.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The variant-head positive set excludes pairs that both received the target augmentation, relying on an assumption the authors themselves flag as potentially confusing in Appendix D.","rationale":"The reader's weakest_assumption identifies the same positive-set construction issue that I consider most load-bearing, and the authors' own Appendix D corroborates it as a real limitation. The central claim that LOEV preserves augmentation information depends on the variant heads receiving the right positive/negative structure; if near-identical transformed views are incorrectly pushed apart, the method's mechanism for preserving continuous parameters is compromised. The concern is concrete and testable through the proposed ablation. However, the empirical results in Table I are strong enough that this is not grounds for rejection, but rather for a conditional acceptance pending a robustness check. The paper also shows good practice by comparing against same-scale MULE baselines and by disclosing architectural and training differences, which supports the credibility of the empirical claims despite the lack of released code or variance estimates. Thus the verdict remains CONDITIONAL, and my stress-test does not alter it; hence verdict_should_be is UNCHANGED.","tokens_in":11147,"tokens_out":11739,"duration_ms":115169,"concrete_test":"Retrain the LOEV-PSTS model with a modified variant-head positive set that additionally includes same-track pairs where both views received Tk and the sampled parameter difference is below a small tolerance (e.g., pitch shift within 0.25 semitones, stretch factor within 0.01). Compare key (Giantsteps), tempo (AllTempo), and tagging (MTAT) probing metrics against the original LOEV-PSTS. If the modified model improves key/tempo accuracy or changes tagging by more than a small margin, the exclusion of similar-parameter pairs is causing measurable information loss; if results are unchanged, the assumption holds and the concern is not load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that LOEV(++) reduces invariance-related information loss rests on the positive set construction in Section II-B. For variant head k, Pk(i) = P(i) ∩ {j | tj,k = ti,k = 0}, i.e., only pairs in which neither view received the target augmentation Tk are attracted. This design has two unexamined consequences. First, anchors that did receive Tk have an empty positive set, so the variant head never directly supervises the representation of transformed views; it only learns to separate Tk=0 from Tk=1, which may preserve a binary flag rather than the continuous shift/stretch parameter that key and tempo probing require. Second, pairs where both views received Tk with similar sampled parameters are treated as negatives, and the paper's justification ('stochastic uniform sampling of continuous parameters... sufficient guarantee') is an assumption, not a result. The authors themselves admit in Appendix D that 'perceptually-similar augmented samples are treated the same as radically different augmented samples with our strategy, which might introduce confusion into the training objective.' If near-identical transformed views are pushed apart, the preserved-information objective degrades exactly in the regime the method is meant to protect, and the claimed mechanism for maintaining continuous augmentation information is weakened. The empirical results may still hold, but the theoretical foundation of the positive set is undersupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Leave-One-EquiVariant (LOEV) and its variant LOEV++ to reduce invariance-related information loss in contrastive music representations. In addition to the standard all-invariant projection head, LOEV trains one extra projection head per selected augmentation, with the positive set for head k restricted to pairs in which neither view received augmentation Tk. The authors argue that this forces the shared encoder to preserve information about all tracked augmentations. LOEV++ additionally resorbs part of the last encoder block into the parallel projection heads to create a concatenated representation space. The paper evaluates frozen representations from models pretrained on MTG-Jamendo on automatic tagging, key estimation, pitch estimation, tempo estimation, and retrieval tasks. The main claimed results are that LOEV and LOEV++ recover much of the key and tempo accuracy lost by standard MULE when pitch shifting and time stretching are used, without harming tagging, and that LOEV++ yields somewhat specialized subspaces for pitch and tempo information.","tokens_in":11438,"tokens_out":4020,"duration_ms":42422,"significance":"If supported, the method would be a practical and reasonably light-weight way to make contrastive music representations task-adaptive, addressing a real limitation of augmentation chains. The paper also transfers the LOOC idea from vision to music with an explicit link to semantically meaningful attributes (key and tempo), which is a useful contribution. The appendices are unusually candid about design limitations, and the paper ships a substantial set of experiments spanning probing and retrieval. However, the empirical support is weakened by the absence of error bars or repeated runs, by per-dataset tuning of probe architectures, and by an underspecified positive-set mechanism that the authors themselves partially question in Appendix D. These issues are fixable and do not, by themselves, invalidate the contribution, but they currently prevent the strong claims in the conclusion from being fully supported.","major_comments":[{"comment":"The construction Pk(i) = P(i) ∩ {j | tj,k = ti,k = 0} means that when an anchor has received the target augmentation Tk, its positive set is empty, so the variant head receives no direct positive supervision for transformed views. As a result, the head may learn only a binary separation between Tk=0 and Tk=1 rather than the continuous shift or stretch parameter that key and tempo probing require. In addition, pairs that both received Tk with nearly equal sampled parameters are treated as negatives, relying on the assertion that uniform sampling of continuous parameters guarantees distinct embeddings. Appendix D explicitly concedes that perceptually similar augmented samples are treated the same as radically different augmented samples, which could introduce confusion into the training objective. The authors should provide an analysis or ablation demonstrating that the variant head encodes the continuous augmentation parameter and that near-identical transformed pairs do not degrade the preserved-information objective.","section":"II-B and Appendix D"},{"comment":"All reported results come from single pretraining and probing runs, with no error bars, and probe architectures, dropout, and learning rates are empirically adjusted per dataset. The conclusion's central claim that LOEV(++) 'reduces information loss ... with no detriment to tagging performance' rests on small tagging differences, e.g., MTAT AUROC 90.6 vs 90.5 and AP 38.4 vs 38.7 between LOEV-PSTS and MULE-PSTS, which cannot be distinguished from run-to-run noise. The authors should report means and standard deviations over multiple seeds, and ideally statistical significance, for the key comparisons: LOEV-PSTS vs MULE-PSTS and LOEV++ vs MULE++ on tagging, key, pitch, and tempo tasks.","section":"III-C, Table I and Conclusion"},{"comment":"The claim that LOEV++ creates a disentangled latent space is supported only by point estimates and qualitative inspection. In Table II, the pitch subspace gives key accuracy 43.0 versus 39.0 for the invariant subspace and 30.0 for the tempo subspace, and the tempo subspace gives tempo acc1 71.5 versus 64.7 for the pitch subspace; Fig. 3 shows cosine-distance curves without uncertainty. Given that these differences motivate the disentanglement claim, the authors should add a quantitative disentanglement evaluation with confidence intervals, or at least repeated runs, rather than relying on single-probe point estimates.","section":"III-D, Table II and Fig. 3"}],"minor_comments":[{"comment":"The word 'defauly' should be 'default' in the in-track sampling strategy paragraph.","section":"Appendix D"},{"comment":"In the time-stretching row, the parameter name 'transpose' should be 'stretch factor'; the current wording is confusing.","section":"Table IV"},{"comment":"The probe time-stretching augmentation is specified as tau ~ U(0.8, 1.2), while the pretraining time-stretch range in Table IV is 0.7 to 1.3; the relationship between these two ranges should be clarified.","section":"III-A"},{"comment":"The notation Pk(i) = P(i) ∩ {j | tj,k = ti,k = 0} is clear in symbols, but the preceding sentence says only 'samples from the same anchor that have not been augmented with Tk'; the requirement that the anchor itself also have ti,k = 0 should be stated explicitly to avoid ambiguity.","section":"II-B"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the central idea is worth publishing if the authors can shore up the evidence. The main risk is overclaiming from single-run experiments and from a positive-set mechanism whose own appendix calls into question. I would not reject, but I would require additional experiments or a clear statistical treatment before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [name],\n\nYou should read this paper if you work on contrastive music representations. It takes the leave-one-out idea from vision and makes it work for music with a simpler, more flexible mechanism: track which augmentations each view received in a binary vector, and for each target augmentation train an extra head whose positives are only pairs that both avoided that augmentation. The pitch-shift and time-stretch heads preserve key, pitch, and tempo information that ordinary invariance training washes out, and tagging performance does not collapse. The tables support that claim, and the retrieval experiments are a good addition.\n\nWhat is new: the binary augmentation tracking, the application to semantically meaningful musical attributes, and the LOEV++ variant that disentangles the latent space into per-augmentation subspaces. The paper also reproduces MULE at smaller scale and compares fairly against equally-scaled baselines, which is more than much of this literature does. The cosine-distance analysis showing pitch-variant structure in the Vp subspace is a nice piece of evidence.\n\nThe soft spots are real but not fatal. All results come from single runs with no error bars, so we cannot tell if the key/tempo gains are significant. The probe architectures are tuned per dataset, which creates some slack. More importantly, the positive set for variant head k is empty for anchors that received Tk, and pairs where both views received the same augmentation with similar parameters are actively repelled. The paper justifies this with the stochastic sampling of continuous parameters, but that is an assumption, and Appendix D concedes that perceptually similar augmented samples are treated as negatives. That could degrade the exact information the method aims to preserve. The authors flag it honestly, but they do not analyze it or provide a fallback.\n\nLOEV++'s disentanglement results are modest, and the abstract oversells them a bit. The tempo anomaly for MULE with both pitch and time stretching is waved off rather than explained.\n\nOverall: the central mechanism is clearly defined, the comparisons are fair, and the results are directionally convincing. It deserves a serious referee. The main revisions I would ask for are multiple seeds or confidence intervals, a precise description of how empty positive sets are handled, and some analysis of the near-identical-positive problem.","headline":"A fair adaptation of LOOC to music with a useful tracking scheme; the empirical case is plausible, but the variant-head positive set has an acknowledged soft spot that needs tightening.","tokens_in":11914,"tokens_out":2567,"would_cite":true,"duration_ms":155575,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A contrastive music encoder can be trained to stay equivariant to pitch shift and time stretch by giving each augmentation its own projection head, so that attribute information survives in the frozen embeddings instead of being erased.","keywords":["contrastive learning","music representation learning","equivariance","learned invariances","pitch shifting","time stretching","disentangled latent space","music information retrieval"],"falsifier":"Train LOEV with pitch-shift amounts drawn from a tiny interval (e.g., ±0.05 semitones) so two independently shifted views are nearly identical; if key-estimation probing then collapses toward the all-invariant baseline, the separation of the variant-head positive set rests on continuous-parameter diversity rather than on a robust principle.","tokens_in":10952,"feed_emoji":"🎵","tokens_out":4988,"duration_ms":51452,"temperature":0.7,"pith_summary":"The paper tries to establish that contrastive music representations do not have to pay the usual price for augmentation robustness. It proposes Leave-One-EquiVariant (LOEV): alongside the standard all-invariant projection head, add one head for each augmentation whose attribute matters downstream, and define that head's positives as views that did not receive that augmentation. The shared encoder must then preserve the augmentation's information, so key, pitch, and tempo survive in frozen embeddings while tagging stays unaffected. A second variant, LOEV++, structurally disentangles the latent space into invariant, pitch-variant, and tempo-variant subspaces, enabling retrieval by those attributes.","feed_headline":"Music AI keeps pitch and tempo it was told to ignore","feed_subtitle":"Leave-one-equivariant heads preserve key and tempo info in frozen embeddings, lifting retrieval while tagging holds.","key_machinery":"The central mechanism is the variant positive set combined with a per-sample augmentation tracking vector. Each sample i carries a binary vector ti indicating which of the K variant augmentations were applied, and for augmentation Tk the positive set is Pk(i) = P(i) ∩ {j | tj,k = ti,k = 0}, so only views that both avoided Tk are pulled together. This forces the k-th projection head to stay equivariant to Tk while remaining invariant to everything else, and the global loss averages the all-invariant head with all variant heads. LOEV++ strengthens the scheme by parallelizing part of the last encoder block into each head, creating separate subspaces Vi, Vp, and Vt that can be probed or concatenated as V++.","core_discovery":"The paper's central claim is that a contrastive encoder can be made to retain attribute information that standard all-invariant training discards, by allocating one extra projection head per augmentation of interest and defining that head's positives to be views that did NOT receive that augmentation. The shared encoder then must keep the augmentation's information in order to satisfy all heads, producing an embedding superspace V from which pitch-key and tempo can still be probed and retrieved. The paper further claims that LOEV++, which resorbs part of the encoder into the heads, yields disentangled subspaces — an invariant space Vi, a pitch-variant space Vp, and a stretch-variant space Vt — with targeted retrieval working best in the corresponding variant space. The reported evidence: on key estimation, LOEV++ PSTS reaches 44.2 weighted accuracy versus 15.1 for MULE++; tempo acc1/acc2 reach 72.6/91.1 versus 63.2/84.1; automatic tagging stays at baseline (MTAT AUROC 90.6).","pith_inferences":["A testable extension is to add same-parameter-augmented pairs as an extra positive set for the variant head, which would directly address the paper's assumption that continuous parameter sampling guarantees distinct views and could tighten the preserved-information objective.","The one-head-per-augmentation cost grows linearly with the number of variant augmentations, and sparse contrastive matrices become a practical limit; this suggests a selection criterion for which augmentations deserve variant heads, a question the paper leaves open.","The same positive-exclusion scheme could be applied to nonparametric semantic transformations such as genre or instrumentation if a continuous proxy parameter can be defined, potentially extending equivariance beyond audio effects.","Because the paper only evaluates 3-second chunks, the method's ability to preserve key and tempo at track level — where key changes and tempo drifts occur — remains untested and is a natural next benchmark."],"forward_implications":["Adding pitch shifting to pretraining no longer sacrifices key and pitch information: LOEV++ with pitch and stretch variants reaches 44.2 weighted key accuracy on Giantsteps versus 15.1 for MULE++ trained with the same augmentations.","Tempo information survives time stretching: LOEV++ PSTS reaches 72.6/91.1 acc1/acc2 on the AllTempo benchmark versus 63.2/84.1 for MULE++.","Automatic tagging is not harmed: MTG-Jamendo and MagnaTagATune AUROC and average precision remain at the MULE++ baseline level.","LOEV++ disentangles the latent space by design: key retrieval is best in the pitch-variant subspace Vp, tempo retrieval is best in the stretch-variant subspace Vt, enabling attribute-targeted retrieval.","The framework is transformation-agnostic, so any augmentation with a semantic downstream attribute can be given its own variant head without changing the training objective's structure."],"supporting_citations":[{"why":"Source of the leave-one-out contrastive idea and the head-duplication scheme that LOEV and LOEV++ adapt to music.","marker":"[16]"},{"why":"MULE is the contrastive music baseline and the encoder/projection architecture that LOEV extends and compares against.","marker":"[5]"},{"why":"SimCLR provides the contrastive objective and pipeline that LOEV builds on for its variant heads.","marker":"[1]"},{"why":"Establishes the local embedding distortion caused by augmentations in music, the phenomenon LOEV addresses, and supplies the cosine-distance analysis method reused here.","marker":"[17]"},{"why":"Provides the F0-SF-NFNet encoder architecture that the paper reproduces for pretraining.","marker":"[19]"},{"why":"First adaptation of contrastive learning to music representation and the source of the canonical data splits used for tagging evaluation.","marker":"[2]"},{"why":"Supplies the AllTempo tempo-estimation setup and the time-stretch robustness augmentation used when probing tempo.","marker":"[31]"},{"why":"MARBLE benchmark provides the key-estimation probing protocol and Giantsteps evaluation setup.","marker":"[25]"}],"fun_headline_variants":["LOEV: Music AI keeps pitch and tempo it once ignored","One extra head helps music AI retain key and tempo","Leave-one-equivariant: music AI remembers what it drops","Music AI now preserves pitch and tempo with LOEV"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's load-bearing premise is that two views that both received the target augmentation are never so similar that they should be positives, because continuous parameters are sampled uniformly; if the sampled shifts or stretch factors coincide or are perceptually identical, the variant head is pushed to pull genuine positives apart and the preserved-information objective degrades.","fun_headline_variants_meta":{"raw":{"variants":["LOEV: Music AI keeps pitch and tempo it once ignored","One extra head helps music AI retain key and tempo","Leave-one-equivariant: music AI remembers what it drops","Music AI now preserves pitch and tempo with LOEV"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000329,"raw_usage":{"total_tokens":1820,"prompt_tokens":913,"completion_tokens":907,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":841}},"tokens_in":529,"tokens_out":907,"duration_ms":10467,"temperature":1.0,"reasoning_tokens":841,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:58:32.356855+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train LOEV with pitch-shift amounts drawn from a tiny interval (e.g., ±0.05 semitones) so two independently shifted views are nearly identical; if key-estimation probing then collapses toward the all-invariant baseline, the separation of the variant-head positive set rests on continuous-parameter diversity rather than on a robust principle.","supporting_citations":[{"cited_title":"What Should Not Be Contrastive in Contrastive Learning,","cited_arxiv_id":null,"evidence_quote":"Source of the leave-one-out contrastive idea and the head-duplication scheme that LOEV and LOEV++ adapt to music."},{"cited_title":"A simple framework for contrastive learning of visual representations,","cited_arxiv_id":null,"evidence_quote":"SimCLR provides the contrastive objective and pipeline that LOEV builds on for its variant heads."},{"cited_title":"On the effect of data-augmentation on local embedding properties in the contrastive learning of music audio representations,","cited_arxiv_id":null,"evidence_quote":"Establishes the local embedding distortion caused by augmentations in music, the phenomenon LOEV addresses, and supplies the cosine-distance analysis method reused here."},{"cited_title":"Towards Learning Universal Audio Representations,","cited_arxiv_id":null,"evidence_quote":"Provides the F0-SF-NFNet encoder architecture that the paper reproduces for pretraining."},{"cited_title":"Similar but Faster: Manipulation of Tempo in Music Audio Embeddings for Tempo Prediction and Search","cited_arxiv_id":"2401.08902","evidence_quote":"Supplies the AllTempo tempo-estimation setup and the time-stretch robustness augmentation used when probing tempo."},{"cited_title":"MARBLE: Music Audio Representation Benchmark for Universal Evaluation,","cited_arxiv_id":null,"evidence_quote":"MARBLE benchmark provides the key-estimation probing protocol and Giantsteps evaluation setup."}],"review_version":1}