Pith. sign in

REVIEW 5 major objections 6 minor 81 references

MixAssist: An Audio-Language Dataset for Co-Creative AI Assistance in Music Mixing

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces MixAssist, a 431-turn audio-grounded dataset, and shows fine-tuned Qwen-Audio-Instruct-7B gives mixing advice that human producers preferred 40% of the time over expert responses.

desk verdict A genuinely new audio-grounded conversational mixing dataset, worth publishing even though the 'significantly outperforming' claim is statistically unsupported and the audio-alignment pipeline is under-validated. read the letter →

arxiv 2507.06329 v1 pith:OFCKOGJE submitted 2025-07-08 cs.SD cs.AIeess.AS

Michael Clemens , Ana Marasović This is my paper
classification cs.SDcs.AIeess.AS
keywords musicmixingaudio-languagedatasetco-creativeAIinstructionaldialogueexpert-amateurcollaborationmodelsLLM-as-a-judgefine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces MixAssist, a dataset of 431 audio-grounded conversational turns extracted from seven live mixing sessions in which expert and amateur producers worked together on real multitrack songs. The authors' aim is to give audio-language models a training and evaluation resource for a task they call music-mixing response generation: given the conversation so far, the amateur's latest utterance, and the audio segment just played, produce advice that is contextually relevant, technically correct, and pedagogically helpful. Fine-tuning Qwen-Audio-Instruct-7B on MixAssist outperforms LTU and MU-LLaMA in LLM-as-a-judge rankings, and in a human preference study its responses beat the original expert responses 40% to 33%. A real-time user study confirms the assistant is conversational and can suggest novel ideas, while exposing limited audio-analysis capability. If the claims hold, MixAssist supplies both a benchmark and a method for building assistants that teach mixing rather than automate it.

What carries the argument

The central object is the MixAssist instance: an input that stacks a generated summary of earlier session context, the current sub-conversation's dialogue history ending with an amateur utterance, and the associated music-only audio segment, with the next expert utterance as the target. The construction pipeline that creates these instances carries the argument: Whisper transcription, manual cleaning and speaker splitting, audio segmentation aligned to the audio played before each amateur's question, topic segmentation into sub-conversations, and the binary has_content filter that keeps only expert turns with substantive, actionable guidance. Audio grounding is what distinguishes this resource from prior caption, tag, or single-turn QA datasets, and the held-out-session split (two complete sessions reserved for testing) is what lets the authors claim generalization to unseen producer pairs and genres.

What would settle it

Have two independent annotators re-apply the has_content label and re-align audio segments to dialogue on a random sample of the seven sessions; if their agreement on has_content is low or a large share of the 19.44-second music-only segments do not match the audio actually played when the amateur asked, the supervision signal is too noisy to support the reported preference results.

Watch

Extended reading notes

Core claim

The central claim is that the missing ingredient for co-creative mixing assistants is situated instructional dialogue grounded in audio, and that MixAssist provides exactly that ingredient. Each of its 431 instances pairs a music-only audio segment (average 19.44 seconds, voices removed) with a conversational context and an expert target response that was manually filtered to contain substantive, actionable guidance. On this data, LoRA fine-tuning of Qwen-Audio-Instruct-7B produces responses that the o3-mini judge ranks first in 50.4% of 250 samples, ahead of LTU (44.4%) and MU-LLaMA (5.2%); human music producers prefer the generated responses to the original expert responses in 40% of 100 pairwise comparisons, versus 33% for the human expert, with 12% both good and 15% both bad. The paper also claims the improvement is caused by the dataset: fine-tuning raises correct actionable guidance from 14% to 35% of cases compared with the base model. It does not claim to have solved audio understanding; instead, it presents the model's limited ability to analyze the uploaded track as a diagnosed limitation that MixAssist is designed to benchmark.

Load-bearing premise

The load-bearing premise is that 431 expert turns, kept or discarded by one author's subjective judgment of whether a turn contains substantive guidance, drawn from seven sessions and twelve producers, are representative and correctly aligned to the audio segments (average 19.44 seconds) so the measured model preferences reflect real mixing instruction rather than annotation noise.

Editorial extensions

If this is right

  • MixAssist gives the field a public benchmark for the new task of music-mixing response generation, with train/dev/test splits that hold out entire sessions so models are evaluated on unseen producer pairs and genres.
  • A fine-tuned 7-billion-parameter audio-language model can produce advice that human producers prefer over the original human expert responses 40% to 33%, making conversational mixing assistance a realistic target for models of this size.
  • Fine-tuning on MixAssist, rather than scale alone, drives the measured gain in audio-grounded guidance: correct actionable advice rises from 14% (base Qwen) to 35% (fine-tuned) in the authors' manual analysis.
  • The release includes raw session recordings, so the resource also supports end-to-end conversational speech recognition and fine-grained interaction-dynamics studies, not only ALM instruction tuning.
  • The real-time interaction results imply that any deployed co-creative mixing assistant needs substantially better audio understanding, and MixAssist provides the benchmark on which that deficiency can be measured and addressed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's reported results, the 40% versus 33% preference gap may be partly an artifact of the evaluation prompt, which explicitly rewards structured, detailed, on-topic answers; a judge or user study that weights conversational naturalness and implicit-context handling could plausibly favor the human expert, so the gap should be read as task-and-criteria dependent.
  • If audio grounding is the true bottleneck, combining MixAssist with the released MixParams parameter annotations offers a direct, testable next step: train a model to condition its advice on both the dialogue and the actual DAW parameter state, and check whether advice quality improves beyond what the 19.44-second audio segments alone support.
  • The topic imbalance (drums 40.4%, overall mix 25.3%, keys 4.2%) means fine-tuned models are likely more reliable on drums and overall-mix questions than on keys or bass; benchmark users should expect and report topic-conditioned performance rather than a single aggregate score.
  • A testable extension suggested by the dataset statistics is to verify the reported learning effect directly: the analysis shows amateurs increase technical-term usage by 42.31% from the first to last third of sessions, so a model that conditions on interaction time might be able to adapt its advice to the amateur's growing vocabulary.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces MixAssist, a dataset of 431 audio-grounded expert-amateur conversational turns collected from seven collaborative music-mixing sessions, together with MixParams, a complementary dataset of DAW parameter settings. The authors fine-tune three audio-language models (Qwen-Audio-Instruct-7B, LTU, and MU-LLaMA) on MixAssist using LoRA, then evaluate them through LLM-as-a-judge ranking, a human preference study, and an in-situ user study, concluding that the Qwen-based model generates promising co-creative mixing advice.

Significance. If the dataset construction holds up, MixAssist is a valuable contribution: it is, to my knowledge, the first publicly released audio-grounded, multi-turn instructional dialogue dataset for music mixing, and the authors are to be credited for planning to release raw session recordings and for documenting the curation pipeline in detail. The three-stage evaluation design (automated judge, human preference, real-time interaction) is thorough in conception, and the paper candidly reports the model's residual limitations in audio understanding. The parameter dataset MixParams is a useful complement with potential for future work linking conversational guidance to technical settings. However, the significance of the contribution hinges on validation of the audio-segment alignment and the content filtering, and on statistical support for the headline model-comparison claim; these issues are the main reasons for requesting a major revision.

major comments (5)
  1. [Abstract and §4.3, Tables 2 and 3] The claim that Qwen 'significantly outperforms' other tested models is not statistically justified: Table 2 reports 50.4% vs. 44.4% top-1 ranks over 250 judge decisions with no significance test or confidence interval, and Table 3 reports 40% vs. 33% over N=100 human preference judgments with no uncertainty quantification; the paper should provide a significance test (or effect-size estimate) or replace the claim with a more measured one.
  2. [§3.2 and Appendix G.1] The audio-segment alignment that grounds every instance is extracted by an undocumented procedure ('capturing the audio played back from the DAW before a conversation turn initiated') and is explicitly acknowledged to have 'inherent subjectivity'; with no inter-annotator agreement, no spot-check validation, and no sensitivity analysis, systematic alignment errors would propagate into both the training signal and the evaluation prompts, so a validation protocol (e.g., a random sample checked by independent annotators) is needed before the audio-grounding claim can be taken at face value.
  3. [§3.2 and Appendix G.1] The binary has_content tag, assigned by a single author, is used both to select target expert responses and to filter the conversational history given to the model, yet no inter-annotator agreement, rubric details, or consistency analysis is reported; because this tag determines the entire supervision signal and shapes the test prompts, its reliability is load-bearing and should be quantified.
  4. [§4.2, Appendix E.1, Table 3] The human preference study is internally inconsistent: §4.2 states that 10 prompts were randomly selected from the test set, while Appendix E.1 states 'We randomly selected 100 prompts from the test set' and reports 100 comparisons (10 participants x 10 rounds); the paper must correct this discrepancy and clarify how many distinct prompts were judged and how many judgments each participant made.
  5. [Appendix D.2 and Table 5] The LLM-as-a-judge validation is too thin to support the headline: the judge-validation step uses only 25 samples with a panel that includes one of the authors, and the main 250-sample ranking is reported only for o3-mini; across the other judges in Table 5 the advantage of Qwen over LTU shrinks (e.g., 38.0% vs. 34.8% for llama3.1:8b), so the claim of consistent 'significant' superiority needs formal comparison across judges and a report of randomization/order effects.
minor comments (6)
  1. [Table 1 vs. Table 10] The topic percentages for 'Overall mix' differ between Table 1 (25.3%) and Table 10 (21.2%), and 'Keys' differs as well (4.2% vs. 1.6%); the text should state whether these are computed over different instance sets (full dataset vs. the 250 evaluation samples) and define the categories consistently.
  2. [Appendix D.2] The phrase 'Author 1 of this paper also ranked these responses' should specify whether this rating was included in the aggregated Table 7 counts and whether any rater-familiarity bias was considered.
  3. [Availability] The Availability section states the dataset 'will be made publicly available upon publication'; please confirm the intended license and provide the Hugging Face, Zenodo, and GitHub links at the time of final submission.
  4. [Figure 1] Figure 1's caption describes 'low shelf around 150Hz' but the figure itself is not visible in the text; ensure the figure is included in the camera-ready version.
  5. [§4.1] The LoRA hyperparameters (rank, alpha, target modules, learning rate) are not reported; provide them in an appendix for reproducibility.
  6. [Appendix G.5] The example instances include a generated summary marked '(Summary)'; clarify whether the test-set prompts also employ the gpt-4o-mini-generated summaries or whether they use the original ungrounded context, since this affects the ecological validity of the evaluation.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: the dataset construction, fine-tuning, and held-out evaluation form a standard supervised pipeline, and the flagged concerns are data-quality and statistical-validity issues rather than self-referential derivations.

full rationale

MIXASSIST is a new dataset, not a derivation from first principles. The paper's central empirical claim is that fine-tuning Qwen-Audio-Instruct-7B on the training split yields helpful mixing advice, evaluated on a test split constructed from held-out sessions. This is conventional supervised machine learning: the model is not fitted to the test labels, and the evaluation does not reduce by construction to the training objective. The has_content filter is a curation step applied before the train/dev/test split; it selects which expert utterances become targets, but it does not encode the later preference outcomes. The LLM-as-a-judge protocol was validated against human rankings, but that is methodological calibration, not a circular reduction. The human preference study compares generated responses with ground-truth expert responses on held-out prompts; the result (40% vs 33%) is an empirical observation, however preliminary. The paper's own admission that audio alignment 'has inherent subjectivity' (Appendix G.1) points to a data-quality limitation, not a logical loop. The 'significantly outperforming' wording is unsupported by significance testing, but an unsupported statistical claim is a correctness risk, not circularity. There is no load-bearing self-citation chain, no imported uniqueness theorem, and no fitted parameter renamed as a prediction. Accordingly, the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim depends on data quality assumptions and a subjective curation tag, not on fitted numerical parameters or new theoretical entities. No free parameters or invented entities are introduced.

assumptions (3)
  • domain assumption Expert-amateur think-aloud mixing sessions elicit natural, pedagogically useful dialogue.
    The entire dataset's usefulness depends on this elicitation method producing instruction that is representative of real co-creative mixing conversations (Section 3.2, Appendix G.2).
  • domain assumption Audio segments of average 19.44 seconds, extracted and aligned manually, are sufficient audio grounding for the response generation task.
    If alignment is off or segments are too short, the 'audio-grounded' supervision for models is weakened (Section 3.2, Appendix G.1).
  • ad hoc to paper The single-author has_content filter separates substantive expert instruction from filler, and filtering input history to substantive turns improves learning.
    This binary label is a paper-specific subjective curation step with no external benchmark, yet it determines both training targets and input history (Section 3.2, Appendix G.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of MixAssist: An Audio-Language Dataset for Co-Creative AI Assistance in Music Mixing." pith.science (2026). https://pith.science/paper/OFCKOGJE

@misc{pith2026250706329,
  author       = {Pith},
  title        = {Pith review of: MixAssist: An Audio-Language Dataset for Co-Creative AI Assistance in Music Mixing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OFCKOGJE}},
  note         = {Machine review of arXiv:2507.06329}
}
read the original abstract

While AI presents significant potential for enhancing music mixing and mastering workflows, current research predominantly emphasizes end-to-end automation or generation, often overlooking the collaborative and instructional dimensions vital for co-creative processes. This gap leaves artists, particularly amateurs seeking to develop expertise, underserved. To bridge this, we introduce MixAssist, a novel audio-language dataset capturing the situated, multi-turn dialogue between expert and amateur music producers during collaborative mixing sessions. Comprising 431 audio-grounded conversational turns derived from 7 in-depth sessions involving 12 producers, MixAssist provides a unique resource for training and evaluating audio-language models that can comprehend and respond to the complexities of real-world music production dialogues. Our evaluations, including automated LLM-as-a-judge assessments and human expert comparisons, demonstrate that fine-tuning models such as Qwen-Audio on MixAssist can yield promising results, with Qwen significantly outperforming other tested models in generating helpful, contextually relevant mixing advice. By focusing on co-creative instruction grounded in audio context, MixAssist enables the development of intelligent AI assistants designed to support and augment the creative process in music mixing.

Figures

Figures reproduced from arXiv: 2507.06329 by the authors.

Figure 1
Figure 1. An example of the audio-grounded, multi-turn instructional dialogue captured [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the MIXASSIST dataset construction pipeline. Raw data from co￾creative mixing sessions between expert (E) and amateur (A) producers is first transcribed. These transcripts are then processed to split utterances by speaker and align them with the corresponding audio segments discussed during the session. Aligned turns are grouped into focused sub-conversations based on the mixing topic (e.g., keys, drums,… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

81 extracted references · 40 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Musiclm: Generating music from text

    Andrea Agostinelli, Timo I Denk, Zal \'a n Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, et al. Musiclm: Generating music from text. arXiv preprint arXiv:2301.11325, 2023

    arXiv 2023

  3. [3]

    Homogenization effects of large language models on human creative ideation

    Barrett R Anderson, Jash Hemant Shah, and Max Kreminski. Homogenization effects of large language models on human creative ideation. In Proceedings of the 16th conference on creativity & cognition, pp.\ 413--425, 2024

    work page 2024

  4. [4]

    METEOR : An automatic metric for MT evaluation with improved correlation with human judgments

    Satanjeev Banerjee and Alon Lavie. METEOR : An automatic metric for MT evaluation with improved correlation with human judgments. In Jade Goldstein, Alon Lavie, Chin-Yew Lin, and Clare Voss (eds.), Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization , pp.\ 65--72, Ann Arbor, Michigan,...

    work page 2005

  5. [5]

    Toward good practice in thematic analysis: Avoiding common problems and be (com) ing a knowing researcher

    Virginia Braun and Victoria Clarke. Toward good practice in thematic analysis: Avoiding common problems and be (com) ing a knowing researcher. International journal of transgender health, 24 0 (1): 0 1--6, 2023

    work page 2023

  6. [6]

    Explainable ai and music

    Nick Bryan-Kinns, Berker Banar, Corey Ford, Courtney N Reed, Yixiao Zhang, and Jack Armitage. Explainable ai and music. In Artificial Intelligence for Art Creation and Understanding, pp.\ 1--29. CRC Press, 2024

    work page 2024

  7. [7]

    Can ai writing be salvaged? mitigating idiosyncrasies and improving human-ai alignment in the writing process through edits

    Tuhin Chakrabarty, Philippe Laban, and Chien-Sheng Wu. Can ai writing be salvaged? mitigating idiosyncrasies and improving human-ai alignment in the writing process through edits. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pp.\ 1--33, 2025

    work page 2025

  8. [8]

    Text2fx: Harnessing clap embeddings for text-guided audio effects

    Annie Chu, Patrick O’Reilly, Julia Barnett, and Bryan Pardo. Text2fx: Harnessing clap embeddings for text-guided audio effects. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 1--5. IEEE, 2025

    work page 2025

Show all 81 references
  1. [9]

    Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models

    Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07919, 2023

    2023 arXiv

  2. [10]

    Towards responsible ai music: an investigation of trustworthy features for creative systems

    Jacopo de Berardinis, Lorenzo Porcaro, Albert Mero \ n o-Pe \ n uela, Angelo Cangelosi, and Tess Buckley. Towards responsible ai music: an investigation of trustworthy features for creative systems. arXiv preprint arXiv:2503.18814, 2025

    2025 arXiv

  3. [11]

    The mix evaluation dataset

    Brecht De Man and Joshua D Reiss. The mix evaluation dataset. In 20th Int. Conf. on Digital Audio Effects (DAFx-17), 2017

    2017

  4. [12]

    On the development and practice of ai technology for contemporary popular music production

    Emmanuel Deruty, Maarten Grachten, Stefan Lattner, Javier Nistal, and Cyran Aouameur. On the development and practice of ai technology for contemporary popular music production. Transactions of the International Society for Music Information Retrieval, 5 0 (1), 2022

    2022

  5. [13]

    Pengi: An audio language model for audio tasks

    Soham Deshmukh, Benjamin Elizalde, Rita Singh, and Huaming Wang. Pengi: An audio language model for audio tasks. Advances in Neural Information Processing Systems, 36: 0 18090--18108, 2023

    2023

  6. [14]

    Adiff: Explaining audio difference using natural language

    Soham Deshmukh, Shuo Han, Rita Singh, and Bhiksha Raj. Adiff: Explaining audio difference using natural language. arXiv preprint arXiv:2502.04476, 2025

    2025 arXiv

  7. [15]

    Jukebox: A generative model for music

    Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, and Ilya Sutskever. Jukebox: A generative model for music. arXiv preprint arXiv:2005.00341, 2020

    2005 arXiv

  8. [16]

    Clotho: An audio captioning dataset

    Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen. Clotho: An audio captioning dataset. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 736--740. IEEE, 2020

    2020

  9. [17]

    Clap learning audio concepts from natural language supervision

    Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. Clap learning audio concepts from natural language supervision. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 1--5. IEEE, 2023

    2023

  10. [18]

    The collaborative music producer: a comparative study of integrated and separated mixing methods

    Nordstr \"o m Emil. The collaborative music producer: a comparative study of integrated and separated mixing methods. 2024

    2024

  11. [19]

    Redefining the user in human-generative ai collaboration: Insights from music composition

    Tron Gianet Eric, Luigi Di Caro, Amon Rapp, et al. Redefining the user in human-generative ai collaboration: Insights from music composition. In CEUR Workshop Proceedings, volume 3685, pp.\ 1--6. CEUR-WS, 2024

    2024

  12. [20]

    Negotiating the swamp: the opportunity and challenge of reflexivity in research practice

    Linda Finlay. Negotiating the swamp: the opportunity and challenge of reflexivity in research practice. Qualitative research, 2 0 (2): 0 209--230, 2002

    2002

  13. [21]

    Fsd50k: an open dataset of human-labeled sound events

    Eduardo Fonseca, Xavier Favory, Jordi Pons, Frederic Font, and Xavier Serra. Fsd50k: an open dataset of human-labeled sound events. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30: 0 829--852, 2021

    2021

  14. [22]

    Exploring the collaborative co-creation process with ai: A case study in novice music production

    Yue Fu, Michele Newman, Lewis Going, Qiuzi Feng, and Jin Ha Lee. Exploring the collaborative co-creation process with ai: A case study in novice music production. arXiv preprint arXiv:2501.15276, 2025

    2025 arXiv

  15. [23]

    The ethics of advanced ai assistants

    Iason Gabriel, Arianna Manzini, Geoff Keeling, Lisa Anne Hendricks, Verena Rieser, Hasan Iqbal, Nenad Toma s ev, Ira Ktena, Zachary Kenton, Mikel Rodriguez, et al. The ethics of advanced ai assistants. arXiv preprint arXiv:2404.16244, 2024

    2024 arXiv

  16. [24]

    Llark: A multimodal instruction-following language model for music

    Josh Gardner, Simon Durand, Daniel Stoller, and Rachel M Bittner. Llark: A multimodal instruction-following language model for music. arXiv preprint arXiv:2310.07160, 2023

    2023 arXiv

  17. [25]

    Audio set: An ontology and human-labeled dataset for audio events

    Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE international conference on acoustics, speech and signal processing (...

    2017

  18. [26]

    Audio flamingo 2: An audio-language model with long-audio understanding and expert reasoning abilities

    Sreyan Ghosh, Zhifeng Kong, Sonal Kumar, S Sakshi, Jaehyeon Kim, Wei Ping, Rafael Valle, Dinesh Manocha, and Bryan Catanzaro. Audio flamingo 2: An audio-language model with long-audio understanding and expert reasoning abilities. arXiv preprint arXiv:2503.03983, 2025

    2025 arXiv

  19. [27]

    Ast: Audio spectrogram transformer

    Yuan Gong, Yu-An Chung, and James Glass. Ast: Audio spectrogram transformer. arXiv preprint arXiv:2104.01778, 2021

    2021 arXiv

  20. [28]

    Listen, think, and understand

    Yuan Gong, Hongyin Luo, Alexander H Liu, Leonid Karlinsky, and James Glass. Listen, think, and understand. arXiv preprint arXiv:2305.10790, 2023

    2023 arXiv

  21. [29]

    A survey on llm-as-a-judge

    Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on llm-as-a-judge. arXiv preprint arXiv:2411.15594, 2024

    2024 arXiv

  22. [30]

    Computer support for collaborative data analysis: augmenting paper affinity diagrams

    Gunnar Harboe, Jonas Minke, Ioana Ilea, and Elaine M Huang. Computer support for collaborative data analysis: augmenting paper affinity diagrams. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, pp.\ 1179--1182, 2012

    2012

  23. [31]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1 0 (2): 0 3, 2022

    2022

  24. [32]

    Evaluating creativity in computational co-creative systems

    Pegah Karimi, Kazjon Grace, Mary Lou Maher, and Nicholas Davis. Evaluating creativity in computational co-creative systems. arXiv preprint arXiv:1807.09886, 2018

    2018 arXiv

  25. [33]

    Audiocaps: Generating captions for audios in the wild

    Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, and Gunhee Kim. Audiocaps: Generating captions for audios in the wild. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (...

    2019

  26. [34]

    Applications, challenges, and future directions of human-in-the-loop learning

    Sushant Kumar, Sumit Datta, Vishakha Singh, Deepanwita Datta, Sanjay Kumar Singh, and Ritesh Sharma. Applications, challenges, and future directions of human-in-the-loop learning. IEEE Access, 2024

    2024

  27. [35]

    Evaluation of algorithms using games: The case of music tagging

    Edith Law, Kris West, Michael I Mandel, Mert Bay, and J Stephen Downie. Evaluation of algorithms using games: The case of music tagging. In ISMIR, pp.\ 387--392. Citeseer, 2009

    2009

  28. [36]

    The pro-am revolution: How enthusiasts are changing our society and economy, 2004

    Charles Leadbeater and Paul Miller. The pro-am revolution: How enthusiasts are changing our society and economy, 2004

    2004

  29. [37]

    Grafx: an open-source library for audio processing graphs in pytorch

    Sungho Lee, Marco Mart \' nez-Ram \' rez, Wei-Hsiang Liao, Stefan Uhlich, Giorgio Fabbro, Kyogu Lee, and Yuki Mitsufuji. Grafx: an open-source library for audio processing graphs in pytorch. arXiv preprint arXiv:2408.03204, 2024

    2024 arXiv

  30. [38]

    Llms-as-judges: a comprehensive survey on llm-based evaluation methods

    Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. Llms-as-judges: a comprehensive survey on llm-based evaluation methods. arXiv preprint arXiv:2412.05579, 2024 a

    2024 arXiv

  31. [39]

    Mert: Acoustic music understanding model with large-scale self-supervised training

    Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghao Xiao, Chenghua Lin, Anton Ragni, Emmanouil Benetos, et al. Mert: Acoustic music understanding model with large-scale self-supervised training. arXiv preprint arXiv:2306.00107, 2023

    2023 arXiv

  32. [40]

    From audio deepfake detection to ai-generated music detection--a pathway and overview

    Yupei Li, Manuel Milling, Lucia Specia, and Bj \"o rn W Schuller. From audio deepfake detection to ai-generated music detection--a pathway and overview. arXiv preprint arXiv:2412.00571, 2024 b

    2024 arXiv

  33. [41]

    ROUGE : A package for automatic evaluation of summaries

    Chin-Yew Lin. ROUGE : A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp.\ 74--81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https://aclanthology.org/W04-1013/

    2004

  34. [42]

    Clotho-aqa: A crowdsourced dataset for audio question answering

    Samuel Lipping, Parthasaarathy Sudarsanam, Konstantinos Drossos, and Tuomas Virtanen. Clotho-aqa: A crowdsourced dataset for audio question answering. In 2022 30th European Signal Processing Conference (EUSIPCO), pp.\ 1140--1144. IEEE, 2022

    2022

  35. [43]

    Music understanding llama: Advancing text-to-music generation with question answering and captioning

    Shansong Liu, Atin Sakkeer Hussain, Chenshuo Sun, and Ying Shan. Music understanding llama: Advancing text-to-music generation with question answering and captioning. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 28...

    2024

  36. [44]

    Ddsp-sfx: Acoustically-guided sound effects generation with differentiable digital signal processing

    Yunyi Liu, Craig Jin, and David Gunawan. Ddsp-sfx: Acoustically-guided sound effects generation with differentiable digital signal processing. arXiv preprint arXiv:2309.08060, 2023

    2023 arXiv

  37. [45]

    The role of collaboration, creativity, and embodiment in ai learning experiences

    Duri Long, Aadarsh Padiyath, Anthony Teachey, and Brian Magerko. The role of collaboration, creativity, and embodiment in ai learning experiences. In Proceedings of the 13th Conference on Creativity and Cognition, pp.\ 1--10, 2021

    2021

  38. [46]

    Musecoco: Generating symbolic music from text

    Peiling Lu, Xin Xu, Chenfei Kang, Botao Yu, Chengyi Xing, Xu Tan, and Jiang Bian. Musecoco: Generating symbolic music from text. arXiv preprint arXiv:2306.00110, 2023

    2023 arXiv

  39. [47]

    Getmusic: Generating any music tracks with a unified representation and diffusion framework

    Ang Lv, Xu Tan, Peiling Lu, Wei Ye, Shikun Zhang, Jiang Bian, and Rui Yan. Getmusic: Generating any music tracks with a unified representation and diffusion framework. arXiv preprint arXiv:2305.10841, 2023

    2023 arXiv

  40. [48]

    Audio-cot: Exploring chain-of-thought reasoning in large audio language model

    Ziyang Ma, Zhuo Chen, Yuping Wang, Eng Siong Chng, and Xie Chen. Audio-cot: Exploring chain-of-thought reasoning in large audio language model. arXiv preprint arXiv:2501.07246, 2025

    2025 arXiv

  41. [49]

    A deep learning approach to intelligent drum mixing with the wave-u-net

    M Martinez Ramirez, Daniel Stoller, and David Moffat. A deep learning approach to intelligent drum mixing with the wave-u-net. Journal of the Audio Engineering Society, 69 0 (3): 0 142, 2021

    2021

  42. [50]

    Design considerations for real-time collaboration with creative artificial intelligence

    Jon McCormack, Patrick Hutchings, Toby Gifford, Matthew Yee-King, Maria Teresa Llano, and Mark D’inverno. Design considerations for real-time collaboration with creative artificial intelligence. Organised Sound, 25 0 (1): 0 41--52, 2020

    2020

  43. [51]

    The artist in the machine: The world of AI-powered creativity

    Arthur I Miller. The artist in the machine: The world of AI-powered creativity. Mit Press, 2019

    2019

  44. [52]

    An automated approach to the application of reverberation

    Dave Moffat and Mark Sandler. An automated approach to the application of reverberation. In Audio Engineering Society Convention 147. Audio Engineering Society, 2019 a

    2019

  45. [53]

    AI Music Mixing Systems

    David Moffat. AI Music Mixing Systems . In Handbook of Artificial Intelligence for Music , pp.\ 345--375. Springer International Publishing, 2021

    2021

  46. [54]

    Approaches in intelligent music production

    David Moffat and Mark B Sandler. Approaches in intelligent music production. Arts & health, 8 0 (4): 0 125, September 2019 b

    2019

  47. [55]

    Meanings of tools, support, and uses for creative design processes

    Kumiyo Nakakoji. Meanings of tools, support, and uses for creative design processes. In International design research symposium, volume 6, pp.\ 156--165, 2006

    2006

  48. [56]

    Collaborative artificial intelligence in music production

    Steven Nicholls, Stuart Cunningham, and Richard Picking. Collaborative artificial intelligence in music production. In Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion, pp.\ 1--4. 2018

    2018

  49. [57]

    Reflexivity and research methodology: A second glance

    Erlinda C Palaganas and Joel C Estacio. Reflexivity and research methodology: A second glance. Philippine Journal of Nursing, 91 0 (1): 0 87--96, 2021

    2021

  50. [58]

    B leu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. B leu: a method for automatic evaluation of machine translation. In Pierre Isabelle, Eugene Charniak, and Dekang Lin (eds.), Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp...

    2002

  51. [59]

    End-to-end speech recognition: A survey

    Rohit Prabhavalkar, Takaaki Hori, Tara N Sainath, Ralf Schl \"u ter, and Shinji Watanabe. End-to-end speech recognition: A survey. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32: 0 325--351, 2023

    2023

  52. [60]

    Robust speech recognition via large-scale weak supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pp.\ 28492--28518. PMLR, 2023

    2023

  53. [61]

    Vygotsky’s creativity options and ideas in 21st-century technology-enhanced learning design

    Kalliopi Rigopouli, Dimitrios Kotsifakos, and Yannis Psaromiligkos. Vygotsky’s creativity options and ideas in 21st-century technology-enhanced learning design. Education Sciences, 15 0 (2): 0 257, 2025

    2025

  54. [62]

    A survey of evaluation metrics used for nlg systems

    Ananya B Sai, Akash Kumar Mohankumar, and Mitesh M Khapra. A survey of evaluation metrics used for nlg systems. ACM Computing Surveys (CSUR), 55 0 (2): 0 1--39, 2022

    2022

  55. [63]

    Knowledge-based programming for music research, volume 13

    John W Schaffer and Deron McGee. Knowledge-based programming for music research, volume 13. AR Editions, Inc., 1997

    1997

  56. [64]

    Creativity support tools

    Ben Shneiderman. Creativity support tools. Communications of the ACM, 45 0 (10): 0 116--120, 2002

    2002

  57. [65]

    Qualitative psychology: A practical guide to research methods

    Jonathan A Smith. Qualitative psychology: A practical guide to research methods. 2024

    2024

  58. [66]

    Towards enhancing ecological validity in user studies: a systematic review of guidelines and implications for qoe research

    Sruti Subramanian, Katrien De Moor, Markus Fiedler, Kamil Koniuch, and Lucjan Janowski. Towards enhancing ecological validity in user studies: a systematic review of guidelines and implications for qoe research. Quality and User Experience, 8 0 (1): 0 6, 2023

    2023

  59. [67]

    Salmonn: Towards generic hearing abilities for large language models

    Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. Salmonn: Towards generic hearing abilities for large language models. arXiv preprint arXiv:2310.13289, 2023

    2023 arXiv

  60. [68]

    Llama: Open and efficient foundation language models, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language...

    2023 arXiv

  61. [69]

    Towards a human-centric design framework for ai assisted music production

    Augoustinos Tsiros and Alessandro Palladini. Towards a human-centric design framework for ai assisted music production. In Proceedings of the International Conference on New Interfaces for Musical Expression, pp.\ 399--404, 2020

    2020

  62. [70]

    The think aloud method: a practical approach to modelling cognitive

    Maarten Van Someren, Yvonne F Barnard, and J Sandberg. The think aloud method: a practical approach to modelling cognitive. London: AcademicPress, 11 0 (6), 1994

    1994

  63. [71]

    Adoption of AI Technology in the Music Mixing Workflow : An Investigation

    Soumya Sai Vanka, Maryam Safi, Jean-Baptiste Rolland, and George Fazekas. Adoption of AI Technology in the Music Mixing Workflow : An Investigation . 2023

    2023

  64. [72]

    Improving llm-as-a-judge inference with the judgment distribution

    Victor Wang, Michael JQ Zhang, and Eunsol Choi. Improving llm-as-a-judge inference with the judgment distribution. arXiv preprint arXiv:2503.03064, 2025

    2025

  65. [73]

    Systematic evaluation of llm-as-a-judge in llm alignment tasks: Explainable metrics and diverse prompt templates

    Hui Wei, Shenghua He, Tian Xia, Andy Wong, Jingyang Lin, and Mei Han. Systematic evaluation of llm-as-a-judge in llm alignment tasks: Explainable metrics and diverse prompt templates. arXiv preprint arXiv:2408.13006, 2024

    2024 arXiv

  66. [74]

    Audio-reasoner: Improving reasoning capability in large audio language models

    Zhifei Xie, Mingbao Lin, Zihang Liu, Pengcheng Wu, Shuicheng Yan, and Chunyan Miao. Audio-reasoner: Improving reasoning capability in large audio language models. arXiv preprint arXiv:2503.02318, 2025

    2025

  67. [75]

    Musicbert: Symbolic music understanding with large-scale pre-training

    Mingliang Zeng, Xu Tan, Rui Wang, Zeqian Ju, Tao Qin, and Tie-Yan Liu. Musicbert: Symbolic music understanding with large-scale pre-training. arXiv preprint arXiv:2106.05630, 2021

    2021 arXiv

  68. [76]

    Bertscore: Evaluating text generation with bert

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019

    1904 arXiv

  69. [77]

    Judging llm-as-a-judge with mt-bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36: 0 46595--46623, 2023

    2023

  70. [78]

    Prosa: Assessing and understanding the prompt sensitivity of llms

    Jingming Zhuo, Songyang Zhang, Xinyu Fang, Haodong Duan, Dahua Lin, and Kai Chen. Prosa: Assessing and understanding the prompt sensitivity of llms. arXiv preprint arXiv:2410.12405, 2024

    2024 arXiv

  71. [79]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  72. [80]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  73. [81]

    We report the percentage of times each model was ranked \#1 by various judges and the average rank (lower is better)

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.