{"id":"9192c554-efe3-4b20-bbe8-395ed7130d34","arxiv_id":"2504.14058","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Calliope is a browser-based system that wraps the MMM transformer model in an interface for multi-track MIDI generation, editing, batch sampling, and DAW streaming.","lead":"Calliope is a web app that lets composers upload, edit, and generate multi-track MIDI music using a transformer model, with bar in-filling, batch generation, and playback streaming to a DAW. It is a practical tool for computer-assisted composition, but it builds on the authors' earlier MMM model and reports no evaluation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MMM generation quality and parameter controllability are cited from prior work, not demonstrated here; the paper's own conclusion calls the system beta with evaluation pending, leaving the central co-creative claim unsupported.","rationale":"The strongest claim is that Calliope is a functional co-creative composition environment, and the reader correctly identifies the load-bearing premise as the unverified quality and controllability of MMM generation. My independent reading confirms this: the manuscript is a system demonstration with no empirical evaluation, no code artifact, and no evidence that the listed parameter controls meaningfully shape the output. The paper itself flags the missing evaluation in the Conclusion by describing the system as beta and future work as including an evaluation study; this is an explicit in-paper limitation that should be weighed, not dismissed. The concern is not an internal inconsistency or a departure from consensus; it is an evidence gap. The descriptive UI claims could all be true, and yet the co-creative workflow could fail if the MMM outputs are incoherent or the controls are ineffective. Because the paper's contribution is precisely the practical packaging of MMM, a stress test on generation quality and controllability is the right next step. That said, I do not think this moves the verdict away from CONDITIONAL. A system-demonstration paper can reasonably be accepted conditionally when the system is publicly released and the core generative model is previously published; rejection would be too strong, and full acceptance is premature without the audit described above. Therefore I recommend no change to the reader's verdict.","tokens_in":4715,"tokens_out":4204,"duration_ms":40213,"concrete_test":"Run an end-to-end controllability audit on the public Calliope deployment (metacreation.net/calliope): load a fixed 8-bar, 4-track seed MIDI; then, holding all other controls fixed, vary one control at a time—Note Density (0, 5, 10), Polyphony Range (1, 3, 6), Note Duration Range (1/32, 1/8, Whole), Instrument Type (piano vs. drums), and Temperature (0.8, 1.0, 1.2)—generating at least 50 outputs per setting. Compute for each condition the corresponding output statistic (note count, maximum simultaneous notes, median note duration, instrument-family histogram, and a style-distance score from Ens and Pasquier 2020b) and test for monotone association (e.g., Spearman rho) between the intended setting and the observed output. If most controls fail to show a significant monotone effect, or if generation fails on more than 10% of runs, the co-creative controllability claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is functional co-creation: users generate, steer, rank, and stream MMM-based multi-track outputs. For that workflow to work, two things must hold: (1) MMM outputs are musically coherent enough for compositional use, and (2) the global and track parameters actually condition the generated content. Neither is evaluated in this manuscript. The section 'Conditioned Music Generation' merely states that generation is achieved with MMM (Ens and Pasquier 2020a) and lists parameters; it provides no output examples, no objective metrics, no listening test, and no user study. The Conclusion explicitly says 'The Calliope system is at the beta phase' and 'future work includes an ongoing evaluation study', which is an in-paper acknowledgment that evaluation is missing. The only stated validation is that the system 'is being used by a variety of composers' and that batches of 5-1000 samples complete in 3 seconds to 10 minutes; both are anecdotal and unreproducible from the text. This is an evidence gap, not an internal inconsistency: the descriptive UI claims could all be true while the co-creative system still fails if the underlying MMM generation is weak or if parameters like Note Density, Polyphony Range, or Instrument Type do not measurably affect output. Because the paper's value proposition is precisely the usable packaging of MMM, this assumption is load-bearing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Calliope, a web-based system for symbolic multi-track music composition built around the Multi-Track Music Machine (MMM), a Transformer model. The system supports MIDI upload and piano-roll visualization, manual editing, bar selection for partial or complete generation, global and track-level generation parameters (temperature, polyphony, note density, instrument type, etc.), batch generation, ranking of outputs against a selected reference, and MIDI streaming to external DAWs. The paper describes these features and the accompanying co-creative workflow, states that the system has been publicly released, and reports anecdotal timing figures for batch generation. The paper does not include any formal evaluation, user study, or quantitative analysis of generation quality or parameter controllability.","tokens_in":4942,"tokens_out":2730,"duration_ms":25395,"significance":"If the system works as described, Calliope would be a practically useful integration of a state-of-the-art symbolic multi-track generative model into an interactive, browser-based environment, offering composers a concrete tool with unusually fine-grained global and local control parameters. The paper is concrete about the system architecture, the parameter ranges, and the workflow, and it credits the external MMM and ranking algorithms appropriately rather than overclaiming novel algorithmic contributions. The main significance is therefore as a system/demonstration contribution. However, its central value proposition—usable co-creation via controllable MMM generation—is not empirically supported in the manuscript: there are no listening tests, no objective metrics on generation quality or parameter steering, and no usability data. The significance is conditional on future validation that the paper itself acknowledges is pending.","major_comments":[{"comment":"The central claim that Calliope enables effective co-creation by letting users steer MMM generation is load-bearing but unsupported. The manuscript states that generation is achieved with MMM (Ens and Pasquier 2020a) and lists parameters such as Note Density, Polyphony Range, and Instrument Type, but it provides no evidence that the generated outputs are musically coherent or that these parameters meaningfully condition the output. The conclusion explicitly says 'The Calliope system is at the beta phase' and lists 'an ongoing evaluation study' as future work, confirming the absence of evaluation. For a claim of a functional co-creative environment, this is a gap that must be filled with at least a minimal user study or objective controllability analysis (e.g., measuring distributional shifts when parameters are changed).","section":"Conditioned Music Generation / Conclusion"},{"comment":"The performance claim in the batch-generation section—'These generations can be done within 3 seconds to 10 minutes on an average computer depending on the total note density of the music input'—is not reproducible as written. It does not specify the hardware, the software versions, the MIDI file sizes, the number of tracks/bars, or the measurement methodology, nor does it define 'average computer' or how 'total note density' was varied. Please provide a controlled benchmark with a concrete environment description, or remove the quantitative claim.","section":"Batch Generation of Music Outputs"},{"comment":"The specification of the generative parameters is underspecified in places, which undermines reproducibility of the core steering mechanism. For instance, the 'Percentage' parameter says 'for tracks per step and bars per step each 4, and percentage at 25, the model will process only 4 out of 16 bars at each generation step,' yet the defaults for tracks per step and bars per step are elsewhere given as 4 and 2, and the computation of '4 out of 16 bars' is not explained (is it 25% of 4×4 bars? 25% of 16 bars?). Similarly, Note Duration Range lists 'Any' as one of the values without saying what model token it maps to, and Note Density's 'zero means random' behavior is described parenthetically. Since controllability claims depend on these parameters, please give precise, unambiguous semantics for each parameter.","section":"Global Parameters and Track Parameters"}],"minor_comments":[{"comment":"The phrase 'node density' should be 'note density' in the first sentence of this section.","section":"Co-Creative Interaction"},{"comment":"In the description of Temperature, the text reads 'it corresponds to the the temperature in the sampling' with a duplicated 'the'.","section":"Global Parameters"},{"comment":"The phrase 'measured of technology acceptance' should be 'measures of technology acceptance'.","section":"Conclusion"},{"comment":"The text refers to 'The connection from steps 9 to 3' but the figure steps are not labelled in the text; please number the steps in Figure 5 or refer to them by descriptive names so the reader can follow the workflow.","section":"Co-Creative Interaction / Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a system description for a tool that appears genuinely useful and is publicly released. The main concern is the gap between the claimed co-creative contribution and the absence of any evaluation; this is a standard expectation for a cs.HC venue. The performance claim and parameter ambiguities are fixable with additional detail. If the authors can provide even a small-scale user study or objective controllability measurements, the paper would be considerably stronger. The current form may be more appropriate for a workshop demo track than a full journal article, but I think major revision rather than rejection is the right call, since the system itself is described concretely and the missing evaluation is explicitly acknowledged as future work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nCalliope is a straightforward system description: a web app that wraps the authors' MMM transformer in a piano-roll editor with bar in-filling, batch generation, ranking, and MIDI streaming to a DAW. That integration is real, and the paper is clear about how the pieces connect. The parameter list for global and track controls is concrete and reproducible. Credit where due: the authors know this is a demo, not a scientific study — they say so in the conclusion, noting the beta status and the ongoing evaluation.\n\nThe soft spots are the ones you'd expect. No user study, no objective metrics, no output examples. The stress-test note is right: the whole co-creative premise rests on MMM's generation quality and whether the attributes like Note Density or Instrument Type actually change the output. That's cited from prior work, and this paper adds no evidence. The only performance figure — 3 seconds to 10 minutes for 5–1000 samples on \"an average computer\" — is basically anecdotal. And the \"unique opportunity\" claim for streaming to a DAW ignores Magenta Studio already living inside Ableton Live. None of this makes the paper bad; it makes it a system demo, not a research result.\n\nWho gets value? Someone evaluating web-based tools for symbolic generation, or a practitioner looking for a browser-based MMM frontend. Not someone looking for evidence about co-creative workflows — there is none here.\n\nI'd send it to a demo or workshop track for peer review, where it can be judged on clarity and usefulness of the system itself. If this is for a full research venue, I'd ask for at least a small user study or some generated examples first. Mixed verdict, leaning toward 'worth reviewing as a demo, not as a full paper.'","headline":"A clear, honest system demo that packages the authors' MMM model into a web-based multi-track workflow; no evaluation, so read it as a tool description, not a scientific result.","tokens_in":5491,"tokens_out":2751,"would_cite":false,"duration_ms":24975,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Calliope presents a browser-based co-creative environment in which a transformer trained on half a million MIDI files handles multi-track generation through bar in-filling, attribute conditioning, batch ranking, and DAW streaming.","keywords":["computer-assisted composition","multi-track music generation","symbolic music","MIDI","interactive music systems","transformer model","co-creative systems","bar in-filling"],"falsifier":"Generate the same seed file under extreme settings of a control that is claimed to matter—for instance, note density 0 versus 10, or temperature 0.8 versus 1.2—and compare distributions of generated note counts or pitch content. If repeated runs show no statistically distinguishable difference, the controllability premise fails; conversely, a clean separation would confirm it.","tokens_in":4502,"feed_emoji":"🎹","tokens_out":4346,"duration_ms":36782,"temperature":0.7,"pith_summary":"This paper presents Calliope, a web application that wraps the Multi-Track Music Machine (MMM) transformer—trained on roughly half a million MIDI files—into a practical multi-track composition environment. A user can upload MIDI, inspect and edit tracks in a piano roll, select specific bars to infill or regenerate, and steer generation through global controls such as temperature and polyphony limits plus per-track controls such as instrument type, note density, and note duration. The system supports batch generation, similarity ranking of generated files, MIDI export, and direct MIDI streaming to a Digital Audio Workstation. The paper argues that combining these features creates a co-creative workflow in which composers set a creative context, generate alternatives, listen, and feed outputs back as new seeds.","feed_headline":"Web app steers a 500k-file music model bar by bar","feed_subtitle":"Upload MIDI, infill selected bars, batch-generate options, rank them, and stream results to a DAW.","key_machinery":"The load-bearing object is the Multi-Track Music Machine (MMM), a Transformer model for conditional multi-track music generation trained on approximately 500,000 MIDI files. Its design combines bar-level in-filling with two levels of conditioning: global parameters that shape the whole generation (temperature, polyphony hard limit, preservation percentage, model dimension in bars, tracks per step, bars per step, max steps, tempo) and per-track parameters that shape individual instrument lines (GM instrument type, note density, polyphony range, note duration range). Calliope's interface exposes these controls and adds batch generation, ranking, and playback/export, so the same machinery supports tasks from bar in-filling to complete multi-track generation.","core_discovery":"The central claim is that Calliope constitutes a complete, browser-based co-creative system for symbolic multi-track composition, not just a single generation function. Generation is achieved by MMM, a transformer that operates on multi-track symbolic music; Calliope makes that model controllable and usable through a visual interface. Partial generation (bar in-filling) constrains newly generated bars by surrounding musical information, while full-file generation creates new multi-track content. Global parameters control sampling conservatism (temperature), maximum polyphony, how much existing content is preserved, the model's bar window, and processing step sizes; per-track parameters condition instrument style, note density, polyphony range, and note duration. Batch generation, statistical ranking against a reference file, and MIDI streaming to external DAWs complete the workflow. If the underlying MMM generation quality holds, Calliope therefore delivers end-to-end assisted composition in the symbolic domain.","pith_inferences":["Beyond the paper: the interface's separation between a browser front end and a Python model runtime means the control surface could be adapted to other symbolic generative back ends, not only MMM.","Beyond the paper: bar in-filling conditioned on preceding bars and neighboring tracks makes tasks like harmonization and orchestration special cases of track-subset selection, a connection the paper does not develop.","Beyond the paper: a direct test of the control claim would be to measure whether note-density and polyphony settings produce statistically distinguishable output distributions; the authors report plans for usability studies but not for such a generative-control test."],"forward_implications":["Composers can use MMM for partial and complete generation without writing code: select bars in a piano roll, set parameters, and generate in batches.","Generated multi-track files can be ranked against a chosen reference, giving an informal quantitative check of style-imitation quality.","Outputs can be streamed to a DAW over a MIDI port or exported as MIDI, letting the generative workflow plug into existing production chains.","Because generated outputs can become seed files for new sessions, iterative 'generate, listen, regenerate' co-creative loops are directly supported.","The same parameter set covers a broad range of compositional tasks, including track infilling, new-track addition, and full-piece generation."],"supporting_citations":[{"why":"Supplies MMM, the transformer model that performs all generation in Calliope.","marker":"Ens and Pasquier 2020a"},{"why":"Describes Apollo, the predecessor interactive environment Calliope builds on for browser-based generation.","marker":"Tchemeube, Ens, and Pasquier 2019"},{"why":"Provides the ranking algorithm used to quantify similarity of generated MIDI files against a selected reference.","marker":"Ens and Pasquier 2020b"},{"why":"Establishes the computer-assisted composition tradition Calliope positions itself within.","marker":"Assayag et al. 1999"},{"why":"Describes Magenta Studio, a DAW plugin baseline for symbolic generative composition that Calliope contrasts with its web workflow.","marker":"Roberts et al. 2019"}],"fun_headline_variants":["Calliope: browser-based generative music for multi-track MIDI","Infill bars, batch-generate options, and stream to your DAW","Co-creative composition: control generation per bar and per track","Symbolic music AI: upload MIDI, generate new tracks, export"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central assumption is that the MMM transformer, trained on about half a million MIDI files, generates musically coherent output and that its attribute controls meaningfully condition that output; the paper cites prior work for this and does not evaluate it here.","fun_headline_variants_meta":{"raw":{"variants":["Calliope: browser-based generative music for multi-track MIDI","Infill bars, batch-generate options, and stream to your DAW","Co-creative composition: control generation per bar and per track","Symbolic music AI: upload MIDI, generate new tracks, export"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000418,"raw_usage":{"total_tokens":2134,"prompt_tokens":903,"completion_tokens":1231,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":519,"completion_tokens_details":{"reasoning_tokens":1154}},"tokens_in":519,"tokens_out":1231,"duration_ms":11128,"temperature":1.0,"reasoning_tokens":1154,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:57:03.350099+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate the same seed file under extreme settings of a control that is claimed to matter—for instance, note density 0 versus 10, or temperature 0.8 versus 1.2—and compare distributions of generated note counts or pitch content. If repeated runs show no statistically distinguishable difference, the controllability premise fails; conversely, a clean separation would confirm it.","supporting_citations":[{"cited_title":"B.; Ens, J.; and Pasquier, P","cited_arxiv_id":null,"evidence_quote":"Describes Apollo, the predecessor interactive environment Calliope builds on for browser-based generation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the computer-assisted composition tradition Calliope positions itself within."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Describes Magenta Studio, a DAW plugin baseline for symbolic generative composition that Calliope contrasts with its web workflow."}],"review_version":1}