{"id":"d47853c1-4745-4aab-8663-931ed3957275","arxiv_id":"2608.08349","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Dramarrator introduces object-based audio editing, where characters and scenes are editable objects that propagate edits across all linked speech, sound effects, music, and ambience.","lead":"Dramarrator turns books into audio dramas by treating characters and scenes as editable objects, so one change, like a character's voice, updates every related audio clip at once. Professional producers in a small study reported lower task load with it, and 300 listeners rated the refined output close to professional quality.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Automatic propagation is never validated: object extraction/linking and LLM-proposed edit propagation have no precision/recall evaluation, so the headline efficiency claim rests on an unmeasured mechanism.","rationale":"The paper is coherent and the listener study (N=300) provides real evidence that refined Dramarrator output approaches professional quality; the authors also disclose the main limitations of the small user study. However, the central conceptual contribution is object-based editing, whose defining promise is that high-level edits automatically propagate to all dependent assets. For that promise to hold, the object model must accurately reflect narrative structure and the propagation mechanism must update all relevant assets without missing or spurious changes. The paper provides no direct measurement of either: extraction and linking are unvalidated, and the propagation step is an LLM suggestion mechanism whose precision and recall are unreported. The user study measures self-reported task load and counts of accepted suggestions, not whether propagation was complete and correct; the listener study measures final output quality, not the propagation mechanism itself. This concern is not an appeal to an outside consensus or an internal contradiction; it is an unmeasured prerequisite for the headline claim, and it is addressable with a targeted evaluation. Because the reader already issued a CONDITIONAL verdict, this concern reinforces the need for additional evidence but does not move the verdict. I marked agreement as partial because the reader's weakest_assumption overlaps with extraction/linking accuracy, while this critique additionally emphasizes that propagation is LLM-proposed and therefore needs precision/recall evaluation, not just voice-consistency evidence.","tokens_in":33426,"tokens_out":6578,"duration_ms":63231,"concrete_test":"Construct a gold-standard set from 5–10 book excerpts with manually annotated character/scene mentions and ground-truth asset dependencies (which speech lines, SFX, ambience belong to which object). Run Algorithm 1 lines 2–7 and compute extraction F1 and link-assignment accuracy. Then issue a standardized object edit (e.g., 'make character X nervous') and have independent annotators label each proposed asset update as correct, missing, or spurious, measuring recall of truly dependent assets and precision of suggestions. If recall of dependent assets is below about 0.9 (or link F1 is low), the 'automatically propagates to all dependent assets' claim is not supported by the current evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism is object-based propagation: Algorithm 1 extracts objects (line 2), links assets to them (line 7), and Section 4.1 states that on an object edit 'Dramarrator uses an LLM to identify all dependent assets that require updates and proposes modification suggestions.' This means propagation is not a deterministic link traversal but an LLM judgment about which assets change and how. The paper reports no accuracy evaluation of (a) character/scene extraction from raw book text, (b) asset-to-object link assignment, or (c) the completeness and correctness of the LLM's propagation suggestions. Section 5.2's quantitative supports—2.6 object edits 'propagating' to 25.0 manual edits and the 16.7-to-3.7 hour reduction—depend on this unvalidated chain. If the LLM misses dependent assets, the claim 'automatically propagates to all dependent assets' is false; if it proposes spurious edits, the 9.7x amplification overstates benefit. P3's doubt about single-prompt voice consistency (Section 5.2) is one instance of this broader gap; Section 5.1.3 discloses study limitations but does not supply a correctness measure for the core abstraction. An end-to-end propagation accuracy test is therefore the missing evidence that would make the central claim secure.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Dramarrator, an end-to-end authoring system that adapts books into multi-track audio dramas using object-based audio editing. Characters and scenes are extracted by an LLM into editable objects with attributes such as voice and ambience design; generated speech, SFX, music, and ambience are linked to these objects, and edits to an object are claimed to propagate automatically to all dependent assets. The system is evaluated in three studies: a within-subjects user study with 8 professional producers comparing Dramarrator to their existing tools, a listener study with 300 participants comparing automatically generated, professionally refined, and existing-tool productions, and an exploratory study with 3 novice storytellers. The reported results are that Dramarrator significantly lowers task load, that a single object edit replaces up to 30 manual edits (a 9.7x amplification), that self-reported creation time falls from 16.7 to 3.7 hours, and that refined output approaches the quality of existing-tool productions on several dimensions.","tokens_in":33643,"tokens_out":4241,"duration_ms":38581,"significance":"If the automatic-propagation mechanism works as described, the contribution is significant for creativity support tools: it offers a genuinely semantic editing abstraction over heterogeneous audio assets, grounded in a careful formative analysis of professional practice. The paper has real strengths: the professional user study collects both quantitative ratings and rich qualitative evidence, the listener study is large (N=300), and the authors are transparent about several study limitations. The central weakness is that the load-bearing mechanism—object extraction, asset-to-object linking, and LLM-proposed edit propagation—is never evaluated for correctness or completeness. The headline efficiency claims therefore rest on an unmeasured mechanism, and the causal attribution to object-based editing is weakened by the fixed condition order and by the absence of a condition that isolates object-based editing from the fully automated pipeline. The paper is a promising systems contribution, but the evidence as presented does not yet make the central claim secure.","major_comments":[{"comment":"The claim that edits 'automatically propagate to all dependent assets' rests on an LLM identifying which assets require updates and proposing modifications, but the paper reports no accuracy evaluation of (a) character and scene extraction from raw book text (Algorithm 1, line 2), (b) asset-to-object linking (Algorithm 1, line 7), or (c) the completeness and correctness of the LLM's propagation suggestions (Section 4.1). Consequently, the quantitative supports in Section 5.2—2.6 object edits 'propagating' into 25.0 manual edits, the 9.7x amplification, and the 16.7-to-3.7-hour time reduction—are not yet tied to a measured mechanism. If the LLM misses dependent assets, the claim is unsupported; if it proposes spurious edits, the amplification overstates the benefit. An end-to-end propagation accuracy test, measuring missed and spurious edits against a ground-truth object model on held-out books, is needed to make the central claim secure.","section":"Section 4.1 / Algorithm 1"},{"comment":"The study cannot cleanly attribute the observed task-load reduction to object-based editing. Participants always completed the existing-tools condition first (Section 5.1), time estimates were self-reported (Section 5.1.3), and Dramarrator bundles object-based editing together with a fully automated script-generation, asset-generation, and mixing pipeline. The limitations paragraph acknowledges the ordering and self-report issues but not the attribution confound: there is no condition with object-based editing disabled, and no condition in which the automated pipeline is used without the object abstraction. A follow-up ablation or counterbalanced design is needed to support the causal claim; until then, the abstract's and Section 5.2's causal wording should be softened.","section":"Section 5.1 / 5.1.3"},{"comment":"The pipeline claims to ensure voice consistency by concatenating all of a character's dialogue lines into a single TTS call (Section 4.3), but no objective or perceptual evaluation of cross-scene voice consistency is reported, and participant P3 explicitly doubted that a single AI voice prompt produces consistent results across all lines (Section 5.2). Since consistency is a promised benefit of object propagation (DG2) and a central part of the listener-evaluation argument, the paper should report a consistency evaluation—for example, per-character listener ratings across different scenes, or acoustic similarity measures—along with the accuracy of character and scene extraction from the raw book text.","section":"Section 4.3 / 5.2"}],"minor_comments":[{"comment":"The '9.7x amplification' is presented as if it were directly measured: '2.6 object edits that propagated into 25.0 manual edits.' As described, this appears to be a projection based on counting object-level edits and estimating the manual edits they would replace. Please label the metric as an estimate and explain how the counterfactual manual-edit count was derived.","section":"Section 5.2"},{"comment":"The table reports raw p-values for many Wilcoxon tests without any multiple-comparison correction. Reporting uncorrected values is acceptable, but it should be stated explicitly so readers can calibrate the number of significant findings.","section":"Table 5 / Section 5.2"},{"comment":"The procedure says participants had 'unlimited time to refine' within five business days, while Figure 5 reports 2.9 hours of final mixing and mastering in existing tools. Please clarify how the 3.7-hour Dramarrator total was measured and whether it includes that 2.9-hour post-refinement mixing phase.","section":"Section 5.1.2 / Figure 5"},{"comment":"The listener study randomly chose 6 stories from the 8-participant user study and assigned each listener one condition per story. The assignment of conditions to stories and listeners is not fully described; please specify the design (e.g., Latin square, number of ratings per condition) so the reader can assess balance.","section":"Section 5.3"},{"comment":"The figure caption and labels appear to contain garbled text tokens (e.g., '/gid00048/gid00065/...') in the provided manuscript. If these are rendering artifacts, the published version should use clean, readable labels.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The paper fits UIST's scope and the object-based editing concept is compelling. My main concern is that the automatic-propagation mechanism is never directly evaluated, so the headline efficiency numbers rest on an unmeasured chain. I would want to see either an accuracy/completeness study of extraction, linking, and propagation, or a substantially weakened set of causal claims, before considering this for acceptance. The fixed-order within-subjects design and the self-reported time estimates further limit the strength of the causal conclusions; these should be addressed in revision or explicitly reframed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this paper is worth taking seriously. It is a coherent systems contribution that demonstrates a new editing abstraction for a real problem, backed by a stronger listener study than most papers in this space. The main caveat is that the central mechanism—automatic propagation of edits—is never directly validated, so the quantitative efficiency claims should be read as suggestive rather than conclusive.\n\nWhat is actually new: object-based audio editing, where characters and scenes become objects with attributes, and each audio asset is linked back to the object it belongs to. A high-level edit, such as changing a character's voice, then propagates across all dependent speech, SFX, and ambience. The extension of object-oriented editing abstractions to heterogeneous audio assets is real, and the authors are honest about their predecessors. The pipeline—extracting objects from a book, generating 25 script candidates, scoring them with an LLM judge, then generating and mixing assets—is solid engineering.\n\nThe strongest part of the paper is the listener study (N=300). It shows that creator-refined output from Dramarrator is rated comparably to professional productions on engagement, overall quality, and character/scene consistency. That gives the whole approach external validity.\n\nThe user study with 8 professionals is typical HCI. The authors disclose the fixed condition order and self-reported time estimates, and the reader's conditional verdict is fair. The stress-test note lands: propagation is defined as an LLM identifying dependent assets and proposing suggestions, but there is no accuracy evaluation of object extraction, asset linking, or propagation completeness. If the LLM misses a dependent asset, 'automatically propagates to all dependent assets' overstates what happens. The accept/decline UI mitigates the risk, so this is a gap, not a fatal flaw. The lack of released code or data also limits reproducibility, though that is common for industry-collaboration systems papers.\n\nWho is this for: HCI researchers working on creativity support tools, generative audio, and narrative media. It will not reshape a field, but it is a solid data point and a good starting point for follow-up work.\n\nRecommendation: send it to peer review. The propagation accuracy gap is addressable with a targeted evaluation, and the core idea is worth that effort. I would ask for either a propagation accuracy study or a more careful framing of what the current evidence does and does not support.","headline":"A genuinely useful systems paper: object-based audio editing is a real abstraction with good evidence, but the headline 'automatic propagation' claim rests on an LLM link that the paper never directly measures.","tokens_in":34232,"tokens_out":2625,"would_cite":true,"duration_ms":23353,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Object-based audio editing, where characters and scenes are editable objects linked to their audio assets, lets a single high-level edit replace dozens of manual timeline edits and cuts professional creation time from 16.7 to 3.7 hours.","keywords":["object-based audio editing","audio drama","creativity support tools","human-AI co-creation","generative audio","edit propagation","text-to-speech","user study"],"falsifier":"Run Dramarrator on several books, apply one object edit such as making a specified character's voice raspier, and count how many linked speech, sound effect, music, and ambience clips actually change; if the changed set is smaller than the linked set, propagation is incomplete. Separately, generate a character's lines batched in one text-to-speech call versus line-by-line and have listeners judge whether both sets sound like the same character; if batched and unbatched lines are equally consistent, the batching rationale for the propagation model is not load-bearing.","tokens_in":33168,"feed_emoji":"🎙️","tokens_out":5434,"duration_ms":45025,"temperature":0.7,"pith_summary":"This paper claims that the real cost of producing an audio drama from a book is not generating individual sounds but managing the ripple effects of creative decisions: changing a character's voice, a scene's mood, or a line's tone forces creators to find and redo every dependent speech clip, sound effect, and music cue by hand. To remove that bottleneck, it introduces object-based audio editing, in which characters and scenes are editable objects with attributes such as voice design and ambience design, and every generated audio asset is linked to the object it belongs to. A single edit to an object automatically propagates to all dependent assets across the whole project. In a within-subjects study with eight professionals, Dramarrator lowered self-reported creation time from 16.7 to 3.7 hours, and one object edit replaced up to 30 manual edits; a listener study with 300 people found professionally refined output statistically comparable to existing-tool production on engagement and consistency. The paper's central assertion is that reifying narrative constructs as propagating objects, rather than clips on a timeline, is what makes fast iterative audio drama production possible.","feed_headline":"Object-based editing cuts audio drama creation from 17 to 4 hours","feed_subtitle":"One change to a character or scene ripples through every linked sound asset automatically.","key_machinery":"The central object is the tripartite object model: objects (narrative constructs such as characters and scenes), attributes (natural-language designs such as a voice design like 'a middle-aged woman with a measured cadence' or an ambience design like 'a busy market square'), and object-dependent assets (each speech clip, sound effect, music cue, and ambience layer on the timeline, each linked to the object or objects it belongs to). The argument runs on propagation: editing an attribute triggers an LLM to identify all linked assets requiring updates and to propose concrete modifications, so one high-level edit replaces many low-level clip edits. Supporting machinery includes best-of-25 script generation scored by an LLM judge against an 11-item rubric, batched text-to-speech synthesis of all of a character's lines in a single call to ensure voice consistency, and anchoring music and ambience to word-level dialogue timestamps so that assets stay aligned with dialogue across regenerations.","core_discovery":"On its own terms, the paper's discovery is that narrative constructs in audio drama—characters and scenes—can be reified as first-class editable objects whose attributes generate and regenerate their dependent assets, and that this structure converts high-level creative decisions from expensive project-wide manual overhauls into single parameter changes. Dramarrator extracts these objects from raw book text with an LLM, instantiates a voice design per character and an ambience design per scene, generates 25 candidate adapted scripts and selects the best by an 11-item rubric scored by an LLM judge, then synthesizes speech, sound effects, music, and ambience layers into a four-track timeline where every clip carries a link to its object or objects. When a creator edits an attribute, such as making a character nervous, an LLM proposes which dependent assets need changes, and accepted proposals regenerate or rewrite them across the whole project. The paper reports that this yields a significant task-load reduction for professionals, a 9.7x average amplification of editing effort (one voice edit propagated to all 30 of a character's lines in one case), and listener ratings for refined output that are statistically comparable to professional productions on engagement, overall quality, and character/scene consistency, while a gap remains on audio element placement, timing, clarity, and distraction.","pith_inferences":["The object model could generalize beyond audio to video production, where character and scene objects would propagate lighting, framing, and expression changes—though visual regeneration risks cross-frame consistency in a way audio regeneration does not; the paper gestures at this possibility but does not test it.","A shared emotion space such as valence and arousal could make cross-modal propagation more transparent and less reliant on LLM judgment: after an edit shifts a scene's emotional valence, assets out of alignment could be flagged deterministically, an extension the paper discusses but does not implement.","Object inheritance (such as child and adult versions of the same character) and time-keyframed attributes would let the object model handle characters and environments that change over a story, which the current always-one-attribute model cannot express.","The measured fourfold time reduction likely understates the gain for full-length books, because the 16.7-hour baseline came from short excerpts; amortizing object creation and prompt tuning over a whole production could make the ratio larger—this is speculative and should be tested."],"forward_implications":["If correct, object-based editing turns late-stage creative revisions in audio drama from near-restarts into cheap experiments, since changing a character or scene attribute regenerates every dependent asset automatically.","A single production can support many alternative versions—different voices for the same character, different ambience for the same scene—because objects parameterize the whole composition rather than individual clips.","The object model makes consistency a structural property: because all of a character's lines are generated from one voice attribute and batched in one text-to-speech call, characters and scenes stay recognizably the same across scenes, which the listener study suggests is achieved.","The paradigm implies a workflow shift for professionals from sequential pre-production, production, and post-production to iterative listen-edit-listen cycles, which participants in the study reported as increased creative exploration.","Object-based editing could extend to other narrative-driven media where characters and scenes cascade across heterogeneous assets, such as game audio and immersive stories, an extension the paper's exploratory study begins to probe."],"supporting_citations":[{"why":"Supplies the instrumental-interaction basis for higher-level abstractions that propagate edits.","marker":"[9]"},{"why":"Establishes content-based text editing for audio stories, the prior paradigm Dramarrator extends to objects.","marker":"[66]"},{"why":"Shows bidirectional script-audio linking, a direct precursor to object-to-asset linking.","marker":"[69]"},{"why":"Demonstrates object-oriented drawing, the main precedent for reifying creative components as editable objects.","marker":"[84]"},{"why":"Contributes the editable layered soundscape and ambience taxonomy Dramarrator adapts for scene sound design.","marker":"[88]"},{"why":"Provides the LLM-as-a-judge approach used to select the best script among 25 candidates.","marker":"[37]"},{"why":"Supplies the calibrated LLM rubric method used for multi-dimensional script scoring.","marker":"[41]"},{"why":"Grounds the audio drama conventions and character/scene consistency requirements that drive the design goals.","marker":"[26]"}],"fun_headline_variants":["Object-based audio editing: change a character, all lines update","From book to audio drama, with edits that ripple automatically","Audio drama production from books, with one-click character updates","Dramarrator: turn books into audio dramas with linked asset editing","Edit an object, update every audio asset: Dramarrator for drama"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole efficiency gain depends on the assumption that one attribute value—a single AI voice prompt for a character, one ambience prompt for a scene—can faithfully represent that character or scene everywhere it appears, so regenerating all linked assets from that attribute preserves consistency; the paper reports no accuracy measurement of this extraction or of voice consistency on the raw pipeline.","fun_headline_variants_meta":{"raw":{"variants":["Object-based audio editing: change a character, all lines update","From book to audio drama, with edits that ripple automatically","Audio drama production from books, with one-click character updates","Dramarrator: turn books into audio dramas with linked asset editing","Edit an object, update every audio asset: Dramarrator for drama"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00027,"raw_usage":{"total_tokens":1659,"prompt_tokens":1011,"completion_tokens":648,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":627,"completion_tokens_details":{"reasoning_tokens":560}},"tokens_in":627,"tokens_out":648,"duration_ms":6539,"temperature":1.0,"reasoning_tokens":560,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:07:10.215248+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Dramarrator on several books, apply one object edit such as making a specified character's voice raspier, and count how many linked speech, sound effect, music, and ambience clips actually change; if the changed set is smaller than the linked set, propagation is incomplete. Separately, generate a character's lines batched in one text-to-speech call versus line-by-line and have listeners judge whether both sets sound like the same character; if batched and unbatched lines are equally consistent, the batching rationale for the propagation model is not load-bearing.","supporting_citations":[{"cited_title":"Does this scene advance the central conflict?","cited_arxiv_id":null,"evidence_quote":"Contributes the editable layered soundscape and ambience taxonomy Dramarrator adapts for scene sound design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the LLM-as-a-judge approach used to select the best script among 25 candidates."},{"cited_title":"2002.Radio drama","cited_arxiv_id":null,"evidence_quote":"Grounds the audio drama conventions and character/scene consistency requirements that drive the design goals."}],"review_version":1}