{"id":"d7b1c100-5802-4a23-81e1-6bce497e1bcf","arxiv_id":"2412.07722","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Emotion values extracted from speech are mapped to haptic, tangible, or proxemic outputs as a new design material for interaction designers.","lead":"This paper proposes converting speech emotion recognition outputs into physical sensations, such as vibrations or spatial robot behavior, to make emotions usable as design material in interactive systems. It describes a starter kit that connects a transformer-based emotion recognition model to Arduino and Processing sketches, and discusses uses for pets, robots, and soma design.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SEC pipeline assumes SER outputs are emotion-faithful enough to drive physical displays, but no accuracy or perception evidence is given; the central claim that emotions become 'directly felt and socially communicated' is therefore unsupported at its most load-bearing point.","rationale":"The reader's verdict is CONDITIONAL, identifying the same weakest assumption: the SER pipeline is treated as a black box with no performance evaluation and no validation that users perceive the mapped physical sensations as emotionally meaningful. My stress-test agrees and sharpens the concern: the claim is not merely about SER accuracy in the abstract, but about the full chain's ability to preserve emotional information through the mapping to haptic, tangible, or proxemic outputs. Since the paper's own design scenarios (especially the robot proxemics example in Section 3.2) make concrete behavioral decisions based on SER outputs, the absence of any accuracy or perceptual evaluation is load-bearing. However, I do not think this rises to REJECT: the paper positions itself as a design exploration and explicitly states that mappings are context-sensitive and not one-solution-fits-all. As a proposal with a starter kit, it can be a valid contribution if framed as an open design tool rather than a validated system. The reader already conditioned acceptance on that framing, so my analysis leaves the verdict unchanged. I would not manufacture a stronger objection, because the conceptual argument is coherent and the cited prior work on SER, haptics, and somaesthetic design provides independent support for the building blocks.","tokens_in":8184,"tokens_out":2454,"duration_ms":29295,"concrete_test":"Run an end-to-end evaluation of the SEC chain: feed a held-out set of naturalistic speech clips with human-annotated valence/arousal (e.g., MSP-PODCAST or IEMOCAP) through the exact command-line tool described in Section 2.2, compute per-window agreement (e.g., CCC or accuracy) between predicted and annotated dimensions, then present the resulting physical outputs (vibration patterns or proxemic distances from the example sketches) to naive participants and measure whether they can reliably distinguish outputs driven by angry versus happy speech. If prediction agreement is near chance or participants cannot distinguish the physical outputs, the central claim that SEC makes emotions directly feelable fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that converting speech emotions into haptic, tangible, or proxemic manifestations makes emotions usable as design material. This claim depends on the full conversion chain preserving emotionally relevant information. Section 2.2.1 delegates sensing to the AffectToolbox's SER model [31] as a black box, and the paper reports no evaluation of that model's predictions in the targeted real-world conditions: live microphone input, varying audio chunk sizes, noisy environments, multiple speakers, and context-sensitive interactions. Yet the design scenarios in Section 3 rely on the mapped physical output being interpretable as the intended emotion. The clearest example is Section 3.2, where a robot's admissible approach distance is set from a person's anger level: if the SER model misclassifies valence or arousal, the robot will make socially wrong or even unsafe proxemic decisions. Similarly, a haptic collar intended to convey calmness may communicate alarm if the SER output is wrong. The paper also does not test whether human or animal users perceive the mapped haptic, tangible, or proxemic signals as emotionally meaningful. Without either SER accuracy numbers in the target conditions or a perception study of the output, the paper's central contribution remains a plausible design proposition rather than a demonstrated one. The 'starter kit' itself is not included in the manuscript and is only planned to be made open source (Section 2), so the method cannot currently be independently inspected or reproduced from the paper.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper argues that converting automatic speech emotion recognition (SER) outputs into physical, haptic, tangible, or proxemic manifestations would make emotions more usable as design material in interaction design. It describes a method and a 'starter kit' for speech emotion conversion (SEC), composed of a command-line SER tool extracted from the AffectToolbox and example Arduino/Processing sketches that display emotion values via vibrotactile or visual feedback. The paper then sketches design opportunities for animal-computer interaction, proxemic interaction, and somaesthetic/artistic research, with concrete examples such as a haptic collar for deaf pets and robot approach distance regulated by detected anger level. The contribution is primarily conceptual: the paper argues for a new design space rather than providing empirical evidence of effectiveness.","tokens_in":8446,"tokens_out":3070,"duration_ms":31535,"significance":"If the argument lands, the paper identifies a genuine and underexplored design material—real-time paralinguistic emotion values as input to physical displays—and connects it to several active HCI research areas (mid-air haptics, somaesthetic design, animal-computer interaction, proxemics). The author leverages prior work on the AffectToolbox and Wagner et al.'s transformer-based SER model, and the proposed pipeline is technically plausible. However, the paper makes no empirical claims and presents no evaluation; its significance rests on the openness of the design space and the availability of the starter kit. Those two pillars are currently weak: the kit is only 'planned' to be open-sourced, and no evidence is provided that the converted physical signals are perceived as emotionally meaningful by humans or animals. The paper is a useful design provocation but not yet a demonstrated contribution.","major_comments":[{"comment":"The abstract states that 'a starter kit for speech emotion conversion is provided,' but Section 2 says 'I plan to make the starter kit available open source' (footnote 1), and no code, binary, instructions, or screenshots of usage beyond a GUI screenshot are included in the manuscript. The central methodological contribution is therefore not actually delivered to the reader. Please either include the kit as a supplement (with installation/run instructions and a minimal demo) or revise the abstract and text to state explicitly that the kit is planned but not yet released, and adjust the contribution claim accordingly.","section":"Abstract and Section 2"},{"comment":"The pipeline delegates emotion sensing to the AffectToolbox's black-box SER model [31], but the paper provides no accuracy or reliability assessment in the target conditions: live microphone input, variable audio chunk sizes, background noise, multiple speakers, or context-dependent interactions. Yet Section 3.2's concrete example sets a robot's approach distance based on detecting anger versus happiness; if the SER model misclassifies valence or arousal, the robot could make socially inappropriate or unsafe proxemic decisions. The claim that SEC lets emotions be 'directly felt and socially communicated' is load-bearing and requires at least a sanity check of SER accuracy in the envisioned scenarios, or an explicit reframing of these examples as speculative design illustrations rather than demonstrated capabilities.","section":"Section 2.2.1 and Section 3.2"},{"comment":"The emotion-to-distance mapping is arbitrary: 'person(s) sounding angry could map to a minimum proximity of 2 meters... if the speech sound changes to happy then the distance could shrink to 1 meter' (Section 3.2). No proxemics literature (e.g., Hall's zones) or user data is cited to justify these thresholds, and the paper itself acknowledges in Section 2.2.2 that 'there is no one solution fits all.' If these numbers are intended only as illustrative, state that explicitly; if they are meant as a design recommendation, support them with evidence or participatory design. Without this distinction, the example is at risk of being read as a validated mapping rather than a placeholder.","section":"Section 3.2"},{"comment":"The animal-interaction scenarios (haptic collar, pet blanket that converts speech emotions to haptic/thermal feedback) assume that companion animals perceive the haptic or thermal output as emotionally meaningful. The paper cites Quaranta et al. for cats' recognition of human emotional valence via visual and auditory signals, and Totakura et al. for animal vocal emotion recognition, but provides no evidence that animals can decode tactile or thermal displays as emotion signals. The phrase 'Haptic feedback is a modality that is already used for pets' refers to training collars, which typically deliver punitive or corrective stimuli, not emotion valence. Please mark these scenarios as open design speculations rather than near-term opportunities, or include relevant evidence about animals' haptic perception.","section":"Section 3.1"}],"minor_comments":[{"comment":"The phrase 'closing the \"valence gab\"' should be 'valence gap.' Also, 'continuos' in Section 2.1 and 'soemasthetic' in the Section 3.3 heading are misspelled; 'proexmic' and 'tangilbe' appear in Section 4. A proofread would improve the paper's polish.","section":"Section 1"},{"comment":"The text repeatedly uses 'audio junks' where 'audio chunks' is meant. This is a terminological error that may confuse readers who are not native speakers.","section":"Section 2.2"},{"comment":"The paper uses the terms 'emotions' and 'affective states' colloquially and deliberately avoids formal definitions. This is acceptable for a design paper, but it should be acknowledged earlier that the validity of the ser model's continuous dimensions (valence, arousal, dominance) is not the focus; otherwise the central argument may be conflated with a claim about the ground truth of emotion.","section":"Section 1"},{"comment":"The description of 'two options' for sending SER results to external boards/apps is slightly ambiguous: one is a serial connection to Arduino, the other a TCP socket to Processing. Clarify which components are wired and which are network-based, as the figure (Figure 1) does not show the connection topology.","section":"Section 2.2.2"},{"comment":"The connection between SEC and somaesthetic interaction design would be strengthened by a more specific example of how the mapped physical sensation (e.g., vibration intensity, warmth) supports first-person or collaborative bodily reflection. As written, the link remains programmatic.","section":"Section 3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a short, idea-oriented contribution with a strong reliance on the author's prior connections to the AffectToolbox and the Augsburg HCI lab. The author self-cites several earlier works, but this is not problematic in a design-research context. My main concern for the editor is the gap between the paper's presentation as a 'starter kit' contribution and the actual absence of the artifact in the review materials. If the journal's scope accepts speculative design papers, the framing could be revised to 'design exploration' with the kit made available as a supplement. However, as a full paper in a serious HCI journal, the lack of any evaluation or even a demonstration of the pipeline's operation is a substantial shortfall. The major comments I raised are all addressable in revision, but they require either supplemental material or a re-scoping of the claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is a design-position paper, not an empirical study. The actual new thing is the SEC kit concept, wrapping the AffectToolbox's SER model in a command-line tool and linking it to Arduino and Processing sketches that drive a vibration motor or visualization. That packaging is modest but real, and the three design directions (animal-computer, proxemic, somaesthetic) are reasonable illustrations of why physical emotion representations might matter. The paper is honest about its status: the kit is planned to be open source, not shipped; the scenarios are explicitly initial ideas; and the author says the domains addressed are far from comprehensive. I appreciate that it does not overclaim. The soft spot is exactly where the stress-test note lands. The central claim that converting SER outputs into physical sensations makes emotions directly felt and socially communicated depends on the whole chain working in real contexts. Section 2.2.1 delegates sensing to the AffectToolbox and Wagner model as a black box, with no accuracy numbers for live microphone input, noise, or varying chunk sizes. Section 3.2's example of a robot adjusting approach distance based on detected anger is the clearest case: if the SER model misfires, the robot makes a socially wrong or unsafe move. And there is no perception study showing users or animals interpret the mapped vibrations as the intended emotion. So the load-bearing claim is unevaluated. But I would resist calling this fatal. The paper frames itself as a starter kit and an argument, not a validated contribution. The mapping is admittedly a designer's choice, and the author explicitly mentions participatory design and auto-ethnographic approaches as future work. For a design-exploration paper at a venue like TEI or alt.chi, this is within scope. The main editorial ask would be to either soften the abstract's claim or add a small proof-of-concept demonstration, even one author-tested scenario would help. The citation pattern looks fine; the author's self-citations are relevant prior design work, not padding. Who gets value from this: interaction designers, educators, and researchers wanting a low-barrier entry into affective haptics or proxemics. It deserves a serious referee, but the referee should push for either a demo or a clearer separation between we built a tool and this tool achieves X. I would be okay with conditional acceptance at an appropriate venue.","headline":"A clear design-position paper that packages existing SER with simple actuator mappings; the argument is plausible and honestly framed, but the central claim rests entirely on untested assumptions about SER reliability and user perception.","tokens_in":769,"tokens_out":746,"would_cite":false,"duration_ms":22829,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that converting automatic speech-emotion recognition outputs into physical, felt, and spatial forms makes emotions usable as interaction design material, and it contributes a starter kit for doing so.","keywords":["speech emotion recognition","affective computing","interaction design","haptic interaction","tangible interaction","proxemic interaction","somaesthetic design","speech emotion conversion"],"falsifier":"A controlled study would settle it: record speech from actors expressing distinct emotions, run the SEC pipeline, have blind participants feel the resulting haptic outputs (or observe the proxemic behaviour), and ask them to classify the intended emotion; if classification is at chance, or if users report the sensations as arbitrary, the central claim that converted speech emotions are meaningful design material fails. A complementary check is to measure the SER model's valence, arousal, and dominance accuracy on noisy, real-world microphone input against its benchmark performance.","tokens_in":7985,"feed_emoji":"🗣️","tokens_out":5336,"duration_ms":47472,"temperature":0.7,"pith_summary":"This paper argues that the labels and values produced by automatic speech emotion recognition are too abstract to work with as design material. The author proposes converting those outputs into bodily sensations and spatial behaviours—vibration, warmth, movement, proximity—so emotions can be directly felt and socially communicated. To make this practical, the paper describes a starter kit that runs a transformer-based emotion recognizer on live speech and sends its valence, arousal, and dominance values to an Arduino or a Processing sketch. The intended payoff is a new design space: emotions as malleable material for tangible, haptic, and proxemic interaction, including interactions with animals and robots.","feed_headline":"Speech emotions become touchable, feelable, and spatial","feed_subtitle":"A starter kit turns speech emotion recognition into haptic, tangible, and proxemic interaction material for designers","key_machinery":"The load-bearing object is the three-part SEC pipeline. Sensing: a wav2vec 2.0-based transformer model, exposed through an affect-analysis toolbox, predicts continuous valence, arousal, and dominance from raw speech audio. Mapping: a configurable command-line tool chunks the audio stream and applies user-specified rules that convert the emotion values into physical or spatial outputs. Display: an Arduino sketch drives a vibrotactile motor and a Processing sketch visualizes the values, both receiving the SER results over a serial or TCP connection. The work of the pipeline is to turn an abstract ML output into a felt, manipulable design material while keeping each stage replaceable.","core_discovery":"The central claim is that speech emotion conversion (SEC) is a useful new design resource: instead of stopping at digital emotion estimates, an interactive system can translate those estimates into physical displays and proxemic actions, letting people feel or see emotions as bodily happenings. The paper's contribution is the method and starter kit, which chain three replaceable parts: a pre-trained transformer-based SER model that predicts continuous valence, arousal, and dominance from raw audio; configurable mapping rules that turn those values into commands; and maker-friendly output devices (a vibrotactile motor on Arduino, a visualization in Processing) that render the commands. The author argues this opens design opportunities such as haptic collars or blankets that let pets feel their owner's tone of voice, social robots that adjust their approach distance based on detected anger or happiness, and somaesthetic or artistic explorations of emotion as a felt material.","pith_inferences":["Because emotion-to-physical mappings are culturally and personally variable, a natural next step is participatory design sessions that elicit preferred mappings for different user groups; the paper suggests such methods but does not test them.","The same pipeline could be paired with emerging animal vocal emotion recognition to create a bidirectional emotional communication channel between humans and companion animals, a direction the paper mentions only as emerging.","A testable extension is using SEC as an accessibility aid that renders emotional tone through touch for deaf or hard-of-hearing users; the paper does not develop this use case explicitly.","Whether the felt physical sensations are emotionally meaningful remains an open empirical question; a user study measuring recognition of mapped emotions would settle it."],"forward_implications":["Designers, artists, and educators can prototype emotion-driven physical interfaces without building or fine-tuning ML models themselves.","Companion animals, including deaf cats and dogs, could receive emotional tone through haptic or thermal feedback in collars or blankets.","Social robots could use detected speech emotion to set proxemic parameters, such as keeping a larger distance when a person sounds angry and a smaller one when they sound happy.","Artistic and somaesthetic researchers can treat speech emotions as raw material and calibrate familiarity or strangeness by adjusting mapping options.","SEC can be extended to media contexts such as gaming, VR/AR, TV, or radio to make emotional content tangible and more accessible."],"supporting_citations":[{"why":"Supplies the transformer-based SER model that predicts continuous valence, arousal, and dominance from raw speech and defines the sensing stage of the pipeline.","marker":"[31]"},{"why":"Provides the open-source affect-analysis toolbox from which the paper extracts and repackages the SER component as a command-line tool.","marker":"[18]"},{"why":"Provides the wav2vec 2.0 self-supervised speech representation architecture that the SER model builds on for raw-audio input.","marker":"[8]"},{"why":"Documents the 15-year evolution and benchmarking of SER methods, motivating the choice of transformer-based state of the art.","marker":"[28]"},{"why":"Demonstrates that cats recognize and respond to emotional valence in human voice, grounding the animal-companion design opportunity.","marker":"[23]"},{"why":"Introduces proxemic interaction as spatial, orientation-aware ubicomp, grounding the robot-distance design examples.","marker":"[14]"},{"why":"Defines somaesthetic interaction design, the design tradition the paper positions SEC within for bodily experience.","marker":"[16]"}],"fun_headline_variants":["Feel speech emotions as touch, vibration, and proxemics","Speech emotion becomes felt: haptic and spatial design","Starter kit makes speech emotions tangible and haptic","Convert speech emotion to vibration, touch, and space","Feel your speech: emotions as haptic and proxemic design"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach assumes the transformer-based model's real-time valence, arousal, and dominance predictions are accurate enough in real, context-sensitive settings that the resulting physical vibrations, warmth, or robot distances will feel emotionally meaningful to users.","fun_headline_variants_meta":{"raw":{"variants":["Feel speech emotions as touch, vibration, and proxemics","Speech emotion becomes felt: haptic and spatial design","Starter kit makes speech emotions tangible and haptic","Convert speech emotion to vibration, touch, and space","Feel your speech: emotions as haptic and proxemic design"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000515,"raw_usage":{"total_tokens":2472,"prompt_tokens":887,"completion_tokens":1585,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":1506}},"tokens_in":503,"tokens_out":1585,"duration_ms":10009,"temperature":1.0,"reasoning_tokens":1506,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:33:13.123412+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled study would settle it: record speech from actors expressing distinct emotions, run the SEC pipeline, have blind participants feel the resulting haptic outputs (or observe the proxemic behaviour), and ask them to classify the intended emotion; if classification is at chance, or if users report the sensations as arbitrary, the central claim that converted speech emotions are meaningful design material fails. A complementary check is to measure the SER model's valence, arousal, and dominance accuracy on noisy, real-world microphone input against its benchmark performance.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the transformer-based SER model that predicts continuous valence, arousal, and dominance from raw speech and defines the sensing stage of the pipeline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the open-source affect-analysis toolbox from which the paper extracts and repackages the SER component as a command-line tool."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates that cats recognize and respond to emotional valence in human voice, grounding the animal-companion design opportunity."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces proxemic interaction as spatial, orientation-aware ubicomp, grounding the robot-distance design examples."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines somaesthetic interaction design, the design tradition the paper positions SEC within for bodily experience."}],"review_version":1}