Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Speejis: Enhancing User Experience of Mobile Voice Messaging with Automatic Visual Speech Emotion Cues

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that automatically generated emojis and colored waveforms, created from speech alone, make voice messaging more attractive, novel, and stimulating, and that all 12 study participants preferred them.

desk verdict A promising design exploration with a real niche, but the emotion-cue claim outruns the evidence and the stats need a cleanup. read the letter →

arxiv 2502.05296 v1 pith:OI7LO3TQ submitted 2025-02-07 cs.HC

Ilhan Aslan , Carla F. Griggio , Henning Pohl , Timothy Merritt , Niels van Berkel This is my paper · ORCID
classification cs.HC
keywords speechemotionrecognitionvoicemessagesemojivisualizationuserexperiencespeech-to-textaffectivecomputingcontinuousdimensions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces "speejis"—emojis and colored-waveform cues generated automatically from the speech signal of a voice message, with no manual selection by the user. The authors built a working prototype and tested it with 12 users, comparing voice messaging with and without the cues on a standard user experience questionnaire plus interviews. They report significantly higher attractiveness, novelty, and stimulation with speejis, a unanimous preference for the augmented version, and a small drop in dependability. If the finding holds, speech emotion recognition can make asynchronous voice messaging more expressive and easier to scan, while raising new questions about trust and AI attribution in interpersonal communication.

What carries the argument

The load-bearing parts are a transformer-based speech emotion recognition model that outputs valence (positivity) and arousal (intensity) values for the full message and for 0.5-second chunks, a published emoji mapping that places each valence-arousal pair onto one of 22 facial emojis, a custom color mapping that colors each waveform bar by its chunk-level emotion values, and a transcription model that provides the text alongside the cues. The chunked analysis carries the paper's central design idea: instead of labeling a message with one emotion, the pipeline can reveal emotion trajectories within the message, treating the ending emotion as a particularly actionable cue directed toward the receiver.

What would settle it

Record voice messages with known intended emotional arcs, run them through the same chunk-level emotion recognition and emoji and color mappings, and compare the output cues against the senders' stated intentions and independent listener judgments; if the cues routinely miss the intended emotion or diverge from what listeners hear, the preference results would not generalize to real use.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that paralinguistic speech emotion recognition is enough to create useful and desirable visual emotion cues for voice messages. The system estimates valence, arousal, and dominance for both 0.5-second chunks and the full message, maps chunk-level values to colors on the waveform and overall and ending values to two emojis, and presents the result alongside a transcription. In the study, the augmented condition outperformed the baseline on attractiveness, novelty, and stimulation, and every participant said they would want to use speejis; the dependability dimension moved in the opposite direction. The paper interprets this as evidence that automatic, AI-driven augmentation of emotional tone is a viable direction for voice messaging, with the caveat that users want control over and clear attribution of the machine-generated cues.

Load-bearing premise

The finding depends on the emotion-reading model correctly interpreting short 0.5-second clips of ordinary voice messages, so the emojis and waveform colors it produces actually match what the sender felt.

Editorial extensions

If this is right

  • Recipients could get an emotional preview of a voice message without listening to it in full, which helps when playback is inappropriate or noisy.
  • Emotion trajectories inside a single message become visible, so a message that shifts from happy to sad can be represented as a change rather than as one static label.
  • Because the emotion estimates come from paralinguistic features only, augmentation can happen without waiting for transcription and can in principle run on the device, keeping the analysis more private.
  • Users are likely to want editorial control over the automated cues, and labeling the cues as AI-generated may address the observed drop in dependability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The study's recording setup used a desktop condenser microphone because the emotion model was trained on podcast audio; real phone-microphone recordings in noisy settings could degrade chunk-level estimates, so the studio setup may inflate the reported preference.
  • If the general result holds, voice messaging could evolve from a linear listening medium into a scannable emotional medium, a shift the paper gestures toward but does not measure.
  • A natural follow-up experiment would vary the 0.5-second chunk length and test whether finer or coarser granularity changes whether users trust the cues.
  • The dependability drop suggests a plausible design fix—sender-editable cues with visible AI attribution—that could preserve the gains in attractiveness and stimulation while restoring trust.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces 'speejis': emojis and a colored waveform overlay that are automatically generated from voice message audio via a speech emotion recognition (SER) pipeline. The system uses the Wagner et al. SER model on 0.5-s audio chunks, maps valence/arousal estimates to a 22-emoji set derived from Kutsuzawa et al., and applies a custom color mapping to the waveform. The authors implemented a Flask/web prototype and report a within-subjects user study (N=12) comparing voice messaging with and without speejis, using the UEQ and semi-structured interviews. They report significant UEQ differences in attractiveness, novelty, and stimulation, a decrease in dependability, and a unanimous participant preference for speejis; they also contribute design implications and iterated design concepts.

Significance. The idea of using continuous emotion dimensions to create fine-grained, automatic visual cues for asynchronous voice messages is timely and relevant to mobile HCI and affective computing. If the reported effects are reliable, the work offers a concrete way to improve glanceability and accessibility of voice messages. The paper has real strengths: it builds on relevant prior work (EmoWear, VoiceMessage++, AniBalloons), uses external SER and emoji norms rather than an in-house fitted model, and reports rich qualitative material with honest limitations. However, the quantitative evidence is currently undermined by an internal inconsistency in the reported test statistics, and the treatment fidelity of the 'emotion cues' is not validated. The results are therefore suggestive rather than conclusive, and the central claims need to be either supported by corrected statistics and additional validation or explicitly softened.

major comments (3)
  1. [Section 5.1, Table 1] The text of Section 5.1 says the paired t-tests found significant differences for 'attractiveness, novelty, perspicuity, and stimulation,' but Table 1 shows Perspicuity p=0.970 (clearly not significant) and Dependability p=0.0097 (significant) is omitted from the list. This is a load-bearing contradiction because the set of significant UEQ dimensions is the main quantitative result. Correct the text and any downstream claims so that the reported significant dimensions match Table 1, and clarify whether any correction for multiple comparisons was applied to the six dimensions tested.
  2. [Table 1, Novelty row] The Novelty row reports t(11)=2.56 and p=0.0002. These two statistics are incompatible: with 11 degrees of freedom, a two-tailed t-statistic of 2.56 corresponds to p approximately 0.026, not 0.0002. Since novelty is one of the headline dimensions in the abstract and Section 5.1, this inconsistency must be resolved. Either the t value or the p value is a typographical error, and the reported means and standard deviations should be checked against the raw data.
  3. [Sections 3.1-3.2] The central claim is that the augmentations are 'speech emotion cues' generated from speech input alone. The paper does not validate that the Wagner et al. SER model, applied to 0.5-s chunks of unpracticed mobile voice messages recorded through a desktop microphone, produces accurate valence/arousal/dominance estimates, nor that the 22-emoji selection and color mapping preserve those estimates. Section 3.1 reports only that the authors 'tested how much we could reduce the size before it started affecting the reliability of the results,' without any quantitative reliability criterion or validation set. Without such evidence, the observed UEQ differences could be attributable to the presence of colorful graphic augmentations rather than to accurate emotion cues. This concern should be addressed by adding a validation component (e.g., human ratings of chunk-level emotion or a comparison of predicted versus intended emotion) or by explicitly reframing the contribution as a study of automatically generated graphic augmentations rather than validated emotion cues.
minor comments (5)
  1. [Section 7.1] The sentence in Section 7.1, 'Another limitation we want to mention is that we did separately measure how much of the fun part etc. was linked with using emojis and how much the effects were to attribute to the coloured waveform,' appears to contain a typo: 'did separately measure' should likely read 'did not separately measure,' since the following sentence describes this as a limitation. Please correct the wording so the acknowledged confound is stated clearly.
  2. [Section 5.2.1] There is a typo in Section 5.2.1: 'speeijis' should be 'speejis.' The same typo appears in the participant-preference heading and in the phrase 'with speeijis.'
  3. [Section 1] In Section 1, 'continuos speech emotion models' should be 'continuous speech emotion models.'
  4. [Section 4.2] The UEQ was administered for both conditions in a single session after free exploration, with no counterbalancing or washout described. This is a minor methodological detail, but reporting the order in which the two UEQ versions were completed, and whether the 'with speejis' condition was always shown first, would strengthen the quantitative comparison.
  5. [Abstract and Section 5.2.1] The abstract's 'clear preference of all participants' is based on self-reports elicited in a semi-structured interview immediately after experiencing the prototype; this is subject to demand characteristics. Consider softening the wording to 'all participants reported preferring' or adding a caveat about the self-report nature of this measure.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an empirical UX comparison built on external SER and emoji-rating sources, not a derivation from fitted parameters or self-citation.

full rationale

The paper's claimed contribution is an empirical user-experience comparison (UEQ scores plus interviews) between voice messaging with and without automatically generated speejis. The speech-emotion values come from the external Wagner et al. SER model and the emoji valence/arousal placements come from the external Kutsuzawa et al. ratings; neither is fitted to the study data, and no equation in the paper reconstructs the UX outcome from the SER values. The 0.5-second chunk size and the 22-emoji subset are design choices whose reliability is a validity concern, not a circularity concern: the paper does not present the UX results as a prediction derived from those choices. The limitations section explicitly flags the missing dominance mapping and the lack of in-the-wild validation, which shows the authors are not presenting assumptions as derived results. Self-citations appear only as related-work context and are not load-bearing for the central empirical claim. No circular step can be exhibited with a quote and a specific reduction, so the appropriate score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 1 invented entities

The central claim depends on the SER model maintaining accuracy on 0.5 second chunks of mobile voice messages and on the emoji/color mappings conveying those values to users; neither is validated beyond the authors' informal testing. The design parameters (chunk size, emoji count, emoji subset, color mapping) are hand-chosen, not fitted from data, so they do not create circularity.

free parameters (4)
  • Audio chunk size for SER = 0.5 seconds
    Chosen by the authors after testing how much chunk size could be reduced before affecting SER reliability (Section 3.1); it determines emotion trajectory granularity and waveform bar resolution.
  • Number and placement of speeji emojis = 2 (overall message and last part)
    Selected based on the authors' explorations and a pilot study (Section 3.1); the first emoji summarizes overall sentiment and the second the final part, with no emoji on the introduction.
  • Emoji subset = 22 of 74 emojis from Kutsuzawa et al.
    The authors selected 22 facial emojis representing a broad valence-arousal range (Figure 3, Section 3.1); the selection affects mapping and is not independently validated.
  • Color mapping for emotional waveform = Oriented by Kutsuzawa et al. valence-arousal graphs
    The mapping from SER values to bar colors is described qualitatively (Figure 4) without exact thresholds or a user validation.
assumptions (3)
  • domain assumption The Wagner et al. SER model, trained on podcast audio, produces reliable valence, arousal, and dominance estimates for 0.5 second chunks of mobile voice messages.
    The entire speejis pipeline feeds these chunk-level values into emoji and color mappings (Section 3.2); no accuracy validation is reported for this domain shift.
  • domain assumption The Kutsuzawa et al. emoji valence/arousal ratings and the custom color mapping preserve the SER outputs well enough for users to perceive them as emotion cues.
    The mapping is described as oriented by Kutsuzawa et al. graphs (Section 3.1), but the selected emojis and colors are not validated against users or ground-truth emotion.
  • domain assumption The users' self-reported UEQ and interview ratings reflect the effect of speejis rather than novelty, demand characteristics, or the simultaneous presence of transcripts.
    The within-subjects protocol showed all designs one below the other and asked participants directly for preferences (Sections 4.2, 5), which invites comparison effects; no control for novelty was included.
invented entities (1)
  • Speejis (automatic visual speech emotion cues)
    purpose: A design concept and functional prototype that automatically adds emojis and colored waveform bars to voice messages based on speech emotion recognition; it is the artifact being evaluated in the user study.
    It is new to this paper as a named concept, but it has no external falsifiable prediction: the only evidence is the subjective user study reported here, not an independently testable handle such as a behavioral metric or a measured performance improvement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Speejis: Enhancing User Experience of Mobile Voice Messaging with Automatic Visual Speech Emotion Cues." pith.science (2026). https://pith.science/paper/OI7LO3TQ

@misc{pith2026250205296,
  author       = {Pith},
  title        = {Pith review of: Speejis: Enhancing User Experience of Mobile Voice Messaging with Automatic Visual Speech Emotion Cues},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OI7LO3TQ}},
  note         = {Machine review of arXiv:2502.05296}
}
read the original abstract

Mobile messaging apps offer an increasing range of emotional expressions, such as emojis to help users manually augment their texting experiences. Accessibility of such augmentations is limited in voice messaging. With the term "speejis" we refer to accessible emojis and other visual speech emotion cues that are created automatically from speech input alone. The paper presents an implementation of speejis and reports on a user study (N=12) comparing the UX of voice messaging with and without speejis. Results show significant differences in measures such as attractiveness and stimulation and a clear preference of all participants for messaging with speejis. We highlight the benefits of using paralinguistic speech processing and continuous emotion models to enable finer grained augmentations of emotion changes and transitions within a single message in addition to augmentations of the overall tone of the message.

Figures

Figures reproduced from arXiv: 2502.05296 by the authors.

Figure 1
Figure 1. Illustration of the basic idea and challenges behind the concept of speejis to augment voice messaging with the help of speech [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of design probes used in the study, representing conditions for baseline voice messages without speejis (left) and the [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The 22 emojis used as speejis in the study to automatically augment voice messages. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Concept for the colour mapping used to augment the audio waveform and create an emotional waveform. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Overview of the speejis system illustrating the components of the system and how they interact with each other to provide [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Illustration of study setup. they wanted. Participants could leave a voice message by using the ‘start recording’ and ‘stop recording’ buttons. The start button started recording and processing the user’s speech data and the stop button resulted in showing the voice me…
Figure 7
Figure 7. Figure 7: Results of the UX questionnaire comparing voice messaging experience with and without visual speech emotion cues. Error [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Iterated speeji design examples, addressing the issue with better connecting emojis with the waveform and dealing with longer [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Iterated speeji design examples, aiming to reduce the issue of complexity with especially long messages by allowing to tab a [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. "How to Explore Biases in Speech Emotion AI with Users?" A Speech-Emotion-Acting Study Exploring Age and Language Biases

    cs.HC 2025-07 conditional novelty 5.0 of 10

    In a 24-person Danish study, a speech emotion recognition model showed no significant age or language differences in recognizing deliberately acted happy, sad, angry, and calm speech, though high-arousal emotions were...

Reference graph

Works this paper leans on

59 extracted references · 38 canonical work pages · cited by 1 Pith paper

  1. [1]

    Jessalyn Alvina, Chengcheng Qu, Joanna McGrenere, and Wendy E. Mackay. 2019. MojiBoard: Generating Parametric Emojis with Gesture Keyboards. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3312771

    arXiv 2019

  2. [2]

    Shahin Amiriparian, Artem Sokolov, Ilhan Aslan, Lukas Christ, Maurice Gerczuk, Tobias Hübner, Dmitry Lamanov, Manuel Milling, Sandra Ottl, Ilya Poduremennykh, et al. 2021. On the impact of word error rate on acoustic-linguistic speech emotion recognition: An update for the deep learning era. arXiv preprint arXiv:2104.10121 (2021)

    arXiv 2021

  3. [3]

    Pengcheng An, Chaoyu Zhang, Haichen Gao, Ziqi Zhou, Yage Xiao, and Jian Zhao. 2025. AniBalloons: Animated chat balloons as affective augmentation for social messaging and chatbot interaction. International Journal of Human-Computer Studies 194 (2025), 103365. https://doi.org/10. 1016/j.ijhcs.2024.103365

    arXiv 2025

  4. [5]

    Pengcheng An, Jiawen Stefanie Zhu, Zibo Zhang, Yifei Yin, Qingyuan Ma, Che Yan, Linghao Du, and Jian Zhao. 2024. EmoWear: Exploring Emotional Teasers for Voice Message Interaction on Smartwatches. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY,...

    arXiv 2024

  5. [6]

    Toshiki Aoki, Rintaro Chujo, Katsufumi Matsui, Saemi Choi, and Ari Hautasaari. 2022. EmoBalloon - Conveying Emotional Arousal in Text Chats with Speech Balloons. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 527, 16 pages. ht...

    arXiv 2022

  6. [7]

    Ilhan Aslan and Elisabeth André. 2017. Pre-touch proxemics: moving the design space of touch targets from still graphics towards proxemic behaviors. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, UK) (ICMI ’17). Association for Computing Machinery, New York, NY, USA, 101–109. https://doi.org/10.1145/3136755.3136808

    work page arXiv 2017

  7. [8]

    Ilhan Aslan, Feiyu Xu, Hans Uszkoreit, Antonio Krüger, and Jörg Steffen. 2005. COMPASS2008: Multimodal, multilingual and crosslingual interaction for mobile tourist guide applications. In International Conference on Intelligent Technologies for Interactive Entertainment . Springer, 3–12. https://doi.org/10.1007/11590323_1

    work page doi:10.1007/11590323_1 2005

  8. [9]

    Björn Bittner, Ilhan Aslan, Chi Tai Dang, and Elisabeth André. 2019. Of Smarthomes, IoT Plants, and Implicit Interaction Design. (2019), 145–154. https://doi.org/10.1145/3294109.3295618

    arXiv 2019

Show all 59 references
  1. [10]

    Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77–101. https: //doi.org/10.1191/1478088706qp063oa

    2006 doi

  2. [11]

    Daniel Buschek, Mariam Hassib, and Florian Alt. 2018. Personal Mobile Messaging in Context: Chat Augmentations for Expressiveness and Awareness. ACM Trans. Comput.-Hum. Interact. 25, 4, Article 23 (Aug. 2018), 33 pages. https://doi.org/10.1145/3201404

    2018 doi

  3. [12]

    Yoonjeong Cha, Jongwon Kim, Sangkeun Park, Mun Yong Yi, and Uichin Lee. 2018. Complex and Ambiguous: Understanding Sticker Misinterpreta- tions in Instant Messaging. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 30 (Nov. 2018), 22 pages. https://doi.org/10.1145/3274299

    2018 doi

  4. [13]

    Qinyue Chen, Yuchun Yan, and Hyeon-Jeong Suk. 2021. Bubble Coloring to Visualize the Speech Emotion. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, A...

    2021

  5. [14]

    Lukas Christ, Shahin Amiriparian, Manuel Milling, Ilhan Aslan, and Björn W Schuller. 2024. Modeling Emotional Trajectories in Written Stories Utilizing Transformers and Weakly-Supervised Learning. arXiv preprint arXiv:2406.02251 (2024)

    2024 arXiv

  6. [15]

    Henriette Cramer, Paloma de Juan, and Joel Tetreault. 2016. Sender-intended functions of emojis in US messaging. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (Florence, Italy) (MobileHCI ’16). Association fo...

    2016

  7. [16]

    Caluã de Lacerda Pataca, Saad Hassan, Nathan Tinker, Roshan Lalintha Peiris, and Matt Huenerfauth. 2024. Caption Royale: Exploring the Design Space of Affective Captions from the Perspective of Deaf and Hard-of-Hearing Individuals. In Proceedings of the 2024 CHI Conference on ...

    2024

  8. [17]

    K Anders Ericsson and Herbert A Simon. 1980. Verbal reports as data.Psychological review 87, 3 (1980), 215. https://doi.org/10.1037/0033-295X.87.3.215

    1980 doi

  9. [18]

    Hans W Gellersen, Albrecht Schmidt, and Michael Beigl. 2002. Multi-sensor context-awareness in mobile devices and smart artifacts. Mobile Networks and Applications 7 (2002), 341–351

    2002

  10. [19]

    Mayank Goel, Leah Findlater, and Jacob Wobbrock. 2012. WalkType: using accelerometer data to accomodate situational impairments in mobile touch screen text entry. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Asso...

    2012

  11. [20]

    Griggio, Benjamin M

    Carla F. Griggio, Benjamin M. Gorman, and Garreth W. Tigwell. 2024. Party Face Congratulations! Exploring Design Ideas to Help Sighted Users with Emoji Accessibility when Messaging with Screen Reader Users. Proc. ACM Hum.-Comput. Interact. 8, CSCW1, Article 175 (April 2024), 3...

    2024 doi

  12. [21]

    Griggio, Arissa J

    Carla F. Griggio, Arissa J. Sato, Wendy E. Mackay, and Koji Yatani. 2021. Mediating Intimacy with DearBoard: a Co-Customizable Keyboard for Everyday Messaging. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Associat...

    2021

  13. [22]

    Jens Grubert, Matthias Heinisch, Aaron Quigley, and Dieter Schmalstieg. 2015. Multifi: Multi fidelity interaction with displays on and around the body. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems . 3933–3942

    2015

  14. [24]

    Gabriel Haas, Jan Gugenheimer, and Enrico Rukzio. 2020. VoiceMessage++: Augmented Voice Recordings for Mobile Instant Messaging. In22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (Oldenburg, Germany) (MobileHCI ’20). Association for...

    2020

  15. [25]

    Chris Harrison, Julia Schwarz, and Scott E. Hudson. 2011. TapSense: enhancing finger interaction on touch surfaces. InProceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (Santa Barbara, California, USA) (UIST ’11). Association for Computing M...

    2011

  16. [26]

    Wozniak, and Florian Alt

    Mariam Hassib, Daniel Buschek, Paweł W. Wozniak, and Florian Alt. 2017. HeartChat: Heart Rate Augmented Mobile Chat to Support Empathy and Awareness. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association ...

    2017

  17. [27]

    Brubaker

    Jialun "Aaron" Jiang, Casey Fiesler, and Jed R. Brubaker. 2018. ’The Perfect One’: Understanding Communication Practices and Challenges with Animated GIFs. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 80 (Nov. 2018), 20 pages. https://doi.org/10.1145/3274349

    2018 doi

  18. [28]

    Matt Jones. 2006. Mobile Interaction Design. John Willey & Sons, Ltd (2006)

    2006

  19. [29]

    Wendy Ju. 2015. The Design of Implicit Interactions (1st ed.). Morgan & Claypool Publishers

    2015

  20. [30]

    Biing-Hwang Juang and Lawrence R Rabiner. 2005. Automatic speech recognition–a brief history of the technology development. Georgia Institute of Technology. Atlanta Rutgers University and the University of California. Santa Barbara 1, 67 (2005), 1

    2005

  21. [31]

    Krause, A

    A. Krause, A. Smailagic, and D.P. Siewiorek. 2006. Context-aware mobile computing: learning context- dependent personal preferences from a wearable sensor array. IEEE Transactions on Mobile Computing 5, 2 (2006), 113–127. https://doi.org/10.1109/TMC.2006.18

    2006 doi

  22. [32]

    Antonio Krüger, Andreas Butz, Christian Müller, Christoph Stahl, Rainer Wasinger, Karl-Ernst Steinberg, and Andreas Dirschl. 2004. The connected user interface: Realizing a personal situated navigation service. In Proceedings of the 9th international conference on Intelligent ...

    2004

  23. [33]

    Leiva, and Antti Oulasvirta

    Utkarsh Kunwar, Sheetal Borar, Moritz Berghofer, Julia Kylmälä, Ilhan Aslan, Luis A. Leiva, and Antti Oulasvirta. 2022. Robust and Deployable Gesture Recognition for Smartwatches. In Proceedings of the 27th International Conference on Intelligent User Interfaces (Helsinki, Fin...

    2022

  24. [34]

    Gaku Kutsuzawa, Hiroyuki Umemura, Koichiro Eto, and Yoshiyuki Kobayashi. 2022. Classification of 74 facial emoji’s emotional states on the valence-arousal axes. Scientific Reports 12, 1 (2022), 398. https://doi.org/10.1038/s41598-021-04357-7

    2022 doi

  25. [35]

    Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and Evaluation of a User Experience Questionnaire. In HCI and Usability for Education and Work, Andreas Holzinger (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 63–76

    2008

  26. [37]

    Marshall McLuhan. 1967. The medium is the massage. A Benthm Bbook (1967)

    1967

  27. [38]

    S. Oviatt. 1996. User-centered modeling for spoken language and multimodal interfaces. IEEE MultiMedia 3, 4 (1996), 26–35. https://doi.org/10. 1109/93.556458

    1996

  28. [39]

    Rosalind W Picard. 2000. Affective computing. MIT press

    2000

  29. [40]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine Mcleavey, and Ilya Sutskever. 2023. Robust Speech Recognition via Large- Scale Weak Supervision. In Proceedings of the 40th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol...

    2023

  30. [41]

    Dimitrios Raptis, Nikolaos Tselios, and Nikolaos Avouris. 2005. Context-based design of mobile applications for museums: a survey of existing practices. In Proceedings of the 7th international conference on Human computer interaction with mobile devices & services . 153–160

    2005

  31. [42]

    Hannes Ritschel, Ilhan Aslan, Silvan Mertes, Andreas Seiderer, and Elisabeth André. 2019. Personalized Synthesis of Intentional and Emotional Non-Verbal Sounds for Social Robots. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII) . 1...

    2019

  32. [43]

    Hannes Ritschel, Ilhan Aslan, David Sedlbauer, and Elisabeth André. 2019. Irony Man: Augmenting a Social Robot with the Ability to Use Irony in Multimodal Communication with Humans. In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems...

    2019

  33. [44]

    Zhanna Sarsenbayeva, Niels van Berkel, Chu Luo, Vassilis Kostakos, and Jorge Goncalves. 2017. Challenges of situational impairments during interaction with mobile devices. In Proceedings of the 29th Australian Conference on Computer-Human Interaction (Brisbane, Queensland, Aus...

    2017

  34. [45]

    Björn Schuller, Stefan Steidl, and Anton Batliner. 2009. The interspeech 2009 emotion challenge. (2009)

    2009

  35. [46]

    Tax, and Hayley Hung

    Stephanie Tan, David M.J. Tax, and Hayley Hung. 2022. Conversation Group Detection With Spatio-Temporal Context. In Proceedings of the 2022 International Conference on Multimodal Interaction (Bengaluru, India) (ICMI ’22). Association for Computing Machinery, New York, NY, USA,...

    2022

  36. [47]

    Tigwell, Benjamin M

    Garreth W. Tigwell, Benjamin M. Gorman, and Rachel Menzies. 2020. Emoji Accessibility for Visually Impaired People. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY,...

    2020

  37. [48]

    Andreas Triantafyllopoulos, Anton Batliner, Simon Rampp, Manuel Milling, and Björn Schuller. 2024. INTERSPEECH 2009 Emotion Challenge Revisited: Benchmarking 15 Years of Progress in Speech Emotion Recognition. arXiv preprint arXiv:2406.06401 (2024)

    2024 arXiv

  38. [49]

    Schuller

    Andreas Triantafyllopoulos, Lukas Christ, Alexander Gebhard, Xin Jing, Alexander Kathan, Manuel Milling, Iosif Tsangko, Shahin Amiriparian, and Björn W. Schuller. 2024. Beyond Deep Learning: Charting the Next Frontiers of Affective Computing. Intelligent Computing 3 (2024), 00...

    2024 doi

  39. [50]

    Kristin Vadas, Nirmal Patel, Kent Lyons, Thad Starner, and Julie Jacko. 2006. Reading on-the-go: a comparison of audio and hand-held displays. In Proceedings of the 8th Conference on Human-Computer Interaction with Mobile Devices and Services (Helsinki, Finland) (MobileHCI ’06...

    2006

  40. [51]

    Schuller

    Johannes Wagner, Andreas Triantafyllopoulos, Hagen Wierstorf, Maximilian Schmitt, Felix Burkhardt, Florian Eyben, and Björn W. Schuller

  41. [52]

    Joseph B. Walther. 2007. Selective self-presentation in computer-mediated communication: Hyperpersonal dimensions of technology, language, and cognition. Computers in Human Behavior 23, 5 (2007), 2538–2557. https://doi.org/10.1016/j.chb.2006.05.002

    2007 doi

  42. [53]

    Joseph B Walther and Kyle P D’Addario. 2001. The impacts of emoticons on message interpretation in computer-mediated communication. Social science computer review 19, 3 (2001), 324–347

    2001

  43. [54]

    Saiwen Wang, Jie Song, Jaime Lien, Ivan Poupyrev, and Otmar Hilliges. 2016. Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, ...

    2016

  44. [55]

    Taiba Majid Wani, Teddy Surya Gunawan, Syed Asif Ahmad Qadri, Mira Kartiwi, and Eliathamby Ambikairajah. 2021. A Comprehensive Review of Speech Emotion Recognition Systems. IEEE Access 9 (2021), 47795–47814. https://doi.org/10.1109/ACCESS.2021.3068045

    2021

  45. [56]

    Klaus Weber, Hannes Ritschel, Ilhan Aslan, Florian Lingenfelser, and Elisabeth André. 2018. How to Shape the Humor of a Robot - Social Behavior Adaptation Based on Reinforcement Learning. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (Boulde...

    2018

  46. [57]

    Philip Weber, Lea Katharina Michel, Lena Koschorreck, and Thomas Ludwig. 2023. Voice Messages Reimagined: Exploring the Design Space of Current Voice Messaging Interfaces. In Proceedings of Mensch Und Computer 2023 (Rapperswil, Switzerland) (MuC ’23). Association for Computing...

    2023

  47. [58]

    Wobbrock

    Jacob O. Wobbrock. 2006. The Future of Mobile Device Research in HCI. https://api.semanticscholar.org/CorpusID:267842432

    2006

  48. [59]

    Yafeng Yin, Lei Xie, Zhiwei Jiang, Fu Xiao, Jiannong Cao, and Sanglu Lu. 2024. A Systematic Review of Human Activity Recognition Based on Mobile Devices: Overview, Progress and Trends.IEEE Communications Surveys & Tutorials26, 2 (2024), 890–929. https://doi.org/10.1109/COMST.2...

    2024

  49. [60]

    Rui Zhou, Jasmine Hentschel, and Neha Kumar. 2017. Goodbye Text, Hello Emoji: Mobile Communication on WeChat in China. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New Yo...

    2017

  50. [61]

    Müller, André N

    Manuela Züger, Sebastian C. Müller, André N. Meyer, and Thomas Fritz. 2018. Sensing Interruptibility in the Office: A Field Study on the Use of Biometric and Computer Interaction Sensors. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal...

    2018

  51. [2023]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 9 (2023), 10745–10759

    Dawn of the Transformer Era in Speech Emotion Recognition: Closing the Valence Gap. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 9 (2023), 10745–10759. https://doi.org/10.1109/TPAMI.2023.3263585

    2023

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.