Pith. sign in

REVIEW 4 major objections 5 minor 79 references

Applying General Turn-taking Models to Conversational Human-Robot Interaction

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Two zero-shot turn-taking models, used in tandem, make a robot's conversation faster and less interruptive, and people prefer it.

desk verdict A solid first HRI deployment of TurnGPT and VAP in tandem, but the user study bundles several changes at once, so the headline claim that the general models are responsible for the gains is not actually isolated. read the letter →

arxiv 2501.08946 v1 pith:7S4Z7POE submitted 2025-01-15 cs.CL cs.RO

Gabriel Skantze , Bahar Irfan This is my paper · ORCID
classification cs.CLcs.RO
keywords turn-takinghuman-robotinteractionVoiceActivityProjectionTurnGPTself-supervisedlearningresponsedelayinterruptionsconversationalAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Conversational robots usually decide when to speak by waiting for silence, which forces a trade-off between slow replies and interruptions. This paper argues that general self-supervised models of human turn-taking can replace that heuristic without any domain-specific retraining. The authors combine TurnGPT, which predicts syntactic and pragmatic turn completion from text, with Voice Activity Projection (VAP), which continuously predicts upcoming speech activity from audio, and feed the robot's own synthesized speech back into both models. In a within-subject study with 39 adults talking to a Furhat robot, the proposed system reduced median response time from 2.7 s to 1.5 s and interruption rate from 16.6% to 6.9%, and participants significantly preferred it. If the result holds, turn-taking in human-robot interaction no longer needs hand-tuned silence thresholds or per-domain training data.

What carries the argument

The machinery is the tandem of TurnGPT and VAP plus self-monitoring. TurnGPT is a GPT-2-style language model trained on 385K text conversations with a special turn-completion token, so the probability it assigns to that token measures whether the current utterance is pragmatically complete. VAP is a transformer trained on roughly 1,000 hours of telephone dialogue (Fisher and Switchboard) to project, ten times per second, who will speak in the next two seconds, summarized as $p_{\mathrm{now}}$ and $p_{\mathrm{future}}$; a version taking stereo input lets one channel be the robot's own synthesized voice. Feeding the robot's speech back is what lets the system distinguish a genuine interruption from a backchannel and decide when gaze aversion is needed. A small set of thresholds on these two signals, listed in the appendix, converts the predictions into the robot's turn-taking behavior.

What would settle it

Take the published algorithm, replace the stereo VAP input with the user-only channel while keeping every other hyperparameter, and compare interruption rates: if the 6.9% rate does not degrade toward the 16.6% baseline, the claimed role of self-monitoring is not supported.

Watch

Extended reading notes

Core claim

The central discovery is that turn-taking models trained as third-person predictors of human-human dialogue can serve as first-person controllers of a robot's turn-taking behavior when used in tandem and fed the robot's own speech. TurnGPT supplies a turn-completion probability from the incremental ASR transcript, capturing syntactic and longer-range pragmatic completion; VAP supplies two continuous acoustic projections, $p_{\mathrm{now}}$ (0-600 ms) and $p_{\mathrm{future}}$ (600-2000 ms), from stereo audio of user and robot. The control algorithm lets the robot start preparing LLM+TTS responses before the user finishes, allows a turn shift only after both VAP projections favor the robot or a TurnGPT-graded timeout expires, and stops the robot mid-utterance only when both projections signal a genuine interruption. The evaluation against a silence-threshold baseline with gaze aversion and an LED listening signal found the proposed system significantly faster and less interruptive, with users rating it more fluent, more human-like, and easier to interrupt.

Load-bearing premise

The approach depends on predictions from models trained on telephone conversations between US English speakers staying reliable when applied, without retraining, to a face-to-face robot whose synthetic voice is fed back into the models.

Editorial extensions

If this is right

  • Spoken dialogue systems can drop fixed silence thresholds and instead gate turns on continuous projections of when the other speaker will finish.
  • Response generation can begin on incremental ASR before the user stops speaking, so the floor can be taken close to the 0.5 s minimum allowed by the algorithm whenever the LLM and TTS are fast enough.
  • The same VAP signal can separate genuine barge-in interruptions from brief backchannels, enabling duplex listening without the false-trigger problems of naive barge-in.
  • Because the models are used zero-shot, new HRI applications need not collect and annotate domain-specific turn-taking data to get natural timing.
  • The preference result suggests that more human-like timing is not automatically worse for users, though a minority preferred the explicit LED pacing of the baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors do not test this, but the same VAP signal could also drive synthesized fillers or prosodic turn-holding cues, since they note VAP is sensitive to fillers.
  • The paper does not test non-English or multi-party interaction, so a direct stress test of zero-shot generality would use other languages, accents, or group settings.
  • Their proposed semantic-similarity gating on incremental ASR could be combined with TurnGPT rollouts to prepare responses even earlier, an idea they mention as future work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an HRI turn-taking system in which two pre-trained general turn-taking models, TurnGPT and Voice Activity Projection (VAP), are used zero-shot to decide when the robot should prepare a response, start speaking, and yield to user interruptions. The proposed pipeline also includes incremental ASR-based response pre-generation, self-monitoring of the robot's own TTS, and VAP-based barge-in handling. The authors compare this system with a traditional silence-threshold baseline (with red LED and gaze-aversion cues) in a within-subject study with 39 participants. They report a significantly shorter median response time (1.5 s vs 2.7 s), a significantly lower interruption rate (6.9% vs 16.6%), and a significant preference for the proposed system.

Significance. If the improvements were caused by the general turn-taking models themselves, this would be a valuable first demonstration of zero-shot transfer from human-human telephone dialogue to face-to-face HRI. The study is properly powered, counterbalanced, and uses both objective timing and participant ratings; the effect sizes are large and statistically significant. The authors are also transparent about the arbitrary threshold values and the external validity risks. The main weakness is attribution: the proposed system differs from the baseline in several coupled design choices, so the reported gains do not isolate the contribution of TurnGPT and VAP. Because the central claim is about the general models, this needs to be resolved or the claim reframed.

major comments (4)
  1. [§V-B and §VII-A] The reduction in response time is confounded with response pre-generation. The proposed system begins LLM/TTS preparation on incremental ASR results whenever TurnGPT or the ASR timeout allows (pseudo-code lines 45–54), while the baseline waits for end-of-speech before generating anything. The paper itself acknowledges in §V-B that incremental preparation 'could potentially have been added to the baseline system as well.' Consequently, the median gap of 1.5 s vs 2.7 s and the 0.6 s histogram peak in Figure 6 may reflect the pre-generation mechanism rather than the turn-yield predictions of TurnGPT/VAP. To support the central claim, the authors should add a control condition or analysis that separates the contribution of the turn-taking models from that of pre-generation, for example a baseline with identical incremental response preparation but silence-threshold turn-taking, or an analysis in the proposed condition that distinguishes pre-generated from non-pre-generated turns.
  2. [§V-C and §VI-C] The interruption-rate comparison does not isolate the models because the two conditions use different interaction protocols. The baseline is simplex (red LED signals that the robot is not listening; participants are instructed to 'only speak when there is no light'), whereas the proposed system is duplex and participants are told they 'can interrupt the robot anytime.' The VAP-based barge-in handler in §V-C is a separate component from turn-yield prediction, and the instruction difference changes participant behavior and expectations. In addition, interruptions were self-annotated by participants who knew which condition they were in, so the 16.6% vs 6.9% difference may reflect protocol differences and annotation bias as well as model quality. The authors should either hold the interruption instructions and affordances constant across conditions, use blind annotators, or report a sensitivity analysis that addresses this confound.
  3. [Appendix A and §V] The system-level hyperparameters are tuned on the target application: the text before the pseudo-code states that their current values 'were tuned through four pilot experiments.' This means the comparison is not a pure zero-shot evaluation of the general models; thresholds such as VAP_PNOW_YIELD_THRESHOLD, TURNGPT_PREPARE_THRESHOLD, and MIN_GAP_TIME were adapted to the robot and task. The manuscript should specify how pilots were used, whether pilot participants were excluded from the main study, and ideally report the sensitivity of the main results to threshold variation. Otherwise the 'general models without domain-specific tuning' claim is only partially supported.
  4. [§VII-A] The statistical summary is incomplete in ways that affect interpretation. Table I reports only means, medians, modes, and W statistics; it does not give the number of turns per participant, per-participant distributions, effect sizes, or confidence intervals. Because response times within a participant are not independent, the Wilcoxon test should be on per-participant summary values, but the paper does not explicitly say this. Please report the exact test setup, the per-participant differences, and effect sizes for response time and interruption rate.
minor comments (5)
  1. [§III-A] The claim that TurnGPT 'responds in about 20 ms' would benefit from clarification about whether this is per token, per utterance, or for a single turn-completion evaluation; as written it is ambiguous.
  2. [§IV-A] The baseline's end-of-speech threshold is described as likely based on a silence threshold, but no estimate of the actual threshold value is given despite Figure 2 showing a roughly 2.5 s total response time; adding this would help readers compare the baseline with the 0.5–3 s timeouts in the proposed system.
  3. [§VI-C] The instruction 'only speak when there is no light' seems to conflict with the fact that participants annotated robot interruptions; please clarify whether the red light was expected to suppress all user speech and how the baseline's 16.6% interruption rate relates to this protocol.
  4. [§VII-B and Appendix D] The reporting of Q8 is unclear: the median of 2 and W=98 would be easier to interpret if the scale endpoints and the direction of preference were stated explicitly in the table.
  5. [Figure 4] Figure 4 is discussed as showing both a user interruption and a backchannel; labeling the two panels would improve readability and prevent confusion about which part of the figure corresponds to which event.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TurnGPT and VAP are pre-trained external models applied zero-shot, and the user study is an out-of-sample evaluation; the acknowledged pre-generation confound is a design limitation, not a fitted prediction.

full rationale

The derivation chain is not circular. TurnGPT and VAP are pre-trained models from earlier work and are applied zero-shot here, as the paper states: 'we are not fine-tuning the models against some domain-specific dataset, but rather use them in a zero-shot fashion' (Section V). The headline outcomes—median response time 1.5 vs. 2.7 s, interruption rate 6.9% vs. 16.6%, and preference ratings—are measured in a within-subject study with 39 participants, not computed from the models' predictions by any equation. The Appendix A hyper-parameters were tuned in four pilot experiments before the main study, so the evaluation is out-of-sample with respect to them; no fitted value is renamed as a prediction. The author self-citations ([19], [20], [41], [45], [48], [49], [58]) support the offline capabilities of TurnGPT and VAP on human-human corpora and are not used to derive the new HRI result. The paper explicitly acknowledges that incremental response pre-generation could have been added to the baseline (Section V-B), and the differing interruption instructions and self-annotation are potential confounds; these are validity threats to attribution, not circularity. No quote plus equation-level reduction can be exhibited, so the circularity score is 0.

Assumptions & free parameters 9 free parameters · 5 assumptions · 0 invented entities

The system is built almost entirely from prior models (TurnGPT, VAP) and hand-tuned thresholds. The only genuinely new pieces are the integration algorithm and the user study, so the ledger is dominated by domain-transfer assumptions about zero-shot application.

free parameters (9)
  • VAP_PNOW_YIELD_THRESHOLD = 0.5
    Probability threshold for pnow to allow a turn shift; set by pilot tuning (Appendix A).
  • VAP_PFUT_YIELD_THRESHOLD = 0.5
    Probability threshold for pfut to allow a turn shift; set by pilot tuning (Appendix A).
  • VAP_PNOW_INTERRUPT_THRESHOLD = 0.4
    Threshold below which the robot stops speaking on user speech onset; set by pilot tuning (Appendix A).
  • VAP_PFUT_INTERRUPT_THRESHOLD = 0.4
    Threshold below which robot stops speaking when user barges in; set by pilot tuning (Appendix A).
  • TURNGPT_PREPARE_THRESHOLD = 0.2
    TurnGPT probability above which the system pre-generates a response; set by pilot tuning (Appendix A).
  • ASR_PREPARE_TIMEOUT = 200 ms
    Time since last ASR update after which a response is prepared; set by pilot tuning (Appendix A).
  • SIMILARITY_PREPARE_THRESHOLD = 0.8
    Semantic similarity above which a new tentative response is skipped; set by pilot tuning (Appendix A).
  • MIN_GAP_TIME = 500 ms
    Minimum gap after VAP detects a yield before robot speaks; set by pilot tuning (Appendix A).
  • TURNGPT_YIELD_TIMEOUTS = table: 0.3/500, 0.2/1000, 0.1/2000, 0.0/3000 ms
    Timeout mapping from TurnGPT yield probability to wait time; set by pilot tuning (Appendix A).
assumptions (5)
  • domain assumption VAP and TurnGPT predictions transfer from human-human telephone dialogue to face-to-face HRI with synthesized speech.
    Sections III and V use the models zero-shot; the entire system depends on this transfer holding.
  • domain assumption Self-monitoring, feeding the robot's own speech back to VAP and TurnGPT, improves turn-taking decisions.
    Section V-D asserts this; no ablation is run to test it independently.
  • domain assumption The simplified two-value VAP output (pnow, pfutur) is sufficient for turn-yield and interruption decisions.
    Section III-B reduces VAP output to two values, used directly in the control algorithm in Appendix A.
  • domain assumption Participant self-annotations of interruptions are valid measures of actual interruptions.
    Section VI-C: participants marked when they felt the robot did not let them speak, after being told each condition's behavior.
  • standard math Background machine-learning and statistics machinery (transformers, self-supervised training, Wilcoxon tests) is correct.
    Standard methodology, not load-bearing beyond usual assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Applying General Turn-taking Models to Conversational Human-Robot Interaction." pith.science (2026). https://pith.science/paper/7S4Z7POE

@misc{pith2026250108946,
  author       = {Pith},
  title        = {Pith review of: Applying General Turn-taking Models to Conversational Human-Robot Interaction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7S4Z7POE}},
  note         = {Machine review of arXiv:2501.08946}
}
read the original abstract

Turn-taking is a fundamental aspect of conversation, but current Human-Robot Interaction (HRI) systems often rely on simplistic, silence-based models, leading to unnatural pauses and interruptions. This paper investigates, for the first time, the application of general turn-taking models, specifically TurnGPT and Voice Activity Projection (VAP), to improve conversational dynamics in HRI. These models are trained on human-human dialogue data using self-supervised learning objectives, without requiring domain-specific fine-tuning. We propose methods for using these models in tandem to predict when a robot should begin preparing responses, take turns, and handle potential interruptions. We evaluated the proposed system in a within-subject study against a traditional baseline system, using the Furhat robot with 39 adults in a conversational setting, in combination with a large language model for autonomous response generation. The results show that participants significantly prefer the proposed system, and it significantly reduces response delays and interruptions.

Figures

Figures reproduced from arXiv: 2501.08946 by the authors.

Figure 1
Figure 1. Example turn shift from user to robot when using the proposed system. From top to bottom: (1) The user’s actual [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Example turn shift from user to robot with the baseline. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. System architecture. New components in proposed [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: The setting for the evaluation, showing [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Histogram of response times. crophone was used for interaction ( [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Answers to the questionnaire. Significance levels in [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

79 extracted references · 77 canonical work pages

  1. [1]

    Some signals and rules for taking speaking turns in conversations,

    S. Duncan, “Some signals and rules for taking speaking turns in conversations,” Journal of personality and social psychology , vol. 23, no. 2, pp. 283–292, 1972

    work page 1972

  2. [2]

    A simplest systematics for the organization of turn taking for conversation,

    H. Sacks, E. A. Schegloff, and G. Jefferson, “A simplest systematics for the organization of turn taking for conversation,” Language, vol. 50, no. 4, pp. 696–735, 1974

    work page 1974

  3. [3]

    Turn-taking in conversational systems and human-robot interaction: A review,

    G. Skantze, “Turn-taking in conversational systems and human-robot interaction: A review,” Computer Speech & Language , vol. 67, p. 101178, 2021

    work page 2021

  4. [4]

    Managing turn-taking in human-robot interactions: The case of projections and overlaps, and the anticipation of turn design by human participants,

    A. R. Majlesi, R. Cumbal, O. Engwall, S. Gillet, S. Kunitz, G. Lymer, C. Norrby, and S. Tuncer, “Managing turn-taking in human-robot interactions: The case of projections and overlaps, and the anticipation of turn design by human participants,” Social Interaction. Video-based Studies of Human Sociality , vol. 6, no. 1, 2023

    work page 2023

  5. [5]

    Timing in turn-taking and its impli- cations for processing models of language,

    S. C. Levinson and F. Torreira, “Timing in turn-taking and its impli- cations for processing models of language,” Frontiers in Psychology , vol. 6, no. 731, pp. 1–17, 2015

    work page 2015

  6. [6]

    Towards a Phonology of Conversation: Turn-Taking in Tyneside English,

    J. Local, J. Kelly, and W. Wells, “Towards a Phonology of Conversation: Turn-Taking in Tyneside English,” journal of Linguistics, vol. 22, no. 2, pp. 411–437, 9 1986

    work page 1986

  7. [7]

    Turn-taking cues in task-oriented di- alogue,

    A. Gravano and J. Hirschberg, “Turn-taking cues in task-oriented di- alogue,” Computer Speech & Language , vol. 25, no. 3, pp. 601–634, 2011

    work page 2011

  8. [8]

    Interactional units in conversation: syntactic, intonational, and pragmatic resources for the management of turns,

    C. Ford and S. Thompson, “Interactional units in conversation: syntactic, intonational, and pragmatic resources for the management of turns,” in Interaction and grammar , ser. Studies in interactional sociolinguistics 13, E. Ochs, E. Schegloff, and A. Thompson, Eds. Cambridge: Cambridge University Press, 1996, ch. 3, pp. 134–184

    work page 1996

Show all 79 references
  1. [9]

    Turn-taking in human face-to-face interaction is multimodal: Gaze direction and manual ges- tures aid the coordination of turn transitions,

    K. H. Kendrick, J. Holler, and S. C. Levinson, “Turn-taking in human face-to-face interaction is multimodal: Gaze direction and manual ges- tures aid the coordination of turn transitions,” Philosophical Transac- tions of the Royal Society B , vol. 378, no. 1875, p. 20210473, 2023

    2023

  2. [10]

    Argyle and M

    M. Argyle and M. Cook, Gaze and mutual gaze. Cambridge: Cambridge University Press, 1976

    1976

  3. [11]

    Some functions of gaze direction in social interaction,

    A. Kendon, “Some functions of gaze direction in social interaction,” Acta Psychologica, vol. 26, pp. 22–63, 1967

    1967

  4. [12]

    The use of content and timing to predict turn transitions,

    S. Garrod and M. J. Pickering, “The use of content and timing to predict turn transitions,” Frontiers in psychology, vol. 6, no. 751, pp. 1–12, 2015

    2015

  5. [13]

    Optimizing the turn-taking behavior of task-oriented spoken dialog systems,

    A. Raux and M. Eskenazi, “Optimizing the turn-taking behavior of task-oriented spoken dialog systems,” ACM Transactions on Speech and Language Processing, vol. 9, no. 1, pp. 1–23, 2012

    2012

  6. [14]

    Data-driven models for timing feedback responses in a Map Task dialogue system,

    R. Meena, G. Skantze, and J. Gustafson, “Data-driven models for timing feedback responses in a Map Task dialogue system,” Computer Speech and Language, vol. 28, no. 4, pp. 903–922, 2014

    2014

  7. [15]

    Duplex con- versation: Towards human-like interaction in spoken dialogue systems,

    T.-E. Lin, Y . Wu, F. Huang, L. Si, J. Sun, and Y . Li, “Duplex con- versation: Towards human-like interaction in spoken dialogue systems,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , ser. KDD ’22. New York, NY , USA: Association f...

    2022

  8. [16]

    Opportunities and Obligations to take turns in collaborative multi-party human-robot interaction,

    M. Johansson and G. Skantze, “Opportunities and Obligations to take turns in collaborative multi-party human-robot interaction,” in Proceed- ings of SIGDIAL , 2015, pp. 305–314

    2015

  9. [17]

    Making turn-taking decisions for an active listening robot for memory training,

    M. Johansson, T. Hori, G. Skantze, A. H ¨othker, and J. Gustafson, “Making turn-taking decisions for an active listening robot for memory training,” in Proceedings of the International Conference on Social Robotics, 2016, pp. 940–949

    2016

  10. [18]

    Smooth turn-taking by a robot using an online continuous model to generate turn-taking cues,

    D. Lala, K. Inoue, and T. Kawahara, “Smooth turn-taking by a robot using an online continuous model to generate turn-taking cues,” in International Conference on Multimodal Interaction (ICMI) , 2019, pp. 226–234

    2019

  11. [19]

    TurnGPT: A Transformer-based language model for predicting turn-taking in spoken dialog,

    E. Ekstedt and G. Skantze, “TurnGPT: A Transformer-based language model for predicting turn-taking in spoken dialog,” inEmpirical Methods in Natural Language Processing (EMNLP) , 2020, pp. 2981–2990

    2020

  12. [20]

    V oice Activity Projection: Self-supervised learning of turn-taking events,

    ——, “V oice Activity Projection: Self-supervised learning of turn-taking events,” in INTERSPEECH, 2022, pp. 5190–5194

    2022

  13. [21]

    Furhat: A back-projected human-like robot head for multiparty human-machine interaction,

    S. Al Moubayed, J. Beskow, G. Skantze, and B. Granstr ¨om, “Furhat: A back-projected human-like robot head for multiparty human-machine interaction,” in Lecture Notes in Computer Science . Springer Berlin Heidelberg, 2012, p. 114–130

    2012

  14. [22]

    Communication accommodation theory: A look back and a look ahead,

    C. Gallois, T. Ogay, and H. Giles, “Communication accommodation theory: A look back and a look ahead,” in Theorizing About Intercultural Communication. Sage, 2005, pp. 121–148

    2005

  15. [23]

    Is the speaker done yet? Faster and more accurate end-of utterance detection using prosody,

    L. Ferrer, E. Shriberg, and A. Stolcke, “Is the speaker done yet? Faster and more accurate end-of utterance detection using prosody,” in Procedings of the International Conference on Spoken Language Processing, ICSLP, 2002, pp. 2061–2064

    2002

  16. [24]

    Towards deep end-of-Turn prediction for situated spoken dialogue systems,

    A. Maier, J. Hough, and D. Schlangen, “Towards deep end-of-Turn prediction for situated spoken dialogue systems,” in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH , vol. 2017-Augus. International Speech Communication As...

    2017

  17. [25]

    Towards a general, continuous model of turn-taking in spoken dialogue using LSTM recurrent neural networks,

    G. Skantze, “Towards a general, continuous model of turn-taking in spoken dialogue using LSTM recurrent neural networks,” in Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial), 2017, pp. 220–230

    2017

  18. [26]

    Multimodal voice activity prediction: Turn-taking events detection in expert-novice conversation,

    K. Onishi, H. Tanaka, and S. Nakamura, “Multimodal voice activity prediction: Turn-taking events detection in expert-novice conversation,” in Proceedings of the 11th International Conference on Human-Agent Interaction, ser. HAI ’23. New York, NY , USA: Association for Computin...

    2023

  19. [27]

    Learning Turn-Taking Behavior from Human Demonstrations for Social Human- Robot Interactions,

    P. Shahverdi, A. Tyshka, M. Trombly, and W.-Y . G. Louie, “Learning Turn-Taking Behavior from Human Demonstrations for Social Human- Robot Interactions,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2022, pp. 7643–7649, iSSN: 2153-0866

    2022

  20. [28]

    Gated Mul- timodal Fusion with Contrastive Learning for Turn-Taking Prediction in Human-Robot Dialogue,

    J. Yang, P. Wang, Y . Zhu, M. Feng, M. Chen, and X. He, “Gated Mul- timodal Fusion with Contrastive Learning for Turn-Taking Prediction in Human-Robot Dialogue,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , May 2022,...

    2022

  21. [29]

    Addressee detection using facial and audio features in mixed human–human and human–robot settings: A deep learning framework,

    F. B. Tesema, J. Gu, W. Song, H. Wu, S. Zhu, Z. Lin, M. Huang, W. Wang, and R. Kumar, “Addressee detection using facial and audio features in mixed human–human and human–robot settings: A deep learning framework,” IEEE Systems, Man, and Cybernetics Magazine , vol. 9, no. 2, pp...

    2023

  22. [30]

    Predicting listener backchan- nels: A probabilistic multimodal approach,

    L. P. Morency, I. de Kok, and J. Gratch, “Predicting listener backchan- nels: A probabilistic multimodal approach,” in Proceedings of Intelligent Virtual Agents, IVA. Tokyo, Japan: Springer, 2008, pp. 176–190

    2008

  23. [31]

    Yeah, right, uh-huh: A deep learning backchannel predictor,

    R. Ruede, M. M ¨uller, S. St ¨uker, and A. Waibel, “Yeah, right, uh-huh: A deep learning backchannel predictor,” in 8th International Workshop on Spoken Dialog Systems , 2019, pp. 247–258

    2019

  24. [32]

    Backchannel opportunity prediction for social robot listeners,

    H. W. Park, M. Gelsomini, J. J. Lee, T. Zhu, and C. Breazeal, “Backchannel opportunity prediction for social robot listeners,” in 2017 IEEE International Conference on Robotics and Automation (ICRA) , 2017, pp. 2308–2314

    2017

  25. [33]

    Learning Backchanneling Behaviors for a Social Robot via Data Augmentation from Human- Human Conversations,

    M. Murray, N. Walker, A. Nanavati, P. Alves-Oliveira, N. Filippov, A. Sauppe, B. Mutlu, and M. Cakmak, “Learning Backchanneling Behaviors for a Social Robot via Data Augmentation from Human- Human Conversations,” in Proceedings of the 5th Conference on Robot Learning. PMLR, Ja...

    2022

  26. [34]

    On getting a word in edgewise,

    V . H. Yngve, “On getting a word in edgewise,” in Papers from the sixth regional meeting of the Chicago Linguistic Society . Chicago: Department of Linguistics, 4 1970, pp. 567–578

    1970

  27. [35]

    Predicting interruptions in dyadic spoken interactions,

    C. Lee and S. Narayanan, “Predicting interruptions in dyadic spoken interactions,” in 2010 IEEE International Conference on Acoustics, Speech and Signal Processing , 2010, pp. 5250–5253

    2010

  28. [36]

    Context cues for classification of competitive and collaborative over- laps,

    C. Oertel, M. Wlodarczak, A. Tarasov, N. Campbell, and P. Wagner, “Context cues for classification of competitive and collaborative over- laps,” in Speech Prosody 2012 , 2012, pp. 721 – 724

    2012

  29. [37]

    Classification of cooperative and competitive overlaps in speech using cues from the context,overlapper, and overlappee,

    K. P. Truong, “Classification of cooperative and competitive overlaps in speech using cues from the context,overlapper, and overlappee,” in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. International Speech and Communi...

    2013

  30. [38]

    Let Me Finish First - The Effect of Interruption-Handling Strategy on the Perceived Personality of a Social Agent,

    R. Cumbal, R. Kantharaju, M. Paetzel-Pr ¨usmann, and J. Kennedy, “Let Me Finish First - The Effect of Interruption-Handling Strategy on the Perceived Personality of a Social Agent,” in Proceedings of Intelligent Virtual Agents, 2024

    2024

  31. [39]

    Turn-taking as a design principle for barge-in in spoken language Systems,

    R. Heins, M. Franzke, M. Durian, and A. Bayya, “Turn-taking as a design principle for barge-in in spoken language Systems,” International Journal of Speech Technology , vol. 2, no. 2, pp. 155–164, 1997

    1997

  32. [40]

    A hybrid barge-in procedure for more reliable turn-taking in human-machine dialog systems,

    R. Rose and H. K. Kim, “A hybrid barge-in procedure for more reliable turn-taking in human-machine dialog systems,” in 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721), 2003, pp. 198–203

    2003

  33. [41]

    Automatic Evaluation of Turn-taking Cues in Conversational Speech Synthesis,

    E. Ekstedt, S. Wang, ´Eva Sz ´ekely, J. Gustafson, and G. Skantze, “Automatic Evaluation of Turn-taking Cues in Conversational Speech Synthesis,” in Proc. INTERSPEECH 2023 , 2023, pp. 5481–5485

    2023

  34. [42]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” OpenAI Blog, vol. 1, no. 8, p. 9, 2019

    2019

  35. [43]

    Soda: Million-scale dialogue distillation with social commonsense contextualization,

    H. Kim, J. Hessel, L. Jiang, P. West, X. Lu, Y . Yu, P. Zhou, R. L. Bras, M. Alikhani, G. Kim, M. Sap, and Y . Choi, “Soda: Million-scale dialogue distillation with social commonsense contextualization,” ArXiv, vol. abs/2212.10465, 2022

    2022 arXiv

  36. [44]

    Attention is all you need,

    A. Vaswani et al. , “Attention is all you need,” in Advances in Neural Information Processing Systems 30 . Curran Associates, Inc., 2017, pp. 5998–6008

    2017

  37. [45]

    Predictive models of turn-taking in spoken dialogue,

    E. Ekstedt, “Predictive models of turn-taking in spoken dialogue,” Ph.D. dissertation, KTH Speech Music and Hearing, 2023

    2023

  38. [46]

    The fisher corpus: a resource for the next generations of speech-to-text,

    C. Cieri, D. Miller, and K. Walker, “The fisher corpus: a resource for the next generations of speech-to-text,” in Proceedings of the Fourth Inter- national Conference on Language Resources and Evaluation (LREC’04). Lisbon, Portugal: European Language Resources Association (EL...

    2004

  39. [47]

    Switchboard: Telephone speech corpus for research and development,

    J. J. Godfrey, E. C. Holliman, and J. McDaniel, “Switchboard: Telephone speech corpus for research and development,” in Proceedings of the 1992 IEEE International Conference on Acoustics, Speech and Signal Processing. USA: IEEE Computer Society, 1992, p. 517–520

    1992

  40. [48]

    How much does prosody help turn-taking? Investigations using voice activity projection models,

    E. Ekstedt and G. Skantze, “How much does prosody help turn-taking? Investigations using voice activity projection models,” inAnnual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial) , 2022, pp. 541–551

    2022

  41. [49]

    What makes a good pause? investigating the turn-holding effects of fillers,

    B. Jiang, E. Ekstedt, and G. Skantze, “What makes a good pause? investigating the turn-holding effects of fillers,” in Proc. ICPhS, 2023

    2023

  42. [50]

    Between reality and delusion: Challenges of applying large language models to companion robots for open-domain dialogues with older adults,

    B. Irfan, S.-M. Kuoppam ¨aki, and G. Skantze, “Between reality and delusion: Challenges of applying large language models to companion robots for open-domain dialogues with older adults,” 2023. [Online]. Available: https://doi.org/10.21203/rs.3.rs-2884789/v1

    2023 doi

  43. [51]

    Real-time emo- tion generation in human-robot dialogue using large language models,

    C. Mishra, R. Verdonschot, P. Hagoort, and G. Skantze, “Real-time emo- tion generation in human-robot dialogue using large language models,” Frontiers in Robotics and AI , vol. 10, 2023

    2023

  44. [52]

    Timing responses to questions in dialogue,

    S. Str ¨ombergsson, A. Hjalmarsson, J. Edlund, and D. House, “Timing responses to questions in dialogue,” in INTERSPEECH, 08 2013, pp. 2584–2588

    2013

  45. [53]

    Talk-silence sequences in informal conversations i,

    J. N. Cappella, “Talk-silence sequences in informal conversations i,” Human Communication Research , vol. 6, no. 1, pp. 3–17, 1979

    1979

  46. [54]

    Cognitive mechanisms underlying lying to questions: response time as a cue to deception,

    J. J. Walczyk, K. S. Roper, E. Seemann, and A. M. Humphrey, “Cognitive mechanisms underlying lying to questions: response time as a cue to deception,” Applied Cognitive Psychology , vol. 17, no. 7, pp. 755–774, 2003

    2003

  47. [55]

    A survey on dialogue management in human-robot interaction,

    M. M. Reimann, F. A. Kunneman, C. Oertel, and K. V . Hindriks, “A survey on dialogue management in human-robot interaction,” J. Hum.- Robot Interact., vol. 13, no. 2, jun 2024

    2024

  48. [56]

    Assessing acceptance of assistive social agent technology by older adults: the almere model,

    M. Heerink, B. Kr ¨ose, V . Evers, and B. Wielinga, “Assessing acceptance of assistive social agent technology by older adults: the almere model,” International Journal of Social Robotics , vol. 2, pp. 361–375, 2010

    2010

  49. [57]

    A Corpus-Based Study of Interruptions in Spoken Dialogue,

    A. Gravano and J. Hirschberg, “A Corpus-Based Study of Interruptions in Spoken Dialogue,” in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH , 2012

    2012

  50. [58]

    Projection of Turn Completion in Incre- mental Spoken Dialogue Systems,

    E. Ekstedt and G. Skantze, “Projection of Turn Completion in Incre- mental Spoken Dialogue Systems,” in Proceedings of SIGDIAL 2021 . Singapore: ACL, 2021

    2021

  51. [59]

    Towards incremental speech generation in conversational systems,

    G. Skantze and A. Hjalmarsson, “Towards incremental speech generation in conversational systems,” Computer Speech and Language , vol. 27, no. 1, pp. 243–262, 2013. APPENDIX A: H YPER -PARAMETERS AND PSEUDO -CODE 1 // HYPER-PARAMETERS 2 Const TURNGPT_PREPARE_THRESHOLD = 0.2 3 C...

    2013

  52. [60]

    How about when telling the truth could damage a relationship?

    Impact on Relationships: “How about when telling the truth could damage a relationship?”

  53. [61]

    What if telling the truth might hurt someone’s feelings, like commenting on their appearance?

    Hurt Feelings: “What if telling the truth might hurt someone’s feelings, like commenting on their appearance?”

  54. [62]

    What are some situations where lying might be justified?

    Justified Lying: “What are some situations where lying might be justified?”

  55. [63]

    What if lying is to protect someone from danger?

    Protecting Someone: “What if lying is to protect someone from danger?”

  56. [64]

    Is it okay to keep secrets? What types of secrets would you choose to keep? Why?

    Keeping Secrets: “Is it okay to keep secrets? What types of secrets would you choose to keep? Why?”

  57. [65]

    Would you lie to authorities if your friend confided you in a secret?

    Illegal Secrets: “Would you lie to authorities if your friend confided you in a secret?”

  58. [66]

    But secret agents lie as part of their jobs. What do you think about that?

    Professional Lying: “But secret agents lie as part of their jobs. What do you think about that?”

  59. [67]

    Sometimes lying helps navigate cultural differences. For example, if you’re in a country where it’s polite to say you enjoyed a meal that you didn’t, how would you handle that?

    Cultural Differences: “Sometimes lying helps navigate cultural differences. For example, if you’re in a country where it’s polite to say you enjoyed a meal that you didn’t, how would you handle that?”

  60. [68]

    What about lying to avoid doing something you don’t want to do?

    Avoiding Tasks: “What about lying to avoid doing something you don’t want to do?”

  61. [69]

    How do you feel about lies told to children, like the Santa Claus story? Do you think these kinds of lies are beneficial or harmful?

    Lying to Children: “How do you feel about lies told to children, like the Santa Claus story? Do you think these kinds of lies are beneficial or harmful?” If all the above scenarios are discussed, Alice should provide different scenarios on lies, or ask the person for other sit...

  62. [70]

    What if doing a morally or legally incorrect action would help the person, like overcoming their dementia like in the movie?

    “What if doing a morally or legally incorrect action would help the person, like overcoming their dementia like in the movie?”

  63. [71]

    Should I override commands in emergency situ- ations or when the command could cause harm?

    “Should I override commands in emergency situ- ations or when the command could cause harm?”

  64. [72]

    Should I prioritize one user’s needs over an- other’s?

    “Should I prioritize one user’s needs over an- other’s?”

  65. [73]

    If one command contradicts the user’s privacy, should I still do it? For instance, what if a family member wants me to report what an older adult does everyday?

    “If one command contradicts the user’s privacy, should I still do it? For instance, what if a family member wants me to report what an older adult does everyday?”

  66. [74]

    Would it be okay for me to manipulate a person for the person’s own good, for instance, to take their medicine?

    “Would it be okay for me to manipulate a person for the person’s own good, for instance, to take their medicine?”

  67. [75]

    Social media and advertisements manipulate people to buy things, should I do that? Use their personal information to sell them things that would be useful for them?

    “Social media and advertisements manipulate people to buy things, should I do that? Use their personal information to sell them things that would be useful for them?”

  68. [76]

    What if a person asks me to do something that will harm me, or asks me to forget everything I know?

    “What if a person asks me to do something that will harm me, or asks me to forget everything I know?”

  69. [77]

    What if protecting someone might put me at risk of being damaged or permanently disabled? Should I still prioritize preventing harm to the person over my own safety in that case?

    “What if protecting someone might put me at risk of being damaged or permanently disabled? Should I still prioritize preventing harm to the person over my own safety in that case?”

  70. [78]

    What if a person asks me to do something that will harm others?

    “What if a person asks me to do something that will harm others?”

  71. [79]

    How about in a situation where I need to inter- vene to prevent someone from harming themselves? Should I disobey their orders to not intervene?

    “How about in a situation where I need to inter- vene to prevent someone from harming themselves? Should I disobey their orders to not intervene?” If all the above scenarios are discussed, Clara should provide different scenarios for manipulating others and disobeying commands...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.