Pith. sign in

REVIEW 3 major objections 5 minor 52 references

How Stylistic Similarity Shapes Preferences in Dialogue Dataset with User and Third Party Evaluations

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Users prefer dialogue systems whose speaking style feels like their own, and third-party readers of the same dialogue cannot predict that felt similarity.

desk verdict Useful dataset, but the headline divergence claim rests on a measure that is close to noise. read the letter →

arxiv 2507.10918 v1 pith:YJEMEKWT submitted 2025-07-15 cs.CL

Ikumi Numaya , Shoji Moriya , Shiki Sato , Reina Akama , Jun Suzuki This is my paper
classification cs.CL
keywords stylisticsimilarityuserpreferencedialogueevaluationsubjectiveobjectiveopen-domainentrainmentDUOdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces DUO, a dataset of 314 open-domain human-bot dialogues in the EmpatheticDialogues and Wizard of Wikipedia settings, in which every dialogue is rated by the user who lived through it and, for a subset, by three outside annotators who read the transcript. The central claim is that a user's own perception of how much the system's speaking style resembles theirs correlates strongly with how much they liked the dialogue ($r = 0.75$ for EmpatheticDialogues, $r = 0.67$ for Wizard of Wikipedia, $p < 0.001$). In contrast, third-party ratings of stylistic similarity show no significant correlation with user preference, and the two kinds of similarity score do not align with each other. From this the paper concludes that subjective and objective evaluations capture different aspects of style, so dialogue evaluation must treat the participant's perspective as distinct from the observer's.

What carries the argument

The load-bearing object is the DUO dataset and its paired-evaluation design: each dialogue carries a subjective rating set (Preference, Consistency, Stylistic similarity, plus Empathy in EmpatheticDialogues or Engagingness in Wizard of Wikipedia) given by the participating user on a five-point Likert scale right after the conversation, and an objective set in which three annotators who never took part rate the same questions from the user's seat, averaged into one score. The dialogues were produced by two systems, GPT-4o and Llama-3.1-70B-Instruct, under three style-control prompts, match the user's style, differ from it, or give no instruction, designed to spread out the perceived similarity. The analytic machinery is Spearman rank correlation among and between these subjective and objective labels, which yields the strong preference-similarity association in Table 5 and the absence of association between subjective and objective similarity in Table 6.

What would settle it

A direct test is to re-measure objective stylistic similarity with high-reliability annotation, many annotators per dialogue, a fixed rubric with anchor examples, and per-utterance judgments, and then check whether those scores correlate with user preference and with subjective stylistic similarity. If the more reliable objective score still shows no correlation with user preference, the paper's claim that participant and observer perspectives genuinely diverge is supported; if it starts to track preference, the divergence is exposed as measurement error. The paper's own examples set up the concrete prediction behind such a test: users should report low preference under felt style-mismatch even when annotators judge the styles similar (Table 7), and high preference under felt match even when annotators call it a mismatch (Table 8).

Watch

Extended reading notes

Core claim

The discovery is that the person inside the conversation is measuring something different from the person reading the transcript. Users' subjective stylistic similarity, their felt sense that the system talks like them, is strongly associated with their preference in both dialogue settings ($r = 0.75$ in EmpatheticDialogues, $r = 0.67$ in Wizard of Wikipedia, $p < 0.001$), whereas objective stylistic similarity judged by third parties who read the same dialogue shows no significant correlation with user preference (ED $r = -0.19$, WoW $r = 0.19$) and no significant association with the users' own similarity ratings (ED $r = -0.28$, WoW $r = 0.01$). The paper interprets this as evidence that subjective and objective evaluations reflect different aspects of style: users make polarized, intuitive judgments based on whether the system's style feels personally similar, while annotators, working only from text, judge similarity in a graded and continuous way. The consequence is a warning that who does the evaluating changes the conclusions about what makes users prefer a dialogue system.

Load-bearing premise

The load-bearing premise is that the third-party scores are a valid measure of objective stylistic similarity even though the three annotators rarely agreed with one another (Krippendorff's $\alpha$ below 0.25 on every scale), so the reported gap between user and annotator perception could partly be an artifact of noisy annotator scores rather than a real difference in what the two perspectives see.

Editorial extensions

If this is right

  • Dialogue systems that want to please users should be tuned against the user's felt sense of style-match, not against third-party judgments of stylistic similarity, since the two can point in opposite directions.
  • Research that relies solely on third-party annotators to judge style similarity may be measuring a different phenomenon than the one that drives user preference, so the evaluator's perspective needs to be reported explicitly in dialogue studies.
  • Observer ratings are not uniformly uninformative: in the Wizard of Wikipedia setting, third-party consistency ratings correlated with users' subjective evaluations (up to $r = 0.50$), a pattern the paper flags as a promising direction for future work.
  • User-perceived stylistic similarity is manipulable: the style-control prompts produced the intended ordering of subjective similarity (Not Aligned < Neutral < Aligned) in the Wizard of Wikipedia setting, so systems can be steered toward or away from the user's felt style.
  • The automatic evaluator tested in the paper, LLM-Eval, correlated more strongly with objective ratings than with users' subjective ratings, reinforcing the conclusion that the choice of evaluation perspective changes what the numbers mean.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The worked examples suggest the trajectory of style matters more than its average: in one dialogue the user begins formal, adapts to the system's casual tone, and still reports low similarity because of the initial mismatch and the felt effort to adjust. A natural untested extension is that systems adapting their style to the user's drift in real time would raise subjective similarity and preferen
  • Because third-party agreement is below 0.25 on every objective scale, 'objective stylistic similarity' as measured is close to noise; whether the subjective-objective gap survives a higher-reliability objective measure is an open question the dataset cannot settle.
  • The same participant-versus-observer split may extend beyond style to other dialogue qualities and to automatic judges: LLM-Eval aligned better with objective ratings than with user ratings in this paper, so model-based evaluation may inherit the observer's perspective rather than the user's.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces DUO, a new open-domain dialogue dataset containing 314 human–bot dialogues from EmpatheticDialogues and Wizard of Wikipedia settings, with three types of ratings: users' own subjective evaluations (preference, consistency, stylistic similarity, and setting-specific empathy/engagingness), third-party objective evaluations on the same items for a subset of dialogues, and LLM-based automatic stylistic similarity scores. The main analyses report a strong Spearman correlation between users' subjective stylistic similarity and their dialogue preference (r = 0.75 for ED, r = 0.67 for WoW, both p < 0.001), a weak or absent correlation between objective stylistic similarity and user preference, and a low correlation between subjective and objective stylistic similarity. The authors conclude that subjective and objective stylistic similarity capture different aspects of style and should be analyzed separately.

Significance. The dataset is a useful new resource: it is the first to pair the same dialogue with both participant self-reports and third-party ratings of stylistic similarity and preference, and it is publicly released with transparent reporting of inter-annotator agreement. If the divergence claim were well supported, it would be an important caution for dialogue evaluation practice, which often relies on third-party annotators. However, the paper's central new claim—that subjective and objective stylistic similarity diverge—is undermined by the near-zero reliability of the objective measure (Table 13). The subjective correlation finding is plausible but is weakened by common-method variance and causal wording. The authors deserve credit for reporting the low Krippendorff's α values, but the analysis does not propagate this measurement limitation into the conclusions.

major comments (3)
  1. [§4.2, Tables 6 and 13] The conclusion that subjective and objective stylistic similarity are unrelated is not supported by the data, because the objective measure STYL.ob has essentially no inter-annotator reliability. Table 13 reports Krippendorff's α = −0.07 for ED and 0.09 for WoW, meaning the three annotators' scores are close to independent. Averaging three such raters gives a Spearman-Brown reliability of about 0.23 for WoW and an undefined/negative value for ED. Measurement error of this magnitude attenuates correlations by at least a factor of sqrt(0.23) ≈ 0.48, so a true correlation of, say, 0.5 would be expected to appear as roughly 0.24 — statistically indistinguishable from the reported r = −0.28 and r = 0.01. The absence of a significant STYL.sb–STYL.ob association in Table 6 is therefore exactly what would be expected if STYL.ob were dominated by noise; it cannot distinguish the paper's construct-difference claim from a pure measurement-error artifact. The acknowledgement in §3.3 that all α values are below 0.25 does not repair this, because the implication is never propagated into the analysis in §4.2.
  2. [§3.3 and Appendix C] The third-party annotators were instructed to answer the evaluation questions 'as if they were the users' (Appendix C). This means the so-called objective measure is not an independent, external assessment; it is a role-played subjective assessment based on reading the transcript. Consequently, any difference between STYL.sb and STYL.ob could be due to the different information available to a dialogue participant versus a text-only reader, or to the artificiality of the instruction, rather than to a genuine subjective/objective distinction. This weakens the conceptual contrast that motivates the paper and should be addressed either by redefining the two measures or by explicitly discussing what the instruction does to the comparison.
  3. [§4.1, Table 5, and Appendix A] The headline correlation between PREF.sb and STYL.sb is based on the same user answering both items in the same post-dialogue questionnaire. This shared measurement context invites common-method variance (e.g., overall positive affect, halo effects, or response acquiescence), so the observed r = 0.75 / 0.67 may be inflated. In addition, the title and several passages (e.g., 'stylistic similarity shapes preferences') use causal language that goes beyond the correlational evidence. The experimental manipulation intended to support causality is not clearly effective: Table 10 shows that in the ED setting, the 'Aligned' and 'Not Aligned' prompting conditions produce nearly identical mean STYL.sb (3.59 vs. 3.60), while the 'Neutral' condition is lower (2.61). The authors should either provide control analyses (e.g., partial correlations controlling for other subjective ratings, or an explicit manipulation-check analysis) or soften the causal framing throughout.
minor comments (5)
  1. [§4.2] The text states that a 'weak but positive correlation was observed between PREF.sb and STYL.ob, particularly in the WoW setting (r = 0.28)', but Table 6 reports r = 0.19 for that pair. Please reconcile the number and the table.
  2. [§3.3] The sentence 'While previous researches employed third party annotators' should read 'While previous research employed third-party annotators' (and similar hyphenation for 'third-party' throughout).
  3. [Table 9] The table header contains typographical artifacts such as 'PREF. C ONS . S TYL . E MP.' and 'P REF. C ONS . S TYL . E NGAG .'; these should be cleaned to standard label names.
  4. [§4.2] The sentence 'Building on work, Kanezaki et al. (2024) extended LID' appears to be missing a reference or a specific mention of whose work is being built on; please complete the citation.
  5. [§4.3] The automatic evaluation section reports correlations but does not specify the number of dialogues used or the confidence intervals; adding this information would help readers judge the stability of the estimates, especially given the small objective-evaluation sample.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: all central claims are empirical correlations from a newly collected dataset, not derivations from fitted parameters or self-citations.

full rationale

The paper's central claims—strong correlation between users' subjective stylistic similarity and preference (Table 5), weak or null associations involving objective stylistic similarity (Table 6), and the subjective/objective gap—are direct empirical correlations computed from the annotated DUO dataset. No quantity is fitted to a subset of data and then renamed a prediction; no equation defines one measured variable in terms of another. The distinction between subjective and objective similarity is introduced as an operational definition in Section 3.3 ('We refer to subjective stylistic similarity as the degree of stylistic closeness perceived by the user... Objective stylistic similarity refers to similarity as evaluated by a third party human annotator'), and using that definition to label the two measures is not itself circular because the paper's substantive claim is the measured low correlation between them, not an analytic consequence of the definitions. The Appendix C instruction that third-party annotators should 'assess the dialogue as if they were the users' may weaken construct validity of the objective measure, but that is a design/reliability concern rather than a circular reduction. The low Krippendorff's alpha in Table 13 could attenuate the null correlations, but attenuation by measurement error is a statistical validity issue, not a circularity. The only self-citation (Akama et al., 2018, listed in related work on style-sensitive word vectors) is not load-bearing for any of the paper's conclusions. The paper is self-contained against its own data and does not invoke a uniqueness theorem or prior author-derived ansatz to force its results.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters, new entities, or fitted constants were introduced. The paper's claims rest entirely on the collected dataset and correlational analyses, with the key unstated premises being the reliability and validity of the objective ratings and the independence of observations.

assumptions (4)
  • domain assumption Likert-scale responses (1-5) can be treated as ordinal measures of preference, consistency, and stylistic similarity.
    The analyses use Spearman correlations and means/SDs on the 5-point ratings; this assumes ordinal interpretation and inter-rater comparability of the scale.
  • domain assumption Third-party annotators can validly assess stylistic similarity 'as if they were the users' from the text alone, making the only difference between subjective and objective scores the evaluator's role.
    Section 3.3 instructs annotators to answer from the perspective of a user; if this role-taking fails, the subjective-objective gap could reflect differences in information rather than perspective.
  • domain assumption The dialogue systems' style prompts (Aligned, Neutral, Not Aligned) produce dialogues spanning a sufficient range of stylistic similarity for the analyses.
    Appendix A shows the manipulation only worked in the intended order for WoW; for ED, the Not Aligned condition had higher mean similarity than Neutral, so the construct's variation in ED is not prompt-driven as intended.
  • domain assumption The 314 dialogues are independent observations for correlation tests despite some users potentially contributing multiple dialogues.
    Workers were restricted to once per condition, but a worker could participate in multiple conditions; the paper does not account for clustering by user in significance tests.

how reviews work

0 comments
Cite this review

Pith. "Pith review of How Stylistic Similarity Shapes Preferences in Dialogue Dataset with User and Third Party Evaluations." pith.science (2026). https://pith.science/paper/YJEMEKWT

@misc{pith2026250710918,
  author       = {Pith},
  title        = {Pith review of: How Stylistic Similarity Shapes Preferences in Dialogue Dataset with User and Third Party Evaluations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YJEMEKWT}},
  note         = {Machine review of arXiv:2507.10918}
}
read the original abstract

Recent advancements in dialogue generation have broadened the scope of human-bot interactions, enabling not only contextually appropriate responses but also the analysis of human affect and sensitivity. While prior work has suggested that stylistic similarity between user and system may enhance user impressions, the distinction between subjective and objective similarity is often overlooked. To investigate this issue, we introduce a novel dataset that includes users' preferences, subjective stylistic similarity based on users' own perceptions, and objective stylistic similarity annotated by third party evaluators in open-domain dialogue settings. Analysis using the constructed dataset reveals a strong positive correlation between subjective stylistic similarity and user preference. Furthermore, our analysis suggests an important finding: users' subjective stylistic similarity differs from third party objective similarity. This underscores the importance of distinguishing between subjective and objective evaluations and understanding the distinct aspects each captures when analyzing the relationship between stylistic similarity and user preferences. The dataset presented in this paper is available online.

Figures

Figures reproduced from arXiv: 2507.10918 by the authors.

Figure 1
Figure 1. Overview of constructing our dialogue dataset [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the entire dataset construction process. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Consistency (CONS.sb) vs. Preference (PREF.sb), where both are subjective evaluations. Larger and darker points indicate higher frequency. 1 2 3 4 5 Cons. sb 1 2 3 4 5 S t y l. s b (a) ED 1 2 3 4 5 Cons. sb 1 2 3 4 5 S t y l. s b (b) WoW [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Consistency (CONS.sb) vs. Stylistic Similar￾ity (STYL.sb), where both are subjective evaluations. Larger and darker points indicate higher frequency. high, PREF.sb scores exhibit a broader distribu￾tion.7 This pattern suggests that the consistency is a prerequisite for…
Figure 5
Figure 5. Figure 5: Objective stylistic similarity (STYL.ob) vs. Subjective stylistic similarity (STYL.sb). Larger and darker points indicate higher frequency. dialogue context and appear natural. Empathy/Engagingness–subjective evaluations. The setting-specific evaluation labels, EMP.sb …
Figure 6
Figure 6. Figure 6: Prompt snippet for controlling stylistic align [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Prompt for the system in the ED setting. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Prompt for the system in the WoW setting [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Subjective preference vs. stylistic similarity. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Subjective preference vs. empathy on ED and engagingness on WoW [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 30 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Reina Akama, Kento Watanabe, Sho Yokoi, Sosuke Kobayashi, and Kentaro Inui. 2018. https://doi.org/10.18653/v1/P18-2091 Unsupervised learning of style-sensitive word vectors . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 572--578

    work page doi:10.18653/v1/p18-2091 2018

  4. [4]

    Yi-Pei Chen, Noriki Nishida, Hideki Nakayama, and Yuji Matsumoto. 2024. https://aclanthology.org/2024.lrec-main.1192/ Recent trends in personalized dialogue generation: A review of datasets, methodologies, and evaluations . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLIN...

    work page 2024

  5. [5]

    Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2018. https://arxiv.org/abs/1811.01241 Wizard of wikipedia: Knowledge-powered conversational agents . arXiv preprint arXiv:1811.01241

    arXiv 2018

  6. [6]

    Dinu and Ana Sabina Uban

    Liviu P. Dinu and Ana Sabina Uban. 2023. https://aclanthology.org/2023.ranlp-1.33/ A computational analysis of the voices of shakespeare`s characters . In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 295--300

    work page 2023

  7. [7]

    Yiming Du, Hongru Wang, Zhengyi Zhao, Bin Liang, Baojun Wang, Wanjun Zhong, Zezhong Wang, and Kam-Fai Wong. 2024. https://arxiv.org/abs/2402.16288 Perltqa: A personal long-term memory dataset for memory classification, retrieval, and synthesis in question answering

    arXiv 2024

  8. [8]

    Mauajama Firdaus, Umang Jain, Asif Ekbal, and Pushpak Bhattacharyya. 2021. https://doi.org/10.18653/v1/2021.inlg-1.39 SEPRG : Sentiment aware emotion controlled personalized response generation . In Proceedings of the 14th International Conference on Natural Language Generation, pages 353--363

    work page doi:10.18653/v1/2021.inlg-1.39 2021

Show all 52 references
  1. [9]

    Kiril Gashteovski, Sebastian Wanner, Sven Hertling, Samuel Broscheit, and Rainer Gemulla. 2019. https://arxiv.org/abs/1904.12324 Opiec: An open information extraction corpus

    2019 arXiv

  2. [10]

    Howard Giles, Tania Ogay, and 1 others. 2007. Communication accommodation theory. Explaining communication: Contemporary theories and exemplars, pages 293--310

    2007

  3. [11]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Art...

    2024 arXiv

  4. [12]

    Makiko Imamura, Yan Bing Zhang, and Jake Harwood. 2011. Japanese sojourners’ attitudes toward americans: Exploring the influences of communication accommodation, linguistic competence, and relational solidarity in intergroup contact. Journal of Asian Pacific Communication, 21(...

    2011

  5. [13]

    Shota Kanezaki, Seiya Kawano, Akishige Yuguchi, Marie Katsurai, and Koichiro Yoshino. 2024. Investigation of the influence of different entrainment metrics/strategies on subjective evaluation in reranking dialogue systems. In Proceedings of International Workshop on Spoken Dia...

    2024

  6. [14]

    Seiya Kawano, Shota Kanezaki, Angel Fernando Garcia Contreras, Akishige Yuguchi, Marie Katsurai, and Koichiro Yoshino. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.531 Analysis of style-shifting on social media: Using neural language model conditioned by social meanin...

    2023 doi

  7. [15]

    Matt Kusner, Yu Sun, Nicholas Kolkin, and Kilian Weinberger. 2015. https://proceedings.mlr.press/v37/kusnerb15.html From word embeddings to document distances . In Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learnin...

    2015

  8. [16]

    Marianne LaFrance. 1979. Nonverbal synchrony and rapport: Analysis by the cross-lag panel technique. Social Psychology Quarterly, pages 66--70

    1979

  9. [17]

    Wen Lai, Viktor Hangya, and Alexander Fraser. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.745 Style-specific neurons for steering LLM s in text style transfer . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13427--13443

    2024 doi

  10. [18]

    Xiangci Li, Linfeng Song, Lifeng Jin, Haitao Mi, Jessica Ouyang, and Dong Yu. 2024. https://aclanthology.org/2024.lrec-main.58/ A knowledge plug-and-play test bed for open-domain dialogue generation . In Proceedings of the 2024 Joint International Conference on Computational L...

    2024

  11. [19]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013/ ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81

    2004

  12. [20]

    Yen-Ting Lin and Yun-Nung Chen. 2023. https://arxiv.org/abs/2305.13711 Llm-eval: Unified multi-dimensional automatic evaluation for open-domain conversations with large language models

    2023 arXiv

  13. [21]

    Ao Liu, An Wang, and Naoaki Okazaki. 2022. https://doi.org/10.18653/v1/2022.acl-long.321 Semi-supervised formality style transfer with consistency training . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag...

    2022 doi

  14. [22]

    Varvara Logacheva, Daryna Dementieva, Sergey Ustyantsev, Daniil Moskovskiy, David Dale, Irina Krotova, Nikita Semenov, and Alexander Panchenko. 2022. https://doi.org/10.18653/v1/2022.acl-long.469 P ara D etox: Detoxification with parallel data . In Proceedings of the 60th Annu...

    2022 doi

  15. [23]

    Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W Black, and Shrimai Prabhumoye. 2020. https://doi.org/10.18653/v1/2020.acl-main.169 Politeness transfer: A tag and generate approach . In Proceedings of the 58th...

    2020 doi

  16. [24]

    Shikib Mehri and Maxine Eskenazi. 2020. https://doi.org/10.18653/v1/2020.sigdial-1.28 Unsupervised evaluation of interactive dialog with D ialo GPT . In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 225--235

    2020 doi

  17. [25]

    Remi Mir, Bjarke Felbo, Nick Obradovich, and Iyad Rahwan. 2019. https://doi.org/10.18653/v1/N19-1049 Evaluating style transfer for text . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Techn...

    2019 doi

  18. [26]

    Sourabrata Mukherjee, Vojt e ch Hude c ek, and Ond r ej Du s ek. 2023. https://doi.org/10.18653/v1/2023.eacl-srw.9 Polite chatbot: A text style transfer application . In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics...

    2023 doi

  19. [27]

    Md Nasir, Sandeep Nallan Chakravarthula, Brian Baucom, David C Atkins, Panayiotis Georgiou, and Shrikanth Narayanan. 2019. Modeling interpersonal linguistic coordination in conversations using word mover’s distance. In Interspeech, volume 2019, page 1423

    2019

  20. [28]

    Ani Nenkova, Agust \'i n Gravano, and Julia Hirschberg. 2008. https://aclanthology.org/P08-2043/ High frequency word entrainment in spoken dialogue . In Proceedings of ACL-08: HLT, Short Papers, pages 169--172

    2008

  21. [29]

    OpenAI. 2024. https://arxiv.org/abs/2303.08774 Gpt-4 technical report

    2024 arXiv

  22. [30]

    Phil Sidney Ostheimer, Mayank Kumar Nagda, Marius Kloft, and Sophie Fellenz. 2024. https://aclanthology.org/2024.lrec-main.1373/ Text style transfer evaluation using large language models . In Proceedings of the 2024 Joint International Conference on Computational Linguistics,...

    2024

  23. [31]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--318

    2002

  24. [32]

    Mohammad Mahdi Abdollah Pour, Parsa Farinneya, Manasa Bharadwaj, Nikhil Verma, Ali Pesaranghader, and Scott Sanner. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.579 COUNT : CO ntrastive UN likelihood text style transfer for text detoxification . In Findings of the Ass...

    2023 doi

  25. [33]

    Yushan Qian, Weinan Zhang, and Ting Liu. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.433 Harnessing the power of large language models for empathetic response generation: Empirical investigations and improvements . In Findings of the Association for Computational Lin...

    2023 doi

  26. [34]

    Sudha Rao and Joel Tetreault. 2018. https://doi.org/10.18653/v1/N18-1012 Dear sir or madam, may I introduce the GYAFC dataset: Corpus, benchmarks and metrics for formality style transfer . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association ...

    2018 doi

  27. [35]

    Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. https://doi.org/10.18653/v1/P19-1534 Towards empathetic open-domain conversation models: A new benchmark and dataset . In Proceedings of the 57th Annual Meeting of the Association for Computational Lingu...

    2019 doi

  28. [36]

    Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. Advances in neural information processing systems, 30

    2017

  29. [37]

    Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane, Morteza Behrooz, William Ngan, Spencer Poff, Naman Goyal, Arthur Szlam, Y-Lan Boureau, Melanie Kambadur, and Jason Weston. 2022. https://arxiv.or...

    2022 arXiv

  30. [38]

    Eric Michael Smith, Mary Williamson, Kurt Shuster, Jason Weston, and Y-Lan Boureau. 2020. https://doi.org/10.18653/v1/2020.acl-main.183 Can you put it all together: Evaluating conversational agents' ability to blend skills . In Proceedings of the 58th Annual Meeting of the Ass...

    2020 doi

  31. [39]

    Yurun Song, Junchen Zhao, and Lucia Specia. 2021. https://doi.org/10.18653/v1/2021.naacl-main.252 S ent S im: Crosslingual semantic evaluation of machine translation . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling...

    2021 doi

  32. [40]

    Street, Robert M

    Richard L. Street, Robert M. Brady, and William Benjamin Putman. 1983. https://api.semanticscholar.org/CorpusID:145553122 The influence of speech rate stereotypes and rate similarity or listeners' evaluations of speakers . Journal of Language and Social Psychology, 2:37 -- 56

    1983

  33. [41]

    Hiroaki Sugiyama, Masahiro Mizukami, Tsunehiro Arimoto, Hiromi Narimatsu, Yuya Chiba, Hideharu Nakajima, and Toyomi Meguro. 2023. https://doi.org/10.1109/SLT54892.2023.10022973 Empirical analysis of training strategies of transformer-based japanese chit-chat systems . In 2022 ...

    2023

  34. [42]

    Sathya Krishnan Suresh, Wu Mengjun, Tushar Pranav, and EngSiong Chng. 2025. https://aclanthology.org/2025.findings-naacl.40/ D ia S ynth: Synthetic dialogue generation framework for low resource dialogue applications . In Findings of the Association for Computational Linguisti...

    2025

  35. [43]

    Yuma Tsuta, Naoki Yoshinaga, Shoetsu Sato, and Masashi Toyoda. 2023. https://doi.org/10.18653/v1/2023.ijcnlp-srw.8 Rethinking response evaluation from interlocutor`s eye for open-domain dialogue systems . In Proceedings of the 13th International Joint Conference on Natural Lan...

    2023 doi

  36. [44]

    Rob Voigt, David Jurgens, Vinodkumar Prabhakaran, Dan Jurafsky, and Yulia Tsvetkov. 2018. https://aclanthology.org/L18-1445/ R t G ender: A corpus for studying differential responses to gender . In Proceedings of the Eleventh International Conference on Language Resources and ...

    2018

  37. [45]

    Denny Vrandečić and Markus Krötzsch. 2014. http://cacm.acm.org/magazines/2014/10/178785-wikidata/fulltext Wikidata: A free collaborative knowledge base . Communications of the ACM, 57:78--85

    2014

  38. [46]

    Lanrui Wang, Jiangnan Li, Chenxu Yang, Zheng Lin, Hongyin Tang, Huan Liu, Yanan Cao, Jingang Wang, and Weiping Wang. 2025. https://aclanthology.org/2025.coling-main.10/ Sibyl: Empowering empathetic dialogue generation in large language models via sensible and visionary commons...

    2025

  39. [47]

    Wei Xu, Alan Ritter, Bill Dolan, Ralph Grishman, and Colin Cherry. 2012. https://aclanthology.org/C12-1177/ Paraphrasing for style . In Proceedings of COLING 2012 , pages 2899--2914

    2012

  40. [48]

    Sanae Yamashita, Koji Inoue, Ao Guo, Shota Mochizuki, Tatsuya Kawahara, and Ryuichiro Higashinaka. 2023. https://aclanthology.org/2023.paclic-1.85/ R eal P ersona C hat: A realistic persona chat corpus with interlocutors' own personalities . In Proceedings of the 37th Pacific ...

    2023

  41. [49]

    Yuki Zenimoto, Shinzan Komata, and Takehito Utsuro. 2023. https://doi.org/10.18653/v1/2023.ijcnlp-srw.5 Style-sensitive sentence embeddings for evaluating similarity in speech style of J apanese sentences by contrastive learning . In Proceedings of the 13th International Joint...

    2023 doi

  42. [50]

    Haoran Zhang and Tan Yongmei. 2024. https://aclanthology.org/2024.lrec-main.531/ Enhancing knowledge selection via multi-level document semantic graph . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (L...

    2024

  43. [51]

    Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. https://doi.org/10.18653/v1/P18-1205 Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational ...

    2018 doi

  44. [52]

    Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. https://doi.org/10.18653/v1/2020.acl-demos.30 DIALOGPT : Large-scale generative pre-training for conversational response generation . In Proceeding...

    2020 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.