Pith. sign in

REVIEW 3 major objections 6 minor 26 references

Real-World En Call Center Transcripts Dataset with PII Redaction

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper releases CallCenterEN, a corpus of 91,706 real-world English call center transcripts with all PII redacted, and claims it is the largest open-source release of its kind.

desk verdict A plausibly valuable call-center transcript dataset whose central PII-redaction claim is unsupported, but the authors are honest about the QA limits. read the letter →

arxiv 2507.02958 v1 pith:HXTO4DAQ submitted 2025-06-30 cs.CL

Ha Dao , Gaurav Chawla , Raghu Banda , Caleb DeLeeuw This is my paper · ORCID
classification cs.CL
keywords callcentertranscriptsconversationalAIPIIredactionASRcorpuscustomersupportsalesdialoguedatasetreleaseEnglishtelephony
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CallCenterEN is a publicly released dataset of 91,706 real-world English conversations between call center agents and customers, corresponding to about 10,448 hours of source audio and tagged by domain, accent, and call direction. The paper's central claim is that this is the largest open-source release of call center transcript data of its kind, filling a gap between older telephone corpora, clean read-speech corpora, and much smaller existing call-center datasets. If the claim is right, researchers get a large unscripted, task-oriented corpus for training and benchmarking customer-support and sales AI, with word-level timestamps and confidence scores. The transcripts are raw and unlabeled, so users add their own labels for intent, sentiment, outcome, or other tasks. The release stands on the assertion that PII was rigorously removed through automated detection plus manual review, while formal human QA covered only 0.1% of the data and reported a 3.87% WER.

What carries the argument

The central object is the dataset itself, released under CC BY-NC 4.0. The mechanism that makes the release legally possible is the PII redaction pipeline: automated entity detection covering more than forty listed categories across personal, financial, government, medical, technical, temporal, and locational identifiers, followed by manual review. The corpus structure is also load-bearing: each JSON file begins with the redacted conversation and metadata, then word-level timestamps and confidence scores, which is what lets it serve both ASR-style error analysis and dialogue-modeling tasks. The distributional skew toward Medicare inbound calls is part of how the corpus is characterized and should be read as a feature of its domain concentration.

What would settle it

An audit would compare a random sample of the pre-redaction source transcripts or re-listened audio against the public JSON and count surviving PII spans; a single surviving phone number, email address, social security number, or account number in the released files would falsify the claim of rigorous removal.

Watch

Extended reading notes

Core claim

On its own terms, the contribution is a corpus rather than a tested model: CallCenterEN delivers 91,706 JSON transcripts of inbound and outbound BPO calls, dominated by Medicare inbound calls at 67.1%, with Indian, Filipino, and American agent accents and US-based customers. Each transcript carries the redacted dialogue, an overall ASR confidence score between 86% and 98%, audio duration, and per-word timestamps with per-word confidence. The paper asserts that previous resources are either outdated non-commercial telephone speech, read studio speech, or small hub datasets, so this release is the largest of its kind and shifts call center transcripts from a scarce to a plentiful commodity for non-commercial research.

Load-bearing premise

The load-bearing premise is that the PII redaction caught every listed identifier in all 91,706 transcripts, even though formal human review covered only 0.1% and no redaction error rate is reported.

Editorial extensions

If this is right

  • A large slice of the 91,706 conversations is concentrated in Medicare customer service, so models built on CallCenterEN will be strongest in insurance and healthcare support dialogue.
  • Researchers can use the word-level confidence scores to downweight uncertain tokens when fine-tuning language models or computing ASR error trends.
  • The non-commercial license lets academic labs train and release open models on the transcripts, while excluding direct commercial product training.
  • Because audio is withheld, the corpus supports transcript-only tasks such as intent detection, summarization, sentiment analysis, and outcome classification, but not speech-signal research on these exact calls.
  • The stated WER of 3.87% on a 0.1% sample, if representative, makes the corpus usable as a benchmark for customer-support agents and AI agent comparisons.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because redaction recall is not measured, an independent audit that re-inspects pre-redaction audio or transcripts against the public JSON would be the natural next step before treating the compliance claim as established.
  • Editorial inference: the redaction removes dates, times, durations, money amounts, and number sequences, so researchers should not expect to recover exact timing, pricing, or policy numbers from the public corpus.
  • Editorial inference: the heavy Medicare and insurance concentration suggests framing CallCenterEN as an insurance-support corpus first; users wanting balanced multi-domain coverage would need to subsample.
  • Editorial inference: the per-word confidence scores could support a testable extension the paper does not pursue: confidence-weighted language model fine-tuning to compensate for ASR noise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces CallCenterEN, a corpus of 91,706 real-world English call center transcripts (corresponding to 10,448 hours of audio) with PII redaction, released publicly under CC BY-NC 4.0. The authors describe sourcing the data from roughly 9-10 BPO centers, transcribing it with AssemblyAI (word-level timestamps and confidence scores), manually reviewing a claimed 0.1% sample for quality, and redacting PII via automated detection plus manual review. The central claims are that this is the largest open-source call center transcript dataset of its kind and that all personally identifiable information has been rigorously removed, making the data compliant with privacy regulations and safe to share.

Significance. If the dataset is genuinely PII-safe and as large as claimed, it would be a valuable community resource for conversational AI research in customer support and sales, filling a documented gap where existing call-center datasets are typically small (under 1,000 examples). The paper's strengths include the public release, the large scale, the real-world provenance, the diversity of accents and domains, and the inclusion of word-level confidence scores and timestamps. The authors also include an explicit limitations section, which is commendable. However, the paper's most consequential claim—that all PII has been removed and that the dataset is legally shareable—is asserted but never evaluated, so the release's primary value is contingent on an unverified assumption. The lack of any redaction recall/precision measurement, adversarial audit, or residual-PII check is a load-bearing omission.

major comments (3)
  1. [Sections 6 and 11] The categorical assertion that "All personally identifiable information (PII) has been rigorously removed" (Abstract, Section 11) and that the dataset "complies with relevant privacy regulations" (Section 6) is a safety claim, not a quality metric, and it is not supported by any quantitative evidence in the paper. No redaction recall, redaction precision, residual-PII counts, or adversarial re-identification audit is reported anywhere; the only quantitative QA disclosed is the WER table (Table 4), which says nothing about PII. Since the dataset is publicly downloadable, a single unredacted phone number, SSN, credit-card number, or medical-record identifier in any of the 91,706 files would invalidate the central claim and create real privacy and legal exposure. The authors should either (i) report an entity-level evaluation of the redaction pipeline on a held-out annotated random sample, including recall and precision per PII category, and (ii) provide a clear statement of the manual-review coverage and inter-annotator agreement, or alternatively scale back the "all PII removed" language to a documented residual-risk statement.
  2. [Section 4, Section 7, Table 4] The paper claims that "0.1% of the dataset was reviewed by human experts" (Abstract and Section 4) and that "in total, we manually tested 0.1% of the total dataset" (Section 7), but Table 4 lists exactly 30 QA files. 30/91,706 is approximately 0.033%, not 0.1%. If Table 4 is only a subset of the QA set, the total number of QA files must be stated; if it is the entire QA set, then the 0.1% claim is internally inconsistent. Furthermore, Section 12 concedes that this sample is "below a statistically significant sample size." Consequently, the dataset-level WER of 3.87% and accuracy of 96.131% in Table 1 are not established for the corpus; they should be reported as descriptive statistics of the QA sample with a confidence interval, and the text should not imply that they characterize the entire dataset.
  3. [Sections 4 and 11] The legal and ethical basis for the release rests on assertions that the data was "collected through authorized partnerships with BPO providers" and that the partnerships were "originally established for commercial data collection purposes" (Section 4). No supporting documentation is provided: there is no evidence of data-use agreements, consent for downstream redistribution under CC BY-NC, or ethical review. Given that the data consists of real-world call recordings (including 61,513 Medicare inbound calls, which are highly sensitive), these provenance and consent details are load-bearing for the claim of compliance with CCPA and DPDP 2023. The authors should provide a data sheet or supplementary statement documenting the legal basis for sharing the transcripts, even if details must be anonymized to protect business agreements.
minor comments (6)
  1. [Title and Abstract] The dataset name is written inconsistently: the title uses "91706" while the abstract and main text use "91,706"; similarly "10448" in the abstract should be "10,448" for readability.
  2. [Section 5, Figure 1] Figure 1's pie and bar charts are not discussed with enough detail in the text; the caption states that Medicare inbound calls are 67.1% of the total, but the figure itself is only referenced in passing. Given that this one domain dominates the dataset, the authors should discuss the implications for the claimed general-purpose call-center coverage.
  3. [Section 8 and Table 1] The intended-use list is duplicated nearly verbatim in Table 1 and Section 8; one of the two should be condensed to avoid redundancy and to make the document easier to read.
  4. [Section 3] The claim that Hugging Face call-center datasets are "nearly always fewer than 1000 observations" is supported only by a link to a search query; the authors should provide a specific comparison table or at least a few concrete dataset names and sizes to substantiate the claim.
  5. [Section 12] The author-role and contact section lists one author's email domain as @columbia.edu while the author list identifies that person as an independent researcher; please clarify the affiliation or remove the outdated address.
  6. [Section 13, Reference [14]] The citation to Hu et al. on Abel Noser data is tangential to the point about dataset availability; a citation directly about speech or dialogue dataset releases would be more appropriate.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper makes no predictive or fitted derivation whose output is defined in terms of its input.

full rationale

This paper is a dataset release and description, not a derivation or modeling claim. There is no equation, fitted parameter, or prediction that is defined in terms of another quantity it is supposed to explain. The reported 3.87% WER is a direct measurement on a 30-file human-reviewed sample computed from manual reference transcripts, which is an external ground truth rather than a quantity constructed from the ASR output. The claim that the dataset is the largest release of its kind is a comparative, externally checkable statement, not a self-referential one. The PII-redaction assertion is an empirical safety claim, and while Section 12 honestly concedes that only 0.1% of the dataset was formally human-reviewed and that this is below a statistically significant sample size, that is a limitation in validation coverage, not a circularity. No load-bearing argument in the paper reduces to its own input, and no cited result is being used tautologically. The paper's central value is the public availability of the transcripts themselves, which stands independent of any derivation chain.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new mathematical constants or invented entities. Its central claims rest on four domain assumptions about ASR quality, PII removal completeness, sample representativeness, and legal authorization. None of these is backed by a measurement, audit, or external verification in the text.

assumptions (4)
  • domain assumption AssemblyAI's ASR confidence scores (86-98%) are a reliable proxy for transcript quality across the whole dataset.
    Invoked in Section 4 to generalize from a 30-file QA sample to all 91,706 transcripts.
  • domain assumption Automated PII detection plus manual review achieves complete removal of the listed PII categories without a formal evaluation.
    Sections 6 and 11 assert 'all personally identifiable information has been removed'; no precision/recall or adversarial test is reported.
  • ad hoc to paper The 0.1% human QA sample (actually 30 files in Table 4) is representative of the full dataset despite being admitted as below statistical significance.
    Section 12 acknowledges the sample is below a statistically significant size but the authors proceed to report dataset-wide WER.
  • domain assumption BPO partnerships originally formed for commercial collection gave sufficient authorization to redistribute redacted transcripts for academic research.
    Section 4 and Section 11 state authorized partnerships and compliance, but no consent or legal documentation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Real-World En Call Center Transcripts Dataset with PII Redaction." pith.science (2026). https://pith.science/paper/HXTO4DAQ

@misc{pith2026250702958,
  author       = {Pith},
  title        = {Pith review of: Real-World En Call Center Transcripts Dataset with PII Redaction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HXTO4DAQ}},
  note         = {Machine review of arXiv:2507.02958}
}
read the original abstract

We introduce CallCenterEN, a large-scale (91,706 conversations, corresponding to 10448 audio hours), real-world English call center transcript dataset designed to support research and development in customer support and sales AI systems. This is the largest release to-date of open source call center transcript data of this kind. The dataset includes inbound and outbound calls between agents and customers, with accents from India, the Philippines and the United States. The dataset includes high-quality, PII-redacted human-readable transcriptions. All personally identifiable information (PII) has been rigorously removed to ensure compliance with global data protection laws. The audio is not included in the public release due to biometric privacy concerns. Given the scarcity of publicly available real-world call center datasets, CallCenterEN fills a critical gap in the landscape of available ASR corpora, and is released under a CC BY-NC 4.0 license for non-commercial research use.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 25 canonical work pages

  1. [1]

    AI systems such as Large Language Models (LLMs) benefit significantly from training on specialized domain-specific data [4]

    Introduction There is a growing need for high-quality, real-world datasets for training AI models in specialized domains like customer service and sales [1, 2, 3] . AI systems such as Large Language Models (LLMs) benefit significantly from training on specialized domain-specific data [4] . However, call center audio data is rarely available due to privacy...

    work page

  2. [2]

    Motivation, Authorship, and Contributions This dataset was created and published by researchers committed to advancing open AI. Our motivation stems from the lack of high-quality, large-scale, and real-world call center data that can be used for training and evaluating conversational AI models, particularly in the customer service and sales domains. By re...

    work page

  3. [3]

    LibriSpeech [5] provides audiobooks with aligned transcripts and remains a foundational dataset

    Related Work Existing open speech datasets have laid critical groundwork for Automatic Speech Recognition (ASR) and speech understanding. LibriSpeech [5] provides audiobooks with aligned transcripts and remains a foundational dataset. Common Voice [6] by Mozilla introduces multilingual coverage with crowdsourced contributions but lacks contextual flow and...

    work page

  4. [4]

    These partnerships were originally established for commercial data collection purposes, with the dataset subsequently being released for academic research

    Methodology The CallCenterEN dataset was sourced through partnerships with approximately 9-10 BPO centers. These partnerships were originally established for commercial data collection purposes, with the dataset subsequently being released for academic research. The selection of BPO centers was driven by domain-specific requirements from previous commerci...

    work page 2020

  5. [5]

    Dataset Overview Table 1: Dataset overview Category Value Total transcripts 91,706 Total hours of corresponding audio 10,448 hours (before removal due to regulatory and biometric privacy concerns) Call types Inbound, Outbound Language English Accents Indian, Filipino, American Annotation method Paid ASR (AssemblyAI) File format JSON Confidence score 86–98...

    work page

  6. [6]

    PII Redaction & Compliance All transcripts have been redacted to remove personally identifiable information (PII) using a combination of automated entity detection and manual review. Below is the list of PII categories that were reviewed and removed from the transcripts when detected: ● Personal Identifiers: person_name, person_age, date_of_birth, 9 phone...

    work page 2023

  7. [7]

    All recordings were downloaded in raw, uncompressed audio format to preserve acoustic fidelity

    Data Acquisition and Processing The dataset was sourced through partnerships with multiple BPO centers operating across various domains. All recordings were downloaded in raw, uncompressed audio format to preserve acoustic fidelity. Calls were then categorized by domain, accent (Indian, American, Filipino), and topic (inbound vs outbound). After domain ta...

    work page

  8. [8]

    the availability of a specialized microstructure dataset propagates across different areas within finance and into other disciplines

    Applications Please note that this is a raw dataset without labels. However, here are some of the intended use cases for this data: CallCenterEN enables a range of research tasks, including: ● Detailed intent detection and classification, ● Additional rare domain-specific raw text training corpus for neural network training especially LLMs and fine-tuning...

    work page

Show all 26 references
  1. [9]

    um” ● “uh

    Transcription Guidelines: The PII categories listed in Section 6 are systematically redacted during transcription. The following filler words are removed by default: ● “um” ● “uh” ● “hmm” ● “mhm” ● “uh-huh” ● “ah” ● “huh” ● “hm” ● “m” In addition, all numbers are converted to ...

  2. [10]

    Use is permitted for academic research and non-commercial model development only

    Access, Licensing, and Author Roles The dataset is publicly available at: https://huggingface.co/datasets/AIxBlock/91706-real-world-call-center-scripts-english License: Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0). Use is permitted for academic r...

  3. [11]

    The publicly released version includes only PII-redacted transcriptions

    Legal Disclaimer: This dataset was collected through authorized partnerships with BPO providers. The publicly released version includes only PII-redacted transcriptions. No audio data is shared to avoid biometric re-identification risks. All personally identifiable information...

  4. [12]

    Although we employed one of the most advanced commercial ASR models available at the time, the transcription accuracy may not be perfect in every case

    Limitations: Due to the real-world nature of call center recordings, the source audio includes background noise, cross-talk, and other natural artifacts that affect transcription quality. Although we employed one of the most advanced commercial ASR models available at the time...

  5. [13]

    Automation of Call Center Workforce Management Using NeuralProphet,

    V. S. S, A. Singh, A. B. Patil, H. S. Arora and T. K. Arcot, "Automation of Call Center Workforce Management Using NeuralProphet," in 2022 IEEE North Karnataka Subsection Flagship International Conference (NKCon), Vijaypur, India, 2022, pp. 1-7, doi: 10.1109/NKCon56289.2022.10126939

    2022

  6. [14]

    Gazquez-Garcia J, Sánchez-Bocanegra C, Sevillano J, AI in the Health Sector: Systematic Review of Key Skills for Future Health Professionals, JMIR Med Educ 2025;11:e58161, URL: https://mededu.jmir.org/2025/1/e58161 DOI: 10.2196/58161

    2025 doi

  7. [15]

    Olyfirenko, Yu., & Sydorenko, G. (2025). Peculiarities of logistics activities of trade enterprises in the conditions of digitalization. Scientific Bulletin of Polissya, 2(29), 52–67. https://doi.org/10.25140/2410-9576-2024-2(29)-52-67

    2025 doi

  8. [16]

    Li, H., Ai, Q., Chen, J., Dong, Q., Wu, Z., & Liu, Y. (2025). BLADE: Enhancing Black-Box Large Language Models with Small Domain-Specific Models. Proceedings of the AAAI Conference on Artificial Intelligence, 39(23), 24422-24430. https://doi.org/10.1609/aaai.v39i23.34620

    2025 doi

  9. [17]

    Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015). Librispeech: An ASR corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5206–5210

    2015

  10. [18]

    Common Voice Dataset

    Mozilla Foundation. Common Voice Dataset. https://commonvoice.mozilla.org

  11. [19]

    The Fisher Corpus: a Resource for the Next Generations of Speech-to-Text

    Christopher Cieri, David Graff, Owen Kimball, Dave Miller, Kevin Walker. The Fisher Corpus: a Resource for the Next Generations of Speech-to-Text. In LREC , 2004. https://catalog.ldc.upenn.edu/LDC2004T19

    2004

  12. [20]

    Paul, D.B., Baker, J.M. (1992). The design for the wall street journal-based CSR corpus. In Proceedings of the 2nd International Conference on Spoken Language Processing (ICSLP 1992), pp. 899-902. doi: 10.21437/ICSLP.1992-277

    1992 doi

  13. [21]

    S., Lamel, L

    Garofolo, J. S., Lamel, L. F., Fisher, W. M., Fiscus, J. G., Pallett, D. S., & Dahlgren, N. L. (1992). TIMIT Acoustic-Phonetic Continuous Speech Corpus. Linguistic Data Consortium. https://www.researchgate.net/publication/243787812_TIMIT_Acoustic-phonetic_Continuous_Speech_Corpus

    1992

  14. [22]

    Cieri, C., Miller, D., & Walker, K. (2004). The Fisher Corpus: a resource for the next generations of speech-to-text. Proceedings of the Fourth International Conference on Language Resources and 16 Evaluation (LREC), 69–71. https://aclanthology.org/L04-1500/

    2004

  15. [23]

    Godfrey, J.J., Holliman, E. (1992). SWITCHBOARD: Telephone speech corpus for research and development. In ICASSP 1992. Available at: https://catalog.ldc.upenn.edu/LDC97S62

    1992

  16. [24]

    Budzianowski, P., Wen, T.-H., Tseng, B.-H., Casanueva, I., Ultes, S., Ramadan, O., & Gasic, M. (2018). MultiWOZ – A large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling. arXiv:1810.00278

    2018 arXiv

  17. [25]

    Call center

    Hugging Face Hub Datasets Search for “Call center” https://huggingface.co/datasets?sort=trending&search=Call+center

  18. [26]

    Gang Hu, Koren M. Jo, Yi Alex Wang, Jing Xie, Institutional trading and Abel Noser data, Journal of Corporate Finance, Volume 52, 2018, Pages 143-167, ISSN 0929-1199, https://doi.org/10.1016/j.jcorpfin.2018.08.005. (https://www.sciencedirect.com/science/article/pii/S0929119917...

    2018 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.