Pith. sign in

REVIEW 3 major objections 6 minor 69 references

Contextualized Token Discrimination for Speech Search Query Correction

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Add a difference vector to BERT; Chinese query correction hits 89.6 F1

desk verdict Plausible ESIM-style composition layer, but the SIGHAN comparison is uninterpretable because the baselines were not given the same MLM fine-tuning. read the letter →

arxiv 2509.04393 v1 pith:CWXYTMQP submitted 2025-09-04 cs.SD cs.CL

Junyu Lu , Di Jiang , Mengze Hong , Victor Junqiu Wei , Qintian Guo , Zhiyang Su This is my paper
classification cs.SDcs.CL
keywords ChinesespellingcorrectionspeechsearchASRerrorcontextualizedtokendiscriminationBERTfine-tuningcompositionlayerSIGHANAAMbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a lightweight layer on top of BERT, a pretrained transformer language model, can fix misrecognized words in Chinese speech-search queries, and that the right test for that fix is a benchmark built from real ASR errors. The central move is to give the corrector three views of each token: its original embedding, its context-aware BERT representation, and the difference between the two, so the model can flag tokens that clash with their surroundings. With this setup the method reaches 89.6% F1 on the standard SIGHAN spelling-correction set and 52.2% F1 on a newly constructed AAM set, outperforming BERT and Soft-Masked BERT. The paper also reports that most of the gain comes from an initial masked-language-model fine-tuning step on domain text, a result it calls surprising; the added composition layer then gives a smaller, consistent boost.

What carries the argument

The load-bearing mechanism is the composition layer. For each token i it forms c_i = [v_C_i; v_I_i; v_C_i - v_I_i], concatenating the contextualized representation from BERT's last layer, the original input token representation, and the difference between the two, and sends this triple through a linear classifier. The difference vector is the key: when a token is misrecognized, its contextualized vector points away from its input vector, giving the classifier a direct, explicit signal of the mismatch. Training uses a loss computed only on error tokens, plus a random sample of correct tokens in a 1:5 ratio, so the model learns to keep correct tokens as well as replace wrong ones.

What would settle it

Re-run the published BERT and Soft-Masked BERT baselines using the same domain-language-model fine-tuning, the same token-level loss, and the same per-character F1 scoring on the same SIGHAN and AAM splits; if fine-tuned BERT alone reaches 89.6% and 52.2% F1, then CTD's composition layer contributes nothing beyond its base model.

Watch

Extended reading notes

Core claim

CTD's core claim is that an erroneous token in an ASR query is detectable by the mismatch between what the token looks like on its own and what BERT thinks it means in context. The paper builds a composition layer that concatenates the input token vector, the contextualized vector from BERT's last layer, and their difference, then feeds that triple to a classifier which selects the correct token from the vocabulary. Because the difference vector makes the mismatch explicit, the classifier can learn to replace exactly the tokens that are out of place while leaving correct tokens alone. The authors report 89.6% F1 on SIGHAN and 52.2% on AAM, and they attribute the largest share of this to doma

Load-bearing premise

The headline improvement over BERT and Soft-Masked BERT rests on those baselines being trained and scored under exactly the same protocol—including the extra domain-language-model fine-tuning and per-character F1—which the paper does not describe for the baselines; the authors themselves note that the base model's 20-point jump was surprising.

Editorial extensions

If this is right

  • With the reported numbers, CTD reaches 89.6% F1 on SIGHAN and 52.2% F1 on the new AAM benchmark, beating BERT (64.9% / 33.2%) and Soft-Masked BERT (66.4% / 33.7%) by a wide margin on both.
  • The biggest part of the gain comes from an initial masked-language-model fine-tuning step on domain text; the base model alone reaches 89.3% on SIGHAN, which the paper calls surprising.
  • The composition layer adds a smaller, consistent boost on both datasets (89.3 to 89.6 and 50.7 to 52.2 F1), so the layer's value is real but modest compared with domain fine-tuning.
  • The method can repair homophone and grammar/logic errors in ASR transcripts, as the examples show, but it still fails on errors that require world knowledge.
  • Because AAM excludes insertion/deletion and stopword errors, the claims are about substitution errors only, not full ASR error correction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not say this, but if the baselines did not receive the same masked-language-model fine-tuning, the roughly 20-point gap likely comes mostly from that fine-tuning rather than from CTD's composition layer.
  • The same input-context-difference composition should transfer to other token-level rewriting tasks, such as grammatical error correction, OCR post-processing, or dialogue repair, where one token is locally inconsistent with its context.
  • A natural extension is to add an alignment or length-prediction head so CTD can handle the insertion and deletion errors the AAM benchmark currently filters out.
  • The paper's own limitations note the reliance on domain-specific training data, so few-shot or zero-shot variants would be the next test of generality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Contextualized Token Discrimination (CTD), a BERT-based method for Chinese speech-search query spelling correction. CTD augments BERT token representations with a composition layer that concatenates the input token embedding, the contextualized embedding from BERT's last layer, and their difference, then applies a linear classifier over the Chinese vocabulary. The method is evaluated on the SIGHAN benchmark and on a newly constructed AAM benchmark built from public Mandarin speech corpora (Aidatatang, AISHELL-1, MagicData) using a Conformer ASR model. Table 1 reports F1 of 89.6% on SIGHAN and 52.2% on AAM, exceeding the cited BERT and Soft-Masked BERT baselines by a large margin, while the proposed CTD improves over the authors' own base model by only 0.3 F1 points on SIGHAN.

Significance. If the reported gains were reliable, CTD would be a simple and practical extension to BERT for ASR query correction, and the AAM dataset would be a useful text-pair benchmark derived from public audio corpora. The architecture is described clearly enough to re-implement, and the use of public speech resources for benchmark construction is a positive step. However, the central empirical claim rests on a single table without variance estimates and on a baseline comparison that mixes a possible training-protocol change with the proposed method. The benchmark's filtering choices further limit the generality of the ASR-correction claims. The contribution is therefore plausible but not yet established at the level needed for publication.

major comments (3)
  1. [Table 1 / §4.1] The 20+ point F1 improvement of the authors' Base Model (89.3) over the reported BERT (64.9) and Soft-Masked BERT (66.4) is explicitly attributed by the authors to an added MLM fine-tuning stage on the training set. The manuscript does not state whether the BERT and Soft-Masked BERT numbers were obtained under the same protocol (same MLM fine-tuning on the same data, same token sampling, same early stopping, same sentence-level F1 computation), nor does it report re-running those baselines. Since the Base Model already reaches 89.3 without the CTD composition layer, the comparison confounds the proposed architecture with training-protocol differences. The 0.3-point CTD-vs-BaseModel increment is small relative to this confound. The claimed superiority over prior BERT-based methods is therefore unsupported unless the baselines are re-run under identical conditions or exact matching citatio
  2. [§4.2 / Limitations] The AAM benchmark explicitly filters to same-length erroneous/ground-truth pairs and removes misrecognized stopwords, which excludes insertion and deletion errors as well as stopword substitutions. As the authors acknowledge in the Limitations section, these are common in ASR transcription. Consequently the reported 52.2% F1 is not a realistic end-to-end ASR-correction figure; it measures a restricted substitution-only, content-word task. The paper nevertheless frames AAM as 'comprehensive' and 'realistic'. The central claim about speech query correction needs to be scoped to this filtered benchmark, or the benchmark needs to include a version that does not filter these error types.
  3. [Table 1 / §4.2] All results are reported as single numbers with no standard deviations, confidence intervals, or significance tests. The advantage of CTD over the authors' Base Model is 0.3 F1 on SIGHAN and 1.5 F1 on AAM. Without multiple seeds or a paired bootstrap, it is not possible to determine whether this small increment is meaningful. At minimum, report mean and std over at least three training runs, and ideally paired significance tests for the CTD-vs-BaseModel comparison.
minor comments (6)
  1. [§4.2] The sentence 'our well-trained methods achieve comparable results to both the previous BERT benchmark and Soft-Masked BERT' is misleading; Table 1 shows substantially higher F1. The text should say 'superior' or 'higher'.
  2. [Figure 2] The caption describes 'the left picture' and 'the right side', but the exact cosine-similarity computation and token mapping are not defined. Please specify how the matrices in the figure are produced, and make the figure legible in grayscale.
  3. [§3.2] Equation (1) uses a semicolon in the concatenation notation without defining it. Also, the set definitions '∀Ti ≠ T'i ∈ Λ' and '∃Ti = T'i ∈ Λ' are mathematically awkward; define Λ1, Λ2, and Λ with standard set notation.
  4. [§4.1] The statement 'we only fine-tuned the MLM using the training set' is confusing because a classification head is also trained. Please separate the two stages: (i) MLM fine-tuning on correct sentences, (ii) supervised training of the correction head, and state which stage is meant in each sentence.
  5. [§4.2] The AAM dataset is said to contain 6965 pairs, but the 'Generated Pairs' column in Table 3 sums to 6965 while the training/validation/test split sums to 6965 as well. Please clarify how the 6965 pairs are partitioned and whether the corpus-specific numbers include all splits.
  6. [References] Several references are formatted inconsistently (e.g., 'CHEN CHEN' in the reference list) and a number of recent citations to the authors' own work appear tangential to the technical content. A pass to tighten the reference list would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CTD is an empirical supervised method evaluated on external benchmarks; the unmatched-baseline issue is an evidence concern, not a circular derivation.

full rationale

The derivation chain is a standard supervised classification setup: BERT contextual representations, a composition layer using concatenation and difference vectors (Eq. 1), a softmax classifier (Eq. 2), and a cross-entropy loss restricted to tokens where the erroneous query differs from the ground truth plus sampled correct tokens (Eq. 3). The target token labels are defined by string comparison between the erroneous query and the ground-truth sentence, not by the model's own outputs, so the training objective is not self-definitional. Evaluation on SIGHAN uses an external public benchmark, and the AAM benchmark is constructed from ASR hypotheses paired with ground-truth transcripts; both are independent of the model's predicted outputs. The paper's many self-citations (e.g., Hong et al. 2025a-d, Wei et al. 2024a-b, Jiang et al. 2013-2021) appear in related work and as motivation/benchmark references, not as load-bearing justification of the CTD mechanism or its uniqueness. The most serious concern is in Section 4.1: the authors report a 'surprising' 20+ point gain over cited BERT/Soft-Masked BERT numbers after adding MLM fine-tuning on the training set, and they do not describe re-running those baselines under the same protocol. That is an unmatched-baseline/evidence problem for the superiority claim, not a circularity, because the superiority is not true by construction and would be resolved by re-running the baselines. The acknowledged same-length and stopword filtering of AAM similarly weakens external validity but does not make the evaluation circular. Therefore no circular step is identified.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends only on standard pretrained models, cited observations, and hand-set hyperparameters; no new theoretical entities are introduced.

free parameters (4)
  • training token sampling ratio Λ1:Λ2 = 1:5
    Ratio of error tokens to randomly sampled correct tokens used in the training objective (Section 3.2, Eq. 3). Hand-set, not tuned against a validation metric.
  • peak learning rate = 5e-4
    Peak learning rate in the Transformer schedule, with warmup over first 5% of steps (Section 4.1). Hand-set.
  • batch size = 512
    Mini-batch size for all experiments (Section 4.1). Hand-set.
  • number of epochs for AAM = 25
    Training epochs for the AAM task (Section 4.2). Hand-set.
assumptions (4)
  • domain assumption BERT-base-chinese pretrained weights provide useful contextualized token representations for Chinese spelling correction.
    The method relies on this pretrained model as the backbone, and the paper fine-tunes it. This is standard practice.
  • domain assumption Upper layers of contextualized models produce context-specific representations (Ethayarajh, 2019).
    The method's motivation rests on this observation, cited in the introduction, that erroneous tokens will have inconsistent representations.
  • domain assumption The ESIM heuristic matching with a difference vector captures token-level compatibility (Chen et al., 2017).
    The composition layer is borrowed from ESIM and assumed to transfer to this task.
  • domain assumption The ground-truth transcripts of the AAM corpora (Aidatatang, AISHELL-1, MagicData) are correct and the Conformer models' CER values in Table 3 are accurate.
    The AAM benchmark is built on these assumptions; wrong transcripts or inaccurate ASR would distort the evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Contextualized Token Discrimination for Speech Search Query Correction." pith.science (2026). https://pith.science/paper/CWXYTMQP

@misc{pith2026250904393,
  author       = {Pith},
  title        = {Pith review of: Contextualized Token Discrimination for Speech Search Query Correction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CWXYTMQP}},
  note         = {Machine review of arXiv:2509.04393}
}
read the original abstract

Query spelling correction is an important function of modern search engines since it effectively helps users express their intentions clearly. With the growing popularity of speech search driven by Automated Speech Recognition (ASR) systems, this paper introduces a novel method named Contextualized Token Discrimination (CTD) to conduct effective speech query correction. In CTD, we first employ BERT to generate token-level contextualized representations and then construct a composition layer to enhance semantic information. Finally, we produce the correct query according to the aggregated token representation, correcting the incorrect tokens by comparing the original token representations and the contextualized representations. Extensive experiments demonstrate the superior performance of our proposed method across all metrics, and we further present a new benchmark dataset with erroneous ASR transcriptions to offer comprehensive evaluations for audio query correction.

Figures

Figures reproduced from arXiv: 2509.04393 by the authors.

Figure 1
Figure 1. Overview: ASR and spelling correction. To enhance the user experience with accurate in￾tent transcription, it is fundamental to enhance the performance of the ASR model (Hong and Jiang, 2025). Typically, the ASR model first generates multiple transcription hypotheses from the audio, then selects the hypothesis with the fewest errors using a language model (LM). Finally, spelling cor￾rection is applied to refine the … view at source ↗
Figure 2
Figure 2. The left picture shows the cosine similarity [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Our proposed Contextualized Token Discrimination (CTD) method consists of two main components: [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Loss of our CTD on the AAM data To demonstrate the effectiveness of our CTD method, we present several cases in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

69 extracted references · 50 canonical work pages

  1. [1]

    C Anantaram, Amit Sangroya, Mrinal Rawat, and Aishwarya Chhabra. 2018. Repairing asr output by artificial development and ontology based learning. In IJCAI, pages 5799--5801

    work page 2018

  2. [2]

    Yuya Asano, Sabit Hassan, Paras Sharma, Anthony B Sicilia, Katherine Atwell, Diane Litman, and Malihe Alikhani. 2025. Contextual asr error handling with llms augmentation for goal-oriented conversational ai. In Proceedings of the 31st International Conference on Computational Linguistics: Industry Track, pages 374--386

    work page 2025

  3. [3]

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in Neural Information Processing Systems, 33

    work page 2020

  4. [4]

    Hui Bu, Jiayu Du, Xingyu Na, Bengu Wu, and Hao Zheng. 2017. Aishell-1: An open-source mandarin speech corpus and a speech recognition baseline. In 2017 20th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment (O-COCOSDA), pages 1--5. IEEE

    work page 2017

  5. [5]

    CHEN CHEN, Yuchen Hu, Chao-Han Huck Yang, Sabato Marco Siniscalchi, Pin-Yu Chen, and EngSiong Chng. 2023. https://openreview.net/forum?id=cAjZ3tMye6 Hyporadise: An open baseline for generative speech recognition with large language models . In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    work page 2023

  6. [6]

    Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. https://doi.org/10.18653/v1/p17-1152 Enhanced lstm for natural language inference . Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    doi:10.18653/v1/p17-1152 2017

  7. [7]

    Zhiyu Chen, Mohamed Trabelsi, Jeff Heflin, Yinan Xu, and Brian D. Davison. 2020. https://doi.org/10.1145/3397271.3401044 Table search using a deep contextualized language model . In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20, page 589–598, New York, NY, USA. Association for Co...

    arXiv 2020

  8. [8]

    Horia Cucu, Andi Buzo, Laurent Besacier, and Corneliu Burileanu. 2013. Statistical error correction methods for domain-specific asr systems. In Statistical Language and Speech Processing, pages 83--92, Berlin, Heidelberg. Springer Berlin Heidelberg

    work page 2013

Show all 69 references
  1. [9]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human...

    2019

  2. [10]

    Huizhong Duan and Bo-June Hsu. 2011. Online spelling correction for query completion. In Proceedings of the 20th international conference on World wide web, pages 117--126

    2011

  3. [11]

    Luis Fernando D’Haro and Rafael E Banchs. 2016. Automatic correction of asr outputs by using machine translation. proceedings Interspeech 2016, pages 3469--3473

    2016

  4. [12]

    Kawin Ethayarajh. 2019. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on N...

    2019

  5. [13]

    Peizhu Gong, Jin Liu, Yurong Xie, Minjie Liu, and Xiliang Zhang. 2023. Enhancing context representations with part-of-speech information and neighboring signals for question classification. Complex & Intelligent Systems, 9(6):6191--6209

    2023

  6. [14]

    Priya Goyal, Piotr Doll \'a r, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. 2017. Accurate, large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677

    2017 arXiv

  7. [15]

    Alex Graves. 2012. https://arxiv.org/abs/1211.3711 Sequence transduction with recurrent neural networks . CoRR, abs/1211.3711

    2012 arXiv

  8. [16]

    Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, and Ruoming Pang. 2020. https://arxiv.org/abs/2005.08100 Conformer: Convolution-augmented transformer for speech recognition . Preprint, arXiv:2005.08100

    2020 arXiv

  9. [17]

    Ryan P Hafen and Michael J Henry. 2012. Speech information retrieval: a review. Multimedia systems, 18(6):499--518

    2012

  10. [18]

    Matthias Hagen, Martin Potthast, Marcel Gohsen, Anja Rathgeber, and Benno Stein. 2017. A large-scale query spelling correction corpus. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1261--1264

    2017

  11. [19]

    Sebastian Hofst \"a tter, Navid Rekabsaz, Mihai Lupu, Carsten Eickhoff, and Allan Hanbury. 2019. Enriching word embeddings for patent retrieval with global context. In European Conference on Information Retrieval, pages 810--818. Springer

    2019

  12. [20]

    Mengze Hong and Di Jiang. 2025. https://arxiv.org/abs/2506.07149 Technical report: A practical guide to kaldi asr optimization . Preprint, arXiv:2506.07149

    2025 arXiv

  13. [21]

    Mengze Hong, Wailing Ng, Chen Jason Zhang, and Di Jiang. 2025 a . Qualbench: Benchmarking chinese LLM s with localized professional qualifications for vertical domain evaluation. In The 2025 Conference on Empirical Methods in Natural Language Processing

    2025

  14. [22]

    Mengze Hong, Wailing Ng, Chen Jason Zhang, Yuanfeng SONG, and Di Jiang. 2025 b . Dial-in LLM : Human-aligned LLM -in-the-loop intent clustering for customer service dialogues. In The 2025 Conference on Empirical Methods in Natural Language Processing

    2025

  15. [23]

    Mengze Hong, Wailing Ng, Chen Jason Zhang, Yifei Wang, Yuanfeng Song, and Di Jiang. 2025 c . Llm-in-the-loop: Replicating human insight with llms for better machine learning applications. Authorea Preprints

    2025

  16. [24]

    Mengze Hong, Chen Jason Zhang, Lingxiao Yang, Yuanfeng SONG, and Di Jiang. 2025 d . https://proceedings.mlr.press/v260/hong25a.html InfantCryNet : A data-driven framework for intelligent analysis of infant cries . In Proceedings of the 16th Asian Conference on Machine Learning...

    2025

  17. [25]

    Di Jiang, Kenneth Wai-Ting Leung, Lingxiao Yang, and Wilfred Ng. 2015. Teii: Topic enhanced inverted index for top-k document retrieval. Knowledge-Based Systems, 89:346--358

    2015

  18. [26]

    Di Jiang, Yuanfeng Song, Rongzhong Lian, Siqi Bao, Jinhua Peng, Huang He, Hua Wu, Chen Zhang, and Lei Chen. 2021. https://doi.org/10.1007/978-3-030-73200-4_36 Familia: A configurable topic modeling framework for industrial text engineering . In Database Systems for Advanced Ap...

    2021 doi

  19. [27]

    Di Jiang, Yongxin Tong, and Yuanfeng Song. 2016. https://doi.org/10.1145/2956235 Cross-lingual topic discovery from multilingual search engine query log . ACM Trans. Inf. Syst., 35(2)

    2016 doi

  20. [28]

    Di Jiang, Jan Vosecky, Kenneth Wai-Ting Leung, and Wilfred Ng. 2013. https://doi.org/10.1145/2452376.2452420 Panorama: a semantic-aware application search framework . In Proceedings of the 16th International Conference on Extending Database Technology, EDBT '13, page 371–382, ...

    2013

  21. [29]

    Taegyun Kwon, Dasaem Jeong, and Juhan Nam. 2017. Audio-to-score alignment of piano music using rnn-based automatic music transcription. arXiv preprint arXiv:1711.04480

    2017 arXiv

  22. [30]

    Yichong Leng, Xu Tan, Rui Wang, Linchen Zhu, Jin Xu, Wenjie Liu, Linquan Liu, Xiang-Yang Li, Tao Qin, Edward Lin, and Tie-Yan Liu. 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.367 F ast C orrect 2: Fast error correction on multiple candidates for automatic speech reco...

    2021 doi

  23. [31]

    Chenda Li, Jing Shi, Wangyou Zhang, Aswin Shanmugam Subramanian, Xuankai Chang, Naoyuki Kamo, Moto Hira, Tomoki Hayashi, Christoph Boeddeker, Zhuo Chen, and Shinji Watanabe. 2021. ESPnet-SE : End-to-end speech enhancement and separation toolkit designed for ASR integration. In...

    2021

  24. [32]

    Yinghui Li, Shang Qin, Haojing Huang, Yangning Li, Libo Qin, Xuming Hu, Wenhao Jiang, Hai-Tao Zheng, and Philip S Yu. 2024. Rethinking the roles of large language models in chinese grammatical error correction. arXiv preprint arXiv:2402.11420

    2024 arXiv

  25. [33]

    Yinghui Li, Qingyu Zhou, Yangning Li, Zhongli Li, Ruiyang Liu, Rongyi Sun, Zizhen Wang, Chao Li, Yunbo Cao, and Hai-Tao Zheng. 2022. https://doi.org/10.18653/v1/2022.findings-acl.252 The past mistake is the future wisdom: Error-driven contrastive probability optimization for C...

    2022 doi

  26. [34]

    Zihong Liang, Xiaojun Quan, and Qifan Wang. 2023. Disentangled phonetic representation for chinese spelling correction. In The 61st Annual Meeting Of The Association For Computational Linguistics

    2023

  27. [35]

    Junwei Liao, Sefik Eskimez, Liyang Lu, Yu Shi, Ming Gong, Linjun Shou, Hong Qu, and Michael Zeng. 2023. Improving readability for automatic speech recognition transcription. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(5):1--23

    2023

  28. [36]

    Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

    2017 arXiv

  29. [37]

    Anirudh Mani, Shruti Palaskar, Nimshi Venkat Meripo, Sandeep Konam, and Florian Metze. 2020. Asr error correction and domain adaptation using machine translation. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6344-...

    2020

  30. [38]

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013 a . https://arxiv.org/abs/1301.3781 Efficient estimation of word representations in vector space . Preprint, arXiv:1301.3781

    2013 arXiv

  31. [39]

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013 b . https://arxiv.org/abs/1310.4546 Distributed representations of words and phrases and their compositionality . Preprint, arXiv:1310.4546

    2013 arXiv

  32. [40]

    Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365

    2018 arXiv

  33. [41]

    Golan Pundak, Tara N Sainath, Rohit Prabhavalkar, Anjuli Kannan, and Ding Zhao. 2018. Deep context: end-to-end contextual speech recognition. In 2018 IEEE spoken language technology workshop (SLT), pages 418--425. IEEE

    2018

  34. [42]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  35. [43]

    Kanishka Rao, Hasim Sak, and Rohit Prabhavalkar. 2018. https://arxiv.org/abs/1801.00841 Exploring architectures, data and units for streaming end-to-end speech recognition with rnn-transducer . CoRR, abs/1801.00841

    2018 arXiv

  36. [44]

    Dhruva Sahrawat, Debanjan Mahata, Haimin Zhang, Mayank Kulkarni, Agniv Sharma, Rakesh Gosangi, Amanda Stent, Yaman Kumar, Rajiv Ratn Shah, and Roger Zimmermann. 2020. Keyphrase extraction as sequence labeling using contextualized embeddings. Advances in Information Retrieval, ...

    2020

  37. [45]

    Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. 2019. wav2vec: Unsupervised pre-training for speech recognition. arXiv preprint arXiv:1904.05862

    2019 arXiv

  38. [46]

    Yuanfeng Song, Di Jiang, Xuefang Zhao, Qian Xu, Raymond Chi-Wing Wong, Lixin Fan, and Qiang Yang. 2021. https://doi.org/10.1145/3474085.3481542 L2rs: A learning-to-rescore mechanism for hybrid speech recognition . In Proceedings of the 29th ACM International Conference on Mult...

    2021

  39. [47]

    End-to-end asr: from supervised to semi-supervised learning with modern architectures

    Gabriel Synnaeve, Qiantong Xu, Jacob Kahn, Tatiana Likhomanenko, Edouard Grave, Vineel Pratap, Anuroop Sriram, Vitaliy Liptchinsky, and Ronan Collobert. End-to-end asr: from supervised to semi-supervised learning with modern architectures. In ICML 2020 Workshop on Self-supervi...

    2020

  40. [48]

    Bruno Taill \'e , Vincent Guigue, and Patrick Gallinari. 2020. Contextualized embeddings in named-entity recognition: An empirical study on generalization. Advances in Information Retrieval, 12036:383

    2020

  41. [49]

    Tomohiro Tanaka, Ryo Masumura, Hirokazu Masataki, and Yushi Aono. 2018. Neural error corrective language models for automatic speech recognition. In INTERSPEECH, pages 401--405

    2018

  42. [50]

    Chenming Tang, Xiuyu Wu, and Yunfang Wu. 2023. Are pre-trained language models useful for model ensemble in chinese grammatical error correction? In The 61st Annual Meeting Of The Association For Computational Linguistics

    2023

  43. [51]

    Raphael Tang, Ferhan Ture, and Jimmy Lin. 2019. Yelling at your TV : An analysis of speech recognition errors and subsequent user behavior on entertainment systems. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retriev...

    2019

  44. [52]

    Ghazaleh H Torbati, Andrew Yates, and Gerhard Weikum. 2021. You get what you chat: Using conversations to personalize search-based recommendations. In European Conference on Information Retrieval, pages 207--223. Springer

    2021

  45. [53]

    Yuen-Hsien Tseng, Lung-Hao Lee, Li-Ping Chang, and Hsin-Hsi Chen. 2015. Introduction to sighan 2015 bake-off for chinese spelling check. In Proceedings of the Eighth SIGHAN Workshop on Chinese Language Processing, pages 32--37

    2015

  46. [54]

    Takuma Udagawa, Masayuki Suzuki, Masayasu Muraoka, and Gakuto Kurata. 2024. Robust asr error correction with conservative data filtering. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 256--266

    2024

  47. [55]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser, and I. Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998--6008

    2017

  48. [56]

    Dingmin Wang, Yan Song, Jing Li, Jialong Han, and Haisong Zhang. 2018. A hybrid approach to automatic corpus generation for chinese spelling check. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2517--2527

    2018

  49. [57]

    Yixuan Wang, Baoxin Wang, Yijun Liu, Dayong Wu, and Wanxiang Che. 2024. Lm-combiner: A contextual rewriting model for chinese grammatical error correction. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation...

    2024

  50. [58]

    Victor Junqiu Wei, Weicheng Wang, Di Jiang, Yuanfeng Song, and Lu Wang. 2024 a . Asr-ec benchmark: Evaluating large language models on chinese asr error correction. arXiv preprint arXiv:2412.03075

    2024 arXiv

  51. [59]

    Victor Junqiu Wei, Weicheng Wang, Di Jiang, Conghui Tan, and Rongzhong Lian. 2024 b . Acoustic model optimization over multiple data sources: Merging and valuation. arXiv preprint arXiv:2410.15620

    2024 arXiv

  52. [60]

    Hongqiu Wu, Shaohua Zhang, Yuchen Zhang, and Hai Zhao. 2023. Rethinking masked language modeling for chinese spelling correction. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10743--10756

    2023

  53. [61]

    Xueyang Wu, Rongzhong Lian, Di Jiang, Yuanfeng Song, Weiwei Zhao, Qian Xu, and Qiang Yang. 2022. A phonetic-semantic pre-training model for robust speech recognition. CAAI Artificial Intelligence Research, 1(1):1--7

    2022

  54. [62]

    Jinhua Xiong, Qiao Zhang, Shuiyuan Zhang, Jianpeng Hou, and Xueqi Cheng. 2015. Hanspeller: a unified framework for chinese spelling correction. In International Journal of Computational Linguistics & Chinese Language Processing, Volume 20, Number 1, June 2015-Special Issue on ...

    2015

  55. [63]

    Junjie Yu and Zhenghua Li. 2014. Chinese spelling error detection and correction based on language model, pronunciation, and shape. In Proceedings of The Third CIPS-SIGHAN Joint Conference on Chinese Language Processing, pages 220--223

    2014

  56. [64]

    Liang-Chih Yu, Lung-Hao Lee, Yuen-Hsien Tseng, and Hsin-Hsi Chen. 2014. Overview of sighan 2014 bake-off for chinese spelling check. In Proceedings of The Third CIPS-SIGHAN Joint Conference on Chinese Language Processing, pages 126--132

    2014

  57. [65]

    Shaohua Zhang, Haoran Huang, Jicong Liu, and Hang Li. 2020 a . Spelling error correction with soft-masked bert. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 882--890

    2020

  58. [66]

    Park, Wei Han, Chung-Cheng Chiu, Ruoming Pang, Quoc V

    Yu Zhang, James Qin, Daniel S. Park, Wei Han, Chung-Cheng Chiu, Ruoming Pang, Quoc V. Le, and Yonghui Wu. 2020 b . https://arxiv.org/abs/2010.10504 Pushing the limits of semi-supervised learning for automatic speech recognition . Preprint, arXiv:2010.10504

    2020 arXiv

  59. [67]

    Yutao Zhu, Jian - Yun Nie, Kun Zhou, Pan Du, and Zhicheng Dou. 2021. https://doi.org/10.1007/978-3-030-72113-8\_50 Content selection network for document-grounded retrieval-based chatbots . In Advances in Information Retrieval - 43rd European Conference on IR Research, ECIR 20...

    2021 doi

  60. [68]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  61. [69]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.