Pith. sign in

REVIEW 3 major objections 4 minor 32 references

Improving OCR using internal document redundancy

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An unsupervised clustering method that learns a document's own character shapes can correct OCR errors and flag mistakes the base system misses.

desk verdict A useful, honest, modest OCR post-processor for weak baselines; the Google flagging claim is inflated by a biased estimator. read the letter →

arxiv 2508.14557 v1 pith:FC2MMLTJ submitted 2025-08-20 cs.CV cs.LGeess.IV

Diego Belzarena , Seginus Mowlavi , Aitor Artola , Camilo Mariño , Marina Gardella , Ignacio Ramírez , Antoine Tadros , Roy He
show 4 more authors
Natalia Bottaioli Boshra Rajaei Gregory Randall Jean-Michel Morel
This is my paper · ORCID
classification cs.CVcs.LGeess.IV
keywords OCRcorrectiondocumentredundancyunsupervisedclusteringGaussianmixturemodelglyphcharacterrecognitionhistoricaldocumentsnormalitytesting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most OCR engines recognize text by matching against patterns learned from huge training corpora, ignoring that a single document repeats the same glyphs many times. This paper argues that this internal visual redundancy is under-used and shows it can be exploited without supervision: group cropped character images into glyph clusters, then let the majority label in each cluster override the minority. To make grouping reliable, the authors extend a Gaussian mixture model with iterative image registration and statistical normality tests that split impure clusters. On historical typewritten and printed documents, the post-processing lowers character error rate by about 1.2 points over a character-level OCR baseline, and on a stronger commercial engine it flags recognitions that are wrong with statistically significant accuracy. This matters because it offers document-specific, interpretable correction that complements language-model-based post-correction.

What carries the argument

The extended Gaussian Mixture Model with iterative realignment and normality testing—called binary tree refinement. Character crops are padded, Poisson-standardized, and segmented via minimal-cost paths; PCA projection speeds clustering; a GMM with Oracle Approximating Shrinkage provides initial clusters; each cluster is realigned with the Inverse Compositional Algorithm and tested for normality on its top principal components; clusters that fail are split by a two-component GMM, recursively, until all leaves pass the test or are too small. This machinery turns noisy, misaligned crops into pure glyph clusters whose majority labels can be trusted.

What would settle it

Run the Anderson-Darling test on hand-verified clusters of identical glyph images from degraded documents: if most genuine glyph clusters fail the normality test at the paper's threshold, the splitting rule cannot recover true clusters. A second check: apply the method to clean, uniform printed text with a near-perfect base OCR; if it still changes labels often and lowers accuracy, the mechanism is not actually discovering document-specific glyph structure.

Watch

Extended reading notes

Core claim

The central claim is that accurate, unsupervised clustering of character images—each cluster corresponding to one glyph as printed or typed in that document—is enough to correct OCR substitution errors by majority vote. The key move is to treat clustering as the discovery of uni-modal Gaussian clusters in image space: after PCA projection, a Gaussian mixture model initializes clusters; each cluster is iteratively realigned to its mean and tested for normality via Anderson-Darling tests, and clusters that fail are split recursively. This yields glyph models learned purely from the input document, with no external training data. Applied to CharNet, the correction reduces CER by 1.16 ± 0.06 poi

Load-bearing premise

The method assumes that every image of the same printed or typed character in a document, after alignment, is drawn from one Gaussian distribution of pixel values; if degradation produces irregular, non-Gaussian variation within a single glyph, clusters will mix different letters and the majority correction will misfire.

Editorial extensions

If this is right

  • Document-internal redundancy can be mined without labels: on degraded typewritten archives and historical newspapers, the method cuts character error rate by more than one point over a character-level OCR baseline.
  • The normality-test refinement makes the final output stable across a wide range of initial cluster counts, so the method does not require careful tuning of K.
  • Even on a strong commercial OCR, the clusters point to recognition errors with statistically significant accuracy, so the method can act as an error-flagging front end for human or language-model review.
  • Because it relies only on image geometry, the approach complements linguistic post-OCR correction rather than competing with it.
  • The method corrects substitution errors only; the paper identifies insertion and deletion errors as the natural next target via pattern matching.
  • The method is not yet a fully automated post-processor on top of very strong OCR engines; its authors present it as evidence that those engines under-use document redundancy.
  • The same glyph models could be reused as per-character confidence scores, giving language-model-based post-correction a visual prior it currently lacks.
  • The Gaussian-glyph assumption likely limits gains on very degraded images, where within-glyph degradation variance can exceed between-glyph shape differences; a robustness strategy would be to detect and quarantine high-variance clusters before voting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes an unsupervised post-processing method to improve OCR output by exploiting the visual redundancy of character shapes within a document. Character images are extracted from OCR bounding boxes, standardized, projected with PCA, clustered with an extended Gaussian Mixture Model that alternates EM with intra-cluster realignment and Anderson-Darling normality tests, and finally relabeled by a super-majority vote within each cluster. Experiments on two historical/typewritten datasets (Berrutti and ENP) report CER reductions of 1.16±0.06 and 1.30±0.06 points when applied to CharNet, and CER increases of 0.53±0.05 and 0.23±0.04 on Google Cloud Vision. The paper interprets the Google results as evidence that the method can flag a statistically significant fraction of SOTA OCR errors, with correction accuracy estimated as 34±2.0 and 31±3.8 via a heuristic formula. The authors provide code and a new public dataset.

Significance. If the central claim holds—that current OCR systems under-use internal document redundancy—the paper identifies a real and relatively unexploited source of signal. The method is unsupervised, parameter choices are stated a priori, and the pipeline is decomposed into clear, independently testable components (preprocessing, clustering, refinement, relabeling). The CharNet CER gains are directly measured and are a genuine, reproducible contribution. The qualitative examples (Fig. 6a) and the public release of code and data strengthen the reproducibility of the work. However, the flagging-accuracy claim for Google, which is the main evidence that a state-of-the-art commercial OCR under-uses redundancy, rests on an estimator (Eq. 3) that is biased in a way the paper does not acknowledge. Since the manuscript's broader claim—that even SOTA systems exhibit untapped redundancy—relies on this estimate, the current evidence is weaker than presented.

major comments (3)
  1. [Section 5.1, Eqs. (2)–(3) and Table 2b] The correction-accuracy estimate Acc is computed from heuristic identities that assume Ncorr = Ntrue + Nfalse. However, a relabeling can also be wrong-to-wrong (Nother), where both the original and the new OCR label disagree with ground truth. In that case Ncorr = Ntrue + Nfalse + Nother, and Eq. (2) still holds (Nother contributes zero to ΔCER), but solving Eq. (3) yields Acc_est = Ntrue/Ncorr + Nother/(2Ncorr), i.e., an upward bias equal to half the wrong-to-wrong fraction. This bias directly affects the headline Google flagging accuracies (34±2.0 on Berrutti, 31±3.8 on ENP) and the ablation study (Fig. 5), which uses Ntrue and Nfalse derived from the same equations. The paper's own failure cases in Fig. 6b (e.g., 'q'→'e', 'u'→'n', 'l'→'i') are exactly of this type when both labels are wrong. Because the Google flagging claim is load-bearing for the conclusion that SOTA OCR under-uses
  2. [Section 5.1, Table 2b and claim of statistical significance] The phrase 'statistically significant accuracy' for Google is based on bootstrap confidence intervals computed on the biased estimator of Eq. (3). With the bias in Acc_est, the true flagging accuracy could be substantially lower; for example, if wrong-to-wrong relabelings are common, the 34% could drop below the random-guess level for many alphabet subsets. The paper does not report any direct measurement of Ntrue for Google, nor does it give a lower bound under a worst-case assumption about Nother. A direct validation using ground truth is needed before the claim that Google under-uses redundancy is supportable. The CharNet CER gains are not affected by this issue, but the Google flagging results are the only quantitative evidence for the SOTA-OCR part of the central claim.
  3. [Section 3.3 and Fig. 6b] The Gaussianity assumption (Section 3) is acknowledged as central, but the failure cases in Fig. 6b show that degraded images of different glyphs can be merged into a single cluster that passes the normality tests. The paper correctly identifies this as a failure mode, but does not quantify how often it occurs. Since the refinement procedure's validity depends on the normality tests separating distinct glyphs, a quantitative estimate of the frequency of such merged clusters (or a diagnostic that detects them) would strengthen the claim that the method's core premise is satisfied in practice. This is not a blocking issue for the CharNet results, which are direct, but it limits the interpretability of the 'explainability' claims.
minor comments (4)
  1. [Supplementary material, Appendix A.1] Typo: 'as wee as' should read 'as well as' in the first paragraph of Appendix A.
  2. [Section 5.1, caption of Table 2] The caption explicitly labels the estimation as 'heuristic', which is honest, but the main text does not discuss the heuristic nature beyond the caption. The reader is left with the impression that the reported numbers are measured accuracies. A sentence acknowledging the wrong-to-wrong bias or a direct-count alternative would resolve this.
  3. [Section 4.4] The parameters H, W, s, q_variance, K, n_min, p_thr, k, f_thr are stated but not justified or varied (except K in the ablation). A short sensitivity discussion for the key thresholds (n_min, p_thr, f_thr) would help assess robustness, even if the ablation study covers K.
  4. [Figure 6] The examples in Fig. 6 are informative, but the figure does not show the ground-truth labels. Adding the ground-truth symbol for each highlighted character would make it easier to judge whether the relabelings are correct or wrong, especially for the wrong-to-wrong cases.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central corrections and CER gains are evaluated against external ground truth, and the self-cited references are not load-bearing.

full rationale

The paper's claimed derivation is not circular. The unsupervised GMM/clustering pipeline operates on character images without using ground-truth labels; the super-majority relabeling (Alg. 3) uses only OCR outputs; and the headline CER reductions (Table 2a) are measured against external ground-truth transcriptions via the standard CER metric. Parameters (H, W, s, qvariance, K, nmin, pthr, k, fthr) are set a priori and not fit to the evaluation data. The 'correction accuracy' in Eq. (3) is explicitly labeled a heuristic estimate derived from ΔCER and Ncorr; while it may be statistically biased (e.g., if wrong-to-wrong relabelings are present), it is not an input to the method and is not used as a prediction. The only relevant self-citation ([5]) is used as background support for attention using global textual information, and does not carry the central argument. No uniqueness theorem, ansatz, or fitted parameter is smuggled in via citation. Therefore the derivation chain is self-contained and no circular step is identifiable.

Assumptions & free parameters 8 free parameters · 4 assumptions · 0 invented entities

The method relies on a set of hand-chosen hyperparameters (image size, PCA variance, cluster count, thresholds) that are not fitted to test data. The domain assumptions of glyph Gaussianity, alignment convergence, and super-majority reliability are stated explicitly in Section 3. No new physical or conceptual entities are introduced.

free parameters (8)
  • Image size H, W = H=48, W=32
    Fixed dimension for normalized character images, chosen by hand.
  • Downscaling factor s = 1.2
    Power-of-s discrete downscaling factor for oversized crops, chosen by hand.
  • PCA retained variance q_variance = 0.9
    Fraction of total variance kept in PCA projection, chosen by hand.
  • Initial GMM components K = 700
    Preset number of components for k-means initialization; robustness across values 200-2000 is tested in the ablation.
  • Minimum cluster size n_min = 20
    Leaves with fewer elements are discarded; chosen by hand.
  • Normality test threshold p_thr = 2*(1-Phi(2)) ~ 0.0455
    Anderson-Darling p-value threshold for the Gaussianity test; chosen by hand as P(|N(0,1)| > 2).
  • Number of PCs for normality testing k = 9
    Number of principal components tested for normality, chosen by hand.
  • Super-majority threshold f_thr = 0.6
    Minimum majority frequency required to override OCR labels, chosen by hand.
assumptions (4)
  • domain assumption Characters of the same symbol, typeface, and font follow a Gaussian distribution in image space after proper alignment
    Explicitly assumed in Section 3, first paragraph. The entire clustering and normality-testing framework rests on this.
  • domain assumption The base OCR has higher-than-random recognition performance when restricted to any subset of the alphabet
    Used in Section 3.4 to justify majority-vote relabeling; essential for the correction rule.
  • domain assumption The Inverse Compositional Algorithm (ICA) can register images of the same glyph to a common template before clustering is reliable
    Used in Section 3.3 as the alignment step; the Gaussianity assumption is conditional on this alignment being correct.
  • standard math Standard statistical and optimization machinery: k-means, EM, PCA, OAS covariance shrinkage, Anderson-Darling test
    Unproved background results used throughout Sections 3.2 and 3.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving OCR using internal document redundancy." pith.science (2026). https://pith.science/paper/FC2MMLTJ

@misc{pith2026250814557,
  author       = {Pith},
  title        = {Pith review of: Improving OCR using internal document redundancy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FC2MMLTJ}},
  note         = {Machine review of arXiv:2508.14557}
}
read the original abstract

Current OCR systems are based on deep learning models trained on large amounts of data. Although they have shown some ability to generalize to unseen data, especially in detection tasks, they can struggle with recognizing low-quality data. This is particularly evident for printed documents, where intra-domain data variability is typically low, but inter-domain data variability is high. In that context, current OCR methods do not fully exploit each document's redundancy. We propose an unsupervised method by leveraging the redundancy of character shapes within a document to correct imperfect outputs of a given OCR system and suggest better clustering. To this aim, we introduce an extended Gaussian Mixture Model (GMM) by alternating an Expectation-Maximization (EM) algorithm with an intra-cluster realignment process and normality statistical testing. We demonstrate improvements in documents with various levels of degradation, including recovered Uruguayan military archives and 17th to mid-20th century European newspapers.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 31 canonical work pages

  1. [1]

    In: CVPR

    Baek, Y., Lee, B., Han, D., Yun, S., Lee, H.: Character region awareness for text detection. In: CVPR. pp. 9365–9374 (2019)

    work page 2019

  2. [2]

    In: CVPR

    Baker, S., Matthews, I.: Equivalence and efficiency of image alignment algorithms. In: CVPR. vol. 1, pp. 1090–1097 (2001)

    work page 2001

  3. [3]

    Berg-Kirkpatrick, T., Durrett, G., Klein, D.: Unsupervised transcription of histor- ical documents. In: ACL. vol. 1, pp. 207–217 (2013)

    work page 2013

  4. [4]

    In: ICDAR

    Bluche, T., Messina, R.: Gated convolutional recurrent neural networks for multi- lingual handwriting recognition. In: ICDAR. vol. 1, pp. 646–651 (2017)

    work page 2017

  5. [5]

    In: Document Analysis and Recognition (DAR) at ICDAR

    Bottaioli, N., Tarride, S., Anger, J., Mowlavi, S., Gardella, M., Tadros, A., Fac- ciolo, G., von Gioi, R.G., Kermorvant, C., Morel, J.M., Preciozzi, J.: Normalized vs diplomatic annotation: a case study of automatic information extraction from handwritten Uruguayan birth certificates. In: Document Analysis and Recognition (DAR) at ICDAR. pp. 40–54 (2024)

    work page 2024

  6. [6]

    Image Processing On Line8, 435–464 (2018)

    Briand, T., Facciolo, G., Sánchez, J.: Improvements of the inverse compositional algorithm for parametric motion estimation. Image Processing On Line8, 435–464 (2018)

    work page 2018

  7. [7]

    TSP58(10), 5016–5029 (2010)

    Chen, Y., Wiesel, A., Eldar, Y.C., Hero, A.O.: Shrinkage algorithms for MMSE covariance estimation. TSP58(10), 5016–5029 (2010)

    work page 2010

  8. [8]

    In: ICDAR

    Constum, T., Preel, L., Larcher, T., Paquet, T., Tranouez, P., Brée, S.: End-to-end information extraction in handwritten documents: understanding Paris marriage records from 1880 to 1940. In: ICDAR. pp. 195–214 (2024)

    work page 1940

Show all 32 references
  1. [9]

    TPAMI 45(7), 8227– 8243 (2023)

    Coquenet, D., Chatelain, C., Paquet, T.: DAN: A segmentation-free document attention network for handwritten document recognition. TPAMI 45(7), 8227– 8243 (2023)

    2023

  2. [10]

    In: Edsger Wybe Dijkstra: his life, work, and legacy, pp

    Dijkstra, E.W.: A note on two problems in connexion with graphs. In: Edsger Wybe Dijkstra: his life, work, and legacy, pp. 287–290. Association for Computing Machinery (2022)

    2022

  3. [11]

    In: ACM MM

    Dutta, A., Zisserman, A.: The VIA annotation software for images, audio and video. In: ACM MM. p. 2276–2279 (2019) Improving OCR using internal document redundancy 17

    2019

  4. [12]

    In: Goodness-of-fit-techniques, pp

    D’Agostino, R.B.: Tests for the normal distribution. In: Goodness-of-fit-techniques, pp. 367–420. Routledge (2017)

    2017

  5. [13]

    TPAMI 21(8), 752–760 (1999)

    El-Yacoubi, A., Gilloux, M., Sabourin, R., Suen, C.Y.: An HMM-based approach for off-line unconstrained handwritten word modeling and recognition. TPAMI 21(8), 752–760 (1999)

    1999

  6. [14]

    https: //cloud.google.com/vision/docs/ocr#optical_character_recognition_ocr

    Google LLC: Google Cloud Vision Document OCR, a Python client library. https: //cloud.google.com/vision/docs/ocr#optical_character_recognition_ocr

  7. [15]

    TPAMI 31(5), 855–868 (2008)

    Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. TPAMI 31(5), 855–868 (2008)

    2008

  8. [16]

    Federal Research Division, Library of Congress (1992)

    Hudson, R.A., Meditz, S.W.: Uruguay: a country study. Federal Research Division, Library of Congress (1992)

    1992

  9. [17]

    TPAMI 19(12), 1313–1324 (1997)

    Kopec, G.E., Lomelin, M.: Supervised template estimation for document image decoding. TPAMI 19(12), 1313–1324 (1997)

    1997

  10. [18]

    In: NeurIPS

    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep con- volutional neural networks. In: NeurIPS. vol. 25 (2012)

    2012

  11. [19]

    Neural Computation 1(4), 541–551 (1989)

    Le Cun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Computation 1(4), 541–551 (1989)

    1989

  12. [20]

    AAAI 37(11), 13094–13102 (2023)

    Li, M., Lv, T., Chen, J., Cui, L., Lu, Y., Florencio, D., Zhang, C., Li, Z., Wei, F.: TrOCR: Transformer-based optical character recognition with pre-trained models. AAAI 37(11), 13094–13102 (2023)

    2023

  13. [21]

    IJCV 129(1), 161–184 (2021)

    Long, S., He, X., Yao, C.: Scene text detection and recognition: the deep learning era. IJCV 129(1), 161–184 (2021)

    2021

  14. [22]

    In: Torres, M., Klapp, J

    Nesmachnow, S., Iturriaga, S.: Cluster-UY: Collaborative scientific high perfor- mance computing in Uruguay. In: Torres, M., Klapp, J. (eds.) Supercomputing. pp. 188–202 (2019), https://cluster.uy/

    2019

  15. [23]

    ACM Computing Surveys (CSUR)54(6), 1–37 (2021)

    Nguyen, T.T.H., Jatowt, A., Coustaty, M., Doucet, A.: Survey of post-OCR pro- cessing approaches. ACM Computing Surveys (CSUR)54(6), 1–37 (2021)

    2021

  16. [24]

    ACM TOG 22(3), 313–318 (2003)

    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM TOG 22(3), 313–318 (2003)

    2003

  17. [25]

    Psychonomic Bulletin & Review21, 1112–1130 (2014)

    Piantadosi, S.T.: Zipf’s word frequency law in natural language: a critical review and future directions. Psychonomic Bulletin & Review21, 1112–1130 (2014)

    2014

  18. [26]

    In: ICDAR

    Rigaud, C., Doucet, A., Coustaty, M., Moreux, J.P.: ICDAR 2019 competition on post-OCR text correction. In: ICDAR. pp. 1588–1593 (2019)

    2019

  19. [27]

    In: Medical Image Computing and Computer-Assisted Intervention (MICCAI)

    Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional networks for biomed- ical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). pp. 234–241 (2015)

    2015

  20. [28]

    In: ICDAR

    Siglidis, I., Gonthier, N., Gaubil, J., Monnier, T., Aubry, M.: The learnable type- writer: a generative approach to text analysis. In: ICDAR. pp. 297–314 (2024)

    2024

  21. [29]

    In: ICDAR

    Smith, R.: An overview of the Tesseract OCR engine. In: ICDAR. vol. 2, pp. 629– 633 (2007)

    2007

  22. [30]

    In: NeurIPS

    Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: NeurIPS. vol. 30 (2017)

    2017

  23. [31]

    In: NeurIPS

    Volpi, R., Namkoong, H., Sener, O., Duchi, J.C., Murino, V., Savarese, S.: Gener- alizing to unseen domains via adversarial data augmentation. In: NeurIPS. vol. 31 (2018)

    2018

  24. [32]

    proto-lines

    Xing, L., Tian, Z., Huang, W., Scott, M.R.: Convolutional character networks. In: ICCV. pp. 9126–9136 (2019) 18 D. Belzarena, S. Mowlavi et al. Supplementary material This supplementary material includes additional quantitative results and details on our methods. Appendix A pr...

    2019

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.