LV-ROVER-MLT: Low-Resource Maltese OCR by Multi-Stream Voting
Pith reviewed 2026-07-03 21:43 UTC · model grok-4.3
The pith
A 5-stream Tesseract voting ensemble plus post-processing reduces Maltese OCR character error rate by 70 percent on the development benchmark.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By training on synthetically generated Maltese pages and combining five Tesseract streams under a lexicon-anchored ROVER-style voting scheme, followed by a two-stage post-processing chain that aligns quote and dash conventions and restores diacritics, the system reaches a character error rate of 0.00700 on the 422-paragraph benchmark, a 70 percent reduction from the fine-tuned single-model baseline of 0.0234.
What carries the argument
Lexicon-anchored ROVER-style voting across five Tesseract streams, followed by quote-alignment and diacritic-recovery post-processing.
If this is right
- Ensemble voting alone produces a 44 percent character error rate reduction to 0.01317 under the same label convention as the baseline.
- The complete pipeline that includes post-processing reaches 0.00700 character error rate.
- The unchanged method yields a 33.7 percent character error rate improvement on Luxembourgish that a bootstrap and permutation audit confirms as statistically significant.
- On Hungarian the same method produces only a 0.8 percent margin that is not statistically significant.
Where Pith is reading between the lines
- The synthetic-data-plus-voting pattern may transfer to other low-resource languages that share similar Latin-script diacritics and quote conventions.
- Gains from the voting scheme are likely to shrink if the synthetic page distribution diverges markedly from the documents encountered at test time.
- A direct comparison of the method against modern end-to-end neural OCR models on the same Maltese benchmark would clarify whether the multi-stream approach remains competitive once more training data becomes available.
Load-bearing premise
The assumption that results measured on the competition's 422-paragraph development set will hold on the held-out real test set and that the synthetic data distribution is close enough to real Maltese documents for the voting scheme to generalize.
What would settle it
Running the full pipeline on the competition's held-out real test set and observing whether the character error rate stays near 0.00700 or rises substantially.
Figures
read the original abstract
Maltese, although a low-resource language, has its own text corpora and pretrained language models, but we are aware of only one real labelled PDF corpus for OCR training, 57 pages, far below what paragraph-level training needs. With no real corpus to train on at scale, we built a synthetic training pipeline and a 5-stream Tesseract ensemble voted under a lexicon-anchored, ROVER-style scheme adapted for a low-resource setting. We call the Maltese submission LV-ROVER-MLT: an engineered adaptation of LV-ROVER's voting algorithm, not a new one, submitted to the DocEng 2026 competition. All results below are dev-set figures from the competition's own benchmark; the held-out real test CER is unknown at the time of writing and this paper does not claim one. We report results on a 422-paragraph benchmark against a fine-tuned Tesseract baseline with a character error rate of 0.0234. Ensemble recognition alone, scored under the same label convention as the baseline, improves character error rate by 44 percent to 0.01317. A post-processing chain that aligns Tesseract's straight-quote and dash output to the benchmark's curly-quote convention, plus one stage that recovers misread diacritics, brings the full pipeline to a character error rate of 0.00700, a 70 percent reduction. We also tested the same method, unchanged, on Hungarian and Luxembourgish: a bootstrap and permutation audit confirms a 33.7 percent character error rate improvement on Luxembourgish, while the Hungarian margin, 0.8 percent, is not statistically significant.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents LV-ROVER-MLT, an adaptation of the LV-ROVER multi-stream voting algorithm for Maltese OCR under low-resource constraints. With only 57 real labelled pages available, the authors train on synthetic data using a 5-stream Tesseract ensemble under a lexicon-anchored ROVER-style voting scheme, followed by post-processing. On the DocEng 2026 competition's 422-paragraph dev set they report a CER reduction from a fine-tuned Tesseract baseline of 0.0234 to 0.00700 (70 percent), with additional unchanged-method results on Hungarian (insignificant 0.8 percent gain) and Luxembourgish (statistically confirmed 33.7 percent gain via bootstrap/permutation audit). All figures are explicitly dev-set only; no held-out test CER is claimed.
Significance. If the dev-set results are taken at face value, the work supplies concrete, auditable CER numbers and a permutation test for a low-resource language OCR task that relies on synthetic data and ensemble voting. These empirical strengths are useful for competition-style engineering. However, because the largest reported gain incorporates benchmark-specific post-processing and because no test-set evaluation is provided, the significance for general methodological advance in low-resource OCR remains limited.
major comments (3)
- [Abstract] Abstract: the 70 percent CER reduction to 0.00700 is achieved only after a post-processing chain that realigns Tesseract straight-quote/dash output to the benchmark's curly-quote convention and recovers diacritics; this step is benchmark-specific and therefore load-bearing for the headline improvement, weakening any claim that the voting scheme alone delivers the gain.
- [Abstract] Abstract: the Hungarian experiment yields only a 0.8 percent margin that the authors themselves state is not statistically significant; this result undercuts the cross-language applicability claim and should be either removed from the summary or accompanied by explicit qualification.
- [Abstract] Abstract (and method description): training relies on synthetic data because only 57 real pages exist, yet no quantitative comparison of synthetic versus real Maltese document distributions (layout, fonts, orthography) is supplied; this match is load-bearing for the reliability of the ROVER-style voting on the dev set.
minor comments (1)
- A compact table listing CER at each pipeline stage (baseline, ensemble, post-processed) for all three languages would improve readability.
Simulated Author's Rebuttal
We thank the referee for the detailed and constructive comments. We address each major comment below, indicating where revisions will be incorporated. All results remain dev-set only as stated in the manuscript, and we do not claim test-set performance.
read point-by-point responses
-
Referee: [Abstract] Abstract: the 70 percent CER reduction to 0.00700 is achieved only after a post-processing chain that realigns Tesseract straight-quote/dash output to the benchmark's curly-quote convention and recovers diacritics; this step is benchmark-specific and therefore load-bearing for the headline improvement, weakening any claim that the voting scheme alone delivers the gain.
Authors: We agree that the post-processing is benchmark-specific and contributes substantially to the reported 70% reduction. The manuscript already separates the ensemble-only result (44% CER reduction to 0.01317 under identical label conventions) from the full pipeline. We will revise the abstract to foreground the ensemble contribution and explicitly qualify the 70% figure as including post-processing. revision: yes
-
Referee: [Abstract] Abstract: the Hungarian experiment yields only a 0.8 percent margin that the authors themselves state is not statistically significant; this result undercuts the cross-language applicability claim and should be either removed from the summary or accompanied by explicit qualification.
Authors: The manuscript already notes that the Hungarian margin is not statistically significant. To prevent any overstatement of cross-language applicability, we will revise the abstract either to remove the Hungarian result from the summary or to add stronger explicit qualification. revision: yes
-
Referee: [Abstract] Abstract (and method description): training relies on synthetic data because only 57 real pages exist, yet no quantitative comparison of synthetic versus real Maltese document distributions (layout, fonts, orthography) is supplied; this match is load-bearing for the reliability of the ROVER-style voting on the dev set.
Authors: A quantitative distributional comparison would be desirable. However, the extremely limited real labelled data (57 pages) precludes a statistically robust comparison of layout, font, and orthographic distributions. The synthetic pipeline was constructed to replicate observable properties of the available real documents. We will add an explicit discussion of this limitation and the constraints of the low-resource setting in the revised manuscript. revision: partial
Circularity Check
No circularity; purely empirical benchmark reporting on dev set
full rationale
The paper presents only empirical CER measurements on the competition's 422-paragraph dev set, with explicit statements that held-out test CER is unknown and unclaimed. No equations, derivations, fitted parameters, or first-principles results are described. The method is described as an engineering adaptation of prior LV-ROVER voting (not claimed as novel), and post-processing steps are rule-based alignments to benchmark conventions rather than learned or self-referential. No load-bearing self-citations or reductions of results to inputs by construction appear. This matches the default case of a self-contained empirical report.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Synthetic pages generated from existing Maltese text corpora produce training distributions sufficiently close to real documents for Tesseract fine-tuning.
- domain assumption A Maltese lexicon can be used to anchor voting decisions without introducing new errors on out-of-vocabulary words.
Reference graph
Works this paper leans on
-
[1]
Merveilles Agbeti-Messan, Pierrick Tranouez, Stéphane Nicolas, Clément Chatelain, and Thierry Paquet. 2026. A Benchmark of State-Space Mod- els vs. Transformers and BiLSTM-based Models for Historical Newspaper OCR. arXiv:2604.00725. arXiv: 2604.00725
work page internal anchor Pith review Pith/arXiv arXiv 2026
-
[2]
Jeonghun Baek, Geewook Kim, Junyeop Lee, Sungrae Park, Dongyoon Han, Sangdoo Yun, Seong Joon Oh, and Hwalsuk Lee. 2019. What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision (ICCV) . IEEE, Seoul, Korea (South), 4715– 4723
work page 2019
-
[3]
Birhanu Belay, Tewodros Habtegebrial, Million Meshesha, Marcus Liwicki, Gebeyehu Belay, and Didier Stricker. 2020. Amharic OCR: An End-to-End Learning. Applied Sciences 10, 3 (2020), 1117. doi:10.3390/app10031117
-
[4]
Edouard Belval. 2019. TRDG: Text Recognition Data Generator. https: //github.com/Belval/TextRecognitionDataGenerator. GitHub repository. Open-source synthetic text image generator for line crops. Accessed 2026- 07-02
work page 2019
-
[5]
Yoav Benjamini and Yosef Hochberg. 1995. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society: Series B (Methodological) 57, 1 (1995), 289–
work page 1995
-
[6]
doi:10.1111/j.2517-6161.1995.tb02031.x
-
[7]
Claudia Borg and Albert Gatt. 2017. Morphological Analysis for the Mal- tese Language: The Challenges of a Hybrid System. In Proceedings of LV-ROVER-MLT: Low-Resource Maltese OCR by Multi-Stream Voting T able 6: F our findings: expected outcome, what the data showed, and what changed. Expected F ound Changed Joiner would dominate hyphen- bucket errors; pl...
work page 2017
-
[8]
Denis Coquenet, Clément Chatelain, and Thierry Paquet. 2023. Faster DAN: Multi-target Queries with Document Positional Encoding for End- to-end Handwritten Document Recognition. In International Conference on Document Analysis and Recognition (ICDAR) . Springer, Cham, 196–
work page 2023
-
[9]
doi:10.1007/978-3-031-41685-9_12
-
[10]
DocEng 2026 Organisers. 2026. DocEng 2026 Competition on Maltese Paragraph OCR. ACM Symposium on Document Engineering. Competi- tion task description and baselines
work page 2026
-
[11]
Bradley Efron and Robert J. Tibshirani. 1993. An Introduction to the Bootstrap. Number 57 in Monographs on Statistics and Applied Probabil- ity. Chapman and Hall/CRC, New York. doi:10.1201/9780429246593
-
[12]
Jonathan G. Fiscus. 1997. A Post-Processing System to Yield Reduced Word Error Rates: Recognizer Output Voting Error Reduction (ROVER). In Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) . IEEE, Santa Barbara, CA, 347–354. doi:10. 1109/ASRU.1997.659110
-
[13]
Ankush Gupta, Andrea Vedaldi, and Andrew Zisserman. 2016. Synthetic Data for Text Localisation in Natural Images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, Las Vegas, NV, 2315–2324
work page 2016
-
[14]
JaidedAI. 2024. EasyOCR: Ready-to-use OCR with 80+ supported lan- guages. https://github.com/JaidedAI/EasyOCR. GitHub repository, v1.7.2, released 2024-09-24. Accessed 2026-07-02
work page 2024
-
[15]
Philipp Koehn. 2004. Statistical Significance Tests for Machine Translation Evaluation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP) . Association for Computational Linguistics, Barcelona, Spain, 388–395. https://aclanthology.org/W04- 3250
work page 2004
-
[16]
Kenton Lee, Mandar Joshi, Iulia Turc, Hexiang Hu, Fangyu Liu, Ju- lian Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. 2023. Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding. In Proceedings of the International Conference on Machine Learning (ICML) . PMLR, Honolulu, HI, 18893– 18912
work page 2023
- [17]
-
[18]
Minghao Li, Tengchao Lv, Jingye Chen, Lei Cui, Yijuan Lu, Dinei Floren- cio, Cha Zhang, Zhoujun Li, and Furu Wei. 2023. TrOCR: Transformer- based Optical Character Recognition with Pre-trained Models. In Proceed- ings of the AAAI Conference on Artificial Intelligence . AAAI Press, Wash- ington, DC, 13094–13102. doi:10.1609/aaai.v37i11.26538
-
[19]
Anna Madarász, Noémi Ligeti-Nagy, András Holl, and Tamás Váradi
-
[20]
In Natural Scientific Language Processing and Research Knowledge Graphs
OCR Cleaning of Scientific Texts with LLMs. In Natural Scientific Language Processing and Research Knowledge Graphs . Springer, Cham. doi:10.1007/978-3-031-65794-8_4
-
[21]
Maltese Language Resource Server. 2025. malti: A Python library for processing text in the Maltese language. https://github.com/MLRS/malti. Accessed 2026-07-02
work page 2025
-
[22]
Yonatan Haile Medhanie and Yuanhua Ni. 2026. Adapting TrOCR for Printed Tigrinya Text Recognition: Word-Aware Loss Weighting for Cross- Script Transfer Learning. arXiv:2604.20813. arXiv: 2604.20813
work page internal anchor Pith review Pith/arXiv arXiv 2026
-
[23]
Kurt Micallef, Albert Gatt, Marc Tanti, Lonneke van der Plas, and Claudia Borg. 2022. Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and BERT Models for Maltese. In Proceedings of the Third Workshop on Deep Learning for Low-Resourced Natural Lan- guage Processing (DeepLo) . Association for Computational Linguistics, Dublin,...
work page 2022
-
[24]
MLRS. 2026. MLRS/korpus_malti: General Maltese corpus, version 4.2. https://huggingface.co/datasets/MLRS/korpus_malti. HuggingFace dataset card. Accessed 2026-07-02
work page 2026
- [25]
-
[26]
Ray Smith, Daria Antonova, and Dar-Shyang Lee. 2009. Adapting the Tesseract Open Source OCR Engine for Multilingual OCR. In Proceedings of the International Workshop on Multilingual OCR . ACM, Barcelona, Spain, 1–8. doi:10.1145/1577802.1577804
-
[27]
Bruno Stuner, Clément Chatelain, and Thierry Paquet. 2020. L V-ROVER: Lexicon Verified Recognizer Output Voting Error Reduction. International Journal on Document Analysis and Recognition (IJDAR) 23, 1 (2020), 1–
work page 2020
-
[28]
doi:10.1007/s10032-019-00344-9
-
[29]
Marc Tanti, Claudia Borg, and Albert Gatt. 2023. NOMOCRAT: New Open Maltese OCR Annotated Text. https://www.systemsandcontrol. com/post/nomocrat-new-open-maltese-ocr-annotated-text . Maltese Lan- guage Resource Server / SystemsAndControl project; no formal proceed- ings venue identified. Accessed 2026-07-02
work page 2023
-
[30]
Nik Vaessen. 2025. jiwer: Evaluate automatic speech recognition systems. https://github.com/jitsi/jiwer. v4.0.0, Python package for CER/WER computation. Accessed 2026-07-02
work page 2025
-
[31]
Moonbin Yim, Yoonsik Kim, Han-Cheol Cho, and Sungrae Park. 2021. Syn- thTIGER: Synthetic Text Image Generator Towards Better Text Recog- nition Models. In International Conference on Document Analysis and Recognition (ICDAR) . Springer, Cham, 109–124. doi:10.1007/978-3-030- 86337-1_8 A L V-ROVER-ML T Inference Algorithm Algorithm 1 gives the full per-imag...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.