{"id":"a1d7a625-1857-4d65-ac42-1e20448b1d84","arxiv_id":"2509.02356","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A new 474-million-word corpus makes two centuries of Danish and Norwegian newspapers searchable for the first time using neural text recognition.","lead":"This paper introduces ENO, a new digital corpus of 474 million words from Danish and Norwegian newspapers printed between 1660 and 1849, reconstructed from microfilm using neural text recognition. The dataset is offered as a searchable resource for historians and language researchers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Validation set is drawn from training newspapers, so the reported 0.6% CER may not generalize to underrepresented corners; an external held-out test is needed before the 'fully searchable' quality claim is accepted.","rationale":"The paper is a data descriptor, and its central value proposition is that a previously illegible corpus has been turned into a high-quality, fully searchable text dataset. The scale (474M words, 565k pages) is plausible given the described pipeline, and the authors are appropriately transparent about coverage choices, the sampling of Københavns Adresseavis, and the known weaknesses of the segmentation and PWA heuristics. The dataset and models are openly released, which is strong evidence of reproducibility. However, the one figure that carries the load—the 0.6% CER—is measured on a validation set that is not independent of the training distribution, and the authors explicitly identify scan/print corners that fall outside that distribution. The PWA indicator is clever but cannot substitute for a direct error-rate measurement on those corners; it measures dictionary-likeness, not fidelity to the source image. A concrete external evaluation on the named problematic sources would either confirm the current claim or force a precise scope restriction. The reader's verdict of CONDITIONAL captures exactly this: the dataset is useful and open, but the headline accuracy figure is not yet established for the full corpus. My read does not change that verdict; if anything, it sharpens the specific test needed before the claim can be upgraded. I agree with the reader that the weakest assumption is the representativeness of the validation data, and I have not identified a separate, more serious flaw in the argument.","tokens_in":6491,"tokens_out":2400,"duration_ms":32202,"concrete_test":"Construct an external held-out evaluation set from the under-represented corners: randomly sample, say, 30 pages from each of the three Norwegian titles and 30 pages from Nyeste Skilderie af Kiøbenhavn (Internet Archive scans). Have a historical-typography expert produce ground-truth line/character transcripts for these pages, ensuring none were used in training. Run the released Transkribus model on these pages and compute CER/WER. If CER on this sample is substantially above 0.6% (e.g., >2–5%), the quality claim must be restricted to the Danish newspaper collection and the dataset's 'fully searchable' characterization qualified per title. As a secondary check, compare the paper's PWA indicator against human-judged word correctness on these same pages to see whether it reliably flags the low-quality corners.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quality claim depends on the 0.6% character error rate reported in the Validation section. That figure comes from 'validation data, made up of additional transcriptions from the newspapers included in the training material'—i.e., the same titles, and likely the same page/scan distributions, used to train the recognition model. This is not an independent test set. The authors themselves flag the gap: 'some corners of the dataset contain both types of print and scans different from those in the validation data,' specifically the three Norwegian papers (newer photographs) and the warped Internet Archive scans of Nyeste Skilderie af Kiøbenhavn. The PWA indicator does not close this gap: it is a dictionary-matching heuristic built in part from an early version of the ENO corpus itself, so it flags whether output words are recognizable period words, not whether they are the words actually on the page; confusable real-word errors and out-of-dictionary proper nouns pass through silently. Thus, the load-bearing assumption is that validation CER transfers to all 28 titles and all scan types. The paper provides no direct evidence for this transfer, so the 'fully searchable' headline and the comparison to a 60M-word corpus rest on an unverified generalization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Enevældens Nyheder Online (ENO), a dataset of approximately 474 million words reconstructed from about 565,000 pages of 28 Danish and Norwegian newspapers from the period 1660–1849. The authors describe a pipeline built on Transkribus neural text-recognition models, a line-level segmentation approach that combines structural features with a BERT-based model, and a word-level predicted-word-accuracy (PWA) indicator. The central claims are that the corpus is 'fully searchable,' that it is 'by far the largest fully searchable text dataset in historical Danish dating from before 1900,' and that the text recognition achieves a 0.6% character error rate. The paper also documents coverage, usage notes, a similar-text recommendation feature, and provides public access to the data and models.","tokens_in":6798,"tokens_out":2885,"duration_ms":33571,"significance":"If the accuracy and coverage claims hold, this is a major infrastructural contribution to Danish and Norwegian historical scholarship, enabling full-text search, text mining, and language modeling on a corpus that was previously largely unusable for computational purposes. The paper is commendable for making the data and models openly available, for providing per-text quality indicators, and for explicitly warning about historical bias in the corpus. The comparison to the next-largest pre-1900 Danish corpus (about 60 million words) makes the potential impact clear. The central quantitative claim, however, rests on a validation set that is not independent of the training data, and the corpus-wide quality indicator is partly constructed from the corpus itself. The significance of the resource is therefore currently clouded by an unverified generalization of the reported error rate.","major_comments":[{"comment":"The reported 0.6% character error rate is computed on 'validation data, made up of additional transcriptions from the newspapers included in the training material.' This is not an independent test set: the same titles and likely the same scan characteristics are over-represented. The authors themselves note that 'some corners of the dataset contain both types of print and scans different from those in the validation data,' specifically the three Norwegian papers and the warped Internet Archive scans of Nyeste Skilderie. Since the headline claims of 'fully searchable' quality and the comparison to a 60M-word corpus depend on corpus-wide accuracy, the paper must either provide a held-out evaluation stratified by title and scan type, or clearly restrict the accuracy claim to the validation subcorpus.","section":"Validation"},{"comment":"The PWA dictionary is built in part from 'the 10.000 most common words in an early version of the corpus' that were not already in canonical literature or census sources. This introduces a circularity: the dictionary can recognize the model's own normalized output, but it cannot detect real-word substitutions that happen to be legitimate period words. The paper acknowledges that the method is 'somewhat simple,' but it is then used to support the statement that 'the data is of a high and consistent quality' and to produce the distribution in Figure 5. The PWA indicator is useful as a descriptive relative measure across texts, but it is not a substitute for independent ground-truth evaluation on atypical scans and titles.","section":"Validation (PWA indicator)"},{"comment":"The segmentation pipeline reports an f1-score of 98.9% on 'a validation dataset of about 7,000 manually tagged lines.' Because the BERT model was trained on 260 million words from the initial segmentation, and the setfit models were trained on a manually tagged dataset of about 30,000 lines, it is not clear whether the 7,000 validation lines are disjoint from both the setfit training data and the BERT pretraining corpus. This matters because the paper itself describes the 'positive feedback loop' in which the initial segmentation informed a better tool. The authors should state explicitly that the validation lines were not used in any training stage and, ideally, report performance broken down by publication and layout type.","section":"From pages to texts"}],"minor_comments":[{"comment":"'Enevaeldens' is spelled inconsistently with the later 'Enevældens'; the abstract also contains the grammar error 'The contributions details this process.'","section":"Abstract"},{"comment":"'Tronheim' should be 'Trondheim'.","section":"Methods"},{"comment":"The newspaper title appears as both 'Nyeste Skilderie af Kiøbenhavn' and 'Nyeste Skilderie after Kiøbenhavn'; the latter appears to be a typo for 'af'.","section":"Validation"},{"comment":"Reference 3 contains a typo: 'Digitsed' should be 'Digitised'.","section":"References"},{"comment":"Figure 5 would benefit from explicit axis labels and a definition of the plotted quantity (e.g., median PWA per text? per publication?), and the text should state the size and coverage of the dictionary used for the PWA indicator.","section":"Validation / Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a data descriptor, so the bar for quantitative verification should be high: the central value of the dataset is its claimed usability for search and text mining. The current validation does not establish corpus-wide accuracy because the test transcripts come from the training newspapers. The authors have been transparent about the limitations, which is to their credit, but the distinction between 'validation' and 'test' needs to be made rigorous. I would support publication after the authors provide an independent held-out evaluation or, failing that, significantly soften the 'fully searchable' and 'similar quality' claims to reflect the fact that the error rate is only measured on the validation subcorpus."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know about this paper because it ships a very large, openly available historical newspaper corpus, and because the central accuracy claim needs a skeptical read.\n\nWhat's new: 474M words from 28 Danish/Norwegian newspapers covering 1660–1849, produced with Transkribus-based neural OCR on microfilm that previous OCR made useless. That is a real contribution. The dataset, models, BERT embeddings, code, and a public search interface are all released, and the integration with the Danish Dynaword corpus is a plus. The authors are also unusually candid about bias and about the corners they expect to be bad—the Norwegian papers from newer photographs and the warped Internet Archive scans of Nyeste Skilderie.\n\nThe soft spots are in the validation story. The reported 0.6% character error rate comes from validation transcriptions drawn from the same newspapers used in training. That is not an independent test set. The in-house PWA indicator is built partly from an early version of the corpus itself, so it can catch gibberish but not real-word errors or rare names. The paper acknowledges both of these limitations, but the headline claim of a 'fully searchable' corpus rests on an unverified transfer from validation to the atypical scans. That transfer might hold, but the paper gives no direct evidence. A proper held-out test sampled across all 28 titles and scan types—including the Norwegian and warped pages—would close the gap. The segmentation F1 of 98.9% on 7,000 manually tagged lines looks more credible, though I'd want confirmation that those lines were not part of the 30,000 training tags.\n\nThe flaws are in the validation narrative, not in the artifact. The dataset is new, openly licensed, and the per-series quality flags give users a way to hedge their own risk. Historians and computational linguists working on early modern Danish or Norwegian will find this transformative despite the open accuracy question.\n\nSend it to peer review. A good reviewer will push for the external held-out test and for precise wording about what 'fully searchable' means. If you work in this area, cite it and use the corpus—just don't repeat the 0.6% figure as if it were measured on independent ground truth.","headline":"A genuinely useful corpus with an honest write-up, but the headline accuracy number is not yet backed by independent validation.","tokens_in":7204,"tokens_out":1762,"would_cite":true,"duration_ms":23784,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims a neural pipeline turns illegible fraktur microfilm into 474 million words of searchable Danish history.","keywords":["historical newspapers","Danish-Norwegian corpus","optical character recognition","fraktur typeface","text segmentation","digital humanities","Absolutism era","data descriptor"],"falsifier":"An independent, stratified sample of pages from the under-validated corners—Norwegian papers, warped or low-contrast scans, capitalised-antiqua advertisements—transcribed by a human reader and compared with the model output character-by-character would settle the quality claim. If the independent character error rate is substantially above 0.6%, the 'fully searchable' description does not hold for the whole corpus.","tokens_in":6438,"feed_emoji":"📰","tokens_out":9018,"duration_ms":94972,"temperature":0.7,"pith_summary":"Most Danish newspapers printed before 1900 were photographed on microfilm decades ago, then run through standard text-recognition software that produced roughly 50% word accuracy—too broken to search. This paper reports a neural text-recognition pipeline that reads the same microfilm with far higher accuracy, and uses it to reconstruct about 474 million words from 28 Danish and Norwegian newspapers published between 1660 and 1849. The authors state this is by far the largest fully searchable text dataset in historical Danish before 1900, dwarfing the previous best of about 60 million words. Alongside the transcribed text, the dataset provides standalone texts split from the page layouts and a per-text word-accuracy score, so users can filter out passages where recognition failed. The dataset is released with its models, and the authors argue it creates a new foundation for computational history of Denmark-Norway.","feed_headline":"474 million words of old Danish print become searchable","feed_subtitle":"Black-letter microfilm that stumped standard readers now opens 1660–1849 newspapers to search and text mining.","key_machinery":"The central machinery is the recognition and segmentation pipeline: (1) a neural text-recognition model trained on about 420,000 words of fraktur print (the black-letter typeface used in Danish newspapers) and intermittent antiqua, which transcribes microfilm scans; (2) a line-classification model that combines structural features—capitalisation, line length, punctuation—with contextual predictions from a masked-language model to decide where a text begins and ends; (3) a dictionary-based word-accuracy indicator that estimates recognition quality per text without ground truth, letting users compare and filter series.","core_discovery":"The dataset's central claim is that the ENO corpus—474 million words from about 565,000 pages across 28 titles—is the largest fully searchable text collection in historical Danish before 1900, and that it is accurate enough to be used as text. The authors report a character error rate of about 0.6% on validation data, a line-level segmentation F1 of 98.9%, and a per-text word-accuracy indicator built from period dictionaries. They also describe a masked-language model trained on an early version of the corpus, used to improve segmentation into standalone texts and, in future, to correct recognition errors.","pith_inferences":["The per-text quality indicator could be reused as a cheap, ground-truth-free quality audit for other historical OCR collections, since it only needs a period dictionary.","If the recognition model generalises beyond the validation set, the same approach may work on other European fraktur newspapers whose scans are of comparable or worse quality; the authors' own caution about Norwegian and warped scans suggests the boundary should be tested explicitly.","A filtered subset of this corpus could serve as training data for historical Danish language models, which in turn could improve search and error correction in the wider digitised archive.","The segmentation-plus-similarity interface tests a workflow in which historians move horizontally from one text to similar texts rather than only keyword search; if it proves useful, similar 'similar-text' layers could be added to other newspaper archives."],"forward_implications":["The full corpus becomes searchable by any word, phrase, or regular expression, letting historians find mentions of people, goods, prices, events, and ideas across 28 newspapers.","Each text carries a quality score, so users can filter to high-accuracy passages for language modelling and quantitative text analysis; the authors recommend aggressive filtering for such uses.","Because texts are segmented into standalone units rather than whole editions, researchers can track genres such as labour advertisements, runaway notices, and consumption ads over time.","The trained recognition and segmentation models are released with the data, so other institutions with similar fraktur or microfilm scans can apply the same pipeline.","The inclusion of Norwegian papers up to 1814 allows comparisons across the two sides of the Denmark-Norway state within one corpus."],"supporting_citations":[{"why":"Documents that re-processed newspaper corpora elsewhere became rich sources for serial information and computational history, motivating the dataset's usefulness.","marker":"3"},{"why":"Provides the comparison baseline—a roughly 60-million-word corpus of novels 1870–1900—against which the 474-million-word claim is the largest before 1900.","marker":"7"},{"why":"The toolchain on which the text recognition models were trained, providing the infrastructure for the method.","marker":"8"},{"why":"Supplies the shared neural text-recognition infrastructure and community method that the authors adapted from handwriting to fraktur print.","marker":"9"},{"why":"Defines the universe of Danish newspapers 1634–1989 that guides which series are included for coverage.","marker":"12"},{"why":"The masked-language model trained on an early corpus version; the segmentation pipeline fine-tunes it to classify line beginnings and boundaries.","marker":"13"},{"why":"Provides canonical period literature used to build the dictionary behind the paper's word-accuracy quality indicator.","marker":"15"}],"fun_headline_variants":["474M words of Danish absolutism-era news now searchable","1660–1849 Danish papers: 474M words finally machine-readable","Black-letter microfilm beaten: 474M words of Danish history","From illegible microfilm to searchable text: 474M Danish words","Danish-Norwegian newspapers 1660–1849: 474M words unlocked"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The claim collapses if the 0.6% character error rate measured on validation pages, which come from the newspapers already in the training data, does not apply to the rest of the corpus—especially Norwegian papers, warped scans, and capitalised antiqua passages, which the authors themselves flag as different.","fun_headline_variants_meta":{"raw":{"variants":["474M words of Danish absolutism-era news now searchable","1660–1849 Danish papers: 474M words finally machine-readable","Black-letter microfilm beaten: 474M words of Danish history","From illegible microfilm to searchable text: 474M Danish words","Danish-Norwegian newspapers 1660–1849: 474M words unlocked"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000143,"raw_usage":{"total_tokens":955,"prompt_tokens":635,"completion_tokens":320,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":379,"completion_tokens_details":{"reasoning_tokens":222}},"tokens_in":379,"tokens_out":320,"duration_ms":4808,"temperature":1.0,"reasoning_tokens":222,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T11:34:13.781281+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An independent, stratified sample of pages from the under-validated corners—Norwegian papers, warped or low-contrast scans, capitalised-antiqua advertisements—transcribed by a human reader and compared with the model output character-by-character would settle the quality claim. If the independent character error rate is substantially above 0.6%, the 'fully searchable' description does not hold for the whole corpus.","supporting_citations":[],"review_version":1}