Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Multilingual Attribute Extraction from News Web Pages

T0 review · 2 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A multilingual DOM-LM extracts news attributes across six languages better than English-only MarkupLM and open-source tools, with no translation step.

desk verdict A useful new multilingual news extraction dataset, but DOM-LM's reported edge over baselines is weakened by unsupervised pre-training on the same pages used for evaluation. read the letter →

arxiv 2502.02167 v1 pith:CR4EWVHK submitted 2025-02-04 cs.CL cs.IR

Pavel Bedrin , Maksim Varlamov , Alexander Yatskov This is my paper · ORCID
classification cs.CLcs.IR
keywords webdataextractioninformationnewspagesmultilingualdatasetDOM-LMMarkupLMXLM-RoBERTatransformerfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper builds a public multilingual benchmark for news-attribute extraction—3,172 labeled article pages in English, German, Russian, Chinese, Korean, and Arabic from 161 sites—and asks whether a single model can extract title, publication date, main text, authors, and tags across languages without translation. It finds that MarkupLM, an English-only pre-trained model, handles non-English pages poorly unless the pages are machine-translated to English first. The authors then re-initialize DOM-LM, a DOM-structure-aware transformer, from a multilingual language model and pre-train it on multilingual news pages. After fine-tuning, this multilingual DOM-LM matches or beats MarkupLM within most languages and, on mixed-language data, beats MarkupLM, a text-only multilingual transformer, and open-source heuristic tools on most attributes, with no translation overhead.

What carries the argument

The load-bearing mechanism is the DOM-LM representation: a cleaned DOM tree is split into subtrees by a DOM Tree Processor, each node is encoded as the concatenation of its HTML tag, attributes, and text plus tree-position features such as depth, index, and parent index, and the whole sequence is passed through XLM-RoBERTa. The model is trained as token classification, with a node's label placed on a BOS token and prediction read from that token. For the multilingual version, the authors initialize from a multilingual language model and pre-train on raw news HTML from the labeled dataset plus a one-day sample of tens of thousands of multilingual news pages. This lets the model use layout and structure cues together with multilingual semantic cues, which the paper argues is what makes cross-lingual extraction work without translating pages.

What would settle it

Re-run the mixed-language 5-fold evaluation with DOM-LM pre-trained only on a corpus that is disjoint from all evaluation pages (for instance, only the sample of raw multilingual news pages or only the training folds). If its F1 on text, author, and tag falls to or below MarkupLM's with translation, the paper's main claim fails; if the score holds, the result is genuine.

Watch

Extended reading notes

Core claim

The paper's central claim is that a DOM-aware transformer can be made multilingual by swapping its English backbone for a multilingual encoder and pre-training on a small multilingual news corpus, and that the resulting model is the best tested open-source approach for extracting news attributes when the page language is unknown. In 5-fold site-level cross-validation on mixed-language data, multilingual DOM-LM reaches F1 scores of 0.93 on title, 0.87 on date, 0.93 on text, 0.41 on author, and 0.73 on tag, the highest among tested neural models and libraries on most attributes. MarkupLM needs English translation to compete, and even then leads only on title and date. Within single-language experiments the paper does not claim a clear leader; the advantage is most pronounced in the mixed-language setting. The dataset itself—3,172 pages across six languages and 161 websites—is contributed as a reusable benchmark.

Load-bearing premise

The comparison assumes that pre-training DOM-LM on the full dataset—including pages that later appear in the 5-fold evaluation—does not inflate its measured quality; if that pre-training leaks information, the reported lead over MarkupLM and the heuristic tools is overstated.

Editorial extensions

If this is right

  • A single multilingual model can replace per-language models in news aggregation and media analysis, since DOM-LM reaches F1 0.93 on title and text, 0.87 on date, 0.41 on author, and 0.73 on tag in mixed-language 5-fold evaluation without translation.
  • Machine translation at inference remains a viable path: MarkupLM with English translation attains 0.95 on title and 0.88 on date, slightly above DOM-LM's 0.93 and 0.87, respectively, at the cost of translation time and compute.
  • Open-source heuristic tools are not sufficient for metadata: the three tested libraries score below the neural models on dates, authors, and tags, though they remain competitive on title and body text.
  • The released 3,172-page, six-language dataset gives researchers a common benchmark for news attribute extraction, with site-level splits that test generalization to unseen websites.
  • On the Zyte article benchmark, fine-tuned DOM-LM reaches F1 0.94, close to reported commercial services and above most open-source libraries.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reported advantage holds up under a clean pre-training split, the practical lesson is that a small, balanced multilingual corpus can substitute for much larger English pre-training data, making DOM-style pre-training affordable for other document extraction domains.
  • The paper's translation experiment implies a trade-off the authors do not state explicitly: when title and date are the priority and translation is cheap, MarkupLM-plus-translation may still be preferable; when author, tag, and text dominate, DOM-LM wins without the translation step.
  • The same recipe—a multilingual encoder backbone plus DOM-tree-aware pre-training on a modest domain corpus—could transfer to product pages, job listings, or other semi-structured page types where labeled data are scarce.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces a new multilingual dataset of 3,172 labeled news web pages from 161 websites in English, German, Russian, Chinese, Korean, and Arabic, with annotations for title, publication date, main text, authors, and tags. The authors fine-tune MarkupLM (pre-trained on English) and DOM-LM (pre-trained on their multilingual dataset plus a CommonCrawl-News sample) for node-level attribute extraction. They compare these models against XLM-RoBERTa and open-source heuristic tools (Trafilatura, Newspaper, News-please) across one-language, cross-lingual, mixed-language, and Zyte benchmark settings. The central claim is that multilingual DOM-LM outperforms other solutions for most attributes when fine-tuned and evaluated on mixed-language data.

Significance. If the claims hold, the paper provides a useful public multilingual benchmark for news-page attribute extraction and a practical result: a DOM-LM model pre-trained on multilingual news pages can match or exceed English-only MarkupLM with a translation pipeline, at lower inference cost. The dataset is a concrete and reusable contribution, and the evaluation protocol, including attribute-specific matching rules and 5-fold site-disjoint splits, is described in sufficient detail to be reproducible. However, the central model-comparison claim is currently undermined by a data-leakage confound, and the absence of variance reporting makes it difficult to assess whether the reported differences are meaningful.

major comments (2)
  1. [§VI.C and §VII.A] DOM-LM pre-training is performed on the full multilingual dataset (Section VI.C: "pre-training was performed on our multilingual news dataset") before the 5-fold cross-validation in Section VII.A divides websites into training and test folds. Because the pre-training phase is unsupervised but sees the raw HTML and DOM structure of every evaluation page, the test pages are not truly unseen to the model at the token-classification stage. The comparison baselines (MarkupLM pre-trained on English CommonCrawl, XLM-RoBERTa pre-trained on general text) did not receive comparable target-domain pre-training, so the reported advantage of DOM-LM in Table VII conflates architecture and pre-training strategy with exposure to evaluation pages. The paper does not report an ablation excluding evaluation sites from pre-training, nor any other evidence that this overlap does not inflate F1. This issue is load-bearing for the Section IX conclusion that multilingual DOM-LM outperforms other solutions on mixed-language data.
  2. [§VII.A and Tables V, VII, VIII] All results are reported as single point estimates of F1, with no standard deviations, per-fold results, confidence intervals, or significance tests. Several differences that underlie the "outperforms" claim are small: in Table VII, DOM-LM versus MarkupLM-EN is 0.93 versus 0.95 for Title, 0.87 versus 0.88 for Date, and 0.41 versus 0.40 for Author. Given the small number of labeled pages for some attributes (e.g., English author and tag have 147 and 98 labeled pages, respectively, in Table III), these differences may well be within run-to-run or fold-to-fold noise. Reporting the five fold-wise results and an estimate of variance is necessary to support the headline comparison.
minor comments (5)
  1. [§II.B] In the Structor description, "The authors retrive a node" should read "retrieve".
  2. [Table III] The two-line-per-language layout in Table III is difficult to parse; the counts of sites/pages with each attribute would be clearer in a standard tabular format with separate rows for sites, pages, and nodes.
  3. [§VIII.B] The new-language experiment reports results only for German and Chinese, although the setup (holding out one language and training on all others) could be applied to all six languages. Reporting all held-out languages would strengthen the cross-lingual transfer claim.
  4. [§VIII.B] The interpretation that Chinese works better than German because the model focuses more on markup than on text is not directly supported by any analysis; consider controlling for markup similarity or reporting additional diagnostic evidence.
  5. [§VII.A] The sentence "Sites used for model training and evaluation did not intersected" contains a grammatical error and should read "did not intersect."

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper's claims are empirical benchmark comparisons checked against external baselines and external benchmark data.

full rationale

The paper makes no formal derivation claims; its results are empirical benchmark comparisons. The multilingual DOM-LM is pre-trained and fine-tuned on a newly created dataset, but its reported superiority is measured against external baselines (MarkupLM, XLM-RoBERTa, Trafilatura, Newspaper, News-please) and against the Zyte Article Extraction Benchmark, which is an external benchmark not constructed by the authors. The only self-citation is [6] (the authors' earlier Russian news dataset and evaluation methodology); it supplies data and evaluation conventions rather than a load-bearing theorem or fitted parameter. No prediction in the paper is defined in terms of another prediction, and no fitted constant is renamed as an independent result. A potential concern is that DOM-LM pre-training (Section VI.C) used the same 'our multilingual news dataset' that later appears in the 5-fold evaluation (Section VII.A), so the test pages' raw HTML may have been seen during unsupervised pre-training; this is a data-contamination/experimental-control issue for soundness, not circularity, because the paper does not define DOM-LM's outputs in terms of the evaluation labels or use the evaluation labels as pre-training targets.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim rests on data collection quality, metric validity, and the assumption that pre-training on evaluation pages does not leak into fine-tuned performance. No new theoretical entities or fitted scientific constants are introduced; the listed hyperparameters are standard training choices.

free parameters (6)
  • max_seq_length = 512
    Chunk length for token classification, chosen as MarkupLM maximum input length; affects node coverage.
  • doc_stride = 170
    Chunk overlap set to one third of max_seq_length; affects node coverage across chunks.
  • pre-training learning rate = 1e-4
    Chosen following the original DOM-LM paper; no grid search reported.
  • fine-tuning learning rate = 3e-5
    Chosen for fine-tuning; no ablation or sensitivity analysis reported.
  • pre-training epochs = 5
    Chosen following the original DOM-LM paper; no ablation reported.
  • CommonCrawl-News pre-training pages = 37,473
    One-day sample of news pages; the size is arbitrary and not justified.
assumptions (6)
  • domain assumption Sitemap-based annotation by annotators produces accurate ground-truth labels.
    Section IV describes manual sitemap creation; no inter-annotator agreement metrics are reported for the new languages.
  • domain assumption The 4-gram bag matching for titles and texts, and set matching for authors and tags, is a valid proxy for extraction quality.
    Section VII.A defines the metric; no validation against exact match or human judgment is provided.
  • domain assumption Site-level 5-fold cross-validation estimates generalization to new websites.
    Section VII.A; with only 9-10 sites per language, fold variance is high and not reported.
  • ad hoc to paper Translating nodes to English with Argos Translate preserves the target attributes for comparison.
    Section VIII.A; the paper does not state how translated predictions are matched to original-language ground truth.
  • ad hoc to paper Pre-training on the same pages later used for evaluation does not inflate downstream extraction quality.
    Section VI.C pre-trains DOM-LM on the full dataset; Section VII.A evaluates on folds of that dataset.
  • domain assumption Excluding attributes with variable locations and captcha-protected pages does not bias the benchmark.
    Section IV details these exclusion rules; they may make the task easier than real-world news extraction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multilingual Attribute Extraction from News Web Pages." pith.science (2026). https://pith.science/paper/CR4EWVHK

@misc{pith2026250202167,
  author       = {Pith},
  title        = {Pith review of: Multilingual Attribute Extraction from News Web Pages},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CR4EWVHK}},
  note         = {Machine review of arXiv:2502.02167}
}
read the original abstract

This paper addresses the challenge of automatically extracting attributes from news article web pages across multiple languages. Recent neural network models have shown high efficacy in extracting information from semi-structured web pages. However, these models are predominantly applied to domains like e-commerce and are pre-trained using English data, complicating their application to web pages in other languages. We prepared a multilingual dataset comprising 3,172 marked-up news web pages across six languages (English, German, Russian, Chinese, Korean, and Arabic) from 161 websites. The dataset is publicly available on GitHub. We fine-tuned the pre-trained state-of-the-art model, MarkupLM, to extract news attributes from these pages and evaluated the impact of translating pages into English on extraction quality. Additionally, we pre-trained another state-of-the-art model, DOM-LM, on multilingual data and fine-tuned it on our dataset. We compared both fine-tuned models to existing open-source news data extraction tools, achieving superior extraction metrics.

Figures

Figures reproduced from arXiv: 2502.02167 by the authors.

Figure 1
Figure 1. Sitemap example in Web Scraper amounts of data. To mark up pages in new languages, we used Web Scraper6 utility. It allows to interactively create website wrappers, called sitemaps, describing the order of resource traversal and CSS selectors of attributes to be extracted from pages. An example of creating a sitemap in the Web Scraper GUI is shown in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. WebLists: Extracting Structured Information From Complex Interactive Websites Using Executable LLM Agents

    cs.AI 2025-04 conditional novelty 6.0 of 10

    BardeenAgent records a single extraction on a webpage, generalizes it with CSS selectors, and replays it across all matching items, reaching 66% recall on a new 200-task WebLists benchmark versus 31% for the best baseline.

Reference graph

Works this paper leans on

14 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    Markuplm: Pre-training of text and markup language for visually rich document understanding,

    J. Li, Y . Xu, L. Cui, and F. Wei, “Markuplm: Pre-training of text and markup language for visually rich document understanding,” in ACL (1). Association for Computational Linguistics, 2022, pp. 6078–6087

    work page 2022

  2. [2]

    Dom- lm: Learning generalizable representations for html documents,

    X. Deng, P. Shiralkar, C. Lockard, B. Huang, and H. Sun, “Dom- lm: Learning generalizable representations for html documents,” arXiv preprint arXiv:2201.10608, 2022

    arXiv 2022

  3. [3]

    From one tree to a forest: a unified solution for structured web data extraction,

    Q. Hao, R. Cai, Y . Pang, and L. Zhang, “From one tree to a forest: a unified solution for structured web data extraction,” inProceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval , 2011, pp. 775–784

    work page 2011

  4. [4]

    The Klarna Product Page Dataset: Web Element Nomination with Graph Neural Networks and Large Language Models

    A. Hotti, R. S. Risuleo, S. Magureanu, A. Moradi, and J. Lagergren, “The klarna product page dataset: A realisticbenchmark for web repre- sentation learning,” arXiv preprint arXiv:2111.02168 , 2021

    work page Pith review arXiv 2021

  5. [5]

    CoVA: Context-aware Visual Attention for Webpage Information Extraction

    A. Kumar, K. Morabia, J. Wang, K. C.-C. Chang, and A. Schwing, “Cova: Context-aware visual attention for webpage information extrac- tion,” arXiv preprint arXiv:2110.12320 , 2021

    work page Pith review arXiv 2021

  6. [6]

    A dataset for information extraction from news web pages,

    M. Varlamov, D. Galanin, P. Bedrin, S. Duda, V . Lazarev, and A. Yatskov, “A dataset for information extraction from news web pages,” 2022 Ivannikov Ispras Open Conference (ISPRAS) , pp. 100–106, 2022

    work page 2022

  7. [7]

    Roberta: A robustly optimized bert pretraining approach,

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” ArXiv, vol. abs/1907.11692, 2019

    arXiv 1907

  8. [8]

    Learning structural co-occurrences for structured web data extraction in low- resource settings,

    Z. Zhang, B. Yu, T. Liu, T. Liu, Y . Wang, and L. Guo, “Learning structural co-occurrences for structured web data extraction in low- resource settings,” in Proceedings of the ACM Web Conference 2023 , 2023, pp. 1683–1692

    work page 2023

Show all 14 references
  1. [9]

    Hierarchi- cal multimodal pre-training for visually rich webpage understanding,

    H. Xu, L. Chen, Z. Zhao, D. Ma, R. Cao, Z. Zhu, and K. Yu, “Hierarchi- cal multimodal pre-training for visually rich webpage understanding,” in Proceedings of the 17th ACM International Conference on Web Search and Data Mining , ser. WSDM ’24. Association for Computing Machine...

    2024

  2. [10]

    Label Studio: Data labeling software,

    M. Tkachenko, M. Malyuk, A. Holmanyuk, and N. Liubimov, “Label Studio: Data labeling software,” 2020-2024, open source software available from https://github.com/HumanSignal/label-studio. [Online]. Available: https://github.com/HumanSignal/label-studio

    2020

  3. [11]

    Un- supervised cross-lingual representation learning at scale,

    A. Conneau, K. Khandelwal, N. Goyal, V . Chaudhary, G. Wenzek, F. Guzm ´an, E. Grave, M. Ott, L. Zettlemoyer, and V . Stoyanov, “Un- supervised cross-lingual representation learning at scale,” in Annual Meeting of the Association for Computational Linguistics , 2019

    2019

  4. [12]

    Neural machine translation with byte- level subwords,

    C. Wang, K. Cho, and J. Gu, “Neural machine translation with byte- level subwords,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 9154–9160, 04 2020

    2020

  5. [13]

    Trafilatura: A Web Scraping Library and Command-Line Tool for Text Discovery and Extraction,

    A. Barbaresi, “Trafilatura: A Web Scraping Library and Command-Line Tool for Text Discovery and Extraction,” in Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natura...

    2021

  6. [14]

    news-please: A generic news crawler and extractor,

    F. Hamborg, N. Meuschke, C. Breitinger, and B. Gipp, “news-please: A generic news crawler and extractor,” in Proceedings of the 15th International Symposium of Information Science , March 2017, pp. 218– 223

    2017

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.