REVIEW 6 cited by
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Pre-training of text and layout has proved effective in a variety of visually-rich document understanding tasks due to its effective model architecture and the advantage of large-scale unlabeled scanned/digital-born documents. We propose LayoutLMv2 architecture with new pre-training tasks to model the interaction among text, layout, and image in a single multi-modal framework. Specifically, with a two-stream multi-modal Transformer encoder, LayoutLMv2 uses not only the existing masked visual-language modeling task but also the new text-image alignment and text-image matching tasks, which make it better capture the cross-modality interaction in the pre-training stage. Meanwhile, it also integrates a spatial-aware self-attention mechanism into the Transformer architecture so that the model can fully understand the relative positional relationship among different text blocks. Experiment results show that LayoutLMv2 outperforms LayoutLM by a large margin and achieves new state-of-the-art results on a wide variety of downstream visually-rich document understanding tasks, including FUNSD (0.7895 $\to$ 0.8420), CORD (0.9493 $\to$ 0.9601), SROIE (0.9524 $\to$ 0.9781), Kleister-NDA (0.8340 $\to$ 0.8520), RVL-CDIP (0.9443 $\to$ 0.9564), and DocVQA (0.7295 $\to$ 0.8672). We made our model and code publicly available at \url{https://aka.ms/layoutlmv2}.
Forward citations
Cited by 6 Pith papers
-
SlideAgent: Hierarchical Agentic Framework for Multi-Page Visual Document Understanding
SARA combines natural-language snippets with semantic compression vectors in RAG to improve answer relevance, correctness, and similarity on 9 datasets across 5 LLMs.
-
Animation Needs Attention: A Holistic Approach to Slides Animation Comprehension with Visual-Language Models
A vision-language model fine-tuned on a new synthetic slide-animation dataset outperforms GPT-4.1 and Gemini-2.5-Pro at describing slide animations, especially on synthetic evaluation data.
-
Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting
A two-stage model that analyzes page layout first, then parses text, tables, and formulas in parallel, reports state-of-the-art accuracy and speed on the benchmarks it evaluates.
-
Finding Needles in Images: Can Multimodal LLMs Locate Fine Details?
A new benchmark and method (Spot-IT) aim to improve multimodal LLMs' ability to locate fine details in documents, with reported significant gains.
-
Spatial ModernBERT: Spatial-Aware Transformer for Table and Key-Value Extraction in Financial Documents at Scale
Spatial ModernBERT is a token-classification model that adds layout coordinates to ModernBERT to extract tables and key-value fields, with benchmark scores that fall short of the claimed state of the art.
-
DocAnnot -- Accelerating the Creation of Key Information Extraction Datasets with GenAI-Powered Auto-annotation
DocAnnot combines an LVLM, OCR, and a spatial matching heuristic to auto-annotate KIE documents at F1 0.68–0.85, and models trained on that data reach roughly 0.68 F1 on CORD.
Discussion (0). Continue with ORCID to comment.