Pith. sign in

REVIEW 4 major objections 5 minor 44 references

A new vision-native benchmark for splitting packets of Bangladeshi government forms shows that multimodal AI models can group pages into the right documents but cannot reliably reconstruct the original page order once pages are shuffled—and

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 06:44 UTC pith:XUZUROJA

load-bearing objection A genuinely new, carefully built benchmark whose central finding — clustering survives shuffling while page-order reconstruction collapses — is solid, but the two controlled analyses are not as clean as the prose claims, so the paper needs a revision pass, not a desk rejection. the 4 major comments →

arxiv 2607.21780 v1 pith:XUZUROJA submitted 2026-07-23 cs.CL cs.AIcs.CV

Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms

Abu Tyeb Azad , Fahim Ahmed , Ishita Sur Apan , Ezharuddin Jubaer , Sumaiya Karim Katha , Armun Alam , Amin Ahsan Ali , Aman Chadha
show 2 more authors
Md Mofijul Islam AKM Mahbubur Rahman
This is my paper
classification cs.CL cs.AIcs.CV
keywords document packet splittingpage-order reconstructionmultimodal LLM evaluationBangla document understandinglow-resource document benchmarkvision-language modelsBangladeshi government formspage stream segmentation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the bottleneck in splitting scanned government-form packets is not deciding which pages belong to which document—it is reconstructing the original page order after pages have been shuffled. To make this case, it introduces Khondo, the first vision-native packet-splitting benchmark for a low-resource language: 1,950 packets assembled from 423 real Bangladeshi government forms across 14 domains, in five concatenation schemes from sequential to fully shuffled. On Khondo, flagship multimodal models cluster pages reasonably well (clustering scores of roughly 0.55–0.85 on shuffled variants) but order them poorly: the best model drops from about 0.95 to 0.68–0.82 when pages are shuffled, and small open models reach only about 0.32 even after fine-tuning. Two controlled experiments attribute part of the gap to prompt instructions (order-aware prompts recover 0.43–0.67 of the lost ordering accuracy but no more) and part to language (identical English packets are ordered 0.13–0.36 better than Bangla ones). If the finding is right, progress in low-resource document automation hinges on teaching models to read visual order cues—printed page numbers, field flow, section continuity—in Bangla, not on better page grouping.

Core claim

The central claim, stated the way the authors would state it: on Khondo, a bilingual (Bangla–English), vision-native benchmark built from authentic Bangladeshi government forms, zero-shot multimodal language models can group pages into their source documents fairly well, but page-order reconstruction is the primary bottleneck. Clustering stays in the 0.55–0.85 range even when documents are interleaved or globally shuffled, whereas ordering collapses from Sord ≈ 0.95 on sequential packets to 0.68–0.82 for the best flagship model, and to roughly 0.05–0.15 for weaker models that effectively copy the presented order. Two paired manipulations show the deficit is not merely prompting: explicit reo

What carries the argument

The instrument is the benchmark itself, designed to control the sources of difficulty. Khondo assembles packets from real Bangladeshi government forms under five concatenation variants: MonoSeq and PolySeq keep pages in original order (ordering is trivial), MonoRand shuffles within each document's block, PolyInt interleaves documents round-robin, and PolyRand globally shuffles. The variants isolate stressors—interleaving and shared domains stress clustering; shuffling stresses ordering. Two further controls, the paired order-aware versus order-agnostic prompts and the size-matched Bangla-only versus English-only packets, separate prompting effects from language effects. The scoring is equall

Load-bearing premise

The load-bearing premise is that a packet's ground-truth page order—taken from the source PDF's order at assembly time—is genuinely recoverable from the page images alone; if a sizable share of pages (legacy-font glyphs, maps, blank continuation sheets, roughly 2% labeled Uncertain) provide no visual order cue, then part of the measured ordering bottleneck is an underdetermined task, not a model failure.

What would settle it

Give careful human annotators the same shuffled packet pages (images only, no text search) and measure their Sord; if humans also plateau near 0.7–0.8, the claim that models face an intrinsic ordering difficulty weakens in favor of a task that is partly impossible. A sharper test: re-score the frontier models on a Khondo subset from which all Uncertain and legacy-font pages are removed; if the English-over-Bangla ordering gap shrinks substantially, the gap is driven by unreadable script cues rather than by the ordering operation itself.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the ordering bottleneck is intrinsic, document-packet-splitting systems for low-resource administrative corpora should be judged primarily on their page-order reconstruction ability, not clustering accuracy.
  • Order-aware prompting is a necessary component of any practical deployment—without it models simply 'copy the input order'—but deployment will also need trainable order-reconstruction capacity.
  • Because English packets are ordered 0.13–0.36 better than Bangla ones, English-only evaluation overstates readiness for the real Bangla-dominant distribution; benchmarks must preserve the natural language skew.
  • Fine-tuning small open MLLMs lifts shuffled-variant ordering from near zero to roughly 0.32, and scaling to 32B pushes packet scores toward but not past the zero-shot flagship level, so the task is learnable but not yet solved.
  • Khondo provides a concrete target for future work: closing the shuffled-variant Sord gap toward the roughly 0.95 achieved when pages arrive in sequence.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: roughly 2% of Khondo pages (legacy non-Unicode Bangla fonts, maps, near-blank sheets) carry little or no image-only order cue, so part of the measured ordering gap is likely task underdetermination; a human-annotation ceiling on shuffled packets would partition the deficit into recoverable-versus-unknowable order.
  • Beyond the paper: the English-over-Bangla advantage might stem from models reading Bangla text as low-level texture rather than structured cues—a testable hypothesis: translate field labels or provide in-prompt OCR for Bangla pages and see whether the ordering gap closes.
  • Beyond the paper: the same five-variant design could be applied to other low-resource scripts to test whether the ordering bottleneck tracks script complexity or generalizes across languages.
  • Beyond the paper: a page's position in a form often follows deterministic cues (printed page numbers, tables continuing across pages); an order-from-layout-only evaluation could isolate whether the deficit is linguistic or purely visual.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Khondo, a benchmark for document packet splitting on Bangladeshi government forms, containing 1,950 packets over five concatenation schemes (MonoSeq, MonoRand, PolySeq, PolyInt, PolyRand), with ground-truth document boundaries, domain labels, and page order recorded at assembly time. The authors evaluate five zero-shot MLLMs, fine-tune three/four small open-weight MLLMs, and run two controlled analyses: a prompt-sensitivity study (order-aware vs. order-agnostic prompts) and a cross-lingual study (Bangla-only vs. English-only packets). Their main finding is that MLLMs cluster pages into source documents reasonably well but fail to reconstruct original page order once pages are shuffled; the ordering gap is reduced, but not eliminated, by explicit ordering instructions, and is larger for Bangla than for English packets.

Significance. If the results hold, Khondo is a valuable first benchmark for vision-native packet splitting in a low-resource language. The construction has clear strengths: form-level disjoint train/validation/test splits prevent content leakage, domain balance is within ~1.6 pp of uniform, ground truth is recorded at assembly time, language labels use blind dual-human review with Cohen's κ=0.90, and the dataset/code are public. The paper also ships a scaling study for fine-tuned Qwen3-VL. However, the two headline controlled analyses contain load-bearing framing flaws — the 'order-agnostic' prompt actually contains an order instruction, and the cross-lingual comparison does not hold language fixed — so the central interpretive claims require additional work before the benchmark's conclusions can be accepted as stated.

major comments (4)
  1. [§5.4 and Fig. 5] The OAg prompt is not order-agnostic. Figure 5 explicitly states: 'page_ordinals within each subdocument must be in the original page order.' This is an instruction about page order. The paper's claim that the OAg prompt 'contains no instruction regarding page order' is therefore false, and the paired comparison OAw vs. OAg does not isolate the effect of order instructions. Additionally, the OAg prompt tells the model to 'Group consecutive pages from the same sub-document together,' while the OAw prompt explicitly permits non-adjacent pages; this confounds grouping constraints with ordering instructions. The conclusion that 'order-specific instructions are necessary, but not sufficient' is not supported by this experiment. The authors should re-run the prompt-sensitivity study with an OAg prompt that truly removes all references to original page order and that keeps the grouping constrai
  2. [§4.1 and A.4] The ground-truth page order is taken as the order of pages in the source PDFs from forms.portal.gov.bd, but the paper does not verify that these PDFs are internally correctly ordered, nor that the order is recoverable from page images alone. Appendix A.4 concedes that ~2.2% of pages are labeled Uncertain (legacy non-Unicode fonts, maps/diagrams, blank/near-empty scans) and that these pages are retained in packets. With mean packet length 14.6 pages, roughly a quarter of packets contain at least one such page. For such packets, no image-only cue fixes the position of those pages, so part of the measured Sord deficit on MonoRand/PolyInt/PolyRand may reflect task underdetermination rather than model failure. The assertion in A.4 that these pages 'do not materially affect' the ordering ground truth is unsupported. The authors should provide a human-ceiling measurement of Sord on shuffled var
  3. [§5.1, Eq. (1)-(2)] The definition of Sord is incomplete for impure predicted clusters. The paper says Sord compares the predicted page order with the ground-truth order over the m pages of each predicted multi-page sub-document, but if a predicted cluster merges pages from two or more ground-truth documents, there is no single ground-truth page order for that set. The manuscript does not describe how such predicted clusters are matched to ground-truth documents or how τa is computed in this case. Also, defining Sord as 1 when a packet contains no predicted multi-page sub-document can inflate the overall score for degenerate outputs (e.g., all pages as singletons). These choices affect every ordering result in Tables 2, 3, 5, and 6 and need to be specified and justified.
  4. [§5.5 and Table 6] The cross-lingual experiment is described as producing conditions that 'differ only in language,' but the Bangla-only and English-only packets are assembled from different source forms. Language is therefore confounded with form identity, layout, content, and possibly page complexity. Figure 15 shows one matched example, but the controlled comparison is not based on translated versions of the same forms. The claim that English packets are ordered 0.13–0.36 better than Bangla packets because of language requires matched or counterbalanced forms. At minimum, the authors should control for form identity (e.g., by using translated pairs) or provide evidence that the form pools are equivalent in difficulty.
minor comments (5)
  1. [Figures 2, 3, 7, 8] The axis labels and some text in these figures appear as unicode-escaped tokens (e.g., '/uni00000030/...'), making them unreadable in the submitted PDF. Please replace with rendered text.
  2. [§5.2] The proprietary model names (Gemini-3.5-Flash, GPT-5.4, Qwen 3.6 Plus, etc.) and open-weight models should be accompanied by exact version identifiers and access dates for reproducibility, as API versions can change behavior.
  3. [§A.3.1] A paired permutation test is mentioned but no p-values or effect sizes are reported. Please include them in Table 5 or the caption.
  4. [Abstract] Minor typo: 'vision-native; where models operate directly on page images' should read 'vision-native, where models operate directly on page images.'
  5. [Eq. (2)] The formula for Kendall's τa is written as (nc − nd)/C(m,2); this is correct only when there are no ties, but the notation should be defined explicitly and the handling of ties stated.

Circularity Check

0 steps flagged

No circularity: Khondo's ground truth is recorded at packet assembly time, and its central ordering-bottleneck claim is an external empirical measurement rather than a quantity fitted from model outputs.

full rationale

Khondo is a benchmark-construction and evaluation paper, not a derivation whose conclusions reduce to its inputs. Ground-truth document membership, domain, and page order are 'recorded as ground truth at assembly time' (§4.1), i.e., before any model is run, and the reported Sclu and Sord scores are computed against that external standard (§5.1). There is no fitted parameter that is later renamed as a prediction: the supervised fine-tuning results are standard held-out test-set evaluations (§5.3), and the prompt-sensitivity and cross-lingual analyses are paired measurements on identical or controlled packets (§5.4–5.5). The only author-overlap citation is DocSplit [16], from which the paper adopts the problem formulation, metrics, and split ratio; this supplies framing and experimental design, not evidence for the headline finding, so it is not load-bearing circularity. A.4's assertion that the ~2.2% Uncertain pages 'do not materially affect' ordering ground truth lacks a human-ceiling check and is a benchmark-validity concern, but it is a correctness/robustness question, not a circular reduction. No specific circular step can be exhibited, so the score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 7 axioms · 0 invented entities

The central claims rest on design choices rather than fitted parameters: packet length, resolution cap, split ratio, and sampling temperature are hand-set dataset/evaluation hyperparameters, not quantities fitted to make a result appear. The load-bearing assumptions are domain-level: ground-truth order equals source-PDF order without verification; whole forms are atomic; page images are assumed to contain recoverable order cues (contradicted in part by the paper's own Uncertain pages); the 14-domain taxonomy is treated as exhaustive and single-label; and the model-generated language labels feed the cross-lingual conditions. No invented entities (no new particles, forces, or theoretical constructs) are introduced.

free parameters (4)
  • Packet target length range = 5–20 pages, mean 14.6
    Hand-chosen (§4.1) to fit MLLM context limits ('a range every evaluated model can process reliably'); sets task difficulty but is not fitted to model scores.
  • Per-page image resolution cap for fine-tuning = ≈512×512
    Chosen in §5.3 to equalize per-page token budgets and keep packets within GPU memory; affects all supervised fine-tuning results.
  • Train/validation/test split ratio = 55:20:25
    Inherited from DocSplit (§4.1) with an evaluation-first design (210 test packets per variant); a design choice, not fitted to outcomes.
  • Decoding temperature and JSON retry limit = T=1.0, k=3
    Evaluation hyperparameters (§5.2); temperature 1.0 makes single-run scores stochastic, so every reported number is a single draw.
axioms (7)
  • domain assumption Ground-truth page order of each document equals the page order of the source PDF downloaded from the government portal; no verification that source scans are internally correctly ordered.
    §4.1: 'Document membership, domain type, and page order are recorded as ground truth at assembly time.' If any source form PDF is itself a mis-ordered scan, the 'original order' ground truth is corrupted and ordering scores against it are mismeasured.
  • domain assumption Whole forms are atomic units; packets never contain partial forms, so the splitting task never requires cutting inside a form.
    §4.1: 'We append whole forms (each used at most once) to the packet.' This makes documents well-defined but simplifies the real-world problem, where packets may contain partial documents.
  • domain assumption Order reconstruction from page images alone is possible; pages contain readable, recoverable order cues such as printed page numbers, field flow, and continuations.
    §3 frames ordering as a recoverable structure; §6 interprets residual failure as 'a genuine difficulty... rather than prompt wording'. A.4 acknowledges ~2.2% Uncertain pages (legacy Bijoy/SutonnyMJ fonts, maps, near-blank) where no readable order cue exists, so part of the benchmark may be underdetermined from images.
  • domain assumption The 14-domain taxonomy with fixed descriptions is a faithful, mutually exclusive labeling of Bangladeshi government forms (single label per form).
    Fig. 4 defines the label block; §4.1 reports domain balance within ~1.6pp of uniform. The taxonomy constrains typing accuracy (PgAcc never exceeds 0.57 even for the best fine-tuned model).
  • standard math V-measure, Rand index, and Kendall's τa are valid external clustering/order metrics with the standard properties assumed in §5.1.
    Eqs. 1–2 define Sclu and Sord; the metrics are standard, but the paper does not define the ground-truth order for a predicted cluster that mixes pages from multiple true documents.
  • domain assumption Language labels (Bangla/English/Mixed/Uncertain) assigned by the dual-MLLM-plus-human pipeline are correct; they drive the cross-lingual §5.5 conditions.
    A.4 describes the labeling cascade with human κ=0.90 on the ambiguous subset; label errors would propagate into the monolingual packet construction of §5.5.
  • ad hoc to paper The OAg prompt's output constraint 'page_ordinals within each subdocument must be in the original page order' does not itself constitute an order instruction.
    Fig. 5; this characterization underlies the 'necessary but insufficient' conclusion, yet the constraint already requests original order, so OAg is not truly order-agnostic.

pith-pipeline@v1.3.0-alltime-deepseek · 25515 in / 24520 out tokens · 232580 ms · 2026-08-01T06:44:02.404865+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms." pith.science (2026). https://pith.science/paper/XUZUROJA

@misc{pith2026260721780,
  author       = {Pith},
  title        = {Pith review of: Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XUZUROJA}},
  note         = {Machine review of arXiv:2607.21780}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Document packets, multiple documents concatenated into a single file, are common in government and administrative workflows, yet splitting them into their constituent documents is difficult, especially for low-resource languages. We introduce Khondo (Bangla for split/segment), the first benchmark for document packet splitting on Bangladeshi government forms. Unlike prior English and OCR-text-based datasets, Khondo is bilingual (Bangla--English) and vision-native; where models operate directly on page images. It spans five concatenation schemes, from sequential to fully shuffled, across 14 administrative domains, with ground-truth boundaries, domain types, and page order. Zero-shot evaluation of MLLMs shows they cluster pages into their source documents fairly well but struggle in restoring the original page order once shuffled. To isolate what drives this difficulty, we run two controlled analyses, varying the prompt instruction and then the packet language. Both primarily affect ordering rather than clustering: (a) explicit page-order instructions are necessary but insufficient, and (b) English packets are ordered more reliably than Bangla, making page arrangement the dominant challenge and language a secondary but consistent factor. Khondo establishes page-order reconstruction as a key open problem in vision-based, low-resource document understanding, and provides a controlled benchmark for measuring progress toward solving it. Our dataset and code is available at https://huggingface.co/datasets/Mausul/khondo

Figures

Figures reproduced from arXiv: 2607.21780 by Abu Tyeb Azad, AKM Mahbubur Rahman, Aman Chadha, Amin Ahsan Ali, Armun Alam, Ezharuddin Jubaer, Fahim Ahmed, Ishita Sur Apan, Md Mofijul Islam, Sumaiya Karim Katha.

Figure 1
Figure 1. Figure 1: Overview of the Khondo dataset construction pipeline. Document packets are created by concatenating authentic Bangladeshi government forms, enabling ground-truth document membership and page order to be recorded during assembly. From left to right, the pipeline consists of: (1) document collection and curation; (2) selection of eligible multi-page forms; (3) partitioning into disjoint, leakage-free train/v… view at source ↗
Figure 2
Figure 2. Figure 2: Change in ordering score, ∆Sord = S OAw ord − S OAg ord , for each model and packet variant. On the shuffled variants, the order-aware (OAw) prompt recovers 0.43–0.67 of the order￾ing performance lost by the order-agnostic (OAg) prompt, which otherwise tends to reproduce the presented page order. On the sequential variants, where the input pages are already correctly ordered, the explicit ordering cue slig… view at source ↗
Figure 3
Figure 3. Figure 3: Cross-lingual performance gap, ∆ = En − Bn, on size-matched monolingual packets for each model and packet variant (shared scale). (a) Ordering: English packets are ordered substantially more accurately than Bangla packets, particularly on the shuffled variants, with the magnitude of the gap varying across models. (b) Clustering: the language effect is considerably smaller (∆ ≈ 0.04–0.08), although it remai… view at source ↗
Figure 4
Figure 4. Figure 4: Administrative domain labels of the government forms in Khondo. For use in the inference prompts [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Order Agnostic Prompt (OAg) used for the packet splitting experiments using MLLMs. Compared to the Order-Aware Prompt [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Order-Aware Prompt (OAw) used for the packet splitting experiments using MLLMs. Compared to the Order-Agnostic Prompt [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: shows the clustering counterpart of the ordering results on the same scale. Whereas ordering changes by 0.43–0.67 on the shuffled variants, every clustering bar re￾mains within 0.09 of zero. This indicates that order-aware instructions have only a minimal effect on clustering com￾pared with their substantial impact on ordering. The small residual changes are systematic rather than random noise. A paired pe… view at source ↗
Figure 7
Figure 7. Figure 7: Domain distribution of Khondo. Each bar gives a do [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: Per-page language labeling/classifying pipeline. An automated cascade labels each form page: two small MLLMs (Gemma 4 [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Prompt used to classify forms into English/Bangla/Mixed/Unsure language groups. See Figure [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Examples of pages labeled Uncertain. (a) A Bengali government form in a legacy, non-Unicode font (the Bijoy/SutonnyMJ family) rather than a Unicode-compatible method such as Avro), so it’s rendered as unreadable text. (b) A ward map filed within a city￾corporation specific application packet, that has no key-value pairs. (c) A near-empty continuation sheet with only a header and footer. Such pages hold li… view at source ↗
Figure 12
Figure 12. Figure 12: Packet score Spkt of fine-tuned Qwen3-VL (using order-aware prompt, OAw) against backbone sizes 4B, 8B, and 32B. One panel per concatenation variant (top row: sequential; bottom row: shuffled). The solid blue curve is for fine-tuned Qwen3-VL; the dashed and dotted lines are for training-free inferences of Gemini-3.5-Flash (best zero-shot) and Qwen 3.6 Plus (same-family zero-shot) respectively, which coinc… view at source ↗
Figure 13
Figure 13. Figure 13: Ordering bottleneck (Gemini 3.5 Flash): Four pages of one Bangla employment form from a shuffled packet, shown in the model’s output order. The badge under each page is its true position (green = correct slot, red = out of place): a correct prediction reads 1 → 2 → 3 → 4, but the model produces 1 → 4 → 3 → 2, swapping 2nd and 4th page of ground truth order. The model assigns the packet’s pages to the corr… view at source ↗
Figure 14
Figure 14. Figure 14: Prompt sensitivity (Gemini 3.5 Flash): Four pages of one Bangla local-government form from a shuffled packet, shown in the model’s output order under two prompts. The badge under each page is its true position (green = correct slot, red = out of place). Under the order-agnostic (OAg) prompt the model reproduces the presented page order, reading 4 → 2 → 3 → 1; under the order-aware (OAw) prompt the same mo… view at source ↗
Figure 15
Figure 15. Figure 15: Cross-lingual ordering gap (Qwen 3.6 Plus): Two education application forms of the same length from shuffled packets, one Bangla (top) and one English (bottom), each shown in the model’s output order. The badge under each page is its true position (green = correct slot, red = out of place). The model orders the English form correctly, 1 → 2 → 3 → 4, but misorders the Bangla one, 1 → 2 → 4 → 3. This Englis… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 1 linked inside Pith

  1. [1]

    Sheikh Iftekhar Ahmed, Muhammad Zubair Hasan, Abrar Jahin Niloy, Syed Mostofa Monsur, and Mark V . Albert. Scaling down, powering up: A survey on the advancements of small vision-language models. Inf. Fusion, 127(PB), 2026. 2

    2026

  2. [2]

    Semantic parsing of interpage relations

    Mehmet Arif Demirtas ¸, Berke Oral, Mehmet Yasin Akpınar, and Onur Deniz. Semantic parsing of interpage relations. In ICPR, pages 1579–1585, 2022. 2

    2022

  3. [3]

    Sohel Rahman, and Rifat Shahriyar

    Abhik Bhattacharjee, Tahmid Hasan, Wasi Ahmad, Kazi Samin Mubasshir, Md Saiful Islam, Anindya Iqbal, M. Sohel Rahman, and Rifat Shahriyar. BanglaBERT: Lan- guage model pretraining and benchmarks for low-resource language understanding evaluation in Bangla. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1318–1327, Seattle, ...

    2022

  4. [4]

    BanglaNLG and BanglaT5: Bench- marks and resources for evaluating low-resource natural lan- guage generation in Bangla

    Abhik Bhattacharjee, Tahmid Hasan, Wasi Uddin Ahmad, and Rifat Shahriyar. BanglaNLG and BanglaT5: Bench- marks and resources for evaluating low-resource natural lan- guage generation in Bangla. In Findings of the Association for Computational Linguistics: EACL 2023, pages 726–735, Dubrovnik, Croatia, 2023. Association for Computational Linguistics. 3

    2023

  5. [5]

    Us- ing deep-learned vector representations for page stream seg- mentation by agglomerative clustering

    Lukas Busch, Ruben van Heusden, and Maarten Marx. Us- ing deep-learned vector representations for page stream seg- mentation by agglomerative clustering. Algorithms, 16(5),

  6. [6]

    The ris- ing costs of training frontier ai models

    Ben Cottier, Robi Rahman, Loredana Fattorini, Nestor Maslej, Tamay Besiroglu, and David Owen. The ris- ing costs of training frontier ai models. arXiv preprint arXiv:2405.21015, 2024. 1

    Pith/arXiv arXiv 2024

  7. [7]

    Khapra, Anoop Kunchukuttan, and Pratyush Kumar

    Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M. Khapra, Anoop Kunchukuttan, and Pratyush Kumar. Towards leaving no Indic language behind: Building monolingual corpora, benchmark and models for Indic languages. In Proceedings of the 61st Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers)...

    2023

  8. [8]

    Bagdanov

    Albert Gordo, Marc ¸al Rusi ˜nol, Dimosthenis Karatzas, and Andrew D. Bagdanov. Document classification and page stream segmentation for digital mailroom applications. In 2013 12th International Conference on Document Analysis and Recognition, pages 621–625, 2013. 2

    2013

  9. [9]

    Docslm: A small vision-language model for long multimodal document understanding

    Tanveer Hannan, Dimitrios Mallios, Parth Pathak, Faegheh Sardari, Thomas Seidl, Gedas Bertasius, Mohsen Fayyaz, and Sunando Sengupta. Docslm: A small vision-language model for long multimodal document understanding. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR) Findings, pages 9337– 9346, 2026. 2

    2026

  10. [10]

    Harley, Alex Ufkes, and Konstantinos G

    Adam W. Harley, Alex Ufkes, and Konstantinos G. Derpanis. Evaluation of deep convolutional nets for document image classification and retrieval. In 2015 13th International Con- ference on Document Analysis and Recognition (ICDAR) , pages 991–995. IEEE, 2015. 2

    2015

  11. [11]

    Large language models for page stream segmentation, 2024

    Hunter Heidenreich, Ratish Dalvi, Rohith Mukku, Nikhil Verma, and Neven Piˇculjan. Large language models for page stream segmentation, 2024. 2

    2024

  12. [12]

    Bros: A pre-trained lan- guage model focusing on text and layout for better key infor- mation extraction from documents

    Teakgyu Hong, Donghyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, and Sungrae Park. Bros: A pre-trained lan- guage model focusing on text and layout for better key infor- mation extraction from documents. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 10767– 10775, 2022. 1

    2022

  13. [13]

    mPLUG-DocOwl2: High-resolution compressing for OCR- free multi-page document understanding

    Anwen Hu, Haiyang Xu, Liang Zhang, Jiabo Ye, Ming Yan, Ji Zhang, Qin Jin, Fei Huang, and Jingren Zhou. mPLUG-DocOwl2: High-resolution compressing for OCR- free multi-page document understanding. In Proceedings of the 63rd Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers) , pages 5817– 5834, Vienna, Austria, 2025. As...

    2025

  14. [14]

    Layoutlmv3: Pre-training for document ai with unified text and image masking

    Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. Layoutlmv3: Pre-training for document ai with unified text and image masking. In ACM MM, 2022. 1

    2022

  15. [15]

    Comparing partitions

    Lawrence Hubert and Phipps Arabie. Comparing partitions. Journal of Classification, 2(1):193–218, 1985. 4

    1985

  16. [16]

    Bishop III, Niharika Jain, Spencer Romo, Bob Strahan, Boyi Xie, and Diego A

    Md Mofijul Islam, Md Sirajus Salekin, Nivedha Balakrish- nan, Vincil C. Bishop III, Niharika Jain, Spencer Romo, Bob Strahan, Boyi Xie, and Diego A. Socolinsky. Docsplit: A comprehensive benchmark dataset and evaluation approach for document packet recognition and splitting, 2026. 1, 2, 3

    2026

  17. [17]

    Maurice G. Kendall. A new measure of rank correlation. Biometrika, 30(1/2):81–93, 1938. 4

    1938

  18. [18]

    Ocr-free document understanding transformer

    Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sang- doo Yun, Dongyoon Han, and Seunghyun Park. Ocr-free document understanding transformer. In European Confer- ence on Computer Vision (ECCV), 2022. 1

    2022

  19. [19]

    Lewis, G

    D. Lewis, G. Agam, S. Argamon, O. Frieder, D. Grossman, and J. Heard. Building a test collection for complex doc- ument information processing. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval , page 665–666, New York, NY , USA, 2006. Association for Computing Ma- chinery. 1

    2006

  20. [20]

    Mmlongbench-doc: bench- marking long-context document understanding with visual- izations

    Yubo Ma, Yuhang Zang, Liangyu Chen, Meiqi Chen, Yizhu Jiao, Xinze Li, Xinyuan Lu, Ziyu Liu, Yan Ma, Xiaoyi Dong, Pan Zhang, Liangming Pan, Yu-Gang Jiang, Jiaqi Wang, Yixin Cao, and Aixin Sun. Mmlongbench-doc: bench- marking long-context document understanding with visual- izations. In NeurIPS, Red Hook, NY , USA, 2024. Curran Associates Inc. 2

    2024

  21. [21]

    Tab this folder of documents: page stream segmentation of business documents

    Thisanaporn Mungmeeprued, Yuxin Ma, Nisarg Mehta, and Aldo Lipani. Tab this folder of documents: page stream segmentation of business documents. In Proceedings of the 22nd ACM Symposium on Document Engineering , New York, NY , USA, 2022. Association for Computing Machin- ery. 2

    2022

  22. [22]

    Cosmo: A multimodal transformer 8 for page stream segmentation in comic books

    Marc Serra Ortega, Emanuele Vivoli, Artemis Llabres, and Dimosthenis Karatzas. Cosmo: A multimodal transformer 8 for page stream segmentation in comic books. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision (ICCV) Workshops, pages 7565–7573, 2025. 2

    2025

  23. [23]

    Bhaasha, bh ¯as.¯a, za- ban: A survey for low-resourced languages in South Asia – current stage and challenges

    Sampoorna Poria and Xiaolei Huang. Bhaasha, bh ¯as.¯a, za- ban: A survey for low-resourced languages in South Asia – current stage and challenges. In Findings of the Association for Computational Linguistics: EMNLP 2025 , pages 1386– 1406, Suzhou, China, 2025. Association for Computational Linguistics. 1

    2025

  24. [24]

    µ gat: Improving single- page document parsing by providing multi-page context

    Fabio Quattrini, Carmine Zaccagnino, Silvia Cascianelli, Laura Righi, and Rita Cucchiara. µ gat: Improving single- page document parsing by providing multi-page context. In ECCV, pages 212–228. Springer, 2024. 2

    2024

  25. [25]

    TigerLLM - a family of Bangla large language models

    Nishat Raihan and Marcos Zampieri. TigerLLM - a family of Bangla large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Lin- guistics (Volume 2: Short Papers) , pages 887–896, Vienna, Austria, 2025. Association for Computational Linguistics. 3

    2025

  26. [26]

    V-measure: A conditional entropy-based external cluster evaluation mea- sure

    Andrew Rosenberg and Julia Hirschberg. V-measure: A conditional entropy-based external cluster evaluation mea- sure. In Proceedings of the 2007 Joint Conference on Em- pirical Methods in Natural Language Processing and Com- putational Natural Language Learning (EMNLP-CoNLL) , pages 410–420, Prague, Czech Republic, 2007. Association for Computational Lingu...

    2007

  27. [27]

    Istiak Hossain Shihab, Md

    Md. Istiak Hossain Shihab, Md. Rakibul Hasan, Mah- fuzur Rahman Emon, Syed Mobassir Hossen, Md. Naz- muddoha Ansary, Intesur Ahmed, Fazle Rabbi Rakib, Shahriar Elahi Dhruvo, Souhardya Saha Dip, Akib Hasan Pavel, Marsia Haque Meghla, Md. Rezwanul Haque, Sayma Sultana Chowdhury, Farig Sadeque, Tahsin Reasat, Ahmed Imtiaz Humayun, and Asif Sushmit. Badlad: A...

    2023

  28. [28]

    MTVQA: Benchmarking multilingual text-centric visual question answering

    Jingqun Tang, Qi Liu, Yongjie Ye, Jinghui Lu, Shu Wei, An-Lan Wang, Chunhui Lin, Hao Feng, Zhen Zhao, Yan- jie Wang, Yuliang Liu, Hao Liu, Xiang Bai, and Can Huang. MTVQA: Benchmarking multilingual text-centric visual question answering. In Findings of the Association for Computational Linguistics: ACL 2025 , pages 7748–7763, Vienna, Austria, 2025. Associ...

    2025

  29. [29]

    Unifying vision, text, and layout for universal document processing

    Zineng Tang, Ziyi Yang, Guoxin Wang, Yuwei Fang, Yang Liu, Chenguang Zhu, Michael Zeng, Cha Zhang, and Mo- hit Bansal. Unifying vision, text, and layout for universal document processing. In CVPR, pages 19254–19264, 2023. 1

    2023

  30. [30]

    Hi- erarchical multimodal transformers for multipage docvqa

    Rub `en Tito, Dimosthenis Karatzas, and Ernest Valveny. Hi- erarchical multimodal transformers for multipage docvqa. Pattern Recognition, 144:109834, 2023. 2

    2023

  31. [31]

    United Nations E-Government Survey 2024: Accelerating Digital Transformation for Sustainable Development - With the addendum on Artificial Intelligence

    United Nations Department of Economic and Social Affairs. United Nations E-Government Survey 2024: Accelerating Digital Transformation for Sustainable Development - With the addendum on Artificial Intelligence . United Nations, New York, 2024. 1

    2024

  32. [32]

    Document understanding dataset and evaluation (dude)

    Jordy Van Landeghem, Rub `en Tito, Łukasz Borchmann, Michał Pietruszka, Pawel Joziak, Rafal Powalski, Dawid Ju- rkiewicz, Mickael Coustaty, Bertrand Anckaert, Ernest Val- veny, Matthew Blaschko, Sien Moens, and Tomasz Stanis- lawek. Document understanding dataset and evaluation (dude). In Proceedings of the IEEE/CVF International Con- ference on Computer ...

  33. [33]

    Beyond document page classi- fication: Design, datasets, and challenges

    Jordy Van Landeghem, Sanket Biswas, Matthew Blaschko, and Marie-Francine Moens. Beyond document page classi- fication: Design, datasets, and challenges. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision (WACV), pages 2962–2972, 2024. 2

    2024

  34. [34]

    Idpflow: A no-code agentic framework for multimodal in- telligent document processing

    Goutham Vignesh, Harikrishnan PM, Siddartha Reddy, Saisubramaniam Gopalakrishnan, and Vishal Vaddina. Idpflow: A no-code agentic framework for multimodal in- telligent document processing. In ACM MM, pages 14359– 14360, 2025. 2

    2025

  35. [35]

    LiLT: A sim- ple yet effective language-independent layout transformer for structured document understanding

    Jiapeng Wang, Lianwen Jin, and Kai Ding. LiLT: A sim- ple yet effective language-independent layout transformer for structured document understanding. In Proceedings of the 60th Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2022. 1

    2022

  36. [36]

    OFA: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework

    Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. OFA: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. In Proceedings of the 39th International Con- ference on Machine Learning, pages 23318–23340. PMLR,

  37. [37]

    Page stream seg- mentation with convolutional neural nets combining textual and visual features

    Gregor Wiedemann and Gerhard Heyer. Page stream seg- mentation with convolutional neural nets combining textual and visual features. In Proceedings of the Eleventh Interna- tional Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, 2018. European Language Resources Association (ELRA). 2

    2018

  38. [38]

    Multi-modal page stream segmentation with convolutional neural networks

    Gregor Wiedemann and Gerhard Heyer. Multi-modal page stream segmentation with convolutional neural networks. Lang. Resour. Eval., 55(1):127–150, 2021. 2

    2021

  39. [39]

    Understanding the performance and estimating the cost of llm fine-tuning

    Yuchen Xia, Jiho Kim, Yuhan Chen, Haojie Ye, Souvik Kundu, Cong Callie Hao, and Nishil Talati. Understanding the performance and estimating the cost of llm fine-tuning. In 2024 IEEE International Symposium on Workload Char- acterization (IISWC), pages 210–223. IEEE, 2024. 1

    2024

  40. [40]

    XFUND: A benchmark dataset for multilingual visually rich form under- standing

    Yiheng Xu, Tengchao Lv, Lei Cui, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, and Furu Wei. XFUND: A benchmark dataset for multilingual visually rich form under- standing. In Findings of the Association for Computational Linguistics: ACL 2022 , pages 3214–3224, Dublin, Ireland,

    2022

  41. [42]

    Pages of one form need NOT be adjacent

    Assign each page to its form, grouping by the cues above. Pages of one form need NOT be adjacent

  42. [43]

    Within each form, reconstruct the original page order : begin from the page that opens the form, then iterate over its remaining pages to find the best sequential continuation -- using printed page numbers, section/heading continuity, text or tables that carry across pages, form-field flow, and closings/signatures that mark a final page

  43. [44]

    subdocuments

    Classify each form by its DOMAIN, using ONLY the domain values listed below. Do not create, invent, or use any value not in the list. ## Output Return ONLY a single JSON object with this exact schema: { "subdocuments": [ { "doc_type_id": "<a domain slug from the list below>", "local_doc_id": "<doc_type_id>-<2-digit ordinal starting at 01>", "page_ordinals...

  44. [2022]

    subdocuments

    Association for Computational Linguistics. 1, 3 9 A. Appendix/Supplementary Materials A.1. Inference Prompt Details Prompts used for the packet splitting experiments are detailed in this section. Domain Label Set: {label block} agriculture agriculture, farming, fertilizer, and seed forms city corp city corporation (urban municipal) administrative forms co...