REVIEW 3 major objections 7 minor 17 references
CoSMo: A Multimodal Transformer for Page Stream Segmentation in Comic Books
T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A compact Transformer that splits comic books into stories, ads, and covers reaches 98.10 F1-Macro on a new 20,800-page benchmark.
desk verdict Useful new task and dataset for comic-book page stream segmentation, but the missing train/test split makes the headline numbers unverifiable as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a dual-token Transformer encoder: each page is represented by a frozen pretrained visual embedding and, in the multimodal variant, a text embedding of structured OCR, and the two tokens are interleaved so the encoder can attend over the whole page sequence. Four Transformer layers with absolute positional encoding model narrative flow, and only one token per page feeds a three-layer classification head. A cost-sensitive cross-entropy loss reweights classes by inverse frequency. The same architecture with only visual tokens forms the lightweight vision-only variant; the design point is that decisions come from cross-page context rather than page-local features alone.
What would settle it
Have several independent annotators re-label a random sample of about 50 books with the same protocol, measure agreement specifically on First-Page boundaries, and retrain and evaluate CoSMo on majority-vote labels. If F1-Macro or Panoptic Quality falls materially, or if annotators disagree on a large share of story starts, the reported state of the art is partly an artifact of one annotation protocol.
Extended reading notes
Core claim
The central discovery is that comic-book page-stream segmentation can be recast as multiclass sequence labeling over a page stream, with each page assigned one of Cover, Advertisement, Text Story, Story, or First-Page, the last being a derived label for the first page of a narrative block. A compact encoder-only Transformer that attends across the whole book solves this task to near-ceiling accuracy on the curated dataset: the multimodal variant reaches 98.10 F1-Macro, 95.08 Panoptic Quality, and a minimum-drags metric of 0.437, while the vision-only variant is nearly as strong. The authors interpret this as evidence that visual layout and style carry most of the segmentation signal, with text helping specifically on ambiguous pages such as ads that resemble stories or first pages with weak title cues. They also position the work as the first method that ingests an entire volume, in contrast to prior single-page or panel-sequence comic understanding work.
Load-bearing premise
The results rest on the assumption that the manually curated, archive-aligned labels, especially the derived First-Page class, are accurate and consistent across all 20,800 pages; quality checks are reported, but no inter-annotator agreement or annotation-error rate is given, so systematic label noise would propagate into every headline number.
Editorial extensions
If this is right
- If the reported accuracy holds, archives can auto-generate story-boundary metadata for backlogs of scanned comic books at roughly one percent error instead of tagging pages by hand.
- A task-specific model with a small encoder outperforming a much larger zero-shot vision-language model suggests that general-purpose models are not the cheapest route for structured document segmentation.
- The strong vision-only results imply that OCR is not required for most pages, so segmentation can run cheaply where text extraction is costly or impossible.
- Reliable page-stream segmentation unlocks downstream analysis, character tracking, story indexing, and metadata enrichment, on story blocks rather than on whole issues.
- First-Page detection is the hardest class and the main source of residual error; further gains depend on better long-range context modeling for story-boundary cues.
Reading between the lines
- The dataset is drawn from one public-domain Golden Age archive; whether the roughly 98 percent F1 transfers to modern comics, manga, or European albums is untested and would be a natural domain-shift study.
- A minimum-drags value below one suggests a human annotator would need fewer than one drag-and-drop correction per book on average, but how sensitive that metric is to label noise is not examined.
- A concrete extension would be fine-tuning on a small sample of modern books to measure transfer; the paper's own failure cases already point to atypical first pages and ad-like mixed content as the error clusters to attack.
- Since First-Page is a derived label, publishing the exact derivation protocol is what would make the benchmark reproducible; a different story-boundary definition would change labels and headline numbers.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces CoSMo, a transformer-encoder architecture for page stream segmentation (PSS) in comic books, with vision-only and multimodal variants. The authors formalize the task, curate a 430-book/20,800-page dataset labeled into Cover, Advertisement, Text Story, Story, and the derived First-Page classes, and report experiments in which CoSMo outperforms XGBoost, linear probes, zero-shot CLIP/SigLIP, and zero-shot Qwen2.5-VL-32B across F1-Macro, Panoptic Quality (PQ), and the stream-level MnDD metric. The central claim is that a lightweight, task-specific sequential model beats larger general-purpose vision-language models on this task.
Significance. If the reported results withstand scrutiny, CoSMo would be a strong new baseline for comic PSS, and the released dataset would be a useful public resource for a previously absent domain in the PSS literature. The paper's strengths include a clear task formalization, a modular architecture description, systematic ablations over visual backbones and fusion strategies, and the stated intention to release code and data. The finding that visual features dominate while text helps resolve ambiguous cases is plausible and actionable. The main weakness is that the evaluation protocol is incompletely specified, so the generalization claims are currently unverifiable.
major comments (3)
- [Section 6.4; Tables 1-3] The paper never specifies the train/validation/test split: it does not state the number of books per split, whether splits are performed at book level, or whether a book used for training can also appear in evaluation. Because CoSMo encodes the full book stream with positional encoding and attention over the entire sequence, any overlap between training and test books would permit memorization of page-transition structure, including the derived First-Page boundaries. All headline numbers (e.g., F1-Macro 98.10, PQ 95.08, MnDD 0.437 in Table 2) therefore lack provenance as measures of generalization. Please state the exact split, guarantee book-level separation, and report results over multiple seeds with mean and standard deviation.
- [Section 5; First-Page label] First-Page is described as a derived label marking the first page in a narrative block, but the derivation protocol is not given and no annotation-quality metrics are reported. The paper mentions 'manual annotation' and 'quality checks' without inter-annotator agreement, label error rate, or adjudication details. Since First-Page is consistently the hardest class and stream-level metrics depend on segment boundaries, systematic ambiguity or noise in this derived label propagates into every comparison in Tables 2 and 3. Please provide the annotation protocol and a quantitative assessment of label reliability.
- [Section 6.4 and Tables 1-3] All experimental results are single numbers without variance. Given the class imbalance (Section 5) and the modest differences between some variants, such as multimodal versus vision-only CoSMo (F1-Macro 98.10 vs 97.30; PQ 95.08 vs 94.50), error bars or repeated-run statistics are needed to support the claims of consistent improvement and of the multimodal variant being 'best overall.' Report mean and standard deviation over at least three seeds or a book-level bootstrap.
minor comments (7)
- [Abstract and Section 1] The abstract and the introductory summary say CoSMo outperforms 'significantly larger general-purpose vision-language models' without consistently adding that the LLM comparisons are zero-shot. Section 6.1 is explicit about zero-shot evaluation, so the high-level statements should carry the same qualifier to avoid overstatement.
- [Table 2; Section 6.2] The two Multimodal rows in Table 2 are not labeled by fusion strategy, even though Section 6.2 describes two distinct strategies, fused and multitoken. Add explicit row labels so the reader can map the table to the described variants.
- [Equation (4)] The MnDD formula as printed, MnDD = N - sum_{i,j} max_i |Gi ∩ Pj|, is not well-formed because the maximization index appears over the same index used in the summation. Please align the notation with reference [8].
- [Section 6.4] The training protocol lists only the learning rate and early stopping. Please also report the optimizer, batch size, number of epochs, and the exact inverse-frequency weighting used in the cost-sensitive cross-entropy loss.
- [Section 4.1; Section 7.1] The manuscript uses 'lightweight' to describe CoSMo without giving parameter counts. Add parameter counts for the vision-only and multimodal variants to support the comparison with Qwen2.5-VL-32B.
- [Section 7.1] The single-page CoSMo evaluation is reported without describing how a sequence-trained model is applied to an isolated page. Specify whether the page is fed as a one-page sequence, whether context is masked, or whether a separate inference procedure is used.
- [Section 1 and Table 2] The introduction states that CoSMo has 'an impressive error rate of approximately 1%,' but Table 2 reports accuracy of 98.65% for the best model, corresponding to a 1.35% error rate. Please reconcile the wording with the reported numbers.
Circularity Check
No circularity: CoSMo's results come from an externally benchmarked supervised model; self-citations are confined to framing and do not carry the derivation.
full rationale
The paper's central claim is that CoSMo outperforms baselines on comic-book Page Stream Segmentation. The derivation chain is: collect a manually annotated dataset, define a four-class (plus derived First-Page) labeling task, extract frozen SigLIP visual features and Qwen OCR embeddings, train an encoder-only Transformer with cost-sensitive cross-entropy, and evaluate against external baselines (XGBoost, CLIP/SigLIP zero-shot and linear probes, Qwen2.5-VL zero-shot) using metrics adopted from OpenPSS and prior PSS work. No equation in the paper maps fitted parameters to the reported F1-Macro, PQ, or MnDD values; the loss is defined on the same class labels used for evaluation, which is standard supervised learning rather than circularity. The 'First-Page' derived label encodes story boundaries, but that is the task definition, not a smuggled input. The LoCU survey [15] is authored by members of the present team, but it is used only as conceptual framing in the introduction and conclusion and does not determine the architecture, training procedure, or benchmark numbers. The reported results depend on external architectures and a freshly curated dataset, and no self-citation is load-bearing. The missing train/validation/test split and the choice of SigLIP based on the reported test table are reproducibility and selection concerns, not circular reasoning. Therefore, the paper shows no significant circularity.
Assumptions & free parameters
free parameters (3)
- learning rate =
1e-6
- dropout rate =
0.4
- cost-sensitive class weights =
inverse class frequency
assumptions (5)
- domain assumption Frozen SigLIP and Qwen-derived embeddings transfer to comic pages without task-specific adaptation.
- domain assumption The manually curated comics.org-aligned labels are accurate, consistent ground truth for story, cover, advertisement, and text-story boundaries.
- domain assumption The derived First-Page label, defined as the first page of a narrative block, is a well-defined and learnable target.
- domain assumption Cost-sensitive cross-entropy with inverse-frequency weights is an adequate treatment of the class imbalance.
- standard math Standard segment-matching assumptions, such as IoU threshold 0.5, apply to comic page streams.
Cite this review
Pith. "Pith review of CoSMo: A Multimodal Transformer for Page Stream Segmentation in Comic Books." pith.science (2026). https://pith.science/paper/WYNOXRRO
@misc{pith2026250710053,
author = {Pith},
title = {Pith review of: CoSMo: A Multimodal Transformer for Page Stream Segmentation in Comic Books},
year = {2026},
howpublished = {\url{https://pith.science/paper/WYNOXRRO}},
note = {Machine review of arXiv:2507.10053}
}
read the original abstract
This paper introduces CoSMo, a novel multimodal Transformer for Page Stream Segmentation (PSS) in comic books, a critical task for automated content understanding, as it is a necessary first stage for many downstream tasks like character analysis, story indexing, or metadata enrichment. We formalize PSS for this unique medium and curate a new 20,800-page annotated dataset. CoSMo, developed in vision-only and multimodal variants, consistently outperforms traditional baselines and significantly larger general-purpose vision-language models across F1-Macro, Panoptic Quality, and stream-level metrics. Our findings highlight the dominance of visual features for comic PSS macro-structure, yet demonstrate multimodal benefits in resolving challenging ambiguities. CoSMo establishes a new state-of-the-art, paving the way for scalable comic book analysis.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Qwen2.5-vl technical report, 2025
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025. 4
2025
-
[2]
Semantic parsing of interpage relations
Mehmet Arif Demirtas ¸, Berke Oral, Mehmet Yasin Akpınar, and Onur Deniz. Semantic parsing of interpage relations. In 2022 26th International Conference on Pattern Recognition (ICPR), pages 1579–1585. IEEE, 2022. 2
work page 2022
-
[3]
Document classification and page stream segmentation for digital mailroom applications
Albert Gordo, Marcal Rusinol, Dimosthenis Karatzas, and Andrew Bagdanov. Document classification and page stream segmentation for digital mailroom applications. In Proceed- ings of the International Conference on Document Analysis and Recognition, ICDAR, pages 621–625, 2013. 2
work page 2013
-
[4]
Large language models for page stream segmentation, 2024
Hunter Heidenreich, Ratish Dalvi, Rohith Mukku, Nikhil Verma, and Neven Piˇculjan. Large language models for page stream segmentation, 2024. 3
work page 2024
-
[5]
Openpss: An open page stream segmentation benchmark
Ruben van Heusden, Jaap Kamps, and Maarten Marx. Openpss: An open page stream segmentation benchmark. In Linking Theory and Practice of Digital Libraries: 28th In- ternational Conference on Theory and Practice of Digital Li- braries, TPDL 2024, Ljubljana, Slovenia, September 24–27, 2024, Proceedings, Part I , page 413–429, Berlin, Heidel- berg, 2024. Spri...
work page 2024
-
[6]
Boyd-Graber, Hal Daum ´e, and Larry S
Mohit Iyyer, Varun Manjunatha, Anupam Guha, Yogarshi Vyas, Jordan L. Boyd-Graber, Hal Daum ´e, and Larry S. Davis. The amazing mysteries of the gutter: Drawing in- ferences between panels in comic book narratives. 2017 IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), pages 6478–6487, 2016. 2
work page 2017
-
[7]
Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Doll ´ar. Panoptic segmentation. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9404–9413, 2019. 3, 5
work page 2019
-
[8]
Tab this folder of documents: Page stream seg- mentation of business documents
Thisanaporn Mungmeeprued, Yuxin Ma, Nisarg Mehta, and Aldo Lipani. Tab this folder of documents: Page stream seg- mentation of business documents. 2022. 2, 3, 5
work page 2022
Show all 17 references
-
[9]
The manga whis- perer: Automatically generating transcriptions for comics
Ragav Sachdeva and Andrew Zisserman. The manga whis- perer: Automatically generating transcriptions for comics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12967–12976, 2024. 2
2024
-
[10]
From panels to prose: Generating literary narratives from comics, 2025
Ragav Sachdeva and Andrew Zisserman. From panels to prose: Generating literary narratives from comics, 2025
2025
-
[11]
Tails tell tales: Chapter-wide manga transcriptions with char- acter names
Ragav Sachdeva, Gyungin Shin, and Andrew Zisserman. Tails tell tales: Chapter-wide manga transcriptions with char- acter names. In Proceedings of the Asian Conference on Computer Vision (ACCV), pages 2053–2069, 2024. 2
2024
-
[12]
Comix: A comprehensive benchmark for multi-task comic understanding
Emanuele Vivoli, Marco Bertini, and Dimosthenis Karatzas. Comix: A comprehensive benchmark for multi-task comic understanding. Advances in Neural Information Processing Systems, 37:140828–140846, 2024. 2
2024
-
[13]
Comics datasets framework: Mix of comics datasets for de- tection benchmarking
Emanuele Vivoli, Irene Campaioli, Mariateresa Nardoni, Niccol`o Biondi, Marco Bertini, and Dimosthenis Karatzas. Comics datasets framework: Mix of comics datasets for de- tection benchmarking. In International Conference on Doc- ument Analysis and Recognition , pages 154–167. ...
-
[14]
Multimodal transformer for comics text-cloze
Emanuele Vivoli, Joan Lafuente Baeza, Ernest Valveny Llo- bet, and Dimosthenis Karatzas. Multimodal transformer for comics text-cloze. In International Conference on Document Analysis and Recognition, pages 128–145. Springer, 2024. 2
2024
-
[15]
One missing piece in vision and language: A survey on comics understanding, 2025
Emanuele Vivoli, Mohamed Ali Souibgui, Andrey Barsky, Artemis Llabr´es, Marco Bertini, and Dimosthenis Karatzas. One missing piece in vision and language: A survey on comics understanding, 2025. 2
2025
-
[16]
Page stream seg- mentation with convolutional neural nets combining textual and visual features
Gregor Wiedemann and Gerhard Heyer. Page stream seg- mentation with convolutional neural nets combining textual and visual features. In Proceedings of the Eleventh Interna- tional Conference on Language Resources and Evaluation (LREC 2018), 2018. 2
2018
-
[17]
Qwen3 em- bedding: Advancing text embedding and reranking through foundation models, 2025
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. Qwen3 em- bedding: Advancing text embedding and reranking through foundation models, 2025. 3
2025
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.