Pith. sign in

REVIEW 4 major objections 5 minor 55 references

Automating Iconclass: LLMs and RAG for Large-Scale Classification of Religious Woodcuts

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A zero-fine-tuning LLM pipeline assigns Iconclass codes to early-modern religious woodcuts at up to 92% precision, beating a 52% keyword baseline.

desk verdict A useful, honest empirical study with a clear article-level result, but the headline precision numbers are inflated by test set construction and truncation. read the letter →

arxiv 2510.19986 v1 pith:TWZXFDCZ submitted 2025-10-22 cs.IR cs.CV

Drew B. Thomas This is my paper · ORCID
classification cs.IRcs.CV
keywords IconclasswoodcutsRetrieval-AugmentedGenerationvectorsearchlargelanguagemodelsearlymodernprintsimageclassificationdigitalhumanities
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a general-purpose multimodal language model, used without any fine-tuning, can classify early-modern religious woodcuts into the Iconclass system accurately enough for large-scale archival work. The pipeline generates detailed descriptions of full book pages, retrieves the most similar Iconclass entries from a database restricted to religion and Bible classes, and lets the language model choose the best match. On a 590-image test set of Reformation-era Bibles and recurring biblical themes, it reaches 92% precision at four classification levels and 87% at five, versus 52% for keyword search. If the numbers hold, art historians can label vast visual archives without building training sets or training custom models.

What carries the argument

The full-page description is the load-bearing component: GPT-4o sees the woodcut together with the printed page's text and produces a caption rich in the theological vocabulary that Iconclass entries use. That description is embedded and used to query a vector database of about 12,600 Iconclass entries from classes 1 (Religion and Magic) and 7 (Bible). A hybrid search (lexical BM25 plus semantic vector similarity) or pure vector search retrieves candidates, and a RAG step presents the top five to the LLM, which selects the single best code. The database's restriction to two of Iconclass's ten base classes is what keeps the task manageable.

What would settle it

Re-run the pipeline on a test set drawn from Iconclass classes outside 1 and 7—for example secular allegories or mythological prints—with codes assigned by a second, independent annotator, and compare precision. If precision drops substantially (say below 60%), the result is an artifact of the closed biblical label space and vocabulary leakage rather than a general classification capability.

Watch

Extended reading notes

Core claim

The central claim is that combining full-page image descriptions from GPT-4o with vector retrieval over a hierarchical database of Iconclass descriptions, followed by a retrieval-augmented generation step, yields precision of 87–92% on Iconclass classification of early-modern religious woodcuts. The description from the full page—including surrounding text, headings, and blackletter captions—is the key innovation: it supplies contextual vocabulary that the retrieval step can match against Iconclass entries. The best configuration is RAG over the basic (non-hierarchical) database with vector search on full-page descriptions; truncating predictions to four or five levels trades recall for high

Load-bearing premise

The 87–92% precision numbers rest on ground-truth Iconclass codes that are subjective and were partly derived from the same thematic search terms used to assemble the test set, so if those labels are idiosyncratic or the test set is unrepresentative, the headline precision will not transfer.

Editorial extensions

If this is right

  • If the pipeline generalizes, the 120,000-image corpus of Holy Roman Empire religious illustrations can be fully classified automatically, with human review reserved for ambiguous cases.
  • The four-level truncation rule (precision 92%) is a practical operational guideline for thematic browsing: broad story-level labels are reliable, deeper levels are not.
  • Because full-page context is decisive, digitized book pages with legible surrounding text are a richer input than cropped illustrations—an insight that transfers to other early printed materials.
  • A closed label space of only two Iconclass classes means the method's guaranteed performance ceiling is narrower than an open-set classifier; expanding the database would be the natural next test.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The measured precision is likely inflated relative to open-set conditions: the test set includes 311 images found by querying an image search engine with the exact theme names (Adam and Eve, Noah's Ark, etc.), so the LLM's descriptions naturally contain the vocabulary that matches those Iconclass entries.
  • A clean falsifier would be to apply the same pipeline to Iconclass classes outside 1 and 7, or to secular non-biblical woodcuts, using independently assigned ground truth; a sharp precision drop would show the result is locked to the biblical vocabulary distribution.
  • The paper's own report that cropped images perform well only for iconic scenes suggests the descriptions are doing semantic inference, not visual recognition; a pure text-only baseline using only the page's OCR text could disentangle how much visual content contributes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a zero-fine-tuning pipeline for assigning Iconclass codes to early modern religious woodcuts. GPT-4o generates textual descriptions from either the full page or the cropped illustration; these descriptions are then matched against a vector database of Iconclass entries restricted to classes 1 and 7 (~12,600 codes) using keyword (BM25), vector, hybrid, and RAG-based retrieval. The system is evaluated on 590 test images (282 Luther Bible illustrations and 311 thematic images) with hierarchical precision, recall, F1, and a weighted score. The best configuration (RAG with vector search over the basic database using full-page descriptions) is reported to reach 87% precision at five levels and 92% at four levels, outperforming image and keyword baselines. The paper argues that this method is practical for large-scale iconographic classification and requires no fine-tuning.

Significance. If the reported accuracy holds under a more rigorous evaluation, this would be a practically valuable contribution to digital humanities: it offers a low-cost, scalable alternative to manual Iconclass annotation. The paper's strengths include a broad comparison across 15 model configurations, the use of full-page contextual descriptions, transparent hierarchical evaluation metrics, and an unusually candid limitations section. However, the headline numbers rest on a test set whose selection process is likely to inflate performance, and the comparison with the keyword baseline is not like-for-like. The significance is therefore conditional on correcting these evaluation asymmetries; as presented, the 87–92% figures are best read as an upper bound for a closed, thematically canonical distribution rather than as a general capability estimate for the full 120,000-image corpus.

major comments (4)
  1. [§9, Fig. 4, Fig. 7] The headline comparison is not apples-to-apples. The text says the method achieves '92% precision at four classification levels and 87% at five levels—a significant improvement in prediction accuracy over traditional keyword search methods (52%).' The 52% is the weighted score from Fig. 4 for the best keyword run, not a precision value, and the keyword baseline is not truncated to the same depth as the RAG predictions. To support the claimed improvement, the authors should recompute precision/recall/F1 for all baselines at the same truncation depths used for the RAG results (e.g., truncating to 4 and 5 levels) and report these alongside the weighted scores.
  2. [§3, §4.2.2] The test set is not representative of the full corpus. Of the 590 images, 311 were retrieved by querying the project's WISE/OpenCLIP visual search engine with the target theme names themselves ('Adam and Eve', 'Noah's Ark', 'Annunciation', etc.). These images are by construction canonical, easily recognized instances of the very Iconclass categories used as ground truth, and the closed label space of only classes 1 and 7 (§4.2.2) further reduces task difficulty. This likely inflates the headline precision. The paper should report results separately for the Luther Bible subset (282 images) and the thematic subset, and explicitly state that the 87–92% figures are not estimates for arbitrary pages in the 120k corpus.
  3. [§7.3, Fig. 7] The headline precision numbers are based on post-hoc truncation of the model's own predictions, and the accompanying recall cost is omitted from the conclusion. In Fig. 7, truncating two levels from the RAG-vector basic model gives precision 0.9246 but recall falls to 0.6972; truncating one level gives precision 0.8694 with recall 0.8126. The conclusion should report precision and recall together for each truncation level, and should clarify that '92% at four levels' and '87% at five levels' are achieved only after removing the most detailed (and most error-prone) levels of the output.
  4. [§3, §8] The ground-truth labels themselves are a load-bearing assumption. For the Luther Bibles, the codes are derived from published caption lists rather than from independent iconographic annotation, and §8 concedes that Iconclass assignment is subjective and that different annotators could choose different codes. If the reference codes are wrong or idiosyncratic, the reported precision is correspondingly unreliable. At a minimum, the authors should quantify the uncertainty by having a second annotator code a sample of the Luther Bible images and reporting agreement, and should describe how ambiguous cases were reconciled.
minor comments (5)
  1. [§6.1 / Fig. 4] The text says the image-based search 'precision was 0.4261, with recall and F1-scores of 0.3097 and 0.3017', but Fig. 4 reports precision 0.3763, recall 0.2844, F1 0.3017. Please reconcile these numbers.
  2. [Fig. 4 / Fig. 7] The F1 score for the RAG-vector page basic row is 0.8248 in Fig. 4 but 0.8262 in Fig. 7 (truncation level 0). Please correct the inconsistency.
  3. [§2 / §4.2.2] The embedding model is not specified beyond 'OpenAI text embedding model'. Since vector search results are sensitive to the embedding choice, please provide the exact model identifier and version used.
  4. [General] The paper repeatedly uses 'significant' and 'significantly' (e.g., §1, §6.3.2, §9) without statistical tests. If significance is meant in a statistical sense, error bars or tests should be provided; otherwise, use a different qualifier.
  5. [General] No code/data repository is provided. To support reproducibility of the reported numbers, please include or link to the test set, the ground-truth lists, and the evaluation scripts.

Circularity Check

0 steps flagged · score 0.0 of 10

No construction-level circularity: the headline accuracies are empirical scores against published ground-truth labels; test-selection and truncation choices are validity concerns, not tautological reductions.

full rationale

The paper's central claim is an empirical benchmark, not a derivation, so there is no chain of equations in which an output is equivalent to an input by construction. Ground truth for the Luther Bible subsets comes from external published sources (Christensen 2005 and the Adelaide 2022 catalogue, Section 3), and the image-search baseline uses the external Iconclass AI Test Set; these labels are not derived from the model's own predictions. The 87%/92% figures come from truncating predicted Iconclass codes (Section 7.3, Fig. 7), with the truncation depth chosen from the model's own per-level accuracy curve on the same test set; this is a post-hoc reporting choice that can inflate precision and lower recall, but it is not a fitted parameter renamed as a prediction. Section 3 discloses that 311 of 590 test images were selected by querying the project's WISE engine with theme names, likely making those instances easier for a text-based pipeline, and Section 4.2.2 restricts the label space to Iconclass classes 1 and 7. These are evaluation-design limitations, also acknowledged in Section 8's discussion of subjective ground truth and single-code assignment, but they do not make the classification result equivalent to its inputs: the retrieval decision still depends on the GPT-4o description of the image, not on the ground-truth label itself. No load-bearing self-citation or imported uniqueness theorem is invoked; the Ornamento corpus citation is for data provenance and is not used to justify the accuracy claim. Under the required standard, no specific circular step can be exhibited.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper contributes a pipeline, not a derivation. No parameters were fit to the ground truth in the machine-learning sense, but the label-space restriction, retrieval depths (k=10 image baseline; k=5 RAG), the evaluation weights, and above all the truncation depth producing the headline precision are hand- or data-chosen. The domain axioms are: GPT-4o describes the woodcuts and reads blackletter faithfully; the published caption lists give correct ground truth; the Iconclass AI Test Set labels are correct; the embedding models are adequate; and the thematic test images found by theme-name queries are a valid sample.

free parameters (5)
  • Nearest-neighbor count k for image-search baseline = 10
    Section 4.1: retrieved top-10 nearest neighbors from the filtered Iconclass AI Test Set and took the most frequent code; k is hand-chosen and affects only the image-baseline scores.
  • Retrieval count k for RAG = 5
    Section 4.3.3: top-five results are passed to the LLM; section 9 notes the correct code was sometimes ranked sixth, so k=5 bounds the achievable RAG accuracy.
  • Prediction truncation depth for headline precision = 2 levels (avg depth 4.08)
    Section 7.3/Fig. 7: truncating predictions by two levels yields the '92% at four levels' figure; the depth was suggested by the per-level accuracy drop-off in Fig. 6, making the headline precision a data-informed transform of the model's output.
  • Label-space restriction = Iconclass classes 1 and 7 only (~12,600 entries)
    Section 4.2.2: the vector database contains only Religion and Magic / Bible categories, turning the task into closed-set retrieval and 'limiting the scope of possible errors' (section 9).
  • Weighted scoring base values = 100/90/85/70/60
    Section 5.2: author-defined weights for match types drive the average weighted score comparisons, including the 52% keyword figure cited in the abstract.
assumptions (5)
  • domain assumption GPT-4o produces sufficiently faithful descriptions of early-modern woodcuts and reads blackletter German page text correctly
    Section 4.2.1 and section 9 ('it proved that the LLM can successfully read early modern German text printed in blackletter type'); no independent transcription-accuracy measurement is provided.
  • domain assumption Published Luther Bible illustration catalogs (Christensen 2005; Adelaide 2022) and the author's caption-to-Iconclass conversion yield correct ground truth
    Section 3: ground truth is based on caption lists; section 8 concedes Iconclass assignment is subjective and different annotators plausibly differ.
  • domain assumption Iconclass AI Test Set reference codes are correct for the image-search baseline
    Section 4.1: the image baseline transfers codes from the 21,422-image filtered test set; errors in that set become baseline errors.
  • domain assumption OpenAI text embeddings and OpenCLIP capture the semantic similarity needed for retrieval
    Section 4.3: the pipeline assumes cosine similarity over these embeddings separates correct Iconclass entries from incorrect ones; the embedding model version is not specified.
  • domain assumption Thematic test images identified by querying with theme names constitute a valid, unbiased test set
    Section 3: images for Adam and Eve, Noah's Ark, etc. were found via text queries to the visual-search engine; the query vocabulary overlaps the Iconclass descriptions the system later retrieves.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Automating Iconclass: LLMs and RAG for Large-Scale Classification of Religious Woodcuts." pith.science (2026). https://pith.science/paper/TWZXFDCZ

@misc{pith2026251019986,
  author       = {Pith},
  title        = {Pith review of: Automating Iconclass: LLMs and RAG for Large-Scale Classification of Religious Woodcuts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TWZXFDCZ}},
  note         = {Machine review of arXiv:2510.19986}
}
read the original abstract

This paper presents a novel methodology for classifying early modern religious images by using Large Language Models (LLMs) and vector databases in combination with Retrieval-Augmented Generation (RAG). The approach leverages the full-page context of book illustrations from the Holy Roman Empire, allowing the LLM to generate detailed descriptions that incorporate both visual and textual elements. These descriptions are then matched to relevant Iconclass codes through a hybrid vector search. This method achieves 87% and 92% precision at five and four levels of classification, significantly outperforming traditional image and keyword-based searches. By employing full-page descriptions and RAG, the system enhances classification accuracy, offering a powerful tool for large-scale analysis of early modern visual archives. This interdisciplinary approach demonstrates the growing potential of LLMs and RAG in advancing research within art history and digital humanities.

Figures

Figures reproduced from arXiv: 2510.19986 by the authors.

Figure 1
Figure 1. Iconclass Example Recent advances in multi-modal Large Language Models (LLMs), combined with improvements in semantic embeddings using vector databases, present new opportunities for 2. https://iconclass.org/ 3. Henri van de Waal, Iconclass an iconographic classification system, completed and edited by L.D. Couprie with R.H. Fuchs, E. Tholen & G. Vellekoop (North-Holland, Amsterdam, 1974-1985) [PITH_FULL_IMAGE:figu… view at source ↗
Figure 2
Figure 2. A page from Luther’s Complete Bible from 1534 depicting Samson wrestling a lion. 4. Methodology 4.1 Image-Based Search woodcuts as the Australian edition except for one or two exchanges. These were rectified prior to the analysis [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Diagram of Methodology 5. Model Evaluation In this section, I outline the methodology and scoring system used to evaluate model performance. Given the hierarchical and nuanced nature of Iconclass, this evaluation focuses not only on perfect matches but also on partial matches, which play a crucial role in identifying broader themes in the images. To assess performance, I used a combination of traditional precision, … view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Model evaluation metrics [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Results by Match Type 6.1 Image-Based Search The image-based search method used the Iconclass AI Test Set for classifying the test images. Across the entire corpus, the model achieved 53 full matches, where the predicted Iconclass code exactly aligned with the ground t…
Figure 6
Figure 6. Figure 6: Accuracy by classification level Another method is to recalculate evaluation metrics after truncating the predictions by one level at a time. In the RAG model with vector search on the basic database, truncating the last level from each predicted Iconclass raises the m…
Figure 7
Figure 7. Figure 7: Recalculated evaluation metrics by truncation level 8. Limitations of the Study While the methodology of using LLM-generated descriptions in combination with a vector database has produced encouraging results, several limitations affect the precision and reliability of…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 4 canonical work pages

  1. [1]

    Visualizing Faith: Print, Piety, and Propaganda

    Introduction The advent of the printing press in Early Modern Europe revolutionized the dissemination of graphic design and visual media.1 For the first time, illustrations were reproduced at an unprecedented scale, appearing in millions of copies of books across the continent. These illustrations played a crucial role in shaping public opinion, educating...

  2. [2]

    Digital Humanities and Artificial Intelligence

    This publication has emanated from research conducted with the financial support of Taighde Éireann – Research Ireland, under Grant number 21/PATH-A/9655 at University College Dublin. This work was first presented at the "Digital Humanities and Artificial Intelligence" conference at the University of Reading on 17 June 2024. 2 reinforce theological princi...

    2024

  3. [3]

    Couprie with R.H

    Henri van de Waal, Iconclass an iconographic classification system, completed and edited by L.D. Couprie with R.H. Fuchs, E. Tholen & G. Vellekoop (North-Holland, Amsterdam, 1974-1985). 3 examining the visual content of historical images. This paper introduces a novel methodology that leverages these tools. By feeding an image into an LLM, a detailed text...

    1974

  4. [4]

    https://iconclass.org/

  5. [5]

    Giles Bergel, et al. “Content-Based Image-Recognition on Printed Broadside Ballads: The Bodleian Libraries’ 4 computer vision to build a large database of printers’ ornaments.6 The Ornamento project used a similar methodology to identify illustrations and ornamentation in books printed across Europe and the Spanish Americas up to the year 1600.7 Such proj...

    2013

  6. [6]

    Related Work Recent advancements in object detection and image captioning have demonstrated significant potential in automating the classification and analysis of cultural heritage data. Vision-language models fine-tuned on custom datasets have shown that deep learning can generate meaningful descriptions in art-historical contexts.4 Several projects have...

  7. [7]

    Iconographic Image Captioning for Artworks

    Eva Cetinic, “Iconographic Image Captioning for Artworks,” arXiv, February 7, 2021. https://doi.org/10.48550/arXiv.2102.03942

    work page Pith review arXiv doi:10.48550/arxiv.2102.03942 2021

  8. [8]

    DEArt: Dataset of European Art

    Artem Reshetnikov, et al., “DEArt: Dataset of European Art,” arXiv, November 3, 2022. https://doi.org/10.48550/arXiv.2211.01226

    work page Pith review arXiv doi:10.48550/arxiv.2211.01226 2022

Show all 55 references
  1. [9]

    Computer Vision and the Creation of a Database of Printers’ Ornaments,

    Hazel Wilkinson, et al. “Computer Vision and the Creation of a Database of Printers’ Ornaments,” Digital Humanities Quarterly, 15, no. 1 (May 21, 2021)

    2021

  2. [10]

    Ornamento Europe: Towards an Atlas of the Visual Geography of the Renaissance Book,

    Alexander S. Wilkinson, “Ornamento Europe: Towards an Atlas of the Visual Geography of the Renaissance Book,” in Arthur der Weduwen and Malcolm Walsby (eds.), The Book World of Early Modern Europe (Leiden: Brill, 2022), pp. 547–62

    2022

  3. [11]

    Capturing Captions: Using AI to Identify and Analyse Image Captions in a Large Dataset of Historical Book Illustrations,

    Julia Thomas and Irene Testini, “Capturing Captions: Using AI to Identify and Analyse Image Captions in a Large Dataset of Historical Book Illustrations,” Digital Humanities Quarterly, 18, no. 2 (2024). https://digitalhumanities.org/dhq/vol/18/2/000740/000740.html

    2024

  4. [12]

    OmniArt: Multi-Task Deep Learning for Artistic Data Analysis,

    Gjorgji Strezoski and Marcel Worring, “OmniArt: Multi-Task Deep Learning for Artistic Data Analysis,” arXiv, August 2, 2017. https://doi.org/10.48550/arXiv.1708.00684 ; Gjorgji Strezoski and Marcel Worring, “OmniArt: A Large-Scale Artistic Benchmark,” ACM Trans. Multimedia Com...

    2017 doi arXiv

  5. [13]

    Applying Computer Vision Systems to Historical Book Illustrations: Challenges and First Results,

    Yongho Kim, et al., “Applying Computer Vision Systems to Historical Book Illustrations: Challenges and First Results,” 2021. https://www.researchgate.net/publication/356843079

    2021

  6. [14]

    A Dataset and a Convolutional Model for Iconography Classification in Paintings,

    Federico Milani and Piero Fraternali, “A Dataset and a Convolutional Model for Iconography Classification in Paintings,” J. Comput. Cult. Herit., 14, no. 4 (July 16, 2021), pp. 1-18. https://doi.org/10.1145/3458885

    2021 doi

  7. [15]

    SniffyArt: The Dataset of Smelling Persons,

    Mathias Zinnen, et al., “SniffyArt: The Dataset of Smelling Persons,” in Proceedings of the 5th Workshop on analySis, Understanding and proMotion of heritAge Contents, SUMAC ’23 (New York: Association for Computing Machinery, 2023), pp. 49–58. https://doi.org/10.1145/3607542.3617357

    2023

  8. [16]

    Improving Object Detection in Paintings Based on Time Contexts,

    Maria-Cristina Marinescu, et al., “Improving Object Detection in Paintings Based on Time Contexts,” in 2020 International Conference on Data Mining Workshops (2020), pp. 926–32. https://doi.org/10.1109/ICDMW51313.2020.00133

    2020

  9. [17]

    Sensitivity and Access: Unlocking the Colonial Visual Archive with Machine Learning,

    Jonathan Dentler, et al., “Sensitivity and Access: Unlocking the Colonial Visual Archive with Machine Learning,” Digital Humanities Quarterly, 18, no. 2 (2024). https://digitalhumanities.org/dhq/vol/18/2/000742/000742.html

    2024

  10. [18]

    Automatic Modeling of Social Concepts Evoked by Art Images as Multimodal Frames,

    Delfina Pandiani, et al., “Automatic Modeling of Social Concepts Evoked by Art Images as Multimodal Frames,” arXiv (October 14, 2021). https://doi.org/10.48550/arXiv.2110.07420

    2021 doi arXiv

  11. [19]

    Multimodal Search on Iconclass Using Vision-Language Pre-Trained Models,

    Cristian Santini, et al., “Multimodal Search on Iconclass Using Vision-Language Pre-Trained Models,” in Proceedings of the 2023 ACM/IEEE Joint Conference on Digital Libraries (Santa Fe, New Mexico: IEEE Press, 2024), pp. 285–87. https://doi.org/10.1109/JCDL57899.2023.00061 ; E...

    2023

  12. [20]

    https://saintgeorgeonabike.eu/

    Saint George on a Bike. https://saintgeorgeonabike.eu/

  13. [21]

    Transfer Learning for the Visual Arts: The Multi- Modal Retrieval of Iconclass Codes,

    Nikolay Banar, Walter Daelemans, and Mike Kestemont, “Transfer Learning for the Visual Arts: The Multi- Modal Retrieval of Iconclass Codes,” J. Comput. Cult. Herit., 16, no. 2 (June 24, 2023). https://doi.org/10.1145/3575865

    2023 doi

  14. [22]

    Gauging the Limitations of Natural Language Supervised Text-Image Metrics Learning by Iconclass Visual Concepts,

    Kai Labusch and Clemens Neudecker, “Gauging the Limitations of Natural Language Supervised Text-Image Metrics Learning by Iconclass Visual Concepts,” in Proceedings of the 7th International Workshop on Historical Document Imaging and Processing (New York: Association for Compu...

    2023

  15. [23]

    Yolov3: An incremental improvement

    Joseph Redmon and Ali Farhadi, "Yolov3: An incremental improvement." arXiv preprint arXiv:1804.02767 (2018)

    2018 arXiv

  16. [24]

    Multi-Modal Label Retrieval for the Visual Arts: The Case of Iconclass,

    Nikolay Banar, Walter Daelemans, and Mike Kestemont, “Multi-Modal Label Retrieval for the Visual Arts: The Case of Iconclass,” in Proceedings of the 13th International Conference on Agents and Artificial Intelligence (Vienna, Austria: 7 interpretive context

  17. [25]

    Visualizing Faith

    Corpus The image corpus used in this study was built by the Ornamento project at University College Dublin. Ornamento is a comprehensive archive of visual elements from early modern books. The project, consisting of Prof. Alexander Wilkinson and myself, employed a CNN trained ...

    2021

  18. [26]

    Reproducible Scaling Laws for Contrastive Language-Image Learning,

    Mehdi Cherti, et al., “Reproducible Scaling Laws for Contrastive Language-Image Learning,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023), pp. 2818–29. https://doi.org/10.1109/CVPR52729.2023.00276. 8

    2023

  19. [27]

    https://www.robots.ox.ac.uk/~vgg/software/vise/

    Abhishek Dutta, Relja Arandjelović, and Andrew Zisserman, VGG Image Search Engine (2021). https://www.robots.ox.ac.uk/~vgg/software/vise/

    2021

  20. [28]

    WISE Image Search Engine (WISE),

    Prasanna Sridhar, et al., “WISE Image Search Engine (WISE),” Wiki Workshop (10th Edition) (2023). https://gitlab.com/vgg/wise/wise

    2023

  21. [29]

    Berlin, Staatsbibliothek, 4" Bu 9401

    Biblia das ist die gantze heilige schrifft Deudsch (Wittenberg: Hans Lufft, 1534). Berlin, Staatsbibliothek, 4" Bu 9401. VD16 B 2694. USTC 616653

  22. [30]

    I used a published list of the illustrations with captions to apply Iconclasses for the ground-truth.28 The images used come from the copy held at the Berlin State Library.29

    Illustrations from Martin Luther's 1534 Bible: This group includes 117 illustrations from the 1534 edition of Luther's Bible, his first German translation of the complete Bible and a foundational text of the Reformation. I used a published list of the illustrations with captio...

  23. [31]

    Illustrations from Martin Luther's 1551 Bible: This group comprises 165 illustrations from the 1551 edition of Luther's Bible. The Löhe Memorial Library at Australian Luther University owns a copy acquired in 1954.30 They issued a catalogue of the illustrations with captions w...

    1954

  24. [32]

    Adam and Eve (60 images) b

    Selected Thematic Illustrations: The third group consists of images that represent key biblical narratives, classified according to the following iconographic themes: a. Adam and Eve (60 images) b. Noah's Ark (26 images) c. The Annunciation (49 images) d. The Nativity (45 imag...

  25. [33]

    Ornamento Europe

    Wilkinson, “Ornamento Europe.”

  26. [34]

    Luther and the Woodcuts to the 1534 Bible,

    Carl C. Christensen, “Luther and the Woodcuts to the 1534 Bible,” Lutheran Quarterly, 19, no. 4 (Winter 2005), pp. 392–413

    2005

  27. [35]

    Thanks to the Irish Centre for High-End Computing for providing access to the Meluxina supercomputer for this task

  28. [36]

    auffs new zugericht (Wittenberg: Hans Lufft, 1551 [1550])

    Biblia das ist: die gantze heilige schrifft: Deudsch. auffs new zugericht (Wittenberg: Hans Lufft, 1551 [1550]). Adelaide, Australian Luther College, RB CB72 1551. VD16 B 2730. USTC 616495

  29. [37]

    Luther Bible, 1551 Edition,

    Luther Bible Images Catalogue: Illustrations from the 1551 Luther Bible (Adelaide: Australian Lutheran College, 2022). See also, Trevor Schaefer, “Luther Bible, 1551 Edition,” Lutheran Theological Journal, 49, no. 3 (November 9, 2020): pp. 171–78

    2022

  30. [38]

    Adam and Eve

    Biblia: das ist die gantze heilige schrifft: Deudsch (Wittenberg: Hans Lufft, 1551). Stuttgart, Württembergische Landesbibliohtek, Bb deutsch 155007. VD16 B 2729. USTC 616664. The illustrations in this edition are the same 9 f. The Crucifixion (59 images) These thematic images...

  31. [39]

    Religion and Magic

    Methodology 4.1 Image-Based Search woodcuts as the Australian edition except for one or two exchanges. These were rectified prior to the analysis. 10 To establish a baseline for comparison, I queried my images with the Iconclass AI Test Set,33 distributed by Iconclass and samp...

  32. [40]

    Iconclass AI Test Set, https://iconclass.org/testset/

  33. [41]

    A Word Is Worth a Thousand Pictures: Why the Use of Iconclass Will Make Artificial Intelligence Smarter

    Hans Brandhorst, “A Word Is Worth a Thousand Pictures: Why the Use of Iconclass Will Make Artificial Intelligence Smarter.” https://iconclass.org/testset/ICONCLASS_and_AI.pdf. For the Arkyves database, see https://www.arkyves.org/

  34. [42]

    generalizing

    Thanks to my research assistant, Lena Böse, for her help examining such matches. 24 accurate, with a decrease to 84% at level four, and a large drop-off at level 5 (65%). The RAG model using hybrid search with the hierarchical database is similar though slightly lower. This su...

  35. [43]

    Multimodal Search on Iconclass Using Vision-Language Pre-Trained Models

    Santini, et al., “Multimodal Search on Iconclass Using Vision-Language Pre-Trained Models.” See also, 11  Full-Page Descriptions: These descriptions were generated using the full page from the book, allowing the LLM to consider surrounding text, headings, chapter numbers, and...

  36. [44]

    The specific model was gpt-4o-2024-05-13

    2024

  37. [45]

    https://github.com/weaviate/weaviate

  38. [46]

    Holy Communion

    Only the descriptions, and not the codes, were embedded in the vector database. 12 Genesis from the descendants of Cain and Seth to Abraham; story of Noah; the building of the ark, and the embarkation (Genesis 7:5-9)”) Notice in the example that there is no mention of Noah in ...

  39. [47]

    14 address cases where the top result might not be the most accurate match

    The OpenAI embedding model was used via Weaviate’s text2vec_openai OpenAI integration. 14 address cases where the top result might not be the most accurate match. By incorporating retrieval and generation, this method added an additional layer of analysis to improve classifica...

  40. [48]

    the story of David and Goliath (1 Samuel 17)

    Model Evaluation In this section, I outline the methodology and scoring system used to evaluate model performance. Given the hierarchical and nuanced nature of Iconclass, this evaluation focuses not only on perfect matches but also on partial matches, which play a crucial role...

  41. [49]

    For example, if the ground truth has five levels and the prediction correctly matches three of them, the base score is adjusted to reflect a 60% match

    https://pypi.org/project/iconclass/ 17 match the ground truth. For example, if the ground truth has five levels and the prediction correctly matches three of them, the base score is adjusted to reflect a 60% match. This percentage-based adjustment enables a more detailed pictu...

  42. [50]

    Results Query Image Database Average of Weight Scores Avg Precision Avg Recall Avg F1 Score image illustration image 31.26168 0.3763 0.2844 0.3017 keyword illustration basic 38.36353 0.435 0.4909 0.4561 keyword illustration hierarchical 42.10763 0.444 0.5188 0.4728 keyword pag...

  43. [51]

    This result aligns with the expectation that hybrid search benefits from leveraging both lexical matches and semantic similarities

    Discussion 7.1 Vector and Hybrid Search Methods: What Worked and Why? The hybrid search method generally outperformed vector search when paired with the hierarchical database. This result aligns with the expectation that hybrid search benefits from leveraging both lexical matc...

  44. [53]

    One key limitation is the dependence on the accuracy of the generated descriptions

    Limitations of the Study While the methodology of using LLM-generated descriptions in combination with a vector database has produced encouraging results, several limitations affect the precision and reliability of the assigned Iconclass codes. One key limitation is the depend...

  45. [54]

    The combination of RAG, vector search, and the basic Iconclass database works exceptionally well

    Conclusion One of the main takeaways from this project is the importance of adapting techniques to the nature of the dataset. The combination of RAG, vector search, and the basic Iconclass database works exceptionally well. The success of the full-page descriptions has demonst...

  46. [55]

    28 well-defined images or emphasizing keywords in cases where textual cues are stronger might enhance performance

    Such pages were excluded from the test set. 28 well-defined images or emphasizing keywords in cases where textual cues are stronger might enhance performance. Another area for improvement would be optimizing RAG retrieval strategies. Currently, only the top five results were r...

  47. [73]

    Passion of Christ 73D2 the episode of the Last Supper 73D23 Christ washes the feet of the apostles (John 13:1-20) 73D231 Christ washes Peter's feet Fig

    New Testament 73D. Passion of Christ 73D2 the episode of the Last Supper 73D23 Christ washes the feet of the apostles (John 13:1-20) 73D231 Christ washes Peter's feet Fig. 1: Iconclass Example Recent advances in multi-modal Large Language Models (LLMs), combined with improveme...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.