REVIEW 4 major objections 4 minor 1 cited by
ProtoSnap: Prototype Alignment for Cuneiform Signs
T0 review · 4 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read ProtoSnap claims that an unsupervised alignment of skeleton templates to photographed cuneiform signs recovers stroke-level structure, and that skeleton-conditioned synthetic data from these alignments boosts rare-sign recognition.
desk verdict The alignment task is new and well-executed; the rare-sign OCR claim needs a controlled comparison before it can be believed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the 4D similarity volume $S$, whose entries are cosine similarities between DIFT feature vectors at every pair of regions in the prototype font image and the target photograph. DIFT (diffusion features) are intermediate activations of the denoising U-Net of a Stable Diffusion model fine-tuned on cuneiform scans, denoted SD-, and the volume is the single shared representation that drives both alignment stages. Best-buddy pairs, defined as mutual nearest neighbors in $S$, are fitted with RANSAC to obtain a global affine transform $G$; a per-stroke projective transform $P^{(i)}$ applied on top of $G$ is then optimized by gradient descent using featural similarity from $S$, a saliency map also derived from $S$, and a regularizer that limits deviation from identity and keeps keypoints in the image. The prototype skeleton, a graph of four-keypoint wedges (three head corners and a tail), is the structural prior that the optimization snaps onto the photograph.
What would settle it
A controlled test would render a set of cuneiform-like signs from a known skeleton, apply realistic clay texture, illumination, and damage, then run ProtoSnap and measure keypoint F1 at a fixed threshold against the known ground truth; if the method does not beat the DIFT+RANSAC baseline at the 20-pixel threshold on that controlled set, the central alignment claim is contradicted. For the OCR claim, one can compare rare-sign accuracy after augmenting with images generated from randomly perturbed skeletons versus from ProtoSnap-aligned skeletons while keeping the number of synthetic samples identical; similar accuracy would show the alignment stage, not the skeleton conditioning, is doing the work.
Extended reading notes
Core claim
ProtoSnap's central claim is that a 4D similarity volume $S$ of pairwise cosine similarities between diffusion-feature vectors of a clean prototype font image and a photographed cuneiform sign contains enough information to align a skeleton to the target. Best-buddy pairs (mutual nearest neighbors in $S$) are used with RANSAC to fit a global affine transform, and a local refinement stage optimizes per-stroke projective transforms $P^{(i)}G$ under a loss combining featural similarity read from $S$, a saliency map derived from $S$, and an $L^1$ plus out-of-bounds regularizer. On the annotated benchmark the full pipeline reaches 27.14% F1 at a 20-pixel keypoint threshold, ahead of SIFT, DINOv2, and DIFT baselines, and the refinement stage contributes a clear gain over global alignment alone. For OCR, augmenting the CSDD training set with ControlNet-generated images conditioned on ProtoSnap skeletons raises overall accuracy to 64.14% and rare-sign balanced accuracy to 39.98%, exceeding augmentation with unconditional diffusion-generated images. The paper's conclusion is that this unsupervised prototype alignment is both a paleographic analysis tool and a route to structurally diverse synthetic data.
Load-bearing premise
Everything depends on the 4D similarity volume computed from diffusion features of a cuneiform-fine-tuned generative model giving trustworthy semantic correspondences between clean font prototypes and noisy, textured, often damaged photographs, and the paper further assumes a structurally matching prototype exists because its test set excludes non-matching variants.
Editorial extensions
If this is right
- If the alignment claim holds on uncurated tablets, stroke-level annotations and hand-copy-style overlays can be produced automatically from photographs, reducing manual paleographic work.
- Skeleton-conditioned synthetic data can populate rare sign classes with structurally correct examples, the regime where text-conditioned generation demonstrably fails.
- The expert-annotated benchmark gives future structure-recovery methods a fixed keypoint-threshold target, making the alignment claim quantitatively comparable.
- Because the ControlNet generator accepts any input skeleton, synthetic training sets can include sign variants that are absent or underrepresented in real data.
- The method's dependence on a matching prototype means its practical reach is tied to the coverage of existing cuneiform font inventories.
Reading between the lines
- Not claimed in the paper: the similarity-volume-plus-best-buddies recipe may transfer to other scripts with wedge- or stroke-like primitives, such as oracle bone inscriptions, since the method's core does not use cuneiform-specific geometry.
- Not claimed in the paper: the survey result that Assyriologists prefer aligned overlays suggests a testable extension measuring whether ProtoSnap overlays speed up expert transcription or reduce inter-annotator disagreement, not just preference.
- Not claimed in the paper: because the OCR gain is concentrated on rare signs, an active data-generation loop that requests ProtoSnap alignments for the rarest sign types first could yield further gains; the paper does not test such a curriculum.
- Not claimed in the paper: adding a confidence score or variant-detection step, which the paper lists as future work, would remove the requirement of knowing in advance that a structurally matching prototype exists.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces ProtoSnap, an unsupervised pipeline that aligns a skeletonized font prototype of a cuneiform sign to a photographed sign. It computes a 4D similarity volume from DIFT features of a cuneiform-fine-tuned Stable Diffusion model, extracts best-buddy correspondences to fit a global affine transformation with RANSAC, and then refines per-stroke projective transformations by optimizing a weighted loss combining feature similarity, saliency, and regularization. The method is evaluated on a newly collected expert-annotated benchmark of 272 sign images from 25 sign types, against SIFT, DINOv2, and DIFT baselines. In a downstream OCR experiment, the authors use ProtoSnap alignments to train a ControlNet model (CN-) and generate synthetic training images that, when added to CSDD, improve classification accuracy and balanced accuracy over the CSDD baseline and over synthetic data from the text-conditioned fine-tuned diffusion model (SD-), especially on rare signs.
Significance. The main contributions are a new task formulation for stroke-level cuneiform alignment, a benchmark with expert annotations, and a demonstration that structure-conditioned synthetic data can help cuneiform OCR. The release of code, data, and trained models is a strong asset, and the alignment comparisons in Table 1 include reasonable generic baselines. If the claimed OCR gains are robust, the work would be practically useful for Assyriology and for low-resource script recognition. The central alignment idea is well-specified and does not appear circular: the benchmark is anchored to external expert annotations, and the test sign types were not seen during SD- fine-tuning. However, the downstream claim needs a cleaner controlled comparison before it can be accepted, and the alignment benchmark's curation limits the strength of the 'wide variety' claim.
major comments (4)
- [Section 5.2 / Table 2 / Appendix A.7] The comparison between +CN- and +SD- is not controlled. The two conditions differ in the base generative model's prompt (CN- uses a generic 'cuneiform single ancient icon' prompt, while SD- uses per-sign codes), in the number of sign types covered (124 vs. 180), and in the amount of training data for the synthetic generator. Since no multiple seeds or error bars are reported, the rare-sign balanced-accuracy improvement from 31.13% to 39.98% cannot be attributed to ProtoSnap's aligned structures rather than to the general effect of skeleton-conditioned generation, to the base-model change, or to different sign coverage. A canonical-skeleton control (e.g., a ControlNet trained on untransformed or randomly transformed prototypes, with identical sign coverage, prompt, and training data) and repeated runs are needed to support the abstract's claim of 'significantly boosting' rare signs.
- [Section 5.1 / Appendix A.5] The alignment test set is curated by excluding images that are poor quality or that show sign variants differing from the prototype, and Section 5.3 admits that the method fails under extreme deformation, variant mismatch, and low image quality. This means the F1 numbers in Table 1 are optimistic relative to a deployment setting on uncurated scans. The authors should report the proportion of excluded images, or provide performance on a random subset of the eBL test split that does not require a matching prototype, and quantify failure rates on the excluded cases. Per-sign results in Table 4 already show a wide spread (F1@20 from 12.50% for IGI to 39.45% for ZE2), which makes this concern concrete.
- [Section 4.3 / Table 5] The ablation appears to contradict the design rationale for Lsim. Removing Lsim improves F1 at all reported thresholds (from 27.14% to 27.37% at F1@20, 42.09% to 42.61% at F1@30, and 52.43% to 53.19% at F1@40), while the text states that the removal 'slightly improves metrics' but is a 'qualitative trade-off' with no quantifying evidence. Since feature similarity is one of the two core objectives of the local refinement stage, this discrepancy needs to be resolved with a quantitative analysis (e.g., a perceptual/qualitative scoring protocol or a finer breakdown) rather than an unexplained trade-off.
- [General experimental reporting / Tables 1-5] Tables 1, 2, 3, and 5 report single-run numbers. The pipeline is stochastic in several places: DIFT features are averaged over random noises, RANSAC is randomized, and local refinement samples skeleton points randomly. Without error bars or at least multiple seeds, the reader cannot assess whether the reported margins over baselines (e.g., 27.14% vs. 21.31% in Table 1) are stable. This is especially important for Table 2, where the OCR result is the headline downstream claim.
minor comments (4)
- [Notation throughout] The symbols 'SD-' and 'CN-' are used without a gloss; please define them at first use (e.g., 'fine-tuned Stable Diffusion' and 'ControlNet fine-tuned with ProtoSnap alignments') and avoid notation that reads like a minus sign.
- [Appendix A.7] The sentence 'we have generated 50 samples per each sign in the test set (180 signs in total)' is ambiguous: it is unclear which test set is meant and whether the generated data include sign types that appear in the OCR test set. Please clarify to rule out any accidental leakage of test-set information into training.
- [Appendix C / Section 5.1] Please clarify the annotation protocol for the ground-truth test set: Section C.1 describes MTurk for font annotations, but the GT alignments on real signs in Section 5.1 are said to be by expert archaeologists; no inter-annotator agreement is reported. A brief description and agreement measure would strengthen the benchmark.
- [Typos and figure formatting] There are several typos ('experienced assyriologists', 'asked experiences assyriologists') and placeholder-like strings in figure captions (e.g., 'xPrototype', 'xxxInput'). Please proofread and clean the final version.
Circularity Check
No circular derivation: ProtoSnap's alignment and OCR claims rest on external expert keypoint annotations and real CSDD test data, not on the method's own fitted inputs.
full rationale
ProtoSnap's derivation chain is self-contained and externally anchored. The alignment output is obtained by optimizing prototype-target DIFT feature similarity and a saliency map derived from the same similarity volume, plus regularization; none of these objectives are defined in terms of the expert keypoint annotations used for evaluation. The 272-image alignment test set covers 25 sign types explicitly not seen during SD- fine-tuning, so the reported F1 scores measure generalization rather than reconstruction of training data. The OCR experiment trains ControlNet (CN-) on ProtoSnap alignments from eBL images, but the downstream classifier is evaluated on real CSDD photographs with labels from Dencker et al.; the synthetic images are new samples, not the test images, so the reported accuracy gain is not a fitted quantity renamed as a prediction. The comparison between +SD- and +CN- may have confounds (different base-model prompts, different numbers of generated sign types, and no reported error bars), but those are experimental-control concerns, not circularity. The only overlapping-author citations (Hamplová et al. 2024 in related work; Cobanoglu et al. 2024 as the eBL data source) are background or data attribution and do not carry the paper's central claims. No equation or loss in the paper is defined in terms of the benchmark target, and no result is imported from a same-author uniqueness theorem. Therefore no circular step is present.
Assumptions & free parameters
free parameters (5)
- Loss weights for local refinement =
lambda_sim=1.0, lambda_sal=3e-4, lambda_reg=1e-4
- DIFT noise timestep =
t=261
- RANSAC hyperparameters =
2000 iterations, 8 repeats, 50 px inlier threshold, 5 correspondences per fit
- Local refinement optimization schedule =
lr=0.01, 100 iterations
- Softmax temperature for similarity and saliency sampling =
temperature=100
assumptions (5)
- domain assumption Every cuneiform stroke can be represented by four keypoints, three corners of the triangular head plus one tail point, and signs can be treated as graphs of such strokes.
- domain assumption For each target sign, a prototype font image with a matching structural variant is available, and deviations are modest enough to be captured by affine plus per-stroke projective transforms.
- domain assumption Intermediate activations of the fine-tuned Stable Diffusion model provide reliable local similarity between font prototypes and photographed cuneiform signs after fine-tuning on cuneiform scans.
- domain assumption Stroke-specific deviations from the global alignment can be modeled by independent projective transformations initialized at identity.
- domain assumption Synthetic images generated by ControlNet conditioned on ProtoSnap skeletons are distributionally close enough to real CSDD scans to improve classifier training.
Cite this review
Pith. "Pith review of ProtoSnap: Prototype Alignment for Cuneiform Signs." pith.science (2026). https://pith.science/paper/2JBIOHDY
@misc{pith2026250200129,
author = {Pith},
title = {Pith review of: ProtoSnap: Prototype Alignment for Cuneiform Signs},
year = {2026},
howpublished = {\url{https://pith.science/paper/2JBIOHDY}},
note = {Machine review of arXiv:2502.00129}
}
read the original abstract
The cuneiform writing system served as the medium for transmitting knowledge in the ancient Near East for a period of over three thousand years. Cuneiform signs have a complex internal structure which is the subject of expert paleographic analysis, as variations in sign shapes bear witness to historical developments and transmission of writing and culture over time. However, prior automated techniques mostly treat sign types as categorical and do not explicitly model their highly varied internal configurations. In this work, we present an unsupervised approach for recovering the fine-grained internal configuration of cuneiform signs by leveraging powerful generative models and the appearance and structure of prototype font images as priors. Our approach, ProtoSnap, enforces structural consistency on matches found with deep image features to estimate the diverse configurations of cuneiform characters, snapping a skeleton-based template to photographed cuneiform signs. We provide a new benchmark of expert annotations and evaluate our method on this task. Our evaluation shows that our approach succeeds in aligning prototype skeletons to a wide variety of cuneiform signs. Moreover, we show that conditioning on structures produced by our method allows for generating synthetic data with correct structural configurations, significantly boosting the performance of cuneiform sign recognition beyond existing techniques, in particular over rare signs. Our code, data, and trained models are available at the project page: https://tau-vailab.github.io/ProtoSnap/
Figures
Figures from the paper (8 more)
Forward citations
Cited by 1 Pith paper
-
Signs of the Past, Patterns of the Present: On the Automatic Classification of Old Babylonian Cuneiform Signs
A ResNet50 classifier reaches 87.1% top-1 accuracy on Old Babylonian cuneiform signs, with lighting visualization and training-set diversity as the main drivers of performance.
Reference graph
Works this paper leans on
-
[4]
Obc306: A large- scale oracle bone character recognition dataset
Shuangping Huang, Haobin Wang, Yongge Liu, Xiaosong Shi, and Lianwen Jin. Obc306: A large- scale oracle bone character recognition dataset. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 681–688. IEEE,
work page 2019
-
[7]
Robust template matching via hierarchical convolutional features from a shape biased cnn
Bo Gao and Michael W Spratling. Robust template matching via hierarchical convolutional features from a shape biased cnn. In The International Conference on Image, Vision and Intelligent Systems (ICIVIS 2021), pages 333–344. Springer,
work page 2021
-
[8]
The dataset represents the Akkadian and Sumerian languages used at those eras
The full dataset comprises around 40% from the Neo-Babylonian period (1000–600 BC), around 20% from the Neo-Assyrian (1000-609 BCE), and less than 10% from the following periods: Ur III (2100–2002 BCE), Old Babylonian (2002-1595 BCE), Old Assyrian (1950–1850 BCE), Middle Babylonian (1500–1000 BCE), Late Babylonian (600 BC–100 AD), Persian (539-331 BCE), H...
work page 2020
-
[9]
Vehicle pose estimation via regression of semantic points of interest
Javier García López, Antonio Agudo, and Francesc Moreno-Noguer. Vehicle pose estimation via regression of semantic points of interest. In 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA), pages 209–214. IEEE,
work page 2019
-
[10]
Pose for everything: Towards category-agnostic pose estimation
12 Published as a conference paper at ICLR 2025 Lumin Xu, Sheng Jin, Wang Zeng, Wentao Liu, Chen Qian, Wanli Ouyang, Ping Luo, and Xiao- gang Wang. Pose for everything: Towards category-agnostic pose estimation. In European conference on computer vision, pages 398–416. Springer,
work page 2025
-
[11]
Pose anything: A graph-based approach for category-agnostic pose estimation
Or Hirschorn and Shai Avidan. Pose anything: A graph-based approach for category-agnostic pose estimation. arXiv preprint arXiv:2311.17891,
-
[13]
URL https://doi.org/10.1515/itit-2024-0028
doi: doi:10.1515/itit-2024-0028. URL https://doi.org/10.1515/itit-2024-0028. David G Lowe. Object recognition from local scale-invariant features. In Proceedings of the seventh IEEE international conference on computer vision, volume 2, pages 1150–1157. Ieee,
-
[16]
doi: 10.1109/ICFHR2020.2020.00019. APPENDIX A E XPERIMENTAL DETAILS Below we provide additional experimental details. Our code is also provided (zipped in the supple- mentary material). For all experiments described below, a single A5000 GPU was used. Running the method on a single image takes about 1 minute. A.1 I MAGE AND FONT INFORMATION For all of our...
Show all 22 references
-
[20]
Those results show that our method is robust and can be generalizable to other usages of the cuneiform writing system
, which contains signs from the Hittite language, as opposed to the Akkadian and Sumerian from which the training and test set are composed. Those results show that our method is robust and can be generalizable to other usages of the cuneiform writing system. 16 Published as a...
2025
-
[21]
We can see that our method produces align- ments which are much closer to expert annotations and is generally less sensitive to outliers
shown for comparison. We can see that our method produces align- ments which are much closer to expert annotations and is generally less sensitive to outliers. 17 Published as a conference paper at ICLR 2025 xxPrototype xxxxInput xxProtoSnap Figure 11: ProtoSnap applied on ima...
2025
-
[22]
cuneiform
16.18% 16.10% 16.14% 34.32% 33.27% 33.79% DIFT + RANSAC 13.15% 13.11% 13.13% 30.43% 29.88% 30.15% Ours (w/o refinement) 21.38% 21.23% 21.31% 50.55% 49.73% 50.13% Ours (full) 27.17% 27.10% 27.14% 52.76% 52.10% 52.43% Table 3: Precision and recall metrics for the alignment evalu...
2025
-
[100]
A.5 D ATASET DETAILS Both the training and the test datasets are taken from the eBL classification dataset Cobanoglu et al
To perform optimization, we apply gradient descent for 100 iterations with learning rate 0.01 and Adam optimizer, updating the the parameters of the local transformations of all strokes. A.5 D ATASET DETAILS Both the training and the test datasets are taken from the eBL classi...
2024
-
[1920]
At each iteration, 5 corre- spondences are used to fit a least-squares affine transformation, with a distance threshold of 50 pixels used to identify outliers
A.3 G LOBAL ALIGNMENT DETAILS To fit our global alignment, we apply RANSAC with 2000 iterations. At each iteration, 5 corre- spondences are used to fit a least-squares affine transformation, with a distance threshold of 50 pixels used to identify outliers. The transformation w...
2014
-
[1999]
Dinov2: Learning robust visual features without supervision,
13 Published as a conference paper at ICLR 2025 Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Ya...
2025
-
[2011]
URL https://doi.org/10.1093/oxfordhb/9780199557301.001.0001
doi: 10.1093/oxfordhb/9780199557301.001.0001. URL https://doi.org/10.1093/oxfordhb/9780199557301.001.0001. Michael P. Streck, editor. Sprachen des Alten Orients. wbg Academic, Darmstadt, 4., überarbeitete und aktualisierte auflage edition,
-
[2016]
Fink, and Gerfrid G.W
Eugen Rusakov, Turna Somel, Gernot A. Fink, and Gerfrid G.W. Müller. Towards query-by- expression retrieval of cuneiform signs. In 2020 17th International Conference on Frontiers in Handwriting Recognition (ICFHR), pages 43–48,
2020
-
[2018]
Recurrent temporal revision graph networks
11 Published as a conference paper at ICLR 2025 Yizhou Chen, Anxiang Zeng, Qingtao Yu, Kerui Zhang, Cao Yuanpeng, Kangle Wu, Guangda Huzhang, Han Yu, and Zhiming Zhou. Recurrent temporal revision graph networks. Advances in Neural Information Processing Systems, 36,
2025
-
[2019]
Decipherment of historical manuscript images
Xusen Yin, Nada Aldarrab, Beáta Megyesi, and Kevin Knight. Decipherment of historical manuscript images. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 78–85. IEEE,
2019
-
[2020]
Cnn based cuneiform sign detection learned from annotated 3d renderings and mapped photographs with illumination augmentation
Ernst Stötzner, Timo Homburg, and Hubert Mara. Cnn based cuneiform sign detection learned from annotated 3d renderings and mapped photographs with illumination augmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1680–1688, 2023a. Tal...
-
[2022]
Carfusion: Combining point tracking and part detection for dynamic 3d reconstruction of vehicles
N Dinesh Reddy, Minh V o, and Srinivasa G Narasimhan. Carfusion: Combining point tracking and part detection for dynamic 3d reconstruction of vehicles. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1906–1915,
1906
-
[2023]
Ocr-free transcript alignment
Tal Hassner, Lior Wolf, and Nachum Dershowitz. Ocr-free transcript alignment. In 2013 12th International Conference on Document Analysis and Recognition, pages 1310–1314. IEEE, 2013b. Tal Hassner, Lior Wolf, Nachum Dershowitz, Gil Sadeh, and Daniel Stökl Ben-Ezra. Dense cor- r...
2013
-
[2024]
Automating transliteration of cuneiform from parallel lines with sparse data
Bartosz Bogacz, Maximilian Klingmann, and Hubert Mara. Automating transliteration of cuneiform from parallel lines with sparse data. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), volume 1, pages 615–620. IEEE,
2017
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.