{"id":"d6c4cbb9-92e5-4247-8767-3c4555e69ca9","arxiv_id":"2502.00129","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"ProtoSnap recovers the internal stroke structure of photographed cuneiform signs by snapping font-derived skeleton prototypes onto target images, and uses the resulting alignments to generate synthetic data that improves cuneiform OCR, especially for rare signs.","lead":"This paper introduces ProtoSnap, an unsupervised method that aligns a skeleton template of a cuneiform sign to a photographed sign by matching deep diffusion features and refining each stroke's position. It also shows that synthetic training data generated from these aligned structures improves cuneiform sign recognition, particularly for rare signs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The rare-sign OCR gain in Table 2 is not yet isolated from the CN- base model, generated sign-type coverage, and single-seed variance; a canonical-skeleton control is needed before the abstract's 'significantly boosting rare signs' claim is supported.","rationale":"ProtoSnap is a technically coherent and genuinely useful contribution: it defines a new task, builds an expert-annotated benchmark, proposes a multi-stage alignment procedure with a sensible DIFT-based similarity volume, best-buddy global alignment, and per-stroke projective refinement, and it reports ablations plus a user study. The released code, data, and trained models are real assets, and the alignment results are clearly better than the generic correspondence baselines. My stress-test focuses on the second half of the central claim, the OCR boost. Table 2 is the only quantitative evidence for 'significantly boosting ... rare signs,' and as detailed in the attack, the +CN- condition differs from +SD- in base model prompt, number of generated sign types, and seed/resampling behavior, with no variance reported. A control using canonical font skeletons without ProtoSnap alignment would isolate whether the improvement comes from ProtoSnap's alignment to real scans or merely from skeleton-conditioned synthesis. The benchmark curation in Section 5.1, which excludes non-matching variants, is a second limitation but it is explicitly acknowledged in Section 5.3 and does not by itself invalidate the alignment contribution. Because the identified issue is missing evidence rather than a demonstrated error, the appropriate response is to keep the reader's CONDITIONAL verdict and require the proposed control before treating the rare-sign OCR claim as established.","tokens_in":16360,"tokens_out":12882,"duration_ms":142219,"concrete_test":"Run the OCR pipeline (Appendix A.7) with at least 10 seeds under three conditions: (1) +SD- exactly as reported; (2) +CN- exactly as reported; (3) +CN-canonical, where ControlNet is trained on pairs of canonical font skeleton and rasterized font image with no ProtoSnap alignment, then used to generate the same 124 sign types with the same random skeleton augmentations. Report per-sign-type rare accuracy, mean and standard deviation, and a paired bootstrap test comparing conditions (2) and (3) and (2) versus (1). If +CN-canonical matches or exceeds +CN-, or if +CN- does not beat +SD- with p<0.05, the claim that ProtoSnap alignments drive the rare-sign OCR boost is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central downstream claim is that conditioning on ProtoSnap structures produces synthetic data with correct structural configurations, 'significantly boosting' cuneiform sign recognition, especially for rare signs. That claim rests on Table 2 and Appendix A.7, but the +CN- condition is not a controlled comparison with +SD-. First, the CN- base Stable Diffusion model was fine-tuned with a generic prompt ('cuneiform single ancient icon') while SD- used per-sign codes, so the two conditions differ in base model behavior as well as in conditioning modality. Second, CN- generated data for 124 sign types while SD- generated for 180 sign types, so the rare-sign subset covered by each augmentation may differ. Third, no error bars, seeds, or significance tests are reported; the rare balanced-accuracy jump from 31.13% to 39.98% is a single-run difference and could be a training-run artifact or an artifact of which sign types had available prototypes. Because the paper's headline contribution includes the OCR boost, this missing control is load-bearing: without it, the improvement cannot be attributed specifically to ProtoSnap's aligned structures rather than to the general benefit of skeleton-conditioned synthesis or to experimental setup differences.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces ProtoSnap, an unsupervised pipeline that aligns a skeletonized font prototype of a cuneiform sign to a photographed sign. It computes a 4D similarity volume from DIFT features of a cuneiform-fine-tuned Stable Diffusion model, extracts best-buddy correspondences to fit a global affine transformation with RANSAC, and then refines per-stroke projective transformations by optimizing a weighted loss combining feature similarity, saliency, and regularization. The method is evaluated on a newly collected expert-annotated benchmark of 272 sign images from 25 sign types, against SIFT, DINOv2, and DIFT baselines. In a downstream OCR experiment, the authors use ProtoSnap alignments to train a ControlNet model (CN-) and generate synthetic training images that, when added to CSDD, improve classification accuracy and balanced accuracy over the CSDD baseline and over synthetic data from the text-conditioned fine-tuned diffusion model (SD-), especially on rare signs.","tokens_in":16574,"tokens_out":6079,"duration_ms":57621,"significance":"The main contributions are a new task formulation for stroke-level cuneiform alignment, a benchmark with expert annotations, and a demonstration that structure-conditioned synthetic data can help cuneiform OCR. The release of code, data, and trained models is a strong asset, and the alignment comparisons in Table 1 include reasonable generic baselines. If the claimed OCR gains are robust, the work would be practically useful for Assyriology and for low-resource script recognition. The central alignment idea is well-specified and does not appear circular: the benchmark is anchored to external expert annotations, and the test sign types were not seen during SD- fine-tuning. However, the downstream claim needs a cleaner controlled comparison before it can be accepted, and the alignment benchmark's curation limits the strength of the 'wide variety' claim.","major_comments":[{"comment":"The comparison between +CN- and +SD- is not controlled. The two conditions differ in the base generative model's prompt (CN- uses a generic 'cuneiform single ancient icon' prompt, while SD- uses per-sign codes), in the number of sign types covered (124 vs. 180), and in the amount of training data for the synthetic generator. Since no multiple seeds or error bars are reported, the rare-sign balanced-accuracy improvement from 31.13% to 39.98% cannot be attributed to ProtoSnap's aligned structures rather than to the general effect of skeleton-conditioned generation, to the base-model change, or to different sign coverage. A canonical-skeleton control (e.g., a ControlNet trained on untransformed or randomly transformed prototypes, with identical sign coverage, prompt, and training data) and repeated runs are needed to support the abstract's claim of 'significantly boosting' rare signs.","section":"Section 5.2 / Table 2 / Appendix A.7"},{"comment":"The alignment test set is curated by excluding images that are poor quality or that show sign variants differing from the prototype, and Section 5.3 admits that the method fails under extreme deformation, variant mismatch, and low image quality. This means the F1 numbers in Table 1 are optimistic relative to a deployment setting on uncurated scans. The authors should report the proportion of excluded images, or provide performance on a random subset of the eBL test split that does not require a matching prototype, and quantify failure rates on the excluded cases. Per-sign results in Table 4 already show a wide spread (F1@20 from 12.50% for IGI to 39.45% for ZE2), which makes this concern concrete.","section":"Section 5.1 / Appendix A.5"},{"comment":"The ablation appears to contradict the design rationale for Lsim. Removing Lsim improves F1 at all reported thresholds (from 27.14% to 27.37% at F1@20, 42.09% to 42.61% at F1@30, and 52.43% to 53.19% at F1@40), while the text states that the removal 'slightly improves metrics' but is a 'qualitative trade-off' with no quantifying evidence. Since feature similarity is one of the two core objectives of the local refinement stage, this discrepancy needs to be resolved with a quantitative analysis (e.g., a perceptual/qualitative scoring protocol or a finer breakdown) rather than an unexplained trade-off.","section":"Section 4.3 / Table 5"},{"comment":"Tables 1, 2, 3, and 5 report single-run numbers. The pipeline is stochastic in several places: DIFT features are averaged over random noises, RANSAC is randomized, and local refinement samples skeleton points randomly. Without error bars or at least multiple seeds, the reader cannot assess whether the reported margins over baselines (e.g., 27.14% vs. 21.31% in Table 1) are stable. This is especially important for Table 2, where the OCR result is the headline downstream claim.","section":"General experimental reporting / Tables 1-5"}],"minor_comments":[{"comment":"The symbols 'SD-' and 'CN-' are used without a gloss; please define them at first use (e.g., 'fine-tuned Stable Diffusion' and 'ControlNet fine-tuned with ProtoSnap alignments') and avoid notation that reads like a minus sign.","section":"Notation throughout"},{"comment":"The sentence 'we have generated 50 samples per each sign in the test set (180 signs in total)' is ambiguous: it is unclear which test set is meant and whether the generated data include sign types that appear in the OCR test set. Please clarify to rule out any accidental leakage of test-set information into training.","section":"Appendix A.7"},{"comment":"Please clarify the annotation protocol for the ground-truth test set: Section C.1 describes MTurk for font annotations, but the GT alignments on real signs in Section 5.1 are said to be by expert archaeologists; no inter-annotator agreement is reported. A brief description and agreement measure would strengthen the benchmark.","section":"Appendix C / Section 5.1"},{"comment":"There are several typos ('experienced assyriologists', 'asked experiences assyriologists') and placeholder-like strings in figure captions (e.g., 'xPrototype', 'xxxInput'). Please proofread and clean the final version.","section":"Typos and figure formatting"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a computer vision venue and the expert benchmark is a valuable asset. The main risk is that the headline OCR gain is not yet isolated from experimental confounds; this is fixable with additional controlled experiments and variance reporting. I would not reject on the current evidence, but the acceptance decision should hinge on the revised OCR comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know this paper is worth reading for the alignment task alone. ProtoSnap introduces an unsupervised method to snap skeleton templates of cuneiform font prototypes onto 2D photographs of real signs, and the authors provide a new expert-annotated benchmark for it. The method is well-specified: DIFT features from a cuneiform-fine-tuned diffusion model, best-buddies correspondences, a RANSAC affine global fit, then per-stroke projective refinement with a combined similarity/saliency/regularization loss. The ablation study shows each component earns its place, and the comparison against generic correspondence baselines (SIFT, DINOv2, DIFT) is fair and clearly won. That part of the paper is solid, and the new task is genuinely new: prior work treated signs categorically or at bounding-box level, not stroke-level alignment from photographs.\n\nThe soft spots are real but concentrated. First, the alignment test set is curated: low-quality or variant-mismatched signs are excluded, so the reported F1 values are optimistic for real-world scans. The authors acknowledge this, but it does mean the headline numbers should not be taken at face value. Second, the OCR claim in the abstract—that ProtoSnap's structures 'significantly boost' recognition, especially for rare signs—is not supported by the comparison in Table 2. The +CN- condition differs from +SD- in two ways at once: the base diffusion model was fine-tuned with a generic prompt instead of per-sign codes, and CN- generated data for 124 sign types versus 180 for SD-. There are also no error bars or seeds. So you cannot attribute the rare-sign balanced-accuracy jump from 31.13% to 39.98% specifically to structural conditioning. The stress-test note is correct: a controlled comparison with matched sign-type coverage and multiple seeds is needed before that claim stands.\n\nThere is a minor oddity in the ablation where removing Lsim slightly improves F1; the authors call it a qualitative trade-off, which is fine, but it deserves a sentence of explanation. Also, the reproduced CSDD baseline differs slightly from the published number; they note it is due to broken URLs, which is plausible.\n\nWho this is for: researchers in digital Assyriology and document analysis will get clear value from the benchmark, the method, and the released code. The paper is a solid applied contribution that deserves a serious referee even though the OCR overclaim needs fixing. If you are in that area, cite it for the alignment benchmark; for the OCR claim, wait for a controlled follow-up.","headline":"The alignment task is new and well-executed; the rare-sign OCR claim needs a controlled comparison before it can be believed.","tokens_in":17169,"tokens_out":3265,"would_cite":true,"duration_ms":29410,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ProtoSnap claims that an unsupervised alignment of skeleton templates to photographed cuneiform signs recovers stroke-level structure, and that skeleton-conditioned synthetic data from these alignments boosts rare-sign recognition.","keywords":["cuneiform sign recognition","unsupervised alignment","skeleton template","diffusion features","best-buddy correspondences","synthetic data generation","ControlNet","paleography"],"falsifier":"A controlled test would render a set of cuneiform-like signs from a known skeleton, apply realistic clay texture, illumination, and damage, then run ProtoSnap and measure keypoint F1 at a fixed threshold against the known ground truth; if the method does not beat the DIFT+RANSAC baseline at the 20-pixel threshold on that controlled set, the central alignment claim is contradicted. For the OCR claim, one can compare rare-sign accuracy after augmenting with images generated from randomly perturbed skeletons versus from ProtoSnap-aligned skeletons while keeping the number of synthetic samples identical; similar accuracy would show the alignment stage, not the skeleton conditioning, is doing the work.","tokens_in":16124,"feed_emoji":"🏺","tokens_out":10045,"duration_ms":91524,"temperature":0.7,"pith_summary":"This paper tries to establish that the internal stroke structure of photographed cuneiform signs can be recovered without any labelled real-sign supervision, by snapping a skeleton-based font prototype onto the photograph. The method builds a four-dimensional similarity volume from deep diffusion features of a generative model fine-tuned on cuneiform scans, extracts mutually nearest matching regions (best buddies) for a global affine fit, and then refines each stroke's position with a per-stroke projective transform. The paper also claims that images generated from these alignments, through a ControlNet conditioned on the resulting skeletons, improve cuneiform sign recognition beyond text-conditioned generation and prior baselines, especially for rare signs. To measure the alignment claim, the authors introduce an expert-annotated benchmark of 272 photographed signs across 25 sign types. If the claims hold, Assyriologists gain an automated stroke-level alignment and hand-copy aid, and OCR systems gain a way to synthesize structurally correct training data for rare variants.","feed_headline":"One pipeline recovers stroke structure of photographed cuneiform signs","feed_subtitle":"ProtoSnap matches clean font prototypes to tablet photos and uses those alignments to boost recognition of rare signs.","key_machinery":"The central object is the 4D similarity volume $S$, whose entries are cosine similarities between DIFT feature vectors at every pair of regions in the prototype font image and the target photograph. DIFT (diffusion features) are intermediate activations of the denoising U-Net of a Stable Diffusion model fine-tuned on cuneiform scans, denoted SD-, and the volume is the single shared representation that drives both alignment stages. Best-buddy pairs, defined as mutual nearest neighbors in $S$, are fitted with RANSAC to obtain a global affine transform $G$; a per-stroke projective transform $P^{(i)}$ applied on top of $G$ is then optimized by gradient descent using featural similarity from $S$, a saliency map also derived from $S$, and a regularizer that limits deviation from identity and keeps keypoints in the image. The prototype skeleton, a graph of four-keypoint wedges (three head corners and a tail), is the structural prior that the optimization snaps onto the photograph.","core_discovery":"ProtoSnap's central claim is that a 4D similarity volume $S$ of pairwise cosine similarities between diffusion-feature vectors of a clean prototype font image and a photographed cuneiform sign contains enough information to align a skeleton to the target. Best-buddy pairs (mutual nearest neighbors in $S$) are used with RANSAC to fit a global affine transform, and a local refinement stage optimizes per-stroke projective transforms $P^{(i)}G$ under a loss combining featural similarity read from $S$, a saliency map derived from $S$, and an $L^1$ plus out-of-bounds regularizer. On the annotated benchmark the full pipeline reaches 27.14% F1 at a 20-pixel keypoint threshold, ahead of SIFT, DINOv2, and DIFT baselines, and the refinement stage contributes a clear gain over global alignment alone. For OCR, augmenting the CSDD training set with ControlNet-generated images conditioned on ProtoSnap skeletons raises overall accuracy to 64.14% and rare-sign balanced accuracy to 39.98%, exceeding augmentation with unconditional diffusion-generated images. The paper's conclusion is that this unsupervised prototype alignment is both a paleographic analysis tool and a route to structurally diverse synthetic data.","pith_inferences":["Not claimed in the paper: the similarity-volume-plus-best-buddies recipe may transfer to other scripts with wedge- or stroke-like primitives, such as oracle bone inscriptions, since the method's core does not use cuneiform-specific geometry.","Not claimed in the paper: the survey result that Assyriologists prefer aligned overlays suggests a testable extension measuring whether ProtoSnap overlays speed up expert transcription or reduce inter-annotator disagreement, not just preference.","Not claimed in the paper: because the OCR gain is concentrated on rare signs, an active data-generation loop that requests ProtoSnap alignments for the rarest sign types first could yield further gains; the paper does not test such a curriculum.","Not claimed in the paper: adding a confidence score or variant-detection step, which the paper lists as future work, would remove the requirement of knowing in advance that a structurally matching prototype exists."],"forward_implications":["If the alignment claim holds on uncurated tablets, stroke-level annotations and hand-copy-style overlays can be produced automatically from photographs, reducing manual paleographic work.","Skeleton-conditioned synthetic data can populate rare sign classes with structurally correct examples, the regime where text-conditioned generation demonstrably fails.","The expert-annotated benchmark gives future structure-recovery methods a fixed keypoint-threshold target, making the alignment claim quantitatively comparable.","Because the ControlNet generator accepts any input skeleton, synthetic training sets can include sign variants that are absent or underrepresented in real data.","The method's dependence on a matching prototype means its practical reach is tied to the coverage of existing cuneiform font inventories."],"supporting_citations":[{"why":"Supplies the DIFT diffusion-feature method that forms the 4D similarity volume.","marker":"Tang et al. (2023)"},{"why":"Supplies the Stable Diffusion model that is fine-tuned into SD- and used as the vision backbone.","marker":"Rombach et al. (2022)"},{"why":"Introduces best-buddies mutual-nearest-neighbor matching used for the global alignment.","marker":"Oron et al. (2017)"},{"why":"Provides ControlNet, fine-tuned into CN- to generate skeleton-conditioned synthetic cuneiform images.","marker":"Zhang et al. (2023)"},{"why":"Provides the CSDD dataset and ResNet18 OCR baseline that the synthetic data augments.","marker":"Dencker et al. (2020)"},{"why":"Supplies the eBL photographed-sign dataset used to fine-tune SD- and to build the test set.","marker":"Cobanoglu et al. (2024)"},{"why":"Serves as the geometry-based SIFT+RANSAC baseline in the alignment evaluation.","marker":"Lowe (1999)"},{"why":"Serves as the DINOv2 deep-feature baseline in the alignment evaluation.","marker":"Oquab et al. (2024)"}],"fun_headline_variants":["ProtoSnap snaps cuneiform signs to font prototypes","Cuneiform sign alignment via prototype snapping","Synthetic data from ProtoSnap boosts rare cuneiform OCR","Unsupervised prototype alignment for cuneiform signs","ProtoSnap aligns photographed cuneiform with clean templates"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything depends on the 4D similarity volume computed from diffusion features of a cuneiform-fine-tuned generative model giving trustworthy semantic correspondences between clean font prototypes and noisy, textured, often damaged photographs, and the paper further assumes a structurally matching prototype exists because its test set excludes non-matching variants.","fun_headline_variants_meta":{"raw":{"variants":["ProtoSnap snaps cuneiform signs to font prototypes","Cuneiform sign alignment via prototype snapping","Synthetic data from ProtoSnap boosts rare cuneiform OCR","Unsupervised prototype alignment for cuneiform signs","ProtoSnap aligns photographed cuneiform with clean templates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000987,"raw_usage":{"total_tokens":4234,"prompt_tokens":1045,"completion_tokens":3189,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":661,"completion_tokens_details":{"reasoning_tokens":3110}},"tokens_in":661,"tokens_out":3189,"duration_ms":23129,"temperature":1.0,"reasoning_tokens":3110,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T20:05:14.017251+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled test would render a set of cuneiform-like signs from a known skeleton, apply realistic clay texture, illumination, and damage, then run ProtoSnap and measure keypoint F1 at a fixed threshold against the known ground truth; if the method does not beat the DIFT+RANSAC baseline at the 20-pixel threshold on that controlled set, the central alignment claim is contradicted. For the OCR claim, one can compare rare-sign accuracy after augmenting with images generated from randomly perturbed skeletons versus from ProtoSnap-aligned skeletons while keeping the number of synthetic samples identical; similar accuracy would show the alignment stage, not the skeleton conditioning, is doing the work.","supporting_citations":[],"review_version":1}