{"id":"abd2188a-c759-4b43-bd5e-5eec1165e97f","arxiv_id":"2501.11895","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A contrastive masked autoencoder achieves 89.7% precision on open-set character-level writer identification on CASIA-OLHWDB, and 81.6% rank-1 on IAM-OnDB.","lead":"This paper combines masked autoencoders with contrastive learning to identify handwriting authors from single character trajectories, reporting 89.7% precision on the CASIA online handwriting dataset and 81.6% rank-1 accuracy on IAM-OnDB. It is a candidate for improving forensic writer identification in open-set scenarios, though the central state-of-the-art claim is currently supported without direct baseline comparison on the primary dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CASIA SOTA claim rests on a bespoke open-set protocol with no baseline comparison; without prior methods run under the same protocol, 89.7% precision is unfalsifiable.","rationale":"The reader identified the absence of CASIA baselines as the weakest assumption. I agree this is the most load-bearing concern because the strongest claim is specifically about state-of-the-art performance on CASIA, and no comparison on that dataset appears anywhere in the paper. The paper does report a baseline comparison on IAM-OnDB (Table I), where CMAE outperforms Chen et al. by 5.4 points, which gives some credibility to the method; however, that does not validate the CASIA number because the protocol and metric are different. In addition, the paper contains internal inconsistencies that the reader also flagged: Eq. (5) omits the cross-entropy loss for the discriminator even though Section III.C lists three losses, and Section IV.A states patch size 52 with 160 patches for an 800-timepoint trajectory, which is arithmetically inconsistent (800/52 is not 160). These issues reduce confidence in the reproducibility of the exact numbers, but the missing CASIA baseline is the single factor that makes the headline claim unfalsifiable. A concrete test running a prior method under the same protocol would settle whether the SOTA claim holds. Because the required test is straightforward and the paper otherwise has a plausible architecture and some positive evidence on IAM, the appropriate verdict remains CONDITIONAL rather than ACCEPT or REJECT; my analysis does not change the reader's verdict.","tokens_in":10871,"tokens_out":4911,"duration_ms":52404,"concrete_test":"Reproduce the CASIA open-set protocol exactly: sample 20 writers and 2 characters per writer from the CASIA-OLHWDB test set, run a published baseline (e.g., Chen et al.'s multi-branch encoder, or at least a standard BiLSTM/ResNet embedding trained with the same pair-classification objective) over the same 100 random trials, and compare average precision. If the baseline reaches or exceeds 89.7%, the claimed state-of-the-art result is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that CMAE achieves state-of-the-art results on CASIA-OLHWDB, with 89.7% average precision. Section IV.B.1 reports this number from a protocol the authors themselves acknowledge 'differs from previous research' (they only follow Chen et al.'s approach on IAM-OnDB). On CASIA, the protocol randomly samples 20 writers and 2 characters per writer, then reports average accuracy and precision of the discriminator over 100 trials. No published method is re-run under this protocol, so there is no basis for the word 'state-of-the-art.' The metric (binary same/different precision) also differs from the rank-1 accuracy used in Table I, making cross-paper comparison apples-to-oranges. This is a falsifiability issue: if a baseline had been run and scored 90%, the 89.7% would be below SOTA. Without such a baseline, the headline number cannot be evaluated. A secondary internal inconsistency compounds this: the training objective is described as combining three losses (reconstruction, supervised contrastive, cross-entropy), but Eq. (5) only includes the first two; given that evaluation uses the discriminator's precision, it must be clear how the cross-entropy loss enters training.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Contrastive Masked Autoencoders (CMAE), a representation-learning framework for character-level open-set writer identification from online handwriting trajectories. The model combines a masked autoencoder (MAE) reconstruction branch with a supervised contrastive-learning branch, followed by a transformer-based discriminator that decides whether two character trajectories come from the same writer. Experiments are reported on CASIA-OLHWDB and IAM-OnDB. The authors claim state-of-the-art performance on CASIA with 89.7% average precision under an open-set protocol, and 81.6% rank-1 accuracy on IAM-OnDB under Chen et al.'s protocol, with ablations on mask ratio, representation-learning pretraining, contrastive learning, MAE, and encoder depth.","tokens_in":11264,"tokens_out":2353,"duration_ms":24969,"significance":"If the central claims hold, the paper would be a useful step toward open-set writer identification from single characters, a practically relevant scenario for forensics and document authentication. The idea of combining reconstruction-based and contrastive self-supervision for online trajectory data is reasonable, and the ablations in Tables II, III, and IV provide useful evidence about which components matter (contrastive learning and pretraining are shown to be critical). However, the headline claim of state-of-the-art on CASIA is not currently falsifiable, because the evaluation protocol is bespoke and no published baseline is rerun under it; the significance of the 89.7% figure therefore cannot be assessed. The paper also has an internal inconsistency in the stated training loss, which must be resolved before the method is fully specified.","major_comments":[{"comment":"The claim of state-of-the-art performance on CASIA (89.7% precision, Abstract and Table II) is not supported by the current evaluation. The protocol randomly selects 20 writers and 2 characters per writer and averages over 100 trials, which the authors acknowledge 'differs from previous research.' No prior method is evaluated under this same protocol, and the metric (binary same/different precision) differs from the rank-1 accuracy used in Table I. As a result, there is no reference point against which 89.7% can be judged as superior, and the claim is unfalsifiable. The authors should run at least one representative prior method (e.g., Chen et al. or a simple embedding baseline) under the identical CASIA protocol, or compare on a protocol already used in the literature.","section":"Section IV.B.1, Abstract, Table II"},{"comment":"The training objective is not fully specified. The text states that the model employs three losses (reconstruction, supervised contrastive, and cross-entropy for the discriminator), but Eq. (5) defines the total loss as ℓ = λℓRE + (1−λ)ℓCL, omitting the cross-entropy term. Since the reported evaluation metrics are the discriminator's accuracy and precision, it is essential to state explicitly how ℓCE enters the optimization: is it added with a third weighting coefficient, optimized only for the discriminator while the encoder is trained with Eq. (5), or included in a different training stage? As written, the objective is ambiguous and the role of the discriminator in training is unclear.","section":"Section III.C, Eq. (5), Section III.B.4"},{"comment":"The patch arithmetic appears inconsistent. The input trajectory is described as '800×2' and the patch size as 52, with 'n patches = 160.' If patching is applied along the time dimension, 800/52 is not an integer and does not equal 160; if the patch size is intended to be 5, the text should say so. This inconsistency affects the reproducibility of the architecture description and should be corrected with a precise definition of the patching operation (including whether patches are non-overlapping, and how the coordinate dimension is handled).","section":"Section III.B.1"}],"minor_comments":[{"comment":"The claim that the paper 'pioneers the use of single characters for writer-id tasks' should be softened, since prior work (Yang et al. 2015, Ref. [17]) already performed character-level writer identification in a closed-set setting; the novelty appears to be the open-set single-character setting, and the wording should say so precisely.","section":"Section I, Contributions"},{"comment":"The notation in the task definition is inconsistent: 'τi and τj' is used in prose, while the formal statement says 'the goal is to learn a function f : (τi and τj) → σ' and later uses τ_n and superscripts. Please define the trajectory notation cleanly and use it consistently throughout.","section":"Section III.A"},{"comment":"In Eq. (2), the reconstructed output is described as 'Pm' while the caption says the decoder reconstructs 'unmasked patches'; this is contradictory. The reconstruction target should be the masked patches, so either the equation or the text should be corrected.","section":"Section III.B.2, Eq. (2)"},{"comment":"The ablation 'CMAE w/o RLP' is presented as part of the mask-ratio ablation, but the difference between 'CMAE' and 'CMAE w/o RLP' is a pretraining stage, not a mask-ratio change. Consider separating the two factors or clarifying the experimental setup so that the effect of mask ratio is not confounded with pretraining.","section":"Section IV.B.2, Table II"},{"comment":"The discussion of trajectory-length effects would be clearer if the authors reported whether the distances are averaged over many writer pairs and how the standard deviation varies; the current text does not state the number of pairs used for each length bin.","section":"Section IV.B.4, Fig. 4"},{"comment":"Some references are incomplete or have formatting artifacts (e.g., Ref. [22] and [42] contain 'V ol-3160' instead of 'Vol-3160'; Ref. [13] is missing the conference name). Please check the reference list against the publisher's styles.","section":"References"},{"comment":"There are several typos and grammatical issues (e.g., 'taun' in Section III.B.1, 'Mask Auto-Encoders' in the Introduction, 'one a popular technique' in Section II.A). A careful proofreading pass is recommended.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The core methodological idea (MAE + contrastive learning for online trajectories) is plausible and the ablations are informative, but the paper's main empirical claim—state-of-the-art on CASIA—is currently not supported because the evaluation protocol is not comparable to any published baseline. This is fixable by adding baselines under the same protocol, but it is a load-bearing issue rather than a cosmetic one. The loss-function inconsistency in Eq. (5) also needs to be resolved. I would not recommend acceptance in the current form; the revision should be substantial enough to warrant a second review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is an incremental but legitimate combination of masked autoencoding and supervised contrastive learning, applied to character-level open-set writer identification. The IAM-OnDB result is the real contribution: it follows Chen et al.'s protocol and beats their rank-1 accuracy (81.6 vs. 76.2). That comparison is meaningful and reproducible in spirit. The ablations are also honest and informative—CL carries the model, while MAE adds only a few points, which is a useful finding for the community.\n\nThat said, the headline CASIA number does not support the SOTA claim. The evaluation protocol there—randomly sampling 20 writers and 2 characters per writer, then reporting discriminator accuracy/precision over 100 trials—is bespoke. No prior method is run under this protocol, so 89.7% precision is not comparable to any published result. If a baseline had been scored this way and got 90%, the claim collapses. The metric is also different from the rank-1 accuracy used in Table I, making the abstract's \"SOTA\" statement doubly unsupported. This is a load-bearing flaw in the paper's central narrative, not a cosmetic issue.\n\nThere are also smaller internal inconsistencies. Equation (5) gives the total loss as a weighted sum of reconstruction and contrastive losses, omitting the cross-entropy term that Section III.C says the discriminator uses. Since the discriminator is what gets evaluated, the training objective needs to be stated correctly. The patch arithmetic is also off: an 800×2 input with patch size 52 does not produce 160 patches (800/52 is not even an integer). Either the patch size, the input length, or the number of patches is misreported. Minor, but the kind of thing a careful reader trips on.\n\nThe phrase \"pioneers the use of single characters\" overstates the novelty—Yang et al. 2015 did character-level closed-set, and Chen et al. 2021 did letter-level open-set. The genuine novelty here is the specific open-set single-character setup, which is a modest but valid step.\n\nWho should read this: the document analysis and forensic handwriting crowd. The IAM result is worth knowing, and the ablation insight about CL vs. MAE is useful. But the paper needs major revision before it can be trusted as a SOTA claim. A referee should require CASIA baselines under the same protocol, a corrected loss equation, and fixed patch-size numbers. I would send it to review, but expect heavy revision.","headline":"A reasonable MAE+CL combination with a solid IAM benchmark, but the CASIA SOTA claim is unfalsifiable without baselines run under the same protocol.","tokens_in":11638,"tokens_out":2606,"would_cite":false,"duration_ms":27222,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A fused masked-autoencoder and contrastive-learning model can identify a writer from a single handwritten character when the writer was never seen in training, reaching 89.7% precision on CASIA-OLHWDB.","keywords":["contrastive masked autoencoders","open-set writer identification","character-level handwriting","online handwriting trajectories","representation learning","CASIA-OLHWDB","IAM-OnDB","self-supervised pretraining"],"falsifier":"Run the strongest published online writer-id baseline under the paper's exact CASIA open-set protocol (20 unseen writers, 2 characters per writer, 100 random trials) and compare precision; if that baseline matches or exceeds 89.7% within the reported spread, the central claim that CMAE is state of the art for single-character open-set identification fails.","tokens_in":10705,"feed_emoji":"✍️","tokens_out":6998,"duration_ms":66578,"temperature":0.7,"pith_summary":"The paper sets out to show that one handwritten character is enough to identify its writer even when that writer's samples never appear in training. It argues this by combining masked autoencoding, which learns the sequential structure of strokes, with contrastive learning, which separates one writer's style from another's, and then testing whether two characters came from the same person. On the CASIA online handwriting database the system reaches 89.7% precision in this open-set setting, and 81.6% rank-1 accuracy on the IAM-OnDB English database, which the paper reports as state-of-the-art results. The practical stake is forensic attribution and document authentication from minimal handwriting evidence.","feed_headline":"Single character reveals unseen writer 89.7% of the time","feed_subtitle":"Contrastive masked autoencoder turns one handwritten glyph into an identity match without ever training on that writer.","key_machinery":"The central object is CMAE, a Transformer-based encoder whose input is a single character trajectory normalized to (0,1], padded to 800 timepoints with pen-down, pen-up, and padding tokens, and segmented into patches of length 52. During pretraining, random patches are masked and a decoder reconstructs them with MSE loss; after the encoder, average pooling and a two-layer MLP projector feed a supervised contrastive loss, and a concatenated pair of embeddings goes to a Transformer-plus-MLP discriminator trained with cross-entropy. The masked reconstruction is tasked with capturing sequential stroke details, while the contrastive objective supplies the style-discrimination that makes unseen writers separable.","core_discovery":"The paper's central claim is that open-set writer identification can be solved at the level of a single character by learning trajectory representations that are at once reconstructable and style-discriminative. The proposed CMAE model masks random patches of a character's coordinate trajectory, reconstructs them with a Transformer decoder, and in parallel uses a supervised contrastive loss to pull embeddings of the same writer together and push different writers apart; a final discriminator reads a pair of embeddings and decides whether they share a writer. The paper reports that this reaches 89.7% ± 7.4% precision under an open-set protocol on CASIA-OLHWDB (20 unseen writers, 2 characters each, averaged over 100 draws) and 81.6% ± 1.6% rank-1 accuracy on IAM-OnDB following the letter-level baseline protocol. Ablations show the contrastive branch is the dominant contributor, removing it drops CASIA precision from 89.7% to 6.5%, while removing the masked reconstruction costs only 4.3 points, and a 0.15 mask ratio beats 0.75.","pith_inferences":["I infer that the single-character capability could extend naturally to writer verification, where the question is only whether two fragments share a writer, and to historical scribe attribution when only one surviving glyph is available.","I infer that the large precision drop without contrastive learning points to a cheaper design: a contrastive-only system over patch embeddings might recover most of the discriminative signal, which would be a simple testable variant.","I infer that the 7.4-point standard deviation across 100 random writer draws means deployed precision will depend heavily on which writers are queried, so practical systems should report per-writer confidence intervals rather than a single number."],"forward_implications":["A single character can serve as the query for open-set writer identification, so forensic or archival systems no longer need long text samples.","Writer identity is treated as a representation-learning problem rather than a closed-set classification problem, since the model is evaluated on writers absent from training.","The contrastive branch carries most of the discriminative power, so future methods can expect large precision losses if they replace it with generation-only objectives.","The same trajectory-patching and tokenization scheme transfers across scripts, as the model trains on Chinese characters and reports results on English IAM-OnDB data.","Mask ratio is a first-order hyperparameter: 0.15 masking outperforms 0.75 by roughly 14 precision points on CASIA."],"supporting_citations":[{"why":"Supplies the masked-autoencoder encoder-decoder design that CMAE adapts to trajectory patches.","marker":"[19]"},{"why":"Supplies the contrastive learning principle for pulling same-writer pairs together and separating different writers.","marker":"[20]"},{"why":"The letter-level online writer-id baseline whose evaluation protocol the paper follows on IAM-OnDB and whose open-set results CMAE is compared with.","marker":"[11]"},{"why":"Provides the CASIA-OLHWDB dataset, including the character classes and disjoint train/test writer splits that produce the 89.7% precision result.","marker":"[47]"},{"why":"Defines the supervised contrastive loss that trains the encoder with same-writer positive and different-writer negative pairs.","marker":"[46]"},{"why":"Provides the supervised contrastive loss formulation the paper cites for its CL loss.","marker":"[41]"},{"why":"Earlier character-level writer-id CNN evaluated only in closed-set settings, marking the gap CMAE addresses.","marker":"[17]"},{"why":"An online DTW-based open-set system that the paper's open-set handling is contrasted with.","marker":"[14]"}],"fun_headline_variants":["Open-set writer ID from single characters hits 89.7%","Contrastive masked autoencoders ID unseen writers per character","One glyph tells writer: 89.7% precision in open set","Character-level open-set writer identification via CMAE","Contrastive MAE achieves open-set writer ID at 89.7%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The state-of-the-art claim depends on the open-set protocol being a fair point of comparison: the paper measures 20 randomly chosen writers with 2 characters each over 100 trials and compares those numbers with earlier models that were evaluated under different protocols, without rerunning a baseline under the same protocol.","fun_headline_variants_meta":{"raw":{"variants":["Open-set writer ID from single characters hits 89.7%","Contrastive masked autoencoders ID unseen writers per character","One glyph tells writer: 89.7% precision in open set","Character-level open-set writer identification via CMAE","Contrastive MAE achieves open-set writer ID at 89.7%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000504,"raw_usage":{"total_tokens":2470,"prompt_tokens":964,"completion_tokens":1506,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":1416}},"tokens_in":580,"tokens_out":1506,"duration_ms":10881,"temperature":1.0,"reasoning_tokens":1416,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:44:17.952644+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the strongest published online writer-id baseline under the paper's exact CASIA open-set protocol (20 unseen writers, 2 characters per writer, 100 random trials) and compare precision; if that baseline matches or exceeds 89.7% within the reported spread, the central claim that CMAE is state of the art for single-character open-set identification fails.","supporting_citations":[{"cited_title":"Masked autoencoders are scalable vision learners,","cited_arxiv_id":null,"evidence_quote":"Supplies the masked-autoencoder encoder-decoder design that CMAE adapts to trajectory patches."},{"cited_title":"Letter-level online writer identification,","cited_arxiv_id":null,"evidence_quote":"The letter-level online writer-id baseline whose evaluation protocol the paper follows on IAM-OnDB and whose open-set results CMAE is compared with."},{"cited_title":"Casia online and of- fline chinese handwriting databases,","cited_arxiv_id":null,"evidence_quote":"Provides the CASIA-OLHWDB dataset, including the character classes and disjoint train/test writer splits that produce the 89.7% precision result."},{"cited_title":"Sclair: Supervised contrastive learning for user and device independent airwriting recog- nition,","cited_arxiv_id":null,"evidence_quote":"Provides the supervised contrastive loss formulation the paper cites for its CL loss."},{"cited_title":"Chinese character-level writer identifi- cation using path signature feature, dropstroke and deep cnn,","cited_arxiv_id":null,"evidence_quote":"Earlier character-level writer-id CNN evaluated only in closed-set settings, marking the gap CMAE addresses."},{"cited_title":"Dsdtw: Local representation learning with deep soft-dtw for dynamic signature verification,","cited_arxiv_id":null,"evidence_quote":"An online DTW-based open-set system that the paper's open-set handling is contrasted with."}],"review_version":1}