{"id":"20a07d7d-cf6c-4ad5-983d-660f6b404af3","arxiv_id":"2602.07049","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An error-based contrastive training loss that pushes sensor features away from near-miss transcripts improves IMU handwriting recognition with zero inference overhead, but gains are split-specific.","lead":"This paper adds a temporary text-embedding branch and two contrastive losses to an IMU handwriting recognizer during training, then removes it for deployment. On a public benchmark it reports lower character error rates with no added inference cost, though the gains require different settings for different test splits.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported 'up to' gains lack error bars and rely on test-fold model selection; the full ECHWR (with EC) improves WD by only 0.41 pp.","rationale":"The reader's weakest_assumption focuses on text-encoder anchor stability, which is a real and visible weakness (Section 4.6). However, the single most load-bearing concern for the central claim is the reliability of the reported improvements: no error bars, hyperparameters/architectures selected on the evaluation folds, and the best WD number requiring a configuration without the paper's novel EC loss. This concern directly affects whether the headline gains are real or selection artifacts. The reader's rationale also mentions these evaluation issues, so I partially agree. The verdict stays CONDITIONAL because the concerns are addressable with proper validation and variance reporting.","tokens_in":10475,"tokens_out":7256,"duration_ms":73140,"concrete_test":"Run 10 independent training seeds of REWI/B and the three objective variants (CTC, +BC, +BC+EC) under both per-split architectures, using a nested validation fold to choose the error-set size for each outer fold. Compute mean and 95% bootstrap CI of the per-fold CER difference relative to REWI/B. If the CI for the full ECHWR on WD includes 0, or if the configuration selected by this protocol differs from the paper's, the claimed 'up to' gains are not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Table 1, the full ECHWR/B configuration (CTC+BC+EC) achieves CER 14.04% on WD and 6.79% on WI, versus REWI/B 14.45% and 7.33% — absolute gains of 0.41 pp and 0.54 pp. The headline 'up to 10.4%' on WD comes not from this full configuration but from the BC-only variant (12.95%). The paper reports no standard deviations or confidence intervals across the 5 folds, and the per-split architectures (RMSNorm+GA vs LayerNorm+GA+Registers) and error-set size (Fig. 2) appear to be selected after inspecting test-fold results. With such small absolute differences and selection over many variants, the observed improvements could be artifacts of optimizer noise or test-set peeking. Section 4.6 itself states the text encoder 'struggles to generalize its embeddings to the unseen word combinations,' consistent with EC being harmful on WD. Thus the central quantitative claim — that ECHWR reduces CER by up to 7.4%/10.4% — is not yet rigorously established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ECHWR, a training framework for IMU-based online handwriting recognition. An auxiliary text-encoder branch is trained alongside the primary CTC-based recognizer using two contrastive objectives: an in-batch contrastive loss (BC) aligning sensor and text embeddings, and an error-based contrastive loss (EC) that separates the ground-truth transcript from synthetic single-edit hard negatives. The auxiliary branch is removed after training, so inference uses the unmodified REWI architecture. Experiments on the right-handed OnHW-Words500 dataset report CER/WER for writer-dependent (WD) and writer-independent (WI) splits, with headline relative CER reductions of up to 10.4% (WD) and 7.4% (WI). The paper also includes ablations on normalization, gated attention, registers, and the number of synthetic error sets.","tokens_in":10744,"tokens_out":4283,"duration_ms":51648,"significance":"If the reported gains were robust, this would be a useful practical contribution: the zero-inference-overhead property is attractive for edge-device deployment, and the idea of using a temporary semantic alignment branch as a regularizer is clearly formulated. The release of code and the explicit loss definitions are also strengths. However, the current evidence is a proof of concept rather than a rigorous state-of-the-art claim: the results come from a single dataset, the best WD number is obtained with a configuration that excludes the error-based contrastive loss, and no uncertainty quantification or statistical testing is reported. With small absolute improvements and per-split architecture selection, the central quantitative claims are not yet established.","major_comments":[{"comment":"The abstract's WD improvement of 10.4% is not achieved by the full ECHWR method as defined by Eq. (1). On the WD split, the full objective CTC+BC+EC gives CER 14.04%, while CTC+BC alone gives 12.95%. Thus the headline WD gain comes from the BC-only variant, not from the proposed error-based contrastive loss. In fact, adding EC degrades WD CER by about 8% relative to BC-only. The authors should clearly separate claims for the BC-only and full frameworks, and either report the full-framework WD result as the main result or justify why the BC-only variant is part of ECHWR.","section":"Section 4.3, Table 1"},{"comment":"All CER/WER numbers appear to be single point estimates over 5-fold cross-validation with no standard deviations, confidence intervals, or statistical tests. Given that the full ECHWR/B WD gain is only 0.41 pp (14.45 to 14.04) and the WI gain is 0.54 pp, these differences could easily be within run-to-run variation. Moreover, the architecture (RMSNorm+GA for WD, LayerNorm+GA+Registers for WI) and the error-set size (Fig. 2) appear to be selected after inspecting test-fold results. The authors should report fold-level means and standard deviations, state how many random seeds were used, and clarify whether validation folds or test folds were used for model selection.","section":"Sections 4.3-4.5"},{"comment":"The paper's own qualitative analysis states that the text encoder 'struggles to generalize its embeddings to the unseen word combinations' on the WD split, and Appendix A shows that the WD train/validation character distributions differ substantially (some characters, e.g., 'q' and 'ä', are absent from the validation set). Since both contrastive losses rely on the auxiliary text branch, this is a direct limitation of the proposed framework's stability on exactly the split where the largest headline gain is reported. The authors should connect this limitation to their quantitative claims and provide evidence that the WD BC-only gain is not an artifact of the misaligned text embeddings.","section":"Section 4.6 and Appendix A"},{"comment":"There is no single ECHWR configuration that performs best on both splits: WD favors RMSNorm+GA, while WI favors LayerNorm+GA+Registers. The current framing suggests that ECHWR is a unified method, but the experiments actually provide per-split recipes. This is acknowledged in the conclusion, but the abstract and introduction nevertheless state that 'ECHWR significantly outperforms' baselines. The authors should either present a unified configuration, or explicitly frame the contribution as a family of contrastive regularizers whose optimal architecture is task-dependent, and adjust the claims accordingly.","section":"Section 4.4, Table 2"}],"minor_comments":[{"comment":"The terms ECHWR/S and ECHWR/B are not defined in the main text. Please define the small and base variants before the table.","section":"Table 1 caption"},{"comment":"The sensitivity plots would benefit from numerical values and error bars; as shown, it is difficult to judge whether the differences between error-set sizes are meaningful. Also, the central legend applies to all subplots, but the legend text is small and hard to read.","section":"Section 4.5, Fig. 2"},{"comment":"The phrase 'zero inference overhead' is repeated several times. The fact that the auxiliary branch is discarded is clear; avoid over-emphasizing a property that follows trivially from the design.","section":"Abstract/Introduction"},{"comment":"The appendix reference appears as 'Appx. A' in Section 4.1; please use 'Appendix A' for consistency.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The core idea is sound and the experimental protocol is fairly standard, but the paper currently overclaims. The WD headline result is from the BC-only variant, not the full ECHWR method, and the lack of error bars plus per-split architecture selection makes the quantitative claims fragile. A revised version that reports variability, separates per-component claims, and clearly limits the scope to a per-split recipe could make the paper acceptable. The authors should also consider whether the 'up to 7.4%/10.4%' phrasing is justified given the full-framework results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: the error-based contrastive loss (EC) that generates Levenshtein-distance-1 synthetic hard negatives from ground-truth text and optimizes them jointly with CTC is a real novelty. I don't know prior work that does that. And the trick of discarding the auxiliary text branch at inference is clean. On the writer-independent split the full method does beat the REWI baseline by 0.54 pp CER (7.33 to 6.79), a 7.4% relative improvement—small but real. The paper is also unusually frank about its ablations: Table 1 and the discussion state that on the writer-dependent split, the full ECHWR (CTC+BC+EC) is worse than BC alone (14.04 vs 12.95), and Section 4.6 says the text encoder 'struggles to generalize its embeddings to the unseen word combinations.' That matches the stress-test's point: the abstract's 'up to 10.4%' on WD comes from the BC-only variant, not the full framework.\n\nThe soft spots are exactly where the stress-test puts them. No error bars or confidence intervals across the five folds. The per-split architectures (RMSNorm+GA for WD; LayerNorm+GA+Registers for WI) and the error-set size S=2 appear to be chosen after inspecting test-fold results. With absolute gains of 0.4-0.5 pp, that is a real test-set peeking risk—selection over many variants can produce those differences by chance. The paper's own conclusion says no universal configuration was found, which is honest but concedes the headline claim is weaker than the abstract implies.\n\nThat said, the central methodological idea is sound and the evaluation protocol is standard. The failure of EC on WD is not a contradiction; the authors explain it as the text encoder's poor anchors on unseen character combinations, which is plausible. The zero-inference-overhead property is a genuine practical advantage for edge deployment, and the code is promised (though I haven't run it).\n\nWho is this for? People working on sensor-based handwriting recognition and anyone interested in auxiliary contrastive regularizers for sequence-to-sequence models. A serious referee should get this, but the revision needs error bars, a proper validation split for hyperparameter selection, and a re-cast of the claims that specifies which configuration achieves each number. I'd accept it for peer review and ask for major revision.","headline":"A genuinely new contrastive training loss with zero inference overhead, but the headline gains are split-dependent and lack error bars.","tokens_in":11244,"tokens_out":3418,"would_cite":false,"duration_ms":34387,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A temporary text-alignment branch, used only during training and discarded afterward, lets an IMU-based handwriting recognizer cut character error by up to 10.4% with zero extra inference cost.","keywords":["Online handwriting recognition","IMU sensors","Contrastive learning","Hard negative mining","CTC loss","Encoder-decoder model","Edge inference"],"falsifier":"Measure CER on a writer-dependent split whose validation words share no character n-grams with the training words. If ECHWR does not beat the CTC-only baseline on that split, the claimed zero-overhead gain on novel vocabulary is falsified.","tokens_in":10346,"feed_emoji":"✍️","tokens_out":9004,"duration_ms":90963,"temperature":0.7,"pith_summary":"The paper proposes a training-only trick for making an IMU-based handwriting recognizer more accurate without changing the model that runs on the device. During training, a temporary 'text branch' embeds the ground-truth transcript and also creates near-miss transcripts that differ by a single character; a dual contrastive loss pulls the sensor signal's embedding toward the correct text and pushes it away from the near-misses. After training the branch is removed, so the deployed model keeps the original architecture and compute cost. On the OnHW-Words500 dataset, the method reduces character error rate by up to 7.4% on the writer-independent split and up to 10.4% on the writer-dependent split relative to the CTC baseline. A central finding is that the two contrastive losses are complementary in a task-dependent way: the hard-negative loss matters when the vocabulary is known but writers are new, while the plain alignment loss matters when writers are known but vocabulary is new.","feed_headline":"Contrastive training cuts handwriting errors 10.4% at zero cost","feed_subtitle":"Discarded after training, the text branch leaves the deployed model fast while improving accuracy.","key_machinery":"The central mechanism is a temporary auxiliary branch consisting of a character-level Transformer text encoder and an attention-pooling module that compresses the CNN's time-series features into a fixed vector. Two contrastive objectives are applied during training: the in-batch contrastive loss, which treats other pairs in the batch as negatives and enforces a soft alignment between sensor and text embeddings, and the error-based contrastive loss, which builds hard negatives by applying single-character delete/insert/substitute edits to the ground-truth transcript and demands a larger distance to those. The auxiliary branch is removed after training, so the original model—with its original","core_discovery":"On the paper's own terms, the discovery is that a sequence-to-sequence handwriting recognizer can be trained to encode a margin between correct and near-correct transcripts. The framework, ECHWR, adds two contrastive objectives to the usual CTC objective: an in-batch loss that aligns each sensor embedding with the embedding of its matching transcript, and an error-based loss that contrasts the sensor embedding against transcripts generated by single-character deletion, insertion, or substitution. The auxiliary text encoder and attention pooling that produce these embeddings are discarded after training, so inference sees exactly the same architecture as the baseline. Evaluations on OnHW-Word","pith_inferences":["If a pretrained or far larger text encoder were used, the error-based loss might also help the writer-dependent split, where the current model's text embeddings fail on unseen words; this would likely unify the two configurations the paper reports.","Hard negatives could be generated from the recognizer's own confusions (e.g., characters that the model often mispredicts) rather than random edits, which would make the margin task better aligned with the model's actual error modes.","The reported gains on OnHW-Words500 come with a training-time memory and compute overhead for the auxiliary branch; a practical follow-up would measure whether the accuracy gain survives when the training budget is fixed and the extra compute is instead spent on longer training of the baseline.","The task-dependent dichotomy suggests a testable hypothesis: on a dataset with both unseen writers and unseen vocabulary, a single configuration with moderate hard-negative weight may be optimal—neither of the two extremes."],"forward_implications":["Because the auxiliary branch is removed after training, the deployed recognizer has exactly the same architecture, memory footprint, and inference latency as the baseline, so the accuracy gains come at zero runtime cost.","On the writer-independent split, where the vocabulary is fixed but writers are unseen, the error-based contrastive loss provides the largest gain, indicating that hard negatives help when the input distribution shifts.","On the writer-dependent split, where writers are known but vocabulary is unseen, the in-batch alignment loss alone gives the largest gain, indicating that semantic alignment helps when the label distribution shifts.","Adding error-based discrimination to a model that already has good alignment can hurt if the text encoder cannot embed unseen words; thus the benefit of hard-negative contrastive learning depends on the stability of the label anchor."],"fun_headline_variants":["10.4% fewer handwriting errors, zero inference overhead","Discarded branch yields 10.4% better IMU handwriting, no slowdown","Contrastive trick boosts handwriting recognition 10.4% with no extra cost","Train with a text branch, deploy without: 10.4% better IMU handwriting","No-inference-overhead contrastive learning shrinks handwriting errors by 10.4%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper assumes that the text encoder trained from scratch on the training split can produce stable anchor embeddings for word combinations it has never seen; its own writer-dependent results show this assumption breaks down (adding the error-based loss raises CER from 12.95% to 14.04%), and the qualitative section states the text encoder 'struggles to generalize its embeddings to the unseen word combinations.'","fun_headline_variants_meta":{"raw":{"variants":["10.4% fewer handwriting errors, zero inference overhead","Discarded branch yields 10.4% better IMU handwriting, no slowdown","Contrastive trick boosts handwriting recognition 10.4% with no extra cost","Train with a text branch, deploy without: 10.4% better IMU handwriting","No-inference-overhead contrastive learning shrinks handwriting errors by 10.4%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001295,"raw_usage":{"total_tokens":5110,"prompt_tokens":722,"completion_tokens":4388,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":466,"completion_tokens_details":{"reasoning_tokens":4293}},"tokens_in":466,"tokens_out":4388,"duration_ms":28875,"temperature":1.0,"reasoning_tokens":4293,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T04:31:52.635173+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure CER on a writer-dependent split whose validation words share no character n-grams with the training words. If ECHWR does not beat the CTC-only baseline on that split, the claimed zero-overhead gain on novel vocabulary is falsified.","supporting_citations":[],"review_version":1}