{"id":"4f32ae60-2dfb-4753-b802-611474b13a3d","arxiv_id":"2606.15883","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A ByT5-small seq2seq model restores Kashmiri diacritics on a new 23.7k-pair dataset, reaching DERm 0.2012, WER 0.2159, and 77.5% native-expert accuracy.","lead":"The paper introduces Koshur Diacritizer, a ByT5-small byte-level model trained on 23.7k Kashmiri sentence pairs to restore omitted diacritics in text written in modified Perso-Arabic script. A smart generalist might read it to see a concrete, released baseline for handling ambiguity in a low-resource Indo-Aryan language's digital text.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Single-expert evaluation and unspecified data sourcing limit support for generalization and correctness claims","rationale":"The reader's weakest assumption matches the load-bearing point exactly. Because the paper releases the dataset and code, the automatic metrics can be re-run, but the human validation and data-representativeness steps remain the weakest link for any claim beyond 'baseline on this split.' This justifies moving from UNVERDICTED to CONDITIONAL rather than full acceptance.","tokens_in":1700,"tokens_out":309,"duration_ms":33870,"concrete_test":"Sample 200 held-out sentences; have two additional independent native Kashmiri speakers annotate model outputs for accuracy; compute Cohen's kappa with the original expert and the fraction of sentences where all three agree. If kappa < 0.7 or unanimous agreement < 70%, the expert figure's reliability is insufficient.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim rests on held-out DERm 0.2012 / WER 0.2159 plus 77.5% mean expert accuracy. The 23.7k pairs are described only as 'aligned' with 'alignment validation' and 'script-aware normalization'; no sourcing details, diversity statistics, or validation protocol appear. The expert result is from one native speaker with no reported sample size, evaluation protocol, or agreement metric. These gaps directly undermine the assumption that test-set numbers and the 77.5% figure demonstrate reliable, generalizable restoration.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces Koshur Diacritizer, a ByT5-small byte-level sequence-to-sequence model for diacritic restoration in Kashmiri text written in modified Perso-Arabic script. It releases a new public dataset of 23.7k aligned undiacritized-diacritized sentence pairs constructed via script-aware normalization and alignment validation, and reports held-out test performance of DERm 0.2012 and WER 0.2159 together with 77.5% mean accuracy from evaluation by a native Kashmiri linguistic expert. The dataset, model, and code are released to support future low-resource language work.","tokens_in":1819,"tokens_out":554,"duration_ms":38475,"significance":"If the empirical results hold under more detailed validation, the work supplies a reproducible baseline and the first public aligned corpus for Kashmiri diacritic restoration, an under-served Indo-Aryan language. The public release of the 23.7k-pair dataset, trained model, and source code is a concrete contribution that lowers the barrier for subsequent research on byte-level models for low-resource script normalization tasks.","major_comments":[{"comment":"Dataset section: the 23.7k aligned sentence pairs are described only as 'aligned' with 'alignment validation' and 'script-aware normalization'; no sourcing details, corpus provenance, diversity statistics, or explicit validation protocol are provided. This directly affects the claim that the held-out test set (and the reported DERm/WER) demonstrates generalization to real-world Kashmiri usage.","section":"Dataset"},{"comment":"Evaluation section: the native-expert result of 77.5% mean accuracy is obtained from a single linguistic expert; the manuscript supplies neither the number of evaluated sentences, the evaluation protocol, nor any agreement metric. Because this figure is presented as additional evidence of correctness alongside the automatic metrics, the missing protocol details are load-bearing for the central generalization claim.","section":"Evaluation"}],"minor_comments":[{"comment":"Abstract and §4: the training procedure, hyperparameter search, error analysis, and statistical significance tests for the reported DERm and WER are not described; adding these would improve reproducibility without altering the central claims.","section":"Abstract"},{"comment":"The paper does not compare against any prior Kashmiri or related-language diacritizer baselines; a brief related-work paragraph would help situate the 0.2012 DERm result.","section":"Related Work"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. We address each major comment below and will revise the manuscript to supply the requested details on dataset construction and evaluation procedures.","responses":[{"response":"We agree that the manuscript would benefit from expanded documentation. In the revision we will add sourcing information for the original Kashmiri texts, corpus provenance, diversity statistics (sentence-length distribution and lexical coverage), and a more explicit account of the alignment-validation steps. These additions will be placed in the Dataset section and will better support the generalization claims for the held-out test set.","revision_made":"yes","referee_comment":"[Dataset] Dataset section: the 23.7k aligned sentence pairs are described only as 'aligned' with 'alignment validation' and 'script-aware normalization'; no sourcing details, corpus provenance, diversity statistics, or explicit validation protocol are provided. This directly affects the claim that the held-out test set (and the reported DERm/WER) demonstrates generalization to real-world Kashmiri usage."},{"response":"We acknowledge the need for greater transparency. The revision will state the number of sentences evaluated by the expert, describe the evaluation protocol (including the exact instructions and scoring method), and note that inter-annotator agreement could not be computed with a single evaluator. The result will be presented more explicitly as supplementary evidence, with the limitation discussed in the text.","revision_made":"yes","referee_comment":"[Evaluation] Evaluation section: the native-expert result of 77.5% mean accuracy is obtained from a single linguistic expert; the manuscript supplies neither the number of evaluated sentences, the evaluation protocol, nor any agreement metric. Because this figure is presented as additional evidence of correctness alongside the automatic metrics, the missing protocol details are load-bearing for the central generalization claim."}],"tokens_in":1429,"tokens_out":399,"duration_ms":42537,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is a new 23.7k sentence dataset and a byte-level model for restoring diacritics in Kashmiri, with reported DERm of 0.2012 and WER of 0.2159 on held-out data plus 77.5% from one expert. No prior work covered this language, so the releases fill an empty spot.\n\nThe paper applies an existing ByT5-small setup with script-aware normalization and skeleton-preserving inference. Releasing the aligned pairs, the trained model, and the code is the practical step that lets others start from something concrete instead of nothing.\n\nThe evaluation side is the weaker part. The abstract gives almost no information on where the sentences came from, how representative they are, or the exact alignment process beyond a mention of validation. The expert check rests on a single native speaker with no sample size, protocol, or agreement numbers reported. Training details such as hyperparameter choices or error breakdowns are also missing. These omissions make it difficult to treat the performance figures as firm evidence of reliable generalization.\n\nThe work targets researchers handling low-resource scripts or Kashmiri specifically. Anyone needing a reproducible starting point for diacritic tasks in similar languages can use the artifacts directly.\n\nThe paper shows straightforward engagement with the task and honest release of resources, so it is worth a serious referee even with the gaps in supporting detail. I would send it to review and ask for more on data collection and evaluation protocol.","headline":"This paper supplies the first public dataset and ByT5 baseline for Kashmiri diacritic restoration, but thin descriptions of data sourcing and evaluation limit how much the numbers can be trusted.","tokens_in":2296,"tokens_out":385,"would_cite":false,"duration_ms":36825,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A byte-level sequence-to-sequence model restores diacritics in Kashmiri text using a released dataset of 23.7k aligned sentences.","keywords":["Kashmiri","diacritic restoration","sequence-to-sequence model","ByT5","low-resource languages","Perso-Arabic script","text normalization","NLP dataset"],"falsifier":"Performance measured on an independently gathered test collection of several thousand Kashmiri sentences whose restored forms are judged by multiple native speakers rather than one expert.","tokens_in":2611,"feed_emoji":"","tokens_out":802,"duration_ms":36184,"temperature":0.7,"pith_summary":"Kashmiri text in modified Perso-Arabic script commonly drops diacritic marks, producing ambiguity that hinders reading and downstream NLP tasks. The paper presents Koshur Diacritizer, a ByT5-small byte-level model trained to restore those marks while preserving the underlying letter sequence. The authors also release a public collection of 23.7k undiacritized-diacritized sentence pairs created through script-aware normalization and alignment validation. On held-out test data the model records a diacritic error rate of 0.2012 and word error rate of 0.2159; a native expert assigns it a mean accuracy of 77.5 percent. The work supplies the first publicly reproducible baseline for this restoration task in Kashmiri.","feed_headline":"Byte-level model restores Kashmiri diacritics at 77.5% expert accuracy","feed_subtitle":"23.7k aligned sentence pairs and skeleton-preserving inference supply the first public baseline for this Indo-Aryan language.","key_machinery":"Byte-level sequence-to-sequence model (ByT5-small) augmented with script-aware normalization, alignment validation, and skeleton-preserving inference to restore diacritics without altering base letters.","core_discovery":"Koshur Diacritizer is a ByT5-small byte-level sequence-to-sequence model that restores diacritics in Kashmiri while keeping the original base-letter sequence intact. It combines script-aware normalization, alignment validation, and skeleton-preserving inference, and is trained on the released set of 23.7k aligned sentence pairs. The model attains a DERm of 0.2012 and WER of 0.2159 on a held-out test set, together with 77.5 percent mean accuracy according to native-speaker expert review.","pith_inferences":["The same byte-level approach and normalization steps could be applied to diacritic restoration in other Perso-Arabic-script languages that also omit marks in digital text.","The skeleton-preserving mechanism offers a template for orthographic normalization tasks that must leave core letter identity unchanged.","An accuracy of 77.5 percent under expert review indicates that larger or more diverse training data would be needed before the model could serve as a fully reliable production tool.","Embedding the diacritizer as a preprocessing step could raise accuracy on downstream Kashmiri tasks such as machine translation or named-entity recognition."],"forward_implications":["The public 23.7k-pair dataset supplies training and evaluation material for any future Kashmiri diacritization system.","Skeleton-preserving inference guarantees that the output never changes the sequence of base letters present in the input.","Reported error rates and expert accuracy establish a concrete performance baseline for Kashmiri diacritic restoration.","The released model, dataset, and code enable direct replication and incremental improvement by other researchers."],"fun_headline_variants":["Byte-level ByT5 restores Kashmiri diacritics at 77.5% accuracy","23.7k aligned pairs train Kashmiri diacritic model","Koshur Diacritizer reaches 0.2012 DERm on held-out test","Skeleton-preserving inference for Kashmiri diacritics","Native expert scores model 77.5% on Kashmiri diacritics"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The 23.7k aligned sentence pairs represent typical real-world Kashmiri usage and the single native expert review sufficiently confirms output correctness.","fun_headline_variants_meta":{"raw":{"variants":["Byte-level ByT5 restores Kashmiri diacritics at 77.5% accuracy","23.7k aligned pairs train Kashmiri diacritic model","Koshur Diacritizer reaches 0.2012 DERm on held-out test","Skeleton-preserving inference for Kashmiri diacritics","Native expert scores model 77.5% on Kashmiri diacritics"]},"model":"grok-4.3","cost_usd":0.007831,"raw_usage":{"total_tokens":3587,"prompt_tokens":694,"num_sources_used":0,"completion_tokens":105,"cost_in_usd_ticks":78312000,"prompt_tokens_details":{"text_tokens":694,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2788,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":694,"tokens_out":105,"duration_ms":18725,"temperature":1.0,"reasoning_tokens":2788,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T04:08:13.641824+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Performance measured on an independently gathered test collection of several thousand Kashmiri sentences whose restored forms are judged by multiple native speakers rather than one expert.","supporting_citations":[],"review_version":1}