{"id":"bd6c2eba-5331-4f0c-b4cb-fa548193ae14","arxiv_id":"2606.26107","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":1,"one_line_summary":"NEST-V1 demonstrates feasibility of emotion-aware Nepali sign language avatar generation from speech with 81.1% ASR accuracy and 79.21% emotion accuracy on a small dataset using an efficient 22.1M parameter model.","lead":"The paper introduces NEST-V1, a proof-of-concept system for translating spoken Nepali words into emotion-conditioned sign language avatars using a shared acoustic encoder. A generalist reader might find it relevant for understanding AI applications in accessibility for low-resource languages and communities.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"No metrics or description of avatar generation quality; results cover only upstream audio encoder.","rationale":"Reader correctly flags the small scale as limiting broader claims. A more direct load-bearing gap is the complete lack of any result on the avatar generation step itself, which is required for the stated multimodal translation objective. This is independent of dataset size and would remain even if the vocabulary were larger.","tokens_in":1769,"tokens_out":268,"duration_ms":17356,"concrete_test":"Add a section reporting avatar output quality (human MOS or automatic pose similarity) on the 600 samples; if the generation step is absent or scores near random, the feasibility demonstration fails.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is feasibility of a multimodal system that generates emotion-conditioned Nepali Sign Language avatars from spoken input. The reported results and architecture details address only the shared acoustic encoder's performance on ASR (81.1%) and emotion classification (79.21%) over 12 classes. No component, loss, or evaluation is described for producing or assessing the actual sign-language output (e.g., pose sequences, gloss accuracy, or emotional fidelity of the avatar). Without this, the pilot does not test the end-to-end claim even on the four-word vocabulary.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.3","summary":"This pilot study introduces NEST-V1, a lightweight multimodal framework using a shared acoustic encoder for simultaneous ASR and emotion classification on four Nepali words across three emotions. It reports 81.1% ASR accuracy and 79.21% emotion recognition accuracy on 600 audio samples from 50 speakers, claims 37% parameter efficiency versus separate models with a 22.1M-parameter footprint, and positions the work as establishing feasibility for generating emotion-conditioned Nepali Sign Language avatars from spoken input in low-resource settings.","tokens_in":1855,"tokens_out":484,"duration_ms":19120,"significance":"If the end-to-end avatar generation were validated with appropriate metrics, the approach could offer a scalable, edge-deployable foundation for emotion-aware sign language systems in low-resource languages, addressing an underexplored area. The current results, however, address only the audio encoder stage and do not substantiate the multimodal translation claim.","major_comments":[{"comment":"The manuscript's central claim is the feasibility of a multimodal system that generates emotion-conditioned Nepali Sign Language avatars from spoken input, yet the reported results, architecture details, and evaluation cover only the shared acoustic encoder's performance on ASR and emotion classification. No component, loss, output representation (e.g., pose sequences or glosses), or quality metric for the avatar generation stage is described or assessed, even on the four-word vocabulary. This leaves the core end-to-end claim untested.","section":"Abstract and results description"},{"comment":"The claim of 37% parameter efficiency compared to separate model architectures is stated without the underlying calculation, the parameter counts of the separate models, or the configuration details used for the comparison. This makes the efficiency figure non-reproducible and dependent on unstated implementation choices.","section":"Abstract"},{"comment":"Evaluation is performed on a small dataset of 600 samples with a limited vocabulary (four words, three emotions) and 12 classes total, but no details are provided on validation method (e.g., speaker-independent splits, cross-validation), baseline comparisons, or overfitting controls. This weakens support for the feasibility claim given the pilot scale.","section":"Abstract and evaluation description"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback on our pilot study. We address each major comment below and will revise the manuscript accordingly to improve clarity, reproducibility, and accurate scoping of the work.","responses":[{"response":"We agree that the abstract overstates the end-to-end multimodal claim. This pilot study validates only the shared acoustic encoder for simultaneous ASR and emotion classification as the core technical foundation; the avatar generation stage (including any pose sequences, glosses, or quality metrics) is not implemented or evaluated here and is explicitly planned as future work. We will revise the abstract, introduction, and conclusion to accurately reflect the scope as a proof-of-concept for the encoder stage that establishes feasibility for subsequent multimodal development.","revision_made":"yes","referee_comment":"[Abstract and results description] The manuscript's central claim is the feasibility of a multimodal system that generates emotion-conditioned Nepali Sign Language avatars from spoken input, yet the reported results, architecture details, and evaluation cover only the shared acoustic encoder's performance on ASR and emotion classification. No component, loss, output representation (e.g., pose sequences or glosses), or quality metric for the avatar generation stage is described or assessed, even on the four-word vocabulary. This leaves the core end-to-end claim untested."},{"response":"The 37% figure is based on the shared 22.1M-parameter encoder versus separate ASR and emotion models, but we acknowledge the absence of explicit calculation details in the abstract. We will add a methods subsection with exact parameter counts for the compared architectures, the formula used, and configuration details to ensure full reproducibility.","revision_made":"yes","referee_comment":"[Abstract] The claim of 37% parameter efficiency compared to separate model architectures is stated without the underlying calculation, the parameter counts of the separate models, or the configuration details used for the comparison. This makes the efficiency figure non-reproducible and dependent on unstated implementation choices."},{"response":"As a pilot on limited data, we will expand the evaluation section to specify the speaker-independent 80/20 split used, note the absence of cross-validation due to dataset size, include baseline comparisons where feasible, and detail overfitting controls (e.g., dropout, early stopping). We will also emphasize the pilot-scale limitations more explicitly while retaining the feasibility demonstration for the encoder.","revision_made":"yes","referee_comment":"[Abstract and evaluation description] Evaluation is performed on a small dataset of 600 samples with a limited vocabulary (four words, three emotions) and 12 classes total, but no details are provided on validation method (e.g., speaker-independent splits, cross-validation), baseline comparisons, or overfitting controls. This weakens support for the feasibility claim given the pilot scale."}],"tokens_in":1493,"tokens_out":599,"duration_ms":35046,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The central fact is that this is a narrow pilot on four Nepali words and three emotions. It reports 81.1% ASR accuracy and 79.21% emotion accuracy from a shared encoder on 600 samples, plus a claim of 37% parameter savings at 22.1M total parameters. That is the entire set of numbers.\n\nWhat is new is the application of a shared acoustic encoder to Nepali speech for these two tasks together. The efficiency angle is reasonable for edge deployment in a low-resource setting, and the choice of language fills a gap where prior sign-language work is scarce.\n\nThe soft spots are straightforward. No part of the paper describes how the sign-language avatar is produced from the encoder outputs, what loss or decoder is used, or how output quality is measured. There are no gloss-level scores, pose metrics, or even qualitative examples of the generated signs. The abstract promises emotion-conditioned avatars from spoken input, but the results stop at the encoder. The dataset is also small enough that overfitting is a real risk, and no baselines or validation splits are detailed.\n\nThis work is mainly for people already building low-resource sign-language tools who want to see one early attempt at adding emotion. It does not have the complete pipeline or evaluation needed to support the main claim, so it is not ready for a serious referee process. I would not send it out for review without the missing avatar component and proper controls.","headline":"The paper only evaluates the shared audio encoder on ASR and emotion classification; the avatar generation step has no results or description at all.","tokens_in":2331,"tokens_out":365,"would_cite":false,"duration_ms":22490,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A lightweight shared-encoder model generates emotion-conditioned Nepali sign language avatars from spoken words.","keywords":["Nepali sign language","emotion-conditioned avatars","shared acoustic encoder","automatic speech recognition","multimodal translation","low-resource languages","sign language generation"],"falsifier":"Testing the system on a new set of ten Nepali words spoken by 100 unseen speakers and finding that either accuracy drops below 70 percent or parameter efficiency is lost would falsify the feasibility claim.","tokens_in":2642,"feed_emoji":"","tokens_out":543,"duration_ms":25554,"temperature":0.7,"pith_summary":"This paper presents a proof-of-concept system called NEST-V1 that translates spoken Nepali into sign language avatars while conveying emotion. It uses one acoustic encoder to perform both speech recognition and emotion detection at the same time. The approach reaches over 80 percent accuracy on both tasks with a model small enough for mobile devices. A sympathetic reader would care because most sign language tools ignore emotional tone, and this could help low-resource languages reach the hearing-impaired community faster.","feed_headline":"Shared encoder delivers 81% accurate Nepali speech to emotion sign avatars","feed_subtitle":"22 million parameter model handles recognition and emotion detection together for low-resource sign language output.","key_machinery":"The shared acoustic encoder processes input audio for both automatic speech recognition and emotion classification in one pass.","core_discovery":"The central claim is that a shared acoustic encoder architecture enables simultaneous automatic speech recognition and emotion classification, achieving 81.1 percent ASR accuracy and 79.21 percent emotion recognition accuracy on 600 audio samples while using 37 percent fewer parameters than separate models and totaling only 22.1 million parameters.","pith_inferences":["Similar shared-encoder designs could apply to other low-resource spoken languages needing sign translation.","Future work might test whether adding more emotions improves avatar naturalness without increasing model size.","Integration with existing sign language avatar rendering engines could be a direct next step."],"forward_implications":["The model supports real-time emotion-aware sign language output on edge hardware.","Expansion to larger vocabularies becomes feasible from this baseline.","Emotion conditioning adds expressiveness missing in current sign language systems for low-resource settings."],"fun_headline_variants":["Shared acoustic encoder hits 81% Nepali ASR and 79% emotion","22.1M params enable Nepali speech to emotion-conditioned signs","37% efficiency gain for low-resource Nepali sign language","Nepali words to avatars: simultaneous ASR and emotion at 81%"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The limited set of four words and three emotions in a 600-sample dataset from 50 speakers is sufficient to demonstrate the broader feasibility of the technical approach.","fun_headline_variants_meta":{"raw":{"variants":["Shared acoustic encoder hits 81% Nepali ASR and 79% emotion","22.1M params enable Nepali speech to emotion-conditioned signs","37% efficiency gain for low-resource Nepali sign language","Nepali words to avatars: simultaneous ASR and emotion at 81%"]},"model":"grok-4.3","cost_usd":0.004494,"raw_usage":{"total_tokens":2239,"prompt_tokens":668,"num_sources_used":0,"completion_tokens":73,"cost_in_usd_ticks":44937000,"prompt_tokens_details":{"text_tokens":668,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1498,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":668,"tokens_out":73,"duration_ms":13904,"temperature":1.0,"reasoning_tokens":1498,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-01T00:26:36.933473+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Testing the system on a new set of ten Nepali words spoken by 100 unseen speakers and finding that either accuracy drops below 70 percent or parameter efficiency is lost would falsify the feasibility claim.","supporting_citations":[],"review_version":1}