{"id":"23ae0ad9-776e-4343-8df2-02ded91e123b","arxiv_id":"2411.15598","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A routine CNN benchmark for 14 hand gestures reports AUC 0.83 and recall 0.85 for an undescribed Ours model, with no error bars or code.","lead":"This paper compares standard CNN models for hand gesture recognition and reports that an unspecified model it calls Ours reaches AUC 0.83 and recall 0.85 on a 14 gesture dataset. Its claimed path to more intuitive user interfaces cannot be assessed because the proposed model, code, and data are missing.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim is unfalsifiable because Table 1 reports an undefined 'Ours' model with no architecture, training details, data split, variance, code, or dataset source; the reported AUC/Recall gains cannot be independently checked.","rationale":"The reader's weakest assumption names the lack of statistical grounding (single split, no variance, no significance test) as the load-bearing issue. My concern is adjacent but more fundamental: the 'Ours' model is never described, so even a properly randomized multi-seed comparison could not be run by anyone else without new information from the authors. The two concerns share the same root—under-specified experimental reporting—but the undefined method is what makes the strongest claim non-checkable, while split/variance issues would remain even if the architecture were disclosed. I therefore partially agree with the reader. I do not find any independent support that changes the picture: no code, no data release, no formal verification, and no parameter-free derivation. The manuscript's own text confirms this by describing 'Ours' only as 'our model further optimizes the feature extraction mechanism and classification strategy' without specifics. Thus the REJECT verdict remains appropriate, and my read does not change it.","tokens_in":6632,"tokens_out":2004,"duration_ms":20197,"concrete_test":"Request from the authors the exact architecture and training configuration of 'Ours' (or the code/weights), the precise HG14 split, and the random seed(s). Then independently re-run the full Table 1 comparison across five random seeds under identical preprocessing and report mean +/- std for AUC and Recall for every model. If the code does not reproduce the reported 0.83/0.85, or Ours's AUC overlaps DenseNet's within one standard deviation, the central superiority claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that a model called 'Ours' achieves AUC 0.83 and Recall 0.85 on the HG14 dataset, outperforming VGG16, ResNet50, EfficientNet, and DenseNet (Section IV.B, Table 1). This claim is load-bearing because the entire contribution of the paper rests on it. Yet the manuscript never specifies what 'Ours' is. No architecture diagram for the proposed model, no layer configuration, no training hyperparameters, no loss-function parameters (e.g., the Focal Loss gamma and alpha in Eq. 4), and no description of the train/validation/test split are provided. The dataset, HG14, is described qualitatively but is not cited or linked, and no code or trained weights are released. Consequently, Table 1 cannot be reproduced even in principle: a reader cannot re-run the 'Ours' model, and the 0.04 AUC / 0.06 Recall margins over DenseNet could easily arise from a favorable split, a different preprocessing pipeline, or post-hoc selection among random seeds. The absence of error bars or significance tests compounds this, but the more fundamental problem is that the method itself is undefined. The claim is therefore an assertion about an unspecified system, not a verifiable scientific result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a gesture-recognition system for UI interaction based on convolutional neural networks, claiming to address class imbalance with Focal Loss. It reports experimental results on a dataset called HG14, comparing VGG16, ResNet50, EfficientNet, DenseNet121, and an 'Ours' model, with the proposed model achieving AUC 0.83 and Recall 0.85, surpassing the other models (Table 1). The paper also includes a user-centered evaluation with 500 participants, reporting task success rate, completion time, and System Usability Scale scores. The central technical contribution is an improved CNN for gesture classification, but the manuscript provides almost no architectural or experimental detail for the proposed model.","tokens_in":6965,"tokens_out":2563,"duration_ms":23187,"significance":"If the reported results were fully substantiated, the contribution would be a modest incremental improvement in gesture-recognition accuracy for UI interaction. However, as written, the experimental core is not verifiable: the proposed model is never described, the dataset is not cited or linked, no experimental protocol is given, and no code or trained weights are released. The paper also lacks any variance or significance testing for the reported performance differences. Because the load-bearing claim cannot be independently checked, the significance of the work in its current form is very limited. There are no machine-checked proofs, reproducible code, parameter-free derivations, or falsifiable predictions to offset these gaps.","major_comments":[{"comment":"The central claim of the paper is that the 'Ours' model achieves AUC 0.83 and Recall 0.85, outperforming VGG16, ResNet50, EfficientNet, and DenseNet121 on the HG14 dataset. However, the paper never specifies what 'Ours' is: no architecture diagram, no layer configuration, no description of the proposed feature-extraction or classification modifications, and no training procedure. A reader cannot reconstruct the model from the manuscript, and therefore cannot reproduce the reported results. This is not a minor omission; it is the central empirical claim of the paper, and it rests entirely on an undefined system.","section":"Section IV.B, Table 1"},{"comment":"The experimental protocol is unspecified. The paper does not state how the HG14 dataset was split into training, validation, and test sets, whether cross-validation was used, or how hyperparameters were selected. Table 1 reports single point estimates of AUC and Recall with no error bars, confidence intervals, or significance tests. The differences between models (e.g., AUC 0.79 for DenseNet versus 0.83 for 'Ours') are small, and without any measure of variance they could plausibly arise from a favorable split or random seed. The comparison therefore does not support the claim of superiority.","section":"Section IV.B, Table 1"},{"comment":"The dataset HG14 is described qualitatively in terms of gestures and environments, but is never cited, linked, or otherwise identified by a source or version. The number of images, class distribution, and exact acquisition conditions are not given. Since class imbalance is a stated motivation for using Focal Loss, the class distribution is directly relevant to the method, and its absence prevents the reader from assessing whether the loss-function choice is appropriate or how the reported metrics should be interpreted.","section":"Section IV.A"},{"comment":"The paper states that Focal Loss is used to address class imbalance, and presents the standard Focal Loss equation, but never reports the values of the balancing factor alpha_t or the focusing parameter gamma that were used in the experiments. Without these hyperparameters, the loss-function description is not actionable, and the reader cannot determine whether the claimed benefit of Focal Loss was actually realized or how sensitive the results are to these choices.","section":"Section III, Eq. (4)"},{"comment":"The user-centered evaluation reports a task success rate of 87%, an average completion time of 6 seconds, and a SUS score of 82, and states that 'statistical comparisons across the five models showed that our proposed model outperformed others in all key metrics'. However, the paper provides no statistical test, no variance measures, no description of how the five models were tested with participants, and no data beyond these point estimates. The claim of statistical superiority is therefore unsupported. Additionally, Figure 3, which is referenced as showing these comparisons, is not present in the manuscript.","section":"Section IV.C"}],"minor_comments":[{"comment":"Table 1 lists models as 'VGG16', 'RESNET', 'EFICIENT', 'DenseNet', and 'Ours', but the text says five models were selected including a 'CNN Baseline'. The table appears to omit the CNN Baseline, and the naming is inconsistent (RESNET and EFICIENT are not spelled out, while DenseNet is). This makes the table difficult to interpret.","section":"Section IV.B"},{"comment":"Equation (1) is typeset in a garbled manner; the mathematical expression for the convolution output is not readable as printed. The equation should be re-typeset with clear notation for the convolution operation, activation function, and bias.","section":"Section III, Eq. (1)"},{"comment":"Figure 1 ('Overall architecture diagram'), Figure 2 ('Evaluation index rising chart'), and Figure 3 ('User-Centered Evaluation') are all referenced in the text but appear to be missing or are not properly embedded in the manuscript. A paper cannot be evaluated without its main figures, especially since Figure 1 is supposed to illustrate the proposed architecture.","section":"Figures"},{"comment":"Several references are unrelated to gesture recognition or UI interaction (e.g., references to survival prediction, time-series event sequences, and RAG systems), and others appear to be arXiv preprints with no clear connection to the claims in the paper. The related-work section would benefit from a more focused and relevant set of citations.","section":"References"},{"comment":"The abstract states that the model is 'optimized', but no optimization method, hyperparameter tuning procedure, or architectural search is described in the paper. Phrases such as 'significant advancement' and 'paves the way for a new era' are also not supported by the reported experimental evidence.","section":"Abstract"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be substantially incomplete: key figures are missing, the proposed model is never specified, the dataset is not cited, and the experimental reporting is limited to single point estimates. The combination of an undefined central model and unreported experimental protocol means the load-bearing claim cannot be checked even in principle. In addition, the reference list contains many apparently unrelated arXiv preprints, which raises concerns about citation relevance. I would recommend rejection on scientific grounds, and the paper would need a complete rewrite with a fully specified model, a reproducible experimental setup, and accessible data before it could be considered for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is not ready for peer review. The central claim rests on an undefined 'Ours' model in Table 1—no architecture, no training details, no dataset split, no code—so the AUC/Recall gains over the baselines cannot be independently checked. That kills it as a scientific contribution.\n\nWhat it does well: the standard CNN pipeline and the Focal Loss equation are presented correctly, and the user evaluation (500 participants, SUS scores) shows the authors care about usability. But those parts are either textbook or too thinly reported to use.\n\nThe soft spots are load-bearing. First, the 'Ours' model is just a label; the stress-test note is exactly right. Second, the dataset HG14 is described qualitatively with no citation or link, so even the baseline numbers are not reproducible. Third, Table 1 gives single point estimates without variance or significance tests; the margin over DenseNet (0.04 AUC) could easily be noise. Fourth, the references include many unrelated arXiv preprints on NLP, recommendation, and RAG, which looks like citation padding and suggests a careless literature review.\n\nThe user study is also problematic: we get headline numbers (87% success, 6 seconds, SUS 82) but no raw data, no confidence intervals, and the promised statistical comparisons across models are absent.\n\nThis is not a case of a promising paper with one weak section; the central method is undefined, so there is nothing for a referee to evaluate. I would desk-reject it, not send it to reviewers. If the authors want another chance, they need to specify the model, release code and data, and report proper statistics. As it stands, the only value is as a cautionary example of irreproducible reporting.","headline":"A gesture recognition paper whose central result is uncheckable: the 'Ours' model is never described, so the reported gains cannot be verified.","tokens_in":7397,"tokens_out":3352,"would_cite":false,"duration_ms":30207,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An optimized CNN with Focal Loss is reported to reach AUC 0.83 and recall 0.85 on the HG14 gesture dataset, beating four standard CNN baselines.","keywords":["gesture recognition","deep learning","convolutional neural network","UI interaction","human-computer interaction","Focal Loss","HG14 dataset"],"falsifier":"A faithful reproduction requires the architecture and training details the paper omits; with those in hand, retrain the five models on several independent stratified splits of HG14 and report AUC and recall with confidence intervals. The claim stands if 'Ours' beats DenseNet by more than split-to-split variation, and falls if the margin disappears or the model cannot be reproduced from the paper.","tokens_in":6408,"feed_emoji":"✋","tokens_out":18722,"duration_ms":157802,"temperature":0.7,"pith_summary":"The paper sets out to show that a convolutional neural network (an image-classifying network that learns visual features from pixels) trained with Focal Loss can recognize fourteen hand gestures from the HG14 dataset well enough to drive a user interface, outperforming four standard CNN architectures. If true, this would give interface designers a practical route to touchless control: users could select, scroll, and trigger commands by moving their hands instead of using a keyboard, mouse, or voice, which matters in noisy or hands-busy settings. The claimed numbers are an AUC (a standard ranking-accuracy score) of 0.83 and a recall (the fraction of true gestures caught) of 0.85 for the authors' 'Ours' model, compared with 0.72/0.73 for VGG16, 0.75/0.77 for ResNet, 0.78/0.79 for EfficientNet, and 0.79/0.78 for DenseNet. A 500-participant user study reported 87% task success, a 6-second average task time, and a System Usability Scale score of 82.","feed_headline":"Gesture CNN scores 0.83 AUC, 0.85 recall for UI control","feed_subtitle":"Trained with Focal Loss on the HG14 hand-gesture set, it outpaces VGG16, ResNet, EfficientNet, and DenseNet.","key_machinery":"The load-bearing machinery is a standard CNN classification pipeline combined with Focal Loss as the training objective. Each gesture image is converted to grayscale, resized to $224\\times224$ pixels, and normalized to pixel values in $[0,1]$; convolution with ReLU activation and pooling extracts features, the feature map is flattened, and a fully connected layer with Softmax produces class probabilities. Focal Loss, with its modulating factor $(1-p_t)^\\gamma$ and class-weighting term $\\alpha_t$, down-weights easy examples so training concentrates on hard, underrepresented gesture classes, which the paper claims corrects class imbalance across the 14 categories. The evidence for the central claim is the five-model comparison table, where the 'Ours' row is the reported winner.","core_discovery":"On its own terms, the central discovery is that an optimized CNN pipeline—grayscale preprocessing, resizing to $224\\times224$ pixels, normalization, convolutional feature extraction, pooling, and a fully connected classifier—trained with Focal Loss achieves the best reported accuracy on the HG14 hand-gesture benchmark. The reported result is AUC 0.83 and recall 0.85, ahead of DenseNet (0.79/0.78), EfficientNet (0.78/0.79), ResNet (0.75/0.77), and VGG16 (0.72/0.73). The authors attribute the gain to richer feature extraction, multi-layer information fusion, and better overfitting control, and they connect it to a user study in which the system reached an 87% task success rate, an average completion time of 6 seconds, and a System Usability Scale score of 82, with higher error rates on complex gestures.","pith_inferences":["The method section never specifies the architecture behind the 'Ours' model—no layer counts, hyperparameters, or training schedule—so the reported 0.83/0.85 cannot yet be reproduced or attributed to a mechanism; publishing that specification is the minimal test of the claim.","Because the paper does not ablate Focal Loss against plain cross-entropy on the same architecture, the gain cannot specifically be credited to Focal Loss; swapping the loss while holding the 'Ours' model fixed would settle that.","The four-point AUC margin over DenseNet comes from one run per model; a multi-split benchmark with confidence intervals would show whether the ranking is stable or a product of the single split.","Adding a mouse-or-touch control condition to the user study would quantify whether the gesture layer reduces the learning curve or simply matches existing input methods."],"forward_implications":["The paper's user results indicate that selection, scrolling, and command triggering can be completed successfully through the gesture system.","The paper's model ordering implies that residual, compound-scaled, and dense architectures are a stronger starting point than plain VGG-style stacks for this gesture task.","The paper claims Focal Loss keeps performance stable across rare gesture classes, making the recipe transferable to other imbalanced interaction datasets.","The reported System Usability Scale score of 82 and 87% task success are taken by the authors as evidence that users need little instruction to operate the interface."],"supporting_citations":[],"fun_headline_variants":["CNN with Focal Loss: 0.83 AUC, 0.85 recall for UI gestures","Hand-gesture CNN beats DenseNet with 0.83 AUC, 0.85 recall","Focal Loss lifts CNN gesture recall to 0.85 for seamless UI","Optimized CNN tops 0.83 AUC on gesture UI, 87% task success"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim assumes that the single reported AUC and recall values in Table 1 are representative—that the comparisons would not change if the data were split differently or the models were retrained with other random seeds—since no variance, significance test, or split description is given.","fun_headline_variants_meta":{"raw":{"variants":["CNN with Focal Loss: 0.83 AUC, 0.85 recall for UI gestures","Hand-gesture CNN beats DenseNet with 0.83 AUC, 0.85 recall","Focal Loss lifts CNN gesture recall to 0.85 for seamless UI","Optimized CNN tops 0.83 AUC on gesture UI, 87% task success"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001032,"raw_usage":{"total_tokens":4375,"prompt_tokens":1000,"completion_tokens":3375,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":3278}},"tokens_in":616,"tokens_out":3375,"duration_ms":23231,"temperature":1.0,"reasoning_tokens":3278,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:06:04.143055+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A faithful reproduction requires the architecture and training details the paper omits; with those in hand, retrain the five models on several independent stratified splits of HG14 and report AUC and recall with confidence intervals. The claim stands if 'Ours' beats DenseNet by more than split-to-split variation, and falls if the margin disappears or the model cannot be reproduced from the paper.","supporting_citations":[],"review_version":1}