{"id":"ce4f51a4-e89c-4eac-bae8-0af3878ea15c","arxiv_id":"2501.10666","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A CNN-LSTM model with MFCC features classifies seven speech emotions on a SAVEE/RAVDESS subset at 61.07% accuracy, with anger (75.31%) and neutral (71.70%) recognized best.","lead":"This paper trains a CNN-LSTM neural network to recognize seven emotions from voice recordings, reaching 61% accuracy on a combined SAVEE and RAVDESS test set. It is a small engineering study that reuses standard methods and datasets, so its value is as a baseline rather than a scientific breakthrough.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported 61.07% accuracy is unverifiable because no train/test split is defined and the test set appears to have been used to monitor training (Fig. 4), so speaker overlap or test-set overfitting could inflate the result.","rationale":"The reader's weakest assumption correctly identifies the missing data partitioning and the risk of speaker overlap. This stress-test agrees that this is the central flaw, but adds a second concrete leak: the paper explicitly plots test-set loss during training (Fig. 4), indicating the test set was used for monitoring, which alone can bias the reported accuracy even without speaker overlap. Both issues threaten the validity of the 61.07% claim, but they are verification concerns rather than demonstrations that the result is false. The paper provides no code, no error bars, and no hyperparameter search description, so the claim is plausible but not currently supported. A conditional verdict is appropriate: the authors should provide the split and re-run with a speaker-disjoint, validation-based protocol. No stronger verdict is justified because the core method—CNN-LSTM with MFCC—is standard and the reported accuracy is within the range of prior SER work, so the result could easily be true under a proper evaluation.","tokens_in":5073,"tokens_out":3128,"duration_ms":34449,"concrete_test":"Request the authors' exact file-level train/test indices, or re-implement the pipeline from the raw SAVEE/RAVDESS files using a deterministic seed. Then re-run the same CNN-LSTM architecture with a strict speaker-disjoint split: e.g., for SAVEE hold out one of the four speakers entirely, and for RAVDESS hold out six of the 24 actors; train on the remaining data, use a separate validation set for early stopping, and report test accuracy on the held-out speakers. If the resulting accuracy is substantially below 61.07% (e.g., below 50%), the original number was inflated by speaker leakage or test-set overfitting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the 61.07% test-set accuracy reported in the Abstract and Section 3. For that number to be an unbiased estimate of generalization, the test set must be completely held out and never used for model selection. The paper never specifies how the 2,459 samples are partitioned into training and test sets (§2.1, §2.3). It also says 'the loss for the test set tends towards stability after the epochs reaching approximately 200' (Fig. 4), implying the test set was used during training, likely for early stopping or hyperparameter choice. If the same speakers (SAVEE: DC, JE, JK, KL; RAVDESS actors) appear in both training and test, CNNs can memorize speaker-specific vocal tract and pitch characteristics, recognizing the speaker rather than the emotion. The removal of calm utterances because of an 'unsatisfactory training result' (§2.1) is an outcome-based decision that alters the task after seeing results, further biasing the reported accuracy. Without a file-level split description and confirmation that the test set was never used for model selection, the central number cannot be reproduced or trusted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a hybrid CNN-LSTM architecture for speech emotion recognition on a combined subset of the SAVEE and RAVDESS datasets. The pipeline extracts MFCC, pitch, and other statistical features using Librosa, then feeds spectrogram-style image inputs through four convolutional layers and three LSTM layers. The authors report a test-set accuracy of 61.07% across seven emotions (calm is excluded from the original RAVDESS labels), with per-class accuracies from 38.33% (disgust) to 75.31% (anger), and they interpret the confusion matrix as evidence that emotion properties influence classifiability. The paper includes no comparison to baselines, no error bars or repeated runs, and no explicit description of the train/test split.","tokens_in":5302,"tokens_out":3481,"duration_ms":33844,"significance":"If the reported accuracy were established through a clean, reproducible evaluation protocol, the result would be an incremental data point for the common CNN-LSTM approach to speech emotion recognition: it would confirm that MFCC-based features plus a convolutional-recurrent stack can reach roughly 60% on a seven-class SAVEE/RAVDESS mix, with anger and neutral being relatively easy classes and disgust being hard. The paper's strength is that it uses two widely available public datasets and reports a full confusion matrix, which facilitates future comparisons. However, the contribution is primarily empirical, and the current evaluation is too underspecified to support the headline number; the absence of baselines also prevents the paper from demonstrating any advantage attributable to the hybrid architecture.","major_comments":[{"comment":"The manuscript never specifies how the 2,459 samples are partitioned into training and test sets, nor whether the partition prevents the same speaker from appearing in both sets; because the reported 61.07% accuracy is the central empirical claim, this missing information leaves the result unreproducible and the risk of speaker-identity leakage unaddressed.","section":"Section 2.1 and Section 2.3"},{"comment":"The statement that 'the loss for the test set tends towards stability after the epochs reaching approximately 200' indicates that the test set was evaluated during training, presumably for early stopping or hyperparameter selection; if the test set influenced any training decision, the reported test accuracy is optimistically biased, and the claim that overfitting was 'reduced as much as possible' is not supported.","section":"Figure 4 and Section 3"},{"comment":"The removal of calm utterances from the dataset because of an 'unsatisfactory training result' is an outcome-based selection: the task is altered after observing performance, which invalidates the reported seven-class accuracy as an estimate of generalization to the original seven-emotion label set and weakens the conclusion that the model performs well on 'common emotions'.","section":"Section 2.1"},{"comment":"The overall accuracy of 61.07% does not match the unweighted mean of the per-class accuracies (approximately 59.8%), and no class sizes or weighted aggregation formula are given; without this information the reader cannot verify the headline number or interpret the per-class rates consistently.","section":"Table 1"},{"comment":"No comparison to any baseline (e.g., a CNN-only model, an LSTM-only model, or a standard MFCC plus shallow classifier) is provided, and no cross-validation or repeated runs with error bars are reported; consequently, the paper does not demonstrate that the CNN-LSTM architecture is responsible for the observed accuracy rather than the feature set or dataset construction choices.","section":"Section 3"}],"minor_comments":[{"comment":"The architecture description gives the number of convolutional and LSTM layers but not the number of filters, kernel sizes, or LSTM units, so the architecture cannot be reproduced from the text.","section":"Section 2.2"},{"comment":"The confusion matrix is described in the text but the axes are not labeled in the figure caption; please add class names and indicate whether the entries are counts or percentages.","section":"Figure 5"},{"comment":"The feature-heuristic selection is stated only as 'heuristics are implemented to score each feature'; without the scoring criterion or the number of features retained, this step is not reproducible.","section":"Section 2.1"},{"comment":"The dataset references point to Kaggle URLs without version identifiers or access dates; please provide a formal citation or a permanent repository identifier.","section":"References [7] and [8]"},{"comment":"The term 'comprehensively' is undefined; specify whether the 61.07% figure is a macro-average, micro-average, or class-weighted accuracy.","section":"Abstract and Section 3"}],"recommendation":"major_revision","confidential_remarks":"The paper reads as a course-style technical report rather than a fully developed research article. The central empirical claim is plausible but currently unverifiable because of the missing split description and the apparent use of the test set during training. If the authors can provide a file-level split protocol, rerun the evaluation with proper speaker-disjoint splits and cross-validation, and add at least one baseline comparison, the work could become a publishable short application note. In its current form, the evaluation is too underspecified to support acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this paper reports 61.07% accuracy on a 7-class SAVEE/RAVDESS subset with a CNN-LSTM on MFCC, but the number is not trustworthy as reported. The train/test split is never defined, and Figure 4 suggests the test set was used to monitor training. That's a load-bearing issue, not a cosmetic one.\n\nWhat's genuinely here: the paper combines selected parts of two public datasets, drops calm, and runs a standard CNN-LSTM with four conv layers and three LSTM layers. The per-class results are the only real content: anger 75.31%, neutral 71.70%, disgust 38.33%. The confusion-matrix observation that negative emotions (sad, fear, disgust) mix with each other is plausible and backed by the matrix, even if the subsequent generalization about emotion semantics is speculative. The writing is clear and the feature-extraction description, while high-level, is coherent.\n\nThe soft spots are significant. First, no data partitioning details: there's no statement about random vs. stratified split, no speaker-independent split, and no indication of whether the same speakers appear in both train and test. With two small datasets, speaker overlap is a real risk and would let the model exploit speaker identity. Second, the test-loss curve in Fig. 4 is described as stabilizing after about 200 epochs, which implies the test set was observed during training, likely for early stopping. That compromises the reported accuracy as an unbiased estimate. Third, calm was removed after seeing an \"unsatisfactory training result\"—this is post-hoc pruning, which alters the task and biases the reported numbers upward. Fourth, there are no error bars, no cross-validation, no baseline comparisons with existing SER systems. The feature selection \"heuristics\" are also left unspecified.\n\nOn the citation side, the paper cites generic SER work but misses the many prior CNN-LSTM SER papers, so it doesn't position itself relative to the actual state of the art.\n\nThe math is not the problem; the reporting is. The paper is internally coherent and the architecture is reasonable, but the empirical claim cannot be reproduced or verified from the text.\n\nThis is a paper for someone who wants a hint of what a CNN-LSTM might get on this dataset mix. It is not a citable result. I would desk reject it, with clear guidance on what would make it salvageable: a speaker-independent split, a written hold-out protocol, error bars or cross-validation, and a baseline comparison.\n\nRecommendation: no refs. Not ready for peer review as is.","headline":"A plausible 61% SER result that is unverifiable as reported, because the data split is undocumented and the test set appears to have been used for model selection.","tokens_in":5807,"tokens_out":3422,"would_cite":false,"duration_ms":34500,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A CNN-LSTM trained on MFCC and spectral features from 2,459 acted utterances classifies seven speech emotions with 61.07% accuracy, with anger and neutral most reliable.","keywords":["speech emotion recognition","CNN-LSTM","MFCC","SAVEE","RAVDESS","audio classification","emotion confusion","spectral features"],"falsifier":"Run the same CNN-LSTM on the same 2,459 utterances using a leave-one-speaker-out split, where no speaker appears in both training and test. If overall accuracy falls well below 61.07%, the original figure partly reflected speaker identity rather than emotion recognition.","tokens_in":4855,"feed_emoji":"🎙️","tokens_out":6290,"duration_ms":61105,"temperature":0.7,"pith_summary":"The paper tries to show that a hybrid CNN-LSTM can recognize seven emotions from speech with about 61% accuracy using MFCC and related spectral features on a combined acted-speech dataset. It matters because speech emotion detection remains an open problem, and a working seven-way classifier with per-emotion breakdown gives a concrete baseline for which emotions are easy or hard to recognize. The author's main empirical finding is that recognition quality is emotion-dependent: anger and neutral are detected well above the average, while disgust falls well below it, and the confusion matrix shows negative emotions mixing with each other and surprise straddling positive and negative contexts. The paper argues this reflects the acoustic properties of the emotions themselves rather than only model limitations.","feed_headline":"CNN-LSTM reads seven emotions from voice at 61%","feed_subtitle":"Anger and neutral are clearest at roughly 75% and 72%; disgust lags at 38%.","key_machinery":"The carrying mechanism is the CNN-LSTM pipeline applied to spectral image inputs. Each utterance is converted to a waveplot or spectrogram and a feature vector containing MFCCs, pitch statistics, formant frequencies, and peak frequency information; four convolutional layers extract local spatial patterns, three LSTM layers model the temporal sequence of those features, and a dense layer outputs one of seven emotion classes. The MFCC step is the load-bearing representation because it maps audio to a Mel-frequency axis that mirrors human pitch perception, which is what makes the classifier's emotion distinctions possible.","core_discovery":"The paper's central claim is that a hybrid CNN-LSTM, trained on MFCCs and related spectral features extracted from 2,459 acted utterances drawn from SAVEE and RAVDESS, classifies seven emotions with 61.07% overall test accuracy, with anger at 75.31% and neutral at 71.70%, and disgust at only 38.33%. The author further argues that these differences are not random: emotions that are frequent and acoustically distinct, such as anger, happy, and neutral, are misclassified least, while context-dependent emotions like surprise are confused with both positive and negative categories, and negative emotions such as sad, fear, and disgust are often mixed with one another. The paper presents this as evidence that emotion recognition accuracy depends on the properties of the emotion itself, not solely on the classifier.","pith_inferences":["Inference: a speaker-disjoint split would likely lower the 61.07% score, because acted corpora like these include highly recognizable voices; the paper's split is unspecified.","Inference: the reported roughly 10% gain from separating male and female voices suggests pitch normalization across gender would make emotion recognition more robust and is a testable next step.","Inference: the paper's confusability story could be checked by training the same CNN-LSTM on continuous valence or arousal ratings; if surprise sits at the boundary, the discrete seven-way task is the reason for its mixed misclassifications."],"forward_implications":["A seven-way speech emotion classifier can reach 61.07% accuracy on a combined acted dataset of 2,459 utterances, providing a concrete baseline for later work.","Anger and neutral are anchor emotions in this feature space, both above 70%, so they can be detected with reasonable confidence in practical systems.","Disgust is the weak class at 38.33%; a deployed system should either gather more disgust-specific data or accept confusion with other negative emotions.","The confusion matrix implies that surprise should not be treated as a single discrete label, since it is split between positive and negative interpretations.","The next improvement target is the negative-emotion cluster (sad, fear, disgust), not the overall architecture."],"supporting_citations":[{"why":"Early GMM-UBM baseline that shows the four-emotion predecessor this work extends.","marker":"[5]"},{"why":"GMM-DNN hybrid whose temporal-sequence weakness motivates the LSTM component.","marker":"[6]"},{"why":"Data source for the SAVEE utterances that anchor the seven emotion labels.","marker":"[7]"},{"why":"Data source for the RAVDESS speech files that make up the rest of the 2,459-sample set.","marker":"[8]"},{"why":"Supplies the audio feature extraction that turns each utterance into spectral input.","marker":"[9]"},{"why":"Defines MFCCs and the Mel-scale mapping used as the core feature.","marker":"[10]"},{"why":"LRCN architecture template for combining CNN spatial features with LSTM temporal modeling.","marker":"[11]"},{"why":"Prior anger and disgust classification work used to explain why disgust is confusable.","marker":"[12]"}],"fun_headline_variants":["Voice emotion AI: 61% accuracy, anger clearest at 75%","CNN-LSTM spots anger and neutral better than other emotions","Mixed emotions: MFCC + CNN-LSTM hits 61% on seven feelings","Surprise is hardest: CNN-LSTM scores 61% on speech emotions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported accuracy presupposes that the training and test split separates speakers and recording sessions and that class imbalance does not distort the average; the paper does not describe this split, so the 61.07% score could be inflated by the model recognizing voices instead of emotions.","fun_headline_variants_meta":{"raw":{"variants":["Voice emotion AI: 61% accuracy, anger clearest at 75%","CNN-LSTM spots anger and neutral better than other emotions","Mixed emotions: MFCC + CNN-LSTM hits 61% on seven feelings","Surprise is hardest: CNN-LSTM scores 61% on speech emotions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000326,"raw_usage":{"total_tokens":1840,"prompt_tokens":972,"completion_tokens":868,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":787}},"tokens_in":588,"tokens_out":868,"duration_ms":7520,"temperature":1.0,"reasoning_tokens":787,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:01:24.736005+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same CNN-LSTM on the same 2,459 utterances using a leave-one-speaker-out split, where no speaker appears in both training and test. If overall accuracy falls well below 61.07%, the original figure partly reflected speaker identity rather than emotion recognition.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Early GMM-UBM baseline that shows the four-emotion predecessor this work extends."},{"cited_title":"2015 Is virtual reality emotionally arousing? Investigating five emotion inducing virtual park scenarios International journal of human-computer studies 82 pp 48-56","cited_arxiv_id":null,"evidence_quote":"GMM-DNN hybrid whose temporal-sequence weakness motivates the LSTM component."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Data source for the SAVEE utterances that anchor the seven emotion labels."},{"cited_title":"2015 Performance evaluation of different support vector machine kernels for face emotion recognition 2015 SAI Intelligent Systems Conference (IntelliSys) pp 804-806","cited_arxiv_id":null,"evidence_quote":"Data source for the RAVDESS speech files that make up the rest of the 2,459-sample set."},{"cited_title":"2015 Improved emotion recognition using GMM-UBMs 2015 International Conference on Signal Processing and Communication Engineering Systems pp 53-57","cited_arxiv_id":null,"evidence_quote":"Supplies the audio feature extraction that turns each utterance into spectral input."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines MFCCs and the Mel-scale mapping used as the core feature."},{"cited_title":"2015 librosa: Audio and music signal analysis in python Proceedings of the 14th python in science conference Vol 8 pp 18-25","cited_arxiv_id":null,"evidence_quote":"LRCN architecture template for combining CNN spatial features with LSTM temporal modeling."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior anger and disgust classification work used to explain why disgust is confusable."}],"review_version":1}