{"id":"0b3faf99-eb6d-4274-9e15-810c599907ed","arxiv_id":"2501.08758","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Combining PhoBERT features with scores from an expanded Vietnamese SentiWordNet achieves F1 of 0.95 and 0.94 on two review datasets, but the lexicon expansion step is not fully defined.","lead":"This paper extends a Vietnamese sentiment lexicon and combines it with the PhoBERT language model to classify product reviews as positive or negative. The authors report strong F1 scores on two benchmarks, but the expansion method is under-specified and the evaluation has several confounds.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"VLSP 2016 is described as a three-class task (positive/negative/neutral) but all reported results and Table 1 are binary; if neutral labels were dropped, the claimed comparison to published baselines is not on the same task, so the headline F1 gains are unsubstantiated.","rationale":"The reader's weakest_assumption is the undefined distance in Section 4.1 Step 3. That is a real reproducibility gap: Equations 5 and 6 depend on 'average distance', but no metric is defined, and the threshold T is unspecified. However, this gap could be repaired by the authors supplying a definition or code, and it does not by itself invalidate the measured numbers. The binary/three-class discrepancy is more load-bearing because it questions whether the reported comparison is to the same task at all. Section 5.1 declares three classes; Table 1 omits neutral; all metrics are binary; and no handling of neutral is described. A quick dataset check can settle it. I therefore keep the reader's REJECT verdict, but my primary concern differs from the reader's stated weakest assumption, hence partial agreement.","tokens_in":9758,"tokens_out":7062,"duration_ms":68149,"concrete_test":"Download the official VLSP 2016 dataset and count labels; then inspect the preprocessing script behind Table 1 to see how 'neutral' is handled. If neutral reviews exist and were removed, rerun CombViSA and the cited baselines on the full three-class split under the official protocol (or rerun baselines on the same binary subset). If CombViSA's 3-class F1 does not exceed the baselines, the headline claim fails; if neutral counts are zero, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is that the reported F1 numbers measure the same classification problem as the baselines. Section 5.1 explicitly states VLSP 2016 is 'categorized into three sentiment classes: negative, positive, and neutral', yet Table 1 lists only positive and negative counts (train 20,439/20,267; test 5,000/5,000, summing to 50,706, not the stated 50,760), and Tables 3 and 4 report only binary-class precision/recall/F1. The paper never says whether neutral reviews were discarded, merged with a bipolar label, or otherwise excluded. The baselines cited from [30] and [31] are results on the official VLSP 2016 task, which is three-class. If the proposed model was trained or evaluated on only positive/negative reviews, its F1 of 0.95 is not comparable to those baselines, and the abstract's claim of 'excellent performance in comparison to other models' is not established. This is a direct validity threat to the central empirical claim, and it is checkable from the dataset files, unlike the undefined distance in Section 4.1 Step 3, which mainly blocks reproducibility.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Vietnamese sentiment analysis system (CombViSA) that combines PhoBERT-V2 representations with lexical-semantic features from an expanded Vietnamese SentiWordNet. The extension procedure (Section 4.1) selects positive and negative seed sets from SentiWordNet using a threshold T, adds synonyms and antonyms from Vietnamese dictionaries, and computes PosScore/NegScore for each new word via an average-distance formula. The resulting scores are used by Algorithm 1 to build PosVec/NegVec vectors, which are concatenated with PhoBERT-V2 features and passed through RCNN and feedforward layers for classification. Experiments on VLSP 2016 and AIVIVN 2019 report F1 scores of 0.95 and 0.94 for CombViSA, outperforming the cited baselines.","tokens_in":9970,"tokens_out":3472,"duration_ms":37184,"significance":"If the claims are correct, the paper makes an incremental but potentially useful contribution: it demonstrates that lexicon-based sentiment features can add modest gains over a strong pretrained language model on Vietnamese review classification, and it attempts to address the small coverage of an existing Vietnamese SentiWordNet. The proposed resource could be of value to downstream Vietnamese sentiment analysis. However, the paper's strengths are limited by serious reproducibility gaps: the distance function in the core extension step is undefined, the threshold T is not reported, and the evaluation setup is inconsistent with the described task. There is no released code or resource, so the claimed gains cannot be verified without additional detail.","major_comments":[{"comment":"The method computes dpos and dneg as the 'average distance' from w to the words in sets P and N, but no distance function over words is ever defined. Because these distances directly determine PosScore and NegScore, and hence the SWVec features used by the model, the core contribution cannot be reproduced or independently validated until the metric is specified (for example, WordNet path similarity, graph distance in VCL, or cosine distance in a pretrained embedding space). Please define the metric explicitly and state how it is computed.","section":"Section 4.1, Step 3, Equation (5)"},{"comment":"The VLSP 2016 dataset is described in Section 5.1 as three-class ('negative', 'positive', and 'neutral'), but Table 1 and all reported results in Tables 3 and 4 are binary. If neutral reviews were discarded, the comparison to the cited baselines SA1/SA2 [31] and the results from [30] is not valid because those baselines were evaluated on the official three-class VLSP 2016 task. Moreover, the counts in Table 1 sum to 50,706, not the stated 50,760. Please state exactly how neutral labels were handled, correct the arithmetic, and either report three-class results or compare against binary-class baselines on the same subset.","section":"Section 5.1, Table 1, Tables 3-4"},{"comment":"The threshold T controls the construction of the seed sets P and N, which in turn generate the features used for classification. The text says T is 'a constant determined through experimentation,' but its value is never reported, and no separation between validation and test data is described. If T was tuned on the same datasets used for evaluation, the reported F1 scores are optimistically biased. Please report the chosen value of T and describe a validation procedure that keeps the test folds untouched during tuning.","section":"Section 4.1, Step 1; Section 5.4"},{"comment":"The discussion paragraph states that 'the best result of the proposed model is 0.92 in terms of F1-score,' but Table 3 reports CombViSA F1 = 0.95 on VLSP 2016 and Table 4 reports F1 = 0.94 on AIVIVN 2019. This inconsistency, together with the missing precision and recall values for SA1 and SA2 in Table 3, undermines confidence in the numerical results. Please correct the text or the tables and report complete metrics for all baselines.","section":"Section 5.4, Tables 3 and 4"}],"minor_comments":[{"comment":"Equation (5) is typeset ambiguously: it should be PosScore = dpos / (dpos + dneg). Please fix the equation and add parentheses.","section":"Section 4.1, Equation (5)"},{"comment":"The sentence 'The Error! Reference source not found. depicts the architecture of our proposed model' contains an unresolved cross-reference; please insert the correct figure number.","section":"Section 4, first paragraph"},{"comment":"The phrase 'As shown in Table and Table' uses unresolved cross-references, and the text states the average comment length for VLSP 2016 is 23.39 while Table 2 reports 25.34. Please correct these inconsistencies.","section":"Section 5.4"},{"comment":"The algorithm relies on VSNO and DVSNO, and on the notion of a 'reverse pattern,' but none of these is formally defined. Please define the normalization operators and give a finite specification of the reversal patterns used in Step 2.","section":"Algorithm 1 and Section 4.2"},{"comment":"Reference [31] is malformed: it begins with a raw author list and does not follow standard citation formatting. Also, the related-work section contains minor typos such as 'Native Bayes' for 'Naive Bayes' and 'the evelopment' for 'the development.'","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a real need—small coverage of Vietnamese SentiWordNet—and the reported gains are plausible, but the current manuscript does not provide enough information to verify the method or the comparison. The undefined distance function, unreported threshold T, and the binary-versus-three-class mismatch on VLSP 2016 are all fixable in principle, but they are not cosmetic: they directly affect the validity of the central empirical claim. If the revision does not resolve these points, I would not recommend acceptance. The novelty over [29] is modest, and the paper would benefit from releasing the expanded lexicon and code to make the contribution reusable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here is my honest read. The paper has a useful empirical hypothesis: the existing Vietnamese SentiWordNet is tiny (1,017 words), and expanding it with synonym/antonym propagation and feeding the result as lexical features into a PhoBERT-V2 pipeline improves review classification. On AIVIVN 2019 the reported gain is 5 F1 points over PhoBERT-V2 (0.94 vs 0.89). That is a real, practically meaningful increment if it reproduces. The negation handling in Algorithm 1 is a sensible detail.\n\nThe problems, in order. First, Equation 5 uses dpos and dneg—average distances from a word to the positive and negative seed sets—but no distance function is ever defined. This is a serious omission, not a minor one: the entire expanded lexicon, and every SentiWordNet feature vector, depends on an unspecified quantity. The resource cannot be reconstructed. Second, the VLSP 2016 evaluation is likely apples-to-oranges. Section 5.1 states the data are three-class (positive, negative, neutral), but Table 1 and Table 3 are binary only, with no statement that neutral was dropped or merged. The published baselines [30] and [31] were evaluated on the official three-class task, so the 0.95 F1 headline is not comparable as reported. Third, the threshold T is tuned by experimentation, never reported, and if tuned on the same test sets this leaks into the evaluation. At minimum we need a dev split and the chosen value. Fourth, the phoBERT-V2 vs phoBERT-V2+ExtViSentiWordNet comparison changes the post-model architecture (top-four-layer concatenation plus MLP vs combined feature vector with MLP), so the lexicon contribution is not cleanly isolated. Fifth, the manuscript has enough small inconsistencies—Table 1 summing to 50,706 instead of 50,760, a 'best result 0.92' that appears in no table, broken cross-references—to corrode trust even before the bigger issues.\n\nThis is not a bad idea. The AIVIVN result is plausible, and expanding the resource is legitimate. But as written the central numbers are not established, and the paper would need heavy revision before I would accept it. For peer review, though, I would rather a serious referee see it than desk-reject it outright: the flaws are fixable, and the underlying question—whether a better SentiWordNet can help a strong PLM on Vietnamese sentiment—is worth a clean answer.","headline":"The paper's lexicon-plus-PLM combination is plausible and shows a consistent gain on AIVIVN, but the undefined distance metric and the binary-vs-three-class VLSP comparison leave the headline claims unsubstantiated.","tokens_in":10522,"tokens_out":4844,"would_cite":false,"duration_ms":46773,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Expanding Vietnamese SentiWordNet lifts review classification F1 to 0.95.","keywords":["Vietnamese sentiment analysis","SentiWordNet expansion","PhoBERT-V2","CombViSA","lexical-semantic features","negation patterns","VLSP 2016","AIVIVN 2019"],"falsifier":"Re-run the pipeline with a named distance function, say shortest path through synonym and antonym relations in WordNet, and recompute the extended lexicon and the CombViSA F1 scores; if the reproduced F1 drops back to the phoBERT-V2-only values of 0.91 and 0.89, or if the distance choice changes which words are scored positive versus negative, the gain claimed for ExtViSentiWordNet does not hold.","tokens_in":9498,"feed_emoji":"📊","tokens_out":6644,"duration_ms":58329,"temperature":0.7,"pith_summary":"The paper tries to establish that Vietnamese sentiment analysis improves when a pre-trained language model is combined with an expanded sentiment lexicon rather than used alone. To that end it grows the 1,017-word Vietnamese SentiWordNet into a larger resource by pulling pure positive and negative words from English SentiWordNet and adding synonyms and antonyms from Vietnamese and WordNet dictionaries, scoring each candidate by the ratio of its average distance to the positive set over the sum of both distances. The combined model, CombViSA, runs PhoBERT-V2 through an RCNN and a feedforward layer to get a language-model vector, concatenates it with sentiment vectors extracted from the expanded lexicon, with negation patterns flipping the scores, and classifies with softmax. On the VLSP 2016 and AIVIVN 2019 review datasets the model reports F1 scores of 0.95 and 0.94, above the phoBERT-V2-only and LSTM baselines.","feed_headline":"Expanded SentiWordNet pushes Vietnamese review F1 to 0.95","feed_subtitle":"Adding lexicon features to PhoBERT-V2 beats language-model-only baselines on two Vietnamese review sets.","key_machinery":"The load-bearing object is the expanded Vietnamese SentiWordNet and the score formula living inside it. For a candidate word w, $PosScore = d_{pos}/(d_{pos}+d_{neg})$ with $NegScore = 1 - PosScore$ forces the two scores to sum to one, so every word carries a normalized polarity between purely positive and purely negative. The sets P and N are grown from pure SentiWordNet entries by synonym and antonym relations, and the SentiVector_Extraction algorithm converts the per-word scores into length-128 PosVec and NegVec vectors for each review, reversing the polarity inside negation patterns. These vectors are concatenated and passed through a feedforward layer to make SWVec, which is then concatenated with the LMVec produced by PhoBERT-V2 plus RCNN; the pair goes through another feedforward layer and softmax.","core_discovery":"The paper's claim, stated on its own terms, is that a larger SentiWordNet plus a way to digest it into fixed-size vectors is what closes the gap between a generic Vietnamese language model and a strong review classifier. The extension procedure defines a pure positive set P and a pure negative set N from SentiWordNet scores, expands them with synonym and antonym relations, and assigns a candidate word w the scores $PosScore = d_{pos}/(d_{pos}+d_{neg})$ and $NegScore = 1 - PosScore$, where $d_{pos}$ and $d_{neg}$ are average distances from w to the two sets. The SentiVector_Extraction algorithm builds PosVec and NegVec from these scores, swapping the two under negation patterns such as không and chẳng. Combining these vectors with PhoBERT-V2 features in CombViSA yields the reported F1 of 0.95 on VLSP 2016 and 0.94 on AIVIVN 2019, which would make the expanded lexicon a working complement to contextual embeddings for Vietnamese.","pith_inferences":["A reproducibility check follows directly from the missing distance definition: any concrete metric substituted into Equation 5 could change which words get which scores, so the reported gains are only meaningful once that function is specified. This is an editorial inference about the paper's reproducibility, not a claim the paper makes.","The improvement attributed to the lexicon might partly come from the extra RCNN and MLP parameters; an ablation that swaps SentiWordNet vectors for random fixed vectors of the same dimension would isolate the lexicon's contribution. This is a suggested extension, not in the paper.","Since P and N are initialized from English SentiWordNet and expanded through dictionary relations, the method's success likely depends on how well those relations cover informal, typo-heavy e-commerce Vietnamese; testing on out-of-vocabulary slang would show the boundary. This is an inference beyond the paper's experiments."],"forward_implications":["On both evaluation sets, adding the expanded-lexicon SentiVec to PhoBERT-V2 raises F1 by 2 points, from 0.91 to 0.93 on VLSP 2016 and from 0.89 to 0.93 on AIVIVN 2019, and the full CombViSA reaches 0.95 and 0.94.","Lexical-semantic features and contextual embeddings are complementary for this task, since the lexicon contributes signal that the language-model-only vector does not already contain.","The negation-pattern handling in SentiVector_Extraction is a concrete mechanism that turns explicit negation into flipped polarity, which matters for review text where phrases like 'not good' are negative.","The expansion procedure, if it works as described, gives a recipe for enlarging a small sentiment lexicon for Vietnamese without manual annotation, using SentiWordNet seeds and synonym and antonym relations."],"supporting_citations":[{"why":"Supplies the PhoBERT-V2 pre-trained model whose output feeds the RCNN and LMVec branch.","marker":"[1]"},{"why":"Source of the initial positive and negative word sets P and N with their sentiment scores.","marker":"[2]"},{"why":"Defines the RCNN architecture used to turn PhoBERT-V2 outputs into the LMVec representation.","marker":"[26]"},{"why":"Provides the original 1,017-word Vietnamese SentiWordNet that the paper extends.","marker":"[29]"},{"why":"Reports the FastText+LSTM and GloVe+LSTM baselines and evaluation setup on both datasets.","marker":"[30]"},{"why":"Supplies SA1 and SA2 baseline results on VLSP 2016 for comparison.","marker":"[31]"}],"fun_headline_variants":["Expanded SentiWordNet + PhoBERT-V2 hits 0.95 F1 on Vietnamese reviews","Lexicon expansion pushes Vietnamese review F1 to 0.95","Combining SentiWordNet and PhoBERT-V2 yields 0.95 F1","Vietnamese sentiment: SentiWordNet expansion gives 0.95 F1"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The expanded lexicon is built by averaging 'distances' from each candidate word to the positive and negative seed sets, but the paper never defines that distance, so Equation 5 has no computable meaning until a distance function is supplied.","fun_headline_variants_meta":{"raw":{"variants":["Expanded SentiWordNet + PhoBERT-V2 hits 0.95 F1 on Vietnamese reviews","Lexicon expansion pushes Vietnamese review F1 to 0.95","Combining SentiWordNet and PhoBERT-V2 yields 0.95 F1","Vietnamese sentiment: SentiWordNet expansion gives 0.95 F1"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000565,"raw_usage":{"total_tokens":2705,"prompt_tokens":997,"completion_tokens":1708,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":1618}},"tokens_in":613,"tokens_out":1708,"duration_ms":12557,"temperature":1.0,"reasoning_tokens":1618,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:17:41.716725+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the pipeline with a named distance function, say shortest path through synonym and antonym relations in WordNet, and recompute the extended lexicon and the CombViSA F1 scores; if the reproduced F1 drops back to the phoBERT-V2-only values of 0.91 and 0.89, or if the distance choice changes which words are scored positive versus negative, the gain claimed for ExtViSentiWordNet does not hold.","supporting_citations":[{"cited_title":"PhoBERT: Pre-trained language models for Vietnamese","cited_arxiv_id":null,"evidence_quote":"Supplies the PhoBERT-V2 pre-trained model whose output feeds the RCNN and LMVec branch."},{"cited_title":"SENTIWORDNET: A publicly available lexical resource for opinion mining","cited_arxiv_id":null,"evidence_quote":"Source of the initial positive and negative word sets P and N with their sentiment scores."},{"cited_title":"Recurrent convolutional neural networks for text classification","cited_arxiv_id":null,"evidence_quote":"Defines the RCNN architecture used to turn PhoBERT-V2 outputs into the LMVec representation."},{"cited_title":"Building a vietnamese sentiwordnet using vietnamese electronic dictionary and string kernel","cited_arxiv_id":null,"evidence_quote":"Provides the original 1,017-word Vietnamese SentiWordNet that the paper extends."},{"cited_title":"Fine-Tuning BERT for Sentiment Analysis of Vietnamese Reviews","cited_arxiv_id":"2011.10426","evidence_quote":"Reports the FastText+LSTM and GloVe+LSTM baselines and evaluation setup on both datasets."},{"cited_title":"Vlsp shared task sen timent analysis","cited_arxiv_id":null,"evidence_quote":"Supplies SA1 and SA2 baseline results on VLSP 2016 for comparison."}],"review_version":1}