Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

BanglishRev: A Large-Scale Bangla-English and Code-mixed Dataset of Product Reviews in E-Commerce

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces BanglishRev, claimed to be the largest e-commerce review dataset for Bengali customers, with 1.74 million written reviews from 3.2 million ratings across about 128,000 products, and reports that a BanglishBERT model…

desk verdict The dataset is real, large, and useful; the 94% accuracy needs an overlap check before it is trusted. read the letter →

arxiv 2412.13161 v2 pith:MEXU66MS submitted 2024-12-17 cs.CL cs.CVcs.LG

Mohammad Nazmush Shamael , Sabila Nawshin , Swakkhar Shatabda , Salekul Islam This is my paper · ORCID
classification cs.CLcs.CVcs.LG
keywords BanglishRevBengalie-commercereviewscode-mixedtexttransliterationsentimentanalysisBERTrating-derivedlabelsDarazBangladesh
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BanglishRev is presented as the largest e-commerce product-review dataset for the Bengali-speaking market: 1.74 million written reviews drawn from 3.2 million ratings across roughly 128,000 products on the Daraz Bangladesh platform. Each review carries rich metadata—the rating, dates of review and purchase, likes and dislikes, seller replies, and image links—so the corpus is positioned to support not only sentiment analysis but also spam detection, market analysis, and consumer-behavior studies. The paper's main experiment tests whether star ratings can substitute for human sentiment labels: a BanglishBERT model trained on BanglishRev with ratings above 3 marked positive and 3 or below marked negative is evaluated on a previously published manually annotated 78,000-review benchmark. The model reaches 94% accuracy and 0.94 F1, which the authors take as evidence that the dataset's scale compensates for the noise in rating-derived labels.

What carries the argument

The load-bearing artifact is the dataset itself, produced by scraping Daraz's three-tier category tree, collecting product URLs, and then extracting review details page by page. The experimental machinery is a normalization pipeline that detects whether a review is Bangla, English, Banglish, or code-mixed, transliterates Banglish words to Bangla phonetics with the avro library, and feeds the text into BanglishBERT, a BERT-based model pre-trained with an ELECTRA-style replaced-token-detection objective. The rating-to-label mapping—positive for ratings greater than 3, negative for ratings 3 or below—is what converts the unlabeled corpus into supervised training data, and the previously published manually annotated benchmark supplies the held-out test set.

What would settle it

Compute the overlap between BanglishRev's training reviews and the manually annotated test set after text normalization; if a material share of test reviews appear in training, the reported accuracy is inflated. A complementary check is to annotate a random sample of BanglishRev reviews and compare the rating-derived labels with human sentiment judgments to see whether the proxy holds outside the test benchmark.

Watch

Extended reading notes

Core claim

The central claim is that BanglishRev is the largest and most metadata-rich public review dataset for Bengali e-commerce, and that it can serve as effective training data for binary sentiment analysis even though it carries no manual sentiment labels. Using rating-derived labels, a BanglishBERT model trained on BanglishRev reaches 94% accuracy and 0.94 F1 on a manually annotated 78,000-review test set, outperforming a baseline model trained and tested on that same manually annotated set (93% accuracy with a weaker negative-class F1 of 0.78). The paper further documents the corpus's language mix—roughly 37.8% Banglish, 31.3% English, 25.0% Bangla, and 5.9% code-mixed—and notes that more than 78% of ratings are five stars, an imbalance it deliberately preserves for downstream tasks.

Load-bearing premise

The evaluation assumes that none of the reviews in the manually annotated test set also appear in BanglishRev's training data, even though both corpora come from the same e-commerce platforms, and the paper reports no de-duplication or overlap check; any leakage would inflate the 94% accuracy.

Editorial extensions

If this is right

  • If BanglishRev is as large as reported, it becomes the default scale reference for Bengali e-commerce NLP, replacing prior datasets that top out at 78k reviews.
  • The metadata—review and purchase dates, likes, dislikes, seller replies, and image links—makes rating prediction, spam review detection, and consumer behavior analysis feasible on the same corpus, not just sentiment.
  • The rating-labeling scheme, if transferable, offers a low-cost path to sentiment classifiers for other low-resource languages where manual annotation is scarce.
  • The observed dense clusters of near-identical five-star reviews provide a concrete starting point for spam-review detection research in Bengali.
  • The language-mix statistics (37.8% Banglish, 31.3% English, 25.0% Bangla, 5.9% code-mixed) give a realistic picture of how Bengali customers actually write, which can inform future tokenizers and transliteration tools.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural but untested consequence is that the rating-proxy model's reported F1, measured on a test set balanced differently than real Daraz traffic, overstates performance on the platform's actual review stream, where positive reviews dominate.
  • The class-wise results after three epochs (positive precision 0.96 and recall 0.98; negative precision 0.84 and recall 0.72) imply asymmetric errors: a deployed system would miss a substantial share of negative reviews unless thresholds are re-tuned.
  • Because the corpus comes from a single platform, the paper's conclusions about Bengali e-commerce language patterns may not transfer to other marketplaces; collecting comparable reviews from booksellers, food delivery, and other verticals would test the generality.
  • The image links, if maintained, could support multimodal analyses—for example, whether photo similarity to the product listing predicts satisfaction—but the dataset's reliance on external URLs makes this capability fragile over time.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces BanglishRev, a large-scale dataset of product reviews from Daraz Bangladesh, containing 1.74 million written reviews drawn from 3.2 million ratings across 128k products, with rich metadata such as review text, rating, dates, likes/dislikes, seller responses, and image links. The authors describe their scraping pipeline, present descriptive statistics and language-distribution analyses, and report a sentiment-classification experiment in which a BanglishBERT model is trained on BanglishRev with ratings converted to binary labels (rating >= 4 positive, <= 3 negative) and evaluated on the externally annotated Rashid et al. (2024) test set, achieving a claimed accuracy of 94% and F1 of 0.94. The paper also discusses limitations, ethical considerations, and several future research directions.

Significance. If the dataset is as described, it is a substantial contribution: it is an order of magnitude larger than existing Bangla e-commerce review datasets, includes metadata that other datasets lack, and is released on HuggingFace for public use. The paper is transparent about its scraping procedure and limitations, and it explicitly anonymizes user and product IDs. The sentiment evaluation, however, is the main supporting evidence for the dataset's utility, and that evaluation currently has unresolved threats to validity (possible train/test leakage, insufficient statistical rigor, and confounded comparisons). The dataset itself is valuable, but the magnitude of the claimed experimental result needs to be verified with additional analysis.

major comments (3)
  1. [§4.1.2, Tables 3 and 5] The claimed superiority of the BanglishRev-trained model over the Rashid et al. baseline is not substantiated because no overlap check is reported between the BanglishRev training reviews and the Rashid et al. test reviews. Both datasets are scraped from Daraz Bangladesh, and the BanglishRev collection period (April–May 2024, §4.2) could overlap with the period from which the Rashid et al. reviews were collected. Since the paper does not report any deduplication or overlap analysis, identical reviews may appear in both the training set and the test set, inflating the reported 94% accuracy and 0.94 F1. Please conduct and report an exact-match and near-duplicate overlap analysis (e.g., MinHash or normalized text hashing) between the BanglishRev training data and the Rashid et al. test set, and rerun the evaluation on the non-overlapping subset or after removing overlapping instances from training. Without this, the central experimental claim is unsupported.
  2. [§4.1, Tables 2–5] The experiments compare models that differ in both training data size and label source, so the observed accuracy difference cannot be attributed to 'rating as sentiment label' (the section's stated focus). The baseline is trained on 80% of the 78k manually labeled Rashid et al. reviews, whereas the BanglishRev models are trained on 1.74M rating-derived labels. To support the claim that rating information is an effective substitute for manual sentiment labels, add a control experiment that trains on a random subsample of BanglishRev of comparable size to the Rashid et al. training split (or trains on the Rashid et al. reviews with rating-derived labels), so that data volume and label source are not confounded.
  3. [§4.1.2, Tables 3 and 5] Results are reported from a single run with no variance estimates, and the numbers are not stable: after one epoch Experiment 2 gives 95% accuracy, while after three epochs it gives 94%. The 1-point gap relative to the baseline's 93% is within likely stochastic noise. Please report the mean and standard deviation over multiple random seeds, and if feasible a paired significance test over the common test set. Additionally, the abstract's 'F1 score of 0.94' is the weighted average, which is dominated by the majority positive class (>78% five-star reviews, §4.2); the negative-class recall of 0.72 in Table 3 is the actual weakness and should be reported in the abstract or at least in the main text near the accuracy claim.
minor comments (5)
  1. [§3.2] The language distribution sentence is internally inconsistent: it states '37.7% of the total reviews are written in Banglish, 21.2% in English, 25.04% in English and only 5.9% in code-mixed language,' but the counts in the next paragraph give 25.04% Bangla, 31.27% English, 5.92% code-mixed, and 37.77% Banglish. Please correct the text and ensure Figure 3 matches.
  2. [§3.1 and §4.3] The number of root categories is inconsistent: §3.1 says 'eight root categories' while the caption of Figure 4 says '12 root categories' and §4.3 also says '8 root categories.' Please clarify the correct number and use it consistently.
  3. [§4.1.2] The sentence 'Experiment 1 and 2 are conducted with the BanglishRev dataset being used for training' is confusing because Experiment 1 is the baseline trained and tested entirely on Rashid et al. Please rephrase to explicitly state the training and test set for each of the three experiments.
  4. [Throughout] There are several typographical errors: 'languague' (Introduction), 'scarped' (§3.1), 'Implemetation' (§4.1.1), 'Google Colaboratory' (§4.1.1), and 'Heath and Beauty' (§3.2). A careful proofread is needed.
  5. [Abstract and dataset link] The dataset is a central contribution, so its long-term availability matters. Please verify that the HuggingFace URL is stable and consider providing a versioned DOI or commit hash, along with a data card that describes the anonymization procedure and the intended use conditions.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the rating-derived training labels are evaluated against an external manual benchmark, and the reported 94% accuracy is not forced by construction.

full rationale

The paper does not attempt a formal derivation; its central claims are a dataset resource claim and an empirical evaluation claim. The sentiment labels used for training are derived from star ratings by a fixed threshold (rating > 3 positive, rating <= 3 negative), while evaluation is performed on the externally curated, manually annotated dataset of Rashid et al. (2024). The threshold is a transparent modeling choice, not a parameter fitted to the test labels, and Experiment 2 and Experiment 3 report different thresholds rather than hiding the choice. The 94% accuracy and 0.94 F1 are measured against manual labels, so they are not obtained by the definition of the training labels. The baseline and the pretrained model are both external works, and no load-bearing self-citation chain is present. The paper candidly reports limitations including class imbalance, computational cost, the Daraz-only collection source, and the April-May 2024 collection window. One non-circular caveat exists: because both BanglishRev and the Rashid et al. test set are scraped from Daraz and no overlap check is reported, some test reviews could also appear in training, which could inflate the reported accuracy; this is an evaluation-risk concern rather than a circular argument, since the test labels are independently annotated and the training labels are not fitted to them.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central resource claim rests on the quality of the scrape and the metadata. The benchmark claim rests on the rating-to-sentiment proxy and on the external test set being clean. No new theoretical entities are introduced.

free parameters (1)
  • rating_to_sentiment_threshold = 3 stars (rating > 3 positive, <= 3 negative)
    Chosen by hand for Experiment-2; the paper also tries other thresholds. The reported result depends on this threshold, which is not tuned on the test set.
assumptions (3)
  • domain assumption Product ratings are a valid proxy for binary sentiment (positive if rating > 3, negative otherwise).
    Invoked in Section 4.1 to generate training labels for 1.74M reviews without manual annotation; the rating distribution is highly skewed (78% five-star), so the proxy is noisy.
  • domain assumption The Rashid et al. (2024) manually annotated dataset is a reliable and non-overlapping evaluation benchmark.
    Used as the sole test set in Section 4.1.2; both corpora were scraped from Daraz and Pickaboo, and the paper does not report any de-duplication or overlap check.
  • domain assumption The text normalization pipeline (NLTK word detection plus Avro transliteration) preserves sentiment-bearing content.
    Described in Section 4.1.1 and the supplementary material; errors in Banglish detection are acknowledged in Section 3.2, so normalized text may differ from the raw review.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BanglishRev: A Large-Scale Bangla-English and Code-mixed Dataset of Product Reviews in E-Commerce." pith.science (2026). https://pith.science/paper/MEXU66MS

@misc{pith2026241213161,
  author       = {Pith},
  title        = {Pith review of: BanglishRev: A Large-Scale Bangla-English and Code-mixed Dataset of Product Reviews in E-Commerce},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MEXU66MS}},
  note         = {Machine review of arXiv:2412.13161}
}
read the original abstract

This work presents the BanglishRev Dataset, the largest e-commerce product review dataset to date for reviews written in Bengali, English, a mixture of both and Banglish, Bengali words written with English alphabets. The dataset comprises of 1.74 million written reviews from 3.2 million ratings information collected from a total of 128k products being sold in online e-commerce platforms targeting the Bengali population. It includes an extensive array of related metadata for each of the reviews including the rating given by the reviewer, date the review was posted and date of purchase, number of likes, dislikes, response from the seller, images associated with the review etc. With sentiment analysis being the most prominent usage of review datasets, experimentation with a binary sentiment analysis model with the review rating serving as an indicator of positive or negative sentiment was conducted to evaluate the effectiveness of the large amount of data presented in BanglishRev for sentiment analysis tasks. A BanglishBERT model is trained on the data from BanglishRev with reviews being considered labeled positive if the rating is greater than 3 and negative if the rating is less than or equal to 3. The model is evaluated by being testing against a previously published manually annotated dataset for e-commerce reviews written in a mixture of Bangla, English and Banglish. The experimental model achieved an exceptional accuracy of 94\% and F1 score of 0.94, demonstrating the dataset's efficacy for sentiment analysis. Some of the intriguing patterns and observations seen within the dataset and future research directions where the dataset can be utilized is also discussed and explored. The dataset can be accessed through https://huggingface.co/datasets/BanglishRev/bangla-english-and-code-mixed-ecommerce-review-dataset.

Figures

Figures reproduced from arXiv: 2412.13161 by the authors.

Figure 1
Figure 1. Sample Banglish, Bangla and Code-mixed texts and their corresponding English translation [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Density plot of product rating [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 5
Figure 5. shows a density plot of the review lengths observed in the dataset. A considerable portion of the dataset contains only 2 word reviews (e.g. "Good product", "Very good", "Nice product"), which may be the case because review rating is negatively correlated to review length (Ghasemaghaei et al., 2018) and BanglishRev contains an overwhelming number of 5 star ratings. Regular expressions are used to identify the Bangla… view at source ↗
Figures from the paper (1 more)
Figure 6
Figure 6. Figure 6: Most used words in Bangla [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MixSarc: A Bangla-English Code-Mixed Corpus for Implicit Meaning Identification

    cs.CL 2026-02 conditional novelty 6.0 of 10

    MixSarc is a new public Bangla–English code-mixed corpus of 9,087 sentences annotated for humor, sarcasm, offensiveness, and vulgarity, with benchmark results showing sarcasm and minority classes remain difficult.

  2. Towards Personalized Bangla Book Recommendation: A Large-Scale Heterogeneous Book Graph Dataset

    cs.IR 2026-02 conditional novelty 6.0 of 10

    RokomariBG is a newly released 127k-book heterogeneous graph dataset from Rokomari.com with benchmarks showing neural two-tower retrieval with relational and textual side features best for Bangla book recommendation (...

Reference graph

Works this paper leans on

25 extracted references · 21 canonical work pages · cited by 2 Pith papers

  1. [1]

    T., Begum, M., and Mustafa, R

    Akter, M. T., Begum, M., and Mustafa, R. (2021). Bengali sentiment analysis of e-commerce product reviews using k-nearest neighbors. In 2021 International conference on information and communication technology for sustainable development (ICICT4SD), pages 40–44. IEEE

    work page 2021

  2. [2]

    Shahriyar, R. (2021). Banglabert: Language model pretraining and benchmarks for low-resource language understanding evaluation in bangla. arXiv preprint arXiv:2101.00204

    arXiv 2021

  3. [3]

    V ., and Manning, C

    Clark, K., Luong, M.-T., Le, Q. V ., and Manning, C. D. (2020). Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555

    arXiv 2020

  4. [4]

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    arXiv 2018

  5. [5]

    and Rajalakshmi, R

    Ganganwar, V . and Rajalakshmi, R. (2023). Enhanced hindi aspect-based sentiment analysis using class balancing approach. International Journal of Information Technology, 15(7):3527–3532

    work page 2023

  6. [6]

    P., Deal, K., and Hassanein, K

    Ghasemaghaei, M., Eslami, S. P., Deal, K., and Hassanein, K. (2018). Reviews’ length and sentiment as correlates of online reviews’ ratings. Internet Research, 28(3):544–563

    work page 2018

  7. [7]

    I., Rahman, M., Ahmed, M

    Hossain, M. I., Rahman, M., Ahmed, M. T., Rahman, M. S., and Islam, A. T. (2021). Rating prediction of product reviews of bangla language using machine learning algorithms. In 2021 International Conference on Artificial Intelligence and Mechatronics Systems (AIMS), pages 1–6. IEEE

    work page 2021

  8. [8]

    J., Joy, D

    Hossain, M. J., Joy, D. D., Das, S., and Mustafa, R. (2022). Sentiment analysis on reviews of e-commerce sites using machine learning algorithms. In 2022 International Conference on Innovations in Science, Engineering and Technology (ICISET), pages 522–527. IEEE

    work page 2022

Show all 25 references
  1. [9]

    S., Ahmed, S., and Khan, R

    Islam, A. S., Ahmed, S., and Khan, R. H. (2022). A review on e-commerce system in bangladesh: an empirical study. In Proceedings of the 2nd International Conference on Computing Advancements, pages 269–276

    2022

  2. [10]

    R., Mukta, S

    Karmakar, D. R., Mukta, S. A., Jahan, B., and Karmakar, J. (2022). Sentiment analysis of customers’ review in bangla using machine learning approaches. In Innovations in Computer Science and Engineering: Proceedings of the Ninth ICICSE, 2021, pages 373–384. Springer

    2022

  3. [11]

    Keung, P., Lu, Y ., Szarvas, G., and Smith, N. A. (2020). The multilingual amazon reviews corpus. arXiv preprint arXiv:2010.02573. 9

    2020 arXiv

  4. [12]

    Klaiman, M. H. and Lahiri, A. (2018). Bengali. In The world’s major languages, pages 427–446. Routledge

    2018

  5. [13]

    Lim, E.-P., Nguyen, V .-A., Jindal, N., Liu, B., and Lauw, H. W. (2010). Detecting product review spammers using rating behaviors. In Proceedings of the 19th ACM international conference on Information and knowledge management, pages 939–948

    2010

  6. [14]

    Lubbe, S. (2002). The economic and social impacts of e-commerce. IGI Global

    2002

  7. [15]

    Mohiuddin, M. (2014). Overview the e-commerce in bangladesh. IOSR Journal of Business and Management, 16(7):01–06

    2014

  8. [16]

    H., Rifat, M

    Munna, M. H., Rifat, M. R. I., and Badrudduza, A. (2020). Sentiment analysis and product review classification in e-commerce platform. In 2020 23rd International Conference on Computer and Information Technology (ICCIT), pages 1–6. IEEE

    2020

  9. [17]

    Q., Vu, T., Pham, S

    Nguyen, D. Q., Vu, T., Pham, S. B., et al. (2014). Sentiment classification on polarity reviews: an empirical study using rating-based features. In Proceedings of the 5th workshop on computational approaches to subjectivity, sentiment and social media analysis, pages 128–135

    2014

  10. [18]

    Noor, F., Bakhtyar, M., and Baber, J. (2019). Sentiment analysis in e-commerce using svm on roman urdu text. In Emerging Technologies in Computing: Second International Conference, iCETiC 2019, London, UK, August 19–20, 2019, Proceedings 2, pages 213–222. Springer

    2019

  11. [19]

    R., Akter, M., Ferdows, R., and Ahmed, F

    Purba, M. R., Akter, M., Ferdows, R., and Ahmed, F. (2022). A hybrid convolutional long short- term memory (cnn-lstm) based natural language processing (nlp) model for sentiment analysis of customer product reviews in bangla. Journal of Discrete Mathematical Sciences and Crypt...

    2022

  12. [20]

    Rashid, M. R. A., Hasan, K. F., Hasan, R., Das, A., Sultana, M., and Hasan, M. (2024). A comprehensive dataset for sentiment and emotion classification from bangladesh e-commerce reviews. Data in Brief, 53:110052

    2024

  13. [21]

    A., Hasan, M

    Shafin, M. A., Hasan, M. M., Alam, M. R., Mithu, M. A., Nur, A. U., and Faruk, M. O. (2020). Product review sentiment analysis by using nlp and machine learning in bangla language. In 2020 23rd International Conference on Computer and Information Technology (ICCIT) , pages 1–5. IEEE

    2020

  14. [22]

    S., Ahmed, Z., and Jony, A

    Shanto, S. S., Ahmed, Z., and Jony, A. I. (2023). Mining user opinions: A balanced bangla sentiment analysis dataset for e-commerce. Malaysian Journal of Science and Advanced Technology, pages 272–279

    2023

  15. [23]

    and Chakma, D

    Sharmin, S. and Chakma, D. (2021). Attention-based convolutional neural network for bangla sentiment analysis. Ai & Society, 36(1):381–396

    2021

  16. [24]

    K., Piryani, R., Uddin, A., and Waila, P

    Singh, V . K., Piryani, R., Uddin, A., and Waila, P. (2013). Sentiment analysis of movie reviews: A new feature-based heuristic for aspect-level sentiment classification. In 2013 International mutli-conference on automation, computing, communication, control and compressed sen...

    2013

  17. [25]

    W., and Isa, S

    Sutoyo, R., Achmad, S., Chowanda, A., Andangsari, E. W., and Isa, S. M. (2022). Prdect-id: Indonesian product reviews dataset for emotions classification tasks. Data in Brief, 44:108554. 10

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.