A Language Model-Driven Semi-Supervised Ensemble Framework for Illicit Market Detection Across Deep/Dark Web and Social Platforms

Amir H. Gandomi; Fang Chen; Hossein Yazdanjouei; Mikko S. Niemela; Mohammad Sadegh Khorshidi; Morteza Rakhshaninejad; Navid Yazdanjue

REVIEW 4 major objections 5 minor 1 cited by

A Language Model-Driven Semi-Supervised Ensemble Framework for Illicit Market Detection Across Deep/Dark Web and Social Platforms

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A two-stage semi-supervised ensemble that combines fine-tuned ModernBERT embeddings with structural document features beats DarkBERT and five other transformer baselines at detecting and categorising illicit marketplace posts across…

desk verdict A credible engineering contribution whose headline cross-dataset gains rest on an unpublished relabeling that needs to be public before the state-of-the-art claim is taken at face value. read the letter →

arxiv 2507.22912 v1 pith:6SBILALS submitted 2025-07-19 cs.CL cs.AIcs.LG

Navid Yazdanjue , Morteza Rakhshaninejad , Hossein Yazdanjouei , Mohammad Sadegh Khorshidi , Mikko S. Niemela , Fang Chen , Amir H. Gandomi This is my paper · ORCID

classification cs.CLcs.AIcs.LG

keywords illicitmarketplacedetectiondarkwebdeepsemi-supervisedlearningensembletransformerembeddingsModernBERTtextclassification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that illicit marketplace content scattered across the deep/dark web, Telegram, Reddit, and Pastebin can be detected and classified into drug, weapon, and credential sales using a two-stage pipeline that works with very few labeled examples. The pipeline pairs domain-fine-tuned ModernBERT embeddings with hand-built structural features, then feeds them into a semi-supervised ensemble of XGBoost, Random Forest, and SVM whose votes are weighted by each model's prediction confidence as measured by entropy. The authors report that this system outperforms BERT, ALBERT, Longformer, BigBird, ModernBERT, and even DarkBERT, a model pretrained on dark-web text, on their own test set and on adjusted versions of two public benchmarks, DUTA and CoDA. If the claim holds, it would show that combining long-context language models, structural cues, and self-training is a practical route to monitoring underground markets under exactly the label scarcity that real monitoring faces.

What carries the argument

The load-bearing mechanism is the two-stage sequential classifier. Stage one is a semi-supervised ensemble of XGBoost, Random Forest, and SVM base learners combined by weights $w_i = (\textit{MEW}_i/\textit{MEC}_i) / \sum_j (\textit{MEW}_j/\textit{MEC}_j)$, where MEC is the mean entropy of a learner's correct predictions and MEW the mean entropy of its wrong predictions; low MEC and high MEW make a learner's vote count more. Confident pseudo-labels, with average class probability above a threshold $\theta$, are added to the training pool across self-training iterations. Stage two uses three self-training XGB classifiers, one per sale category. The input representation is mean-pooled embeddings from ModernBERT, a long-context transformer that processes up to 8192 tokens via alternating local and global attention with rotary positional embeddings, concatenated with manually engineered features: line-width and indentation statistics, counts and relative weights of embedded items such as Bitcoin addresses, emails, IPs, URLs, images, and credit-card numbers, plus source and date metadata.

What would settle it

A concrete check: two independent annotators re-label a random sample of DUTA and CoDA documents into sale, drug, weapon, and credential using only the paper's description, and both the proposed pipeline and DarkBERT are scored against the fresh labels; if agreement is low, or if DarkBERT matches or beats the proposed model on the fresh labels, the reported generalization margin is an artifact of the relabeling.

Watch

Extended reading notes

Core claim

The central claim is that the complete sequential pipeline, fine-tuned ModernBERT embeddings concatenated with layout, pattern-specific item, and metadata features, followed by a self-training ensemble with entropy-based weighted voting for sale detection and three semi-supervised XGB classifiers for category, achieves accuracy of 0.96489, F1-score of 0.93467, and TMCC of 0.95388 on the authors' multi-source test set. On the DUTA, CoDA, and multi-source test sets, the model reports higher mean accuracy, F1-score, and TMCC than BERT, ALBERT, Longformer, BigBird, ModernBERT, and DarkBERT (Table IX). The paper's argued explanation is that the gain comes from the task-specific design, domain adaptation of the embedding model, structural features beyond raw text, hierarchical two-stage classification, and the use of unlabeled data, rather than from any single component such as long-context capacity alone.

Load-bearing premise

The cross-dataset generalization claim depends on the authors' manual relabeling of DUTA and CoDA into the four target categories, and no mapping rules, label guidelines, or inter-annotator agreement are reported; if that relabeling is idiosyncratic, the reported F1 margins measure the authors' label definitions rather than model superiority.

Editorial extensions

If this is right

A monitoring system could start from roughly half the labeled data and still score above 0.82 on all three reported metrics, with the remaining lift coming from unlabeled documents.
Domain fine-tuning of the embedding model contributes more than raw sequence length: fine-tuned ModernBERT ranked first among thirteen representation techniques, while pretrained ModernBERT ranked below GloVe in the Friedman ranking.
Structural signals add value beyond text embeddings, so the full pipeline, not the language model alone, is what beats every LM-only baseline.
The entropy-weighted voting improves over majority voting in the semi-supervised ensemble, with accuracy 0.95250 versus 0.94080, indicating that confidence weighting is doing work beyond simple ensembling.
The model generalizes across distributions, with reported F1 of 0.86568 on adjusted DUTA and 0.94439 on adjusted CoDA after relabeling.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The cross-dataset margins over DarkBERT, about 4.0 F1 points on DUTA and 1.4 points on CoDA, are measured against the authors' own relabeling of those benchmarks; an independent annotation study would show how much of the margin is model performance and how much is label-definition choice.
The MEC/MEW weighting rule is generic: any ensemble of probabilistic classifiers could use it to gate pseudo-labels, so the mechanism is testable in other low-annotation text domains such as fraud, extremism, or abuse detection.
Because the CoDA advantage over DarkBERT is small, the decisive components may be the domain-fine-tuned embeddings and structural features rather than the self-training loop; an ablation that removes only the self-training stage would isolate this.
The framework is evaluated as sequential one-vs-rest categories, while illicit posts often advertise multiple goods; a multi-label variant that skips the sale gate could be a natural next step.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a credible engineering paper, not a paradigm shift. What is new is the multi-source corpus (deep/dark web, Telegram, Reddit, Pastebin), the two-stage semi-supervised pipeline with ModernBERT embeddings plus manual structural features, and the MEC/MEW entropy-based weighting rule. The internal held-out evaluation is clean: a fixed 315-sample test set, 30 runs, a Friedman test, and the gains over DarkBERT and the other baselines on that test set are plausible. The semi-supervised gains are consistent across stages. Credit where due: the authors are careful about validation and report variance.

The load-bearing weakness is the external benchmark comparison. DUTA and CoDA have their own label taxonomies; the paper relabels them into its four sale categories, but gives no mapping rules, no label guidelines, and no inter-annotator agreement. A 'drug' page in CoDA is not necessarily a 'drug sale' page, and a marketplace page can mention weapons without selling them. If the relabeling encodes the authors' interpretation, the Table IX F1 margins over DarkBERT measure that interpretation, not model superiority. That concern looks structurally valid to me. It does not sink the internal result, but it does mean the cross-dataset state-of-the-art claim is not verifiable as written.

Other soft spots are minor by comparison. There is an internal inconsistency in the ModernBERT epoch count (text says 15, Table IV says 10). The 5%-labeled result is near chance—F1 0.49—and while it is honestly reported, the text frames it as a positive, which is a bit generous. There is no ablation isolating the manual features or the MEC/MEW weighting, so we cannot see how much each contributes. The metadata features include source platform, which is constant in both external test sets, so on the internal test set those features could be a shortcut; the paper does not address that.

Who is this for? Researchers working on dark-web and illicit-marketplace detection. It is a useful engineering contribution with reproducible potential, but only if the corpus and relabeling are released. I would send it to review, not desk-reject it. The reviewer should ask for the relabeling mapping, the corpus, and the ablations. If those come through, the paper supports its claims. If not, the external-benchmark section should be downweighted.

Referee Report

4 major / 5 minor

Summary. The paper proposes a two-stage pipeline for detecting and classifying illicit marketplace posts (sale vs. no-sale; drug/weapon/credential) across deep/dark web, Telegram, Reddit, and Pastebin. Features are fine-tuned ModernBERT embeddings concatenated with manually engineered layout, pattern, and metadata features. Stage 1 is a semi-supervised ensemble (XGB, RF, SVM) with a proposed MEC/MEW entropy-based weighted voting; Stage 2 consists of three semi-supervised XGB classifiers for sale-type categorization. The evaluation uses the authors' own 1,575 labeled / 20,000 unlabeled collection, plus DUTA and CoDA as test sets. The authors report top results on their own test set (Acc 0.96489, F1 0.93467, TMCC 0.95388) and state that the full pipeline outperforms BERT, ModernBERT, ALBERT, Longformer, BigBird, and DarkBERT on all three datasets.

Significance. If the results hold, the paper demonstrates a practically useful engineering result: combining long-context domain-adapted embeddings with structural features and self-training can improve detection over strong dark-web baselines under limited supervision. Strengths of the manuscript include a fixed held-out test set, 30 repeated runs, a Friedman test for representation ranking, validation-based hyperparameter selection, and evaluation on external corpora. The MEC/MEW weighting is a simple, clearly motivated idea. However, as detailed in the major comments, the external-benchmark comparisons and the limited-supervision framing need substantial clarification and additional evidence before the claims as stated are fully supportable.

major comments (4)

[Section IV.G, Tables VIII-IX] The cross-dataset generalization claim depends on an undocumented relabeling of DUTA and CoDA. The text says only that samples were 'filtering for English-language documents and relabeling samples according to the four target categories'; no mapping rules, label guidelines, or inter-annotator agreement statistics are provided. DUTA and CoDA have their own taxonomies (e.g., CoDA includes hacking, pornography, violence, and financial categories alongside drugs), and a label such as 'drug' does not automatically map to 'drug sale.' Because the reported F1 margins over DarkBERT are 4.9 points on DUTA and 1.5 points on CoDA, the comparison may reflect the authors' label definitions rather than model superiority. Please provide the full mapping, annotator guidelines, agreement measures, and ideally a comparison on the original labels or on a shared subset.
[Section IV.B and Table IV] The reported optimal number of fine-tuning epochs for ModernBERT is inconsistent: Section IV.B states the grid search identified 15 epochs, while Table IV lists the optimal value as 10. Since the fine-tuned ModernBERT embeddings are a central component and the exact configuration affects reproducibility, please correct the discrepancy and state which value produced the headline results.
[Section IV.G, Table IX] The comparison in Table IX varies several factors at once: the proposed pipeline uses manually engineered features, a two-stage hierarchical design, semi-supervised self-training, and the fine-tuned ModernBERT backbone, while each deep baseline is used as a 'standalone multi-label classifier.' The reported gaps therefore do not isolate the contribution of the model architecture, the feature set, or the semi-supervised mechanism. To support the claim that the framework 'outperforms' these baselines, the baselines should be run with the same concatenated manual features and, where feasible, with the same hierarchical wrapper; otherwise the headline comparison conflates feature engineering with model design.
[Section IV.E, Fig. 5] At 5% labeled data the model achieves F1=0.49176 and TMCC=0.51877, which are essentially at the random baseline (TMCC=0.5 corresponds to random predictions), yet the text says this result 'indicate[s] the model's ability to learn useful patterns even from minimal labeled training set.' This statement is unsupported and conflicts with the abstract's 'robustness under limited supervision.' Please either remove the 5% claim, report statistical confidence intervals relative to a random-majority baseline, or reframe the limited-supervision claim to the settings where the model is clearly above chance (e.g., 25% and above).

minor comments (5)

[Table VI and Section IV.F] In the text, the supervised ensemble using the proposed weighting is labeled 'SSE WV' in the sentence 'best-performing supervised ensemble using the same weighting strategy (SSE WV, 0.89429)'; this should be 'SE WV' to match the table and to avoid confusion with the semi-supervised ensemble.
[Section III.B.2(c)] The list of metadata features contains 'Deep Web Source' twice; remove the duplicate.
[Equation (7)] The pseudo-label decision rule does not specify tie-breaking when TPP_sale equals TPP_no-sale; please add a deterministic rule.
[Abstract] The headline numbers (Acc 0.96489, F1 0.93467, TMCC 0.95388) are only for the authors' own multi-source test set; please state this explicitly in the abstract so readers do not infer they hold on DUTA and CoDA as well.
[Section III.B.1] The fine-tuning objective (e.g., masked language modeling or classification) and the exact corpus split used for fine-tuning ModernBERT are not stated; please specify these details for reproducibility.

Circularity Check

0 steps flagged · score 2.0 of 10

No circularity in the core derivation; the minor self-cited feature set and the author-generated DUTA/CoDA relabeling are validity concerns, not definitional circularity.

full rationale

The paper's central claims are empirical measurements on a held-out test set, not derivations from a fitted quantity. Section IV.A fixes a 60/20/20 split (945/315/315) before training; Section IV.B selects hyperparameters on the validation set; the test set is used only for final evaluation, so the reported accuracy/F1/TMCC values are not forced by the training objective. Equations (1)-(12) define preprocessing counts, Shannon entropy, MEC/MEW validation statistics, weight normalization, and TPP aggregation; none of these equations defines a predicted quantity in terms of itself or in terms of the test labels. The manual feature set is attributed to the authors' previous published study [15], which is an external, verifiable result and is not invoked as an unproven uniqueness theorem. The only ground-truth concern is Section IV.G's relabeling of DUTA and CoDA into the paper's four categories without published mapping rules or inter-annotator agreement, but this makes the external benchmark's validity questionable rather than making the model's prediction equivalent to its input by construction. Accordingly, no circular step is exhibited; the score of 2 reflects the minor self-citation and the author-generated relabeling as non-load-bearing concerns.

Assumptions & free parameters 11 free parameters · 4 assumptions · 1 invented entities

The pipeline rests on a large set of grid-searched hyperparameters (11 entries, including the confidence threshold and iteration counts that control the self-training loop), plus domain assumptions about label consistency, unlabeled-data distribution, and the legitimacy of platform-source features. The only invented algorithmic construct is the MEC/MEW weighting rule, which has no evidence outside this paper. The dependence on the authors' own labels and relabeling choices is the largest unfunded input.

free parameters (11)

Confidence threshold θ = 0.9 (SSE), 0.9 (drug), 0.85 (weapon), 0.9 (credential)
Controls which pseudo-labeled samples enter the training pool; tuned by grid search over [0.8, 0.85, 0.9, 0.95] (Table IV).
Max self-training iterations = 75 (SSE), 25 (drug), 25 (weapon), 50 (credential)
One of the three self-training termination criteria; tuned over [25, 50, 75, 100] (Table IV).
ModernBERT learning rate = 5e-5
Grid-searched over [1e-5, 3e-5, 5e-5, 8e-5]; affects all downstream embeddings (Table IV).
ModernBERT fine-tuning epochs = Text says 15; Table IV lists 10
Internal inconsistency in the paper; the chosen value affects the quality of the domain-adapted embeddings (Section IV.B vs Table IV).
XGB learning rates = 0.1, 0.1, 0.05, 0.1 (SSE, drug, weapon, credential)
Grid-searched per model over [0.01, 0.05, 0.1, 0.2] (Table IV).
XGB n_estimators = 400, 300, 300, 500
Grid-searched over [100, 200, 300, 400, 500] (Table IV).
XGB max depth = 5, 5, 5, 7
Grid-searched over [3, 5, 7, 10] (Table IV).
SVM regularization C = 0.01
Grid-searched over [0.001, 0.01, 0.1, 1, 10] (Table IV).
SVM gamma = 0.1
Grid-searched over [0.001, 0.01, 0.1, 1, 10] (Table IV).
RF max depth = 5
Grid-searched over [3, 5, 7, 10] (Table IV).
RF n_estimators = 400
Grid-searched over [100, 200, 300, 400, 500] (Table IV).

assumptions (4)

domain assumption Manual labels and the DUTA/CoDA relabeling are accurate and semantically consistent across annotators and datasets.
No inter-annotator agreement or relabeling guidelines are given (Sections IV.A, IV.G); all headline metrics are computed against these labels.
domain assumption The 20,000 unlabeled samples share the label semantics and distribution of the labeled subset.
Self-training pseudo-labels remain informative only under this assumption; no distribution or label-prior analysis of the unlabeled pool is provided (Section III.C.1).
ad hoc to paper Platform-source metadata features are legitimate discriminators and do not create a shortcut that replaces content understanding.
Included without reliance analysis; on DUTA/CoDA (all dark web) these features are constant, so transfer behavior may change (Section III.B.2(c), Section IV.G).
standard math Shannon entropy and the Friedman test are valid standard statistics.
Used in Equations (2)-(4) and Section IV.D; uncontroversial background.

invented entities (1)

MEC/MEW entropy-based weighting criteria
purpose: To weight the XGB, RF, and SVM base learners in the first-stage ensemble so that confidently correct and cautiously wrong classifiers dominate the vote.
Defined in Equations (2)-(3) and (5); their benefit is demonstrated only on the authors' test set (Table VI) without external replication or component ablation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Language Model-Driven Semi-Supervised Ensemble Framework for Illicit Market Detection Across Deep/Dark Web and Social Platforms." pith.science (2026). https://pith.science/paper/6SBILALS

@misc{pith2026250722912,
  author       = {Pith},
  title        = {Pith review of: A Language Model-Driven Semi-Supervised Ensemble Framework for Illicit Market Detection Across Deep/Dark Web and Social Platforms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6SBILALS}},
  note         = {Machine review of arXiv:2507.22912}
}

read the original abstract

Illegal marketplaces have increasingly shifted to concealed parts of the internet, including the deep and dark web, as well as platforms such as Telegram, Reddit, and Pastebin. These channels enable the anonymous trade of illicit goods including drugs, weapons, and stolen credentials. Detecting and categorizing such content remains challenging due to limited labeled data, the evolving nature of illicit language, and the structural heterogeneity of online sources. This paper presents a hierarchical classification framework that combines fine-tuned language models with a semi-supervised ensemble learning strategy to detect and classify illicit marketplace content across diverse platforms. We extract semantic representations using ModernBERT, a transformer model for long documents, finetuned on domain-specific data from deep and dark web pages, Telegram channels, Subreddits, and Pastebin pastes to capture specialized jargon and ambiguous linguistic patterns. In addition, we incorporate manually engineered features such as document structure, embedded patterns including Bitcoin addresses, emails, and IPs, and metadata, which complement language model embeddings. The classification pipeline operates in two stages. The first stage uses a semi-supervised ensemble of XGBoost, Random Forest, and SVM with entropy-based weighted voting to detect sales-related documents. The second stage further classifies these into drug, weapon, or credential sales. Experiments on three datasets, including our multi-source corpus, DUTA, and CoDA, show that our model outperforms several baselines, including BERT, ModernBERT, DarkBERT, ALBERT, Longformer, and BigBird. The model achieves an accuracy of 0.96489, an F1-score of 0.93467, and a TMCC of 0.95388, demonstrating strong generalization, robustness under limited supervision, and effectiveness in real-world illicit content detection.

Figures

Figures reproduced from arXiv: 2507.22912 by the authors.

**Figure 2.** The proposed methodology overview feature extraction phase is conducted on each document to generate rich textual and structural representation. These representations serve as the foundation for training our sequential classification framework, which aims to accurately identify and categorize illicit marketplaces. The subsequent subsections provide a detailed explanation of each phase in the proposed pipeline. A. D… view at source ↗

**Figure 3.** Flowchart of the proposed sequential classification [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Performance comparison of text representation tech [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

**Figure 5.** Figure 5: Performance of SSE Model with Varying Labeled Data [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Two-Stage Swarm Intelligence Ensemble Deep Transfer Learning (SI-EDTL) for Vehicle Detection Using Unmanned Aerial Vehicles
cs.CV 2025-09 reject novelty 1.0 of 10

SI-EDTL, a 2022 ensemble of three pre-trained Faster R-CNNs with five classifiers tuned by WOA, is re-presented here with 91.3% accuracy on AU-AIR, but no code, error bars, or fair baseline comparisons are provided.

Reference graph

Works this paper leans on

37 extracted references · 30 canonical work pages · cited by 1 Pith paper

[1]

Information needs of drug users on a local dark web marketplace,

A. Haasio, J. T. Harviainen, and R. Savolainen, “Information needs of drug users on a local dark web marketplace,” Information Processing & Management, vol. 57, no. 2, p. 102080, 2020

work page 2020
[2]

Takedown of notorious hacker marketplace selling your identity to criminals,

EUROPOL, “Takedown of notorious hacker marketplace selling your identity to criminals,” Apr. 2023. [Online]. Available: https://www.europol.europa.eu/media-press/newsroom/news/ takedown-of-notorious-hacker-marketplace-selling-your-identity-to-criminals

work page 2023
[3]

Rydox cybercrime marketplace shut down and three administrators arrested,

United States Department of Justice, “Rydox cybercrime marketplace shut down and three administrators arrested,” Dec

work page
[4]

An unsupervised model for identifying and characterizing dark web forums,

S. Nazah, S. Huda, J. H. Abawajy, and M. M. Hassan, “An unsupervised model for identifying and characterizing dark web forums,” IEEE Access, vol. 9, pp. 112 871–112 892, 2021

work page 2021
[5]

Analyzing the darknetmarkets subreddit for evolutions of tools and trends using lda topic modeling,

K. Porter, “Analyzing the darknetmarkets subreddit for evolutions of tools and trends using lda topic modeling,” Digital Investigation, vol. 26, pp. S87–S97, 2018

work page 2018
[6]

Identifying and categorizing malicious content on paste sites: a neural topic modeling approach,

T. Vahedi, B. Ampel, S. Samtani, and H. Chen, “Identifying and categorizing malicious content on paste sites: a neural topic modeling approach,” in 2021 IEEE International Conference on Intelligence and Security Informatics (ISI) . IEEE, 2021, pp. 1–6

work page 2021
[7]

A vector space model for automatic indexing,

G. Salton, A. Wong, and C.-S. Yang, “A vector space model for automatic indexing,” Communications of the ACM , vol. 18, no. 11, pp. 613–620, 1975

work page 1975
[8]

Distributed representations of sentences and documents,

Q. Le and T. Mikolov, “Distributed representations of sentences and documents,” in International conference on machine learning . PMLR, 2014, pp. 1188–1196

work page 2014

Show all 37 references

[9]

Glove: Global vectors for word representation,

J. Pennington, R. Socher, and C. D. Manning, “Glove: Global vectors for word representation,” in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) , 2014, pp. 1532–1543

2014
[10]

Identification of chinese dark jargons in telegram underground markets using context-oriented and linguistic features,

Y . Hou, H. Wang, and H. Wang, “Identification of chinese dark jargons in telegram underground markets using context-oriented and linguistic features,” Information Processing & Management , vol. 59, no. 5, p. 103033, 2022

2022
[11]

Bert: Pre-training of deep bidirectional transformers for language understanding,

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Pro- ceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technolo...

2019
[12]

Albert: A lite bert for self-supervised learning of language representa- tions,

Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, “Albert: A lite bert for self-supervised learning of language representa- tions,” arXiv preprint arXiv:1909.11942 , 2019

1909 arXiv
[13]

Roberta: A robustly optimized bert pretraining approach,

Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692 , 2019

1907 arXiv
[14]

Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,

B. Warner, A. Chaffin, B. Clavi ´e, O. Weller, O. Hallstr ¨om, S. Taghadouini, A. Gallagher, R. Biswas, F. Ladhak, T. Aarsen et al. , “Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,” arXiv ...

2024 arXiv
[15]

Yazdanjue, M

N. Yazdanjue, M. Rakhshaninejad, H. Yazdanjouei, M. S. Niemel ¨a, F. Chen, and A. H. Gandomi, “Cyber threat management using semi- supervised ensemble learning and enhanced interior search algorithm: applications for illicit marketplace classification in deep/dark web and soci...

2025
[16]

Chapelle, B

O. Chapelle, B. Sch ¨olkopf, and A. Zien, Semi-Supervised Learning in Practice. MIT Press, 2006, pp. 331–331

2006
[17]

Zhu and A

X. Zhu and A. Goldberg, Introduction to semi-supervised learning . Morgan & Claypool Publishers, 2009

2009
[18]

Darknet and deepnet mining for proactive cybersecurity threat intelligence,

E. Nunes, A. Diab, A. Gunn, E. Marin, V . Mishra, V . Paliath, J. Robert- son, J. Shakarian, A. Thart, and P. Shakarian, “Darknet and deepnet mining for proactive cybersecurity threat intelligence,” in 2016 IEEE Conference on Intelligence and Security Informatics (ISI) . IEEE,...

2016
[19]

Classifying illegal activities on tor network based on web textual contents,

M. W. Al Nabki, E. Fidalgo, E. Alegre, and I. De Paz, “Classifying illegal activities on tor network based on web textual contents,” in Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers , 2017, pp. 35–43

2017
[20]

An expert system for classifying harmful content on the dark web,

H. Kobayashi, M. Kadoguchi, S. Hayashi, A. Otsuka, and M. Hashimoto, “An expert system for classifying harmful content on the dark web,” in 2020 IEEE International Conference on Intelligence and Security Informatics (ISI). IEEE, 2020, pp. 1–6

2020
[21]

Building an ai/ml based classification framework for dark web text data,

C. A. Murty, H. Rana, R. Verma, R. Pathak, and P. H. Rughani, “Building an ai/ml based classification framework for dark web text data,” in Proceedings of International Conference on Computing and Communication Networks: ICCCN 2021 . Springer, 2022, pp. 93–111

2021
[22]

A hybrid tf-idf and rnn model for multi-label classification of the deep and dark web,

A. Dalvi, S. Bhoir, N. Naik, A. Kitkaru, I. Siddavatam, and S. Bhirud, “A hybrid tf-idf and rnn model for multi-label classification of the deep and dark web,” International Journal of Advanced Computer Science and Applications, vol. 14, no. 7, 2023

2023
[23]

Prediction search on the threat of darknet based on machine learning,

J. Qian, A. Li, Y . Chen, and X. Zhou, “Prediction search on the threat of darknet based on machine learning,” in 2023 IEEE International Con- 16 ference on Control, Electronics and Computer Technology (ICCECT) . IEEE, 2023, pp. 876–880

2023
[24]

Extracting cyber threat intelligence from hacker forums: Support vector machines versus convolutional neural networks,

I. Deliu, C. Leichter, and K. Franke, “Extracting cyber threat intelligence from hacker forums: Support vector machines versus convolutional neural networks,” in 2017 IEEE International Conference on Big Data (Big Data). IEEE, 2017, pp. 3648–3656

2017
[25]

Darkembed: Exploit prediction with neural language models,

N. Tavabi, P. Goyal, M. Almukaynizi, P. Shakarian, and K. Lerman, “Darkembed: Exploit prediction with neural language models,” in Pro- ceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018

2018
[26]

Exploring the dark web for cyber threat intelligence using machine leaning,

M. Kadoguchi, S. Hayashi, M. Hashimoto, and A. Otsuka, “Exploring the dark web for cyber threat intelligence using machine leaning,” in 2019 IEEE International Conference on Intelligence and Security Informatics (ISI). IEEE, 2019, pp. 200–202

2019
[27]

Distilling contextual embeddings into a static word embedding for improving hacker forum analytics,

B. Ampel and H. Chen, “Distilling contextual embeddings into a static word embedding for improving hacker forum analytics,” in 2021 IEEE International Conference on Intelligence and Security Informatics (ISI) . IEEE, 2021, pp. 1–3

2021
[28]

Identification of cybersecurity specific content using different language models,

O. Mendsaikhan, H. Hasegawa, Y . Yamaguchi, H. Shimada, and E. Bataa, “Identification of cybersecurity specific content using different language models,” Journal of Information Processing, vol. 28, pp. 623– 632, 2020

2020
[29]

Shedding new light on the language of the dark web,

Y . Jin, E. Jang, Y . Lee, S. Shin, and J.-W. Chung, “Shedding new light on the language of the dark web,” arXiv preprint arXiv:2204.06885 , 2022

2022 arXiv
[30]

Illicit darkweb classification via natural-language processing: Classifying illicit con- tent of webpages based on textual information,

G. Cascavilla, G. Catolino, and M. Sangiovanni, “Illicit darkweb classification via natural-language processing: Classifying illicit con- tent of webpages based on textual information,” arXiv preprint arXiv:2312.04944, 2023

2023 arXiv
[31]

Darkbert: A language model for the dark side of the internet,

Y . Jin, E. Jang, J. Cui, J.-W. Chung, Y . Lee, and S. Shin, “Darkbert: A language model for the dark side of the internet,” arXiv preprint arXiv:2305.08596, 2023

2023 arXiv
[32]

Towards safe cyber practices: Developing a proactive cyber-threat intelligence system for dark web forum content by identifying cybercrimes,

K. S. Sangher, A. Singh, H. M. Pandey, and V . Kumar, “Towards safe cyber practices: Developing a proactive cyber-threat intelligence system for dark web forum content by identifying cybercrimes,” Information, vol. 14, no. 6, p. 349, 2023

2023
[33]

An analysis of topic modeling approaches for unlabeled dark web data classification,

A. Dalvi, V . Joshi, A. Warior, D. Nair, and S. Bhirud, “An analysis of topic modeling approaches for unlabeled dark web data classification,” in International Conference on Innovations and Advances in Cognitive Systems. Springer, 2024, pp. 150–162

2024
[34]

A mathematical theory of communication,

C. E. Shannon, “A mathematical theory of communication,” The Bell system technical journal , vol. 27, no. 3, pp. 379–423, 1948

1948
[35]

Longformer: The long- document transformer,

I. Beltagy, M. E. Peters, and A. Cohan, “Longformer: The long- document transformer,” arXiv preprint arXiv:2004.05150 , 2020

2004 arXiv
[36]

Big bird: Transformers for longer sequences,

M. Zaheer, G. Guruganesh, K. A. Dubey, J. Ainslie, C. Alberti, S. Ontanon, P. Pham, A. Ravula, Q. Wang, L. Yang et al. , “Big bird: Transformers for longer sequences,” Advances in neural information processing systems, vol. 33, pp. 17 283–17 297, 2020

2020
[2024]

Available: https://www.justice.gov/archives/opa/pr/ rydox-cybercrime-marketplace-shut-down-and-three-administrators-arrested

[Online]. Available: https://www.justice.gov/archives/opa/pr/ rydox-cybercrime-marketplace-shut-down-and-three-administrators-arrested

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Information needs of drug users on a local dark web marketplace,

A. Haasio, J. T. Harviainen, and R. Savolainen, “Information needs of drug users on a local dark web marketplace,” Information Processing & Management, vol. 57, no. 2, p. 102080, 2020

work page 2020

[2] [2]

Takedown of notorious hacker marketplace selling your identity to criminals,

EUROPOL, “Takedown of notorious hacker marketplace selling your identity to criminals,” Apr. 2023. [Online]. Available: https://www.europol.europa.eu/media-press/newsroom/news/ takedown-of-notorious-hacker-marketplace-selling-your-identity-to-criminals

work page 2023

[3] [3]

Rydox cybercrime marketplace shut down and three administrators arrested,

United States Department of Justice, “Rydox cybercrime marketplace shut down and three administrators arrested,” Dec

work page

[4] [4]

An unsupervised model for identifying and characterizing dark web forums,

S. Nazah, S. Huda, J. H. Abawajy, and M. M. Hassan, “An unsupervised model for identifying and characterizing dark web forums,” IEEE Access, vol. 9, pp. 112 871–112 892, 2021

work page 2021

[5] [5]

Analyzing the darknetmarkets subreddit for evolutions of tools and trends using lda topic modeling,

K. Porter, “Analyzing the darknetmarkets subreddit for evolutions of tools and trends using lda topic modeling,” Digital Investigation, vol. 26, pp. S87–S97, 2018

work page 2018

[6] [6]

Identifying and categorizing malicious content on paste sites: a neural topic modeling approach,

T. Vahedi, B. Ampel, S. Samtani, and H. Chen, “Identifying and categorizing malicious content on paste sites: a neural topic modeling approach,” in 2021 IEEE International Conference on Intelligence and Security Informatics (ISI) . IEEE, 2021, pp. 1–6

work page 2021

[7] [7]

A vector space model for automatic indexing,

G. Salton, A. Wong, and C.-S. Yang, “A vector space model for automatic indexing,” Communications of the ACM , vol. 18, no. 11, pp. 613–620, 1975

work page 1975

[8] [8]

Distributed representations of sentences and documents,

Q. Le and T. Mikolov, “Distributed representations of sentences and documents,” in International conference on machine learning . PMLR, 2014, pp. 1188–1196

work page 2014