Semantic-Aware Contrastive Fine-Tuning: Boosting Multimodal Malware Classification with Discriminative Embeddings

Aritran Piplai; Ivan Montoya Sanchez; Shaswata Mitra; Sudip Mittal

REVIEW 4 major objections 4 minor 24 references

Semantic-Aware Contrastive Fine-Tuning: Boosting Multimodal Malware Classification with Discriminative Embeddings

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that selecting hard negatives by cosine similarity during contrastive fine-tuning makes LLM embeddings discriminative enough to materially improve few-shot malware family classification, reporting 63.15% accuracy with 20…

desk verdict The hard-negative selection idea is sensible, but the missing train/eval split makes the headline gains unverifiable. read the letter →

arxiv 2504.21028 v1 pith:WMCWJGLO submitted 2025-04-25 cs.CR cs.AIcs.LG

Ivan Montoya Sanchez , Shaswata Mitra , Aritran Piplai , Sudip Mittal This is my paper · ORCID

classification cs.CRcs.AIcs.LG

keywords malwareclassificationcontrastivefine-tuninghardnegativesamplingmultimodallearningfew-shotLLMembeddingscybersecurityknowledgedistillation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Malware families keep spawning new variants, and analysts often share text descriptions rather than raw binaries. The paper claims that standard LLM embeddings are too overlapping to tell closely related malware families apart, and that a contrastive fine-tuning step fixes this when the negative examples are chosen by cosine similarity rather than at random. On two public malware datasets, the fine-tuned embeddings, paired with binary features in a few-shot meta-learning classifier, lift accuracy by 11–21 percentage points over an attribute-only baseline, reaching 63.15% accuracy with 20 samples per family. If true, defenders could classify new variants from natural-language threat reports without large labeled sets.

What carries the argument

The load-bearing mechanism is a similarity-ranked negative-selection procedure wrapped around the InfoNCE contrastive loss. Hard negatives are descriptions from other families that look almost identical to the positive in the pre-trained embedding space; the procedure selects the top 20 by cosine similarity and adds 12 randomly drawn mid-similarity negatives. Each anchor-positive pair is trained with five hard and three diverse negatives per contrastive sample. The refined embeddings, obtained by mean pooling final hidden states, are then evaluated in a Model-Agnostic Meta-Learning (MAML) multimodal classifier that fuses them with binary attributes through knowledge distillation. This negative-selection rule is what forces the model to repel near-identical descriptions while retaining enough diversity to generalize.

What would settle it

Run the pipeline again under a strict family-level hold-out, training contrastive fine-tuning only on families excluded from the few-shot evaluation, and compare accuracy with the reported setup. If the 11–21 point gain mostly disappears, the embeddings benefited from exposure to test-family descriptions rather than from the hard-negative selection rule.

Watch

Extended reading notes

Core claim

The central discovery is that negative-sample selection, not the contrastive loss itself, determines whether LLM text embeddings become discriminative for malware families. For each family's expert-written reference description, the authors rank candidate descriptions from other families by cosine similarity and keep the top 20 (scores around 0.85–0.95) as hard negatives, plus 12 mid-tier negatives for diversity. InfoNCE training on anchor-positive-hard-negative triples then separates families whose language is nearly indistinguishable in the pre-trained space. The refined embeddings are fused with binary behavioral features in a Model-Agnostic Meta-Learning classifier with knowledge distillation, and the paper reports consistent gains over random-negative contrastive fine-tuning and over pre-trained embeddings alone. The stated interpretation is that the model learns finer semantic distinctions and generates more attribute-specific descriptions that generalize to unseen samples.

Load-bearing premise

The reported gains assume the contrastive fine-tuning step never saw descriptions from the malware families used in the few-shot evaluation; the paper does not state that such a split was made.

Editorial extensions

If this is right

If the claim holds, LLM-generated malware descriptions become usable as a classification signal, not just analyst reading material, because fine-tuned embeddings separate families that raw embeddings blur.
Few-shot accuracy of 63.15% with 20 samples per family on CIC-AndMal-2020 means new families could be handled with tiny labeled sets, which matters when samples are scarce and reports are abundant.
Similarity-based negative selection generalizes across model sizes and datasets, with larger models gaining more, so the method scales with better base LLMs.
Because the same embeddings improve RAGAS correctness and similarity scores, the fine-tuned models produce descriptions that are both more machine-actionable and more interpretable for analysts.
The multimodal fusion with binary features means the text modality adds information beyond dynamic or static attributes rather than substituting for them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the same cosine-similarity hard-negative rule should transfer to other high-overlap text classification tasks, such as phishing reports, fraud narratives, or medical symptom notes, wherever a similarity-ranked candidate pool can be built.
Editorial inference: a strict temporal evaluation, fine-tuning on older variants and evaluating on newer variants of the same families, would test whether the generalization claim reflects family-level attributes or dataset-specific phrasing.
Editorial inference: publishing per-family confusion matrices would show whether the gains come from separating the hardest overlapping pairs, such as the Android adware families, or from easier families, and would isolate the mechanism.
Editorial inference: the method relies on one expert-written ground-truth description per family; testing multiple or crowd-sourced positives would show whether the hard-negative signal survives positive-sample variation.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The short version: the method is a reasonable extension of known contrastive tricks to LLM-generated malware descriptions, and the evaluation is broad enough to be interesting. But the paper never states whether the contrastive fine-tuning data and the MAML evaluation data are disjoint, and that gap makes the reported 11–21 point gains hard to trust as evidence of generalization.

What's actually new: the two-tier negative selection—taking the top-20 highest cosine-similarity negatives plus a few mid-tier ones—applied to LLM embeddings of malware descriptions before feeding them into a MAML few-shot classifier. That specific combination is not in the cited work, and the consistent direction of results across four LLMs and two datasets suggests the effect is real, not a single-model fluke. The ablation showing similarity-based negatives beating random negatives by 10–20 points is also a useful data point. Credit where due: the method is described well enough to re-implement, and the RAGAS evaluation of description quality goes beyond raw accuracy.

Now the soft spots, in proportion. The load-bearing one is the split question. The paper says 200 anchors per family are generated and each anchor is paired with that family's expert ground-truth description. If those anchors come from the same samples, or even the same families, used later in the MAML support/query sets, then the fine-tuned LLM has effectively seen the family labels during contrastive training. The MAML classifier then only needs to pick up on an embedding space that was explicitly shaped to separate those families. That would make the gains a measure of exposure, not generalization. The paper never clarifies this, and it's easy to fix—just state whether families were held out, and if not, frame the claim as improving same-family few-shot classification rather than generalization to new families.

Minor but real: no error bars or significance tests, single-run accuracies, hand-picked thresholds (T, negative counts) with no sensitivity analysis, and no code release. The claim of being the "first multimodal malware classification" is also an overclaim, contradicted by their own related-work section. There's a duplicated paragraph in Section II.D and an incomplete sentence in Section III.B.

The method might well work; the evidence just doesn't yet support the strong version of the claim. This deserves a serious referee, not a desk reject—but the referee should insist on the data split, variance estimates, and ideally code.

For peer review: yes, send it out. For my own citation list: not in the next year, unless the split question gets resolved.

Referee Report

4 major / 4 minor

Summary. The paper proposes a contrastive fine-tuning (CFT) method that selects hard negative samples by cosine similarity in the embedding space of LLM-generated malware descriptions, and combines the resulting embeddings with binary behavioral features in a MAML-based few-shot multimodal classifier. Experiments on CIC-AndMal-2020 and BODMAS report large accuracy gains over pre-trained embeddings and over random negative sampling, with the best result being 63.15% on CIC-AndMal-2020, and a RAGAS-based evaluation of description quality is also included.

Significance. If the reported results are valid, the paper makes a useful contribution: it demonstrates a concrete way to adapt LLM embeddings for fine-grained malware family discrimination and shows consistent gains across four LLMs and two datasets, which is a nontrivial qualitative pattern. The multimodal integration of fine-tuned text embeddings with behavioral features under MAML is also an interesting direction. However, the central claim depends on the contrastive fine-tuning data being disjoint from the few-shot evaluation data in a way that the manuscript never states, and the quantitative comparisons lack any uncertainty estimation, so the significance of the claimed 11--21 percentage point improvements is currently not established.

major comments (4)

[Sections III.B--III.C and IV.B] The manuscript never describes a data split between the contrastive fine-tuning (CFT) training data and the MAML evaluation data. Section III.B says 200 anchors are generated per family from the same two datasets, and Section III.C selects hard negatives from other families' descriptions using the family-specific ground-truth description. Section IV.B then evaluates support and query sets per family on those same datasets. If the anchors or their source samples overlap with the 10+20 support/query samples, or if the expert ground-truth descriptions of the evaluation families were used as positives during CFT, the reported gains in Tables III--V could reflect exposure to the test families rather than generalization. This is load-bearing for every accuracy comparison in the paper, so the authors must specify the exact partition (sample-level and family-level) and confirm that no information from the evaluation families or samples enters CFT.
[Tables III--V] Each accuracy in Tables III, IV, and V is reported as a single number with no error bars, no number of random trials, and no significance tests. Because the MAML pipeline and the negative sampling procedure involve randomness (batch composition, random mid-tier negatives, initialization), the 11--21 percentage point improvements and the ablation differences of 10--23% could be within run-to-run variation. The authors should report means and standard deviations over multiple runs and, ideally, a paired significance test for the random-vs-similarity ablation.
[Abstract and Section IV] The abstract and introduction claim that the method surpasses 'prior negative sampling strategies,' but the experiments only compare similarity-based negatives with random negatives. No comparison is made to the class-label-based negatives of Supervised Contrastive Learning [7], the semantic clustering approach [10], or the distance-aware approach of Xu et al. [11], despite these being discussed in the related work. Either add these comparisons or remove the claim about surpassing prior strategies.
[Section III.A] The BODMAS feature selection is performed using LightGBM feature importance on the full dataset before any train/test split is described. Selecting the top 64 features (and limiting to 15 families based on sufficient samples) using the entire dataset can leak information from the test portion into the model pipeline. The authors should clarify whether feature selection was done inside a cross-validation or on a separate training fold, and report how the family subset was chosen.

minor comments (4)

[Section II.D] The paragraph beginning 'Meta-learning, particularly Model-Agnostic Meta-Learning (MAML) [6]...' is duplicated verbatim, first in the discussion of meta-adaptation and then again in the knowledge-distillation paragraph.
[Figure 2 caption and text in Section III.B] The sentence referencing Figure 2 is incomplete: 'Figure 2 shows the . Although some of the families...' contains a dangling phrase and a missing noun. This needs to be rewritten for clarity.
[Algorithm 1] Algorithm 1 adds candidate negatives only when s ≤ T, where T is described as the 'Maximum Similarity Threshold,' but the text describes hard negatives as having high cosine similarity in a range 0.85--0.95. It is unclear whether T is an upper bound, a lower bound, or a cutoff that then requires the descending sort to find the top-20 hardest negatives. Clarify the role of T.
[Reference [16]] Reference [16], about multimodal detection of hateful memes, does not appear relevant to malware description generation or low-level feature analysis in the sentence where it is cited. A more germane reference on LLM-based malware analysis would be appropriate.

Circularity Check

1 steps flagged · score 2.0 of 10

RAGAS evaluation partly measures the CFT training objective; no self-citation chain or forced prediction found in the main classification result.

fitted input called prediction [Section III.B/III.D (positive sampling and InfoNCE) vs. Section IV.A (RAGAS evaluation), Table II]
"Each anchor is paired with an expert-generated ground-truth description (positive sample) specific to its malware family. ... we employ RAGAS metrics (Answer Correctness and Similarity) to assess the quality of LLM-generated malware descriptions."

The InfoNCE objective pulls the anchor embedding toward the embedding of the expert ground-truth description selected as the positive sample. The RAGAS Answer Correctness and Similarity metrics then score LLM-generated descriptions against those same expert ground-truth narratives. The reported Table II gains are therefore partly a re-measurement of the training target (similarity/alignment to the positive text), not an independent test of description quality. This does not invalidate the main MAML accuracy comparison, which uses a downstream classification metric, but Table II should not be read as independent validation of the embeddings.

full rationale

The central claim is not circular by construction: the CFT method optimizes a standard InfoNCE loss with negatives selected by cosine similarity, and the headline evaluation is MAML classification accuracy on support/query sets from two external datasets (CIC-AndMal-2020, BODMAS). That accuracy is an independent, externally meaningful metric and could have rejected the method, so the result is not a fitted parameter renamed as a prediction. The paper contains no load-bearing self-citations; its key references (MAML, InfoNCE, the datasets) are external. The only internal-dependency step found is the RAGAS evaluation, whose reference text is identical to the CFT positive-sample text, so Table II partly reports training-objective satisfaction. A separate experimental-control issue, not definitional circularity, is that no split is described between the 200 per-family anchors used in CFT and the 10-support/20-query samples used in MAML evaluation; if the sets overlap, the classification gains could include label exposure. That is a missing-control concern, not a circular reduction, so it is noted but not scored as a circular step.

Assumptions & free parameters 9 free parameters · 4 assumptions · 0 invented entities

The method has no invented physical or conceptual entities. Its central claim depends mainly on modeling assumptions and hand-set selection counts. The most consequential assumption is that the contrastive fine-tuning data and the few-shot evaluation data are separated; this is never stated. The free parameters are mostly standard hyperparameters, but several are data-dependent (similarity range, T, feature count) and lack sensitivity analysis.

free parameters (9)

InfoNCE temperature tau = 0.07
Fixed temperature in the contrastive loss (Section III.D), no sensitivity analysis.
Maximum similarity threshold T (Algorithm 1) = Not reported
Gate for candidate negatives; the paper only notes selected hard negatives typically fall in the 0.85 to 0.95 cosine similarity range, suggesting data-dependent tuning.
High-tier negative count per family = 20
Top 20 most similar negatives selected per family (Algorithm 1).
Mid-tier negative count per family = 12
Randomly selected mid-similarity negatives per family (Section III.C).
Per-anchor contrastive composition = 5 high plus 3 mid negatives, 4 samples per anchor
Each anchor-positive pair is combined into four training samples (Section III.C).
Anchors per family = 200
200 model-inferred descriptions per family for each dataset (Section III.B).
CFT training schedule = 1 epoch, learning rate 1e-5, batch size 32, AdamW
Hyperparameters listed without sensitivity analysis (Section III.D).
BODMAS feature count = 64
Top 64 features by LightGBM importance from 2,380 static features (Section III.A).
MAML hyperparameters = Not reported
Inner and outer loop learning rates, adaptation steps, and task sampling are not given, though they affect the reported accuracies (Section IV.B).

assumptions (4)

domain assumption LLM-generated textual descriptions built only from behavioral attributes carry enough family-discriminative signal for classification.
Anchors are generated from attribute lists (Section III.B) and the classifier relies on these embeddings; if descriptions are not informative, the gains do not transfer.
ad hoc to paper A single expert-written ground-truth description per family is a valid positive anchor for every sample in that family.
Algorithm 1 selects exactly one d_i per family, ignoring intra-family description diversity; this may fail for polymorphic or multi-behavior malware.
domain assumption The MAML support and query protocol measures generalization to unseen malware without leakage from contrastive fine-tuning.
Section IV.B describes support and query sets but never states that evaluation families or samples were excluded from CFT data construction; without this, reported few-shot gains may reflect memorization.
domain assumption InfoNCE contrastive fine-tuning of LLM embeddings transfers to downstream few-shot classifier accuracy.
The paper assumes the standard contrastive-learning transfer premise for malware descriptions; it is plausible but not independently established here.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Semantic-Aware Contrastive Fine-Tuning: Boosting Multimodal Malware Classification with Discriminative Embeddings." pith.science (2026). https://pith.science/paper/WMCWJGLO

@misc{pith2026250421028,
  author       = {Pith},
  title        = {Pith review of: Semantic-Aware Contrastive Fine-Tuning: Boosting Multimodal Malware Classification with Discriminative Embeddings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WMCWJGLO}},
  note         = {Machine review of arXiv:2504.21028}
}

read the original abstract

The rapid evolution of malware variants requires robust classification methods to enhance cybersecurity. While Large Language Models (LLMs) offer potential for generating malware descriptions to aid family classification, their utility is limited by semantic embedding overlaps and misalignment with binary behavioral features. We propose a contrastive fine-tuning (CFT) method that refines LLM embeddings via targeted selection of hard negative samples based on cosine similarity, enabling LLMs to distinguish between closely related malware families. Our approach combines high-similarity negatives to enhance discriminative power and mid-tier negatives to increase embedding diversity, optimizing both precision and generalization. Evaluated on the CIC-AndMal-2020 and BODMAS datasets, our refined embeddings are integrated into a multimodal classifier within a Model-Agnostic Meta-Learning (MAML) framework on a few-shot setting. Experiments demonstrate significant improvements: our method achieves 63.15% classification accuracy with as few as 20 samples on CIC-AndMal-2020, outperforming baselines by 11--21 percentage points and surpassing prior negative sampling strategies. Ablation studies confirm the superiority of similarity-based selection over random sampling, with gains of 10-23%. Additionally, fine-tuned LLMs generate attribute-aware descriptions that generalize to unseen variants, bridging textual and binary feature gaps. This work advances malware classification by enabling nuanced semantic distinctions and provides a scalable framework for adapting LLMs to cybersecurity challenges.

Figures

Figures reproduced from arXiv: 2504.21028 by the authors.

**Figure 1.** Overview of our similarity-based contrastive fine-tuning framework for malware classification. Initially, embeddings from a pre-trained LLM exhibit significant overlap among malware families, leading to ambiguous descriptions and poor classification. In contrast, in similarity-based contrastive fine-tuning, embeddings become discriminative, clearly separating malware families into distinct clusters. This improved e… view at source ↗

**Figure 2.** Visualization of the pre-trained embedding space of malware descriptions generated by LLaMA-3.1-8B, projected into two dimensions using UMAP. Each color represents a different malware family. The significant overlap and lack of clearly defined clusters demonstrate the pre-trained model’s limited capability to semantically distinguish among closely related malware families. split frequency. We then select the top 64 … view at source ↗

**Figure 3.** Distribution of cosine similarity scores between candidate negative samples and ground-truth descriptions for LLaMA-3.2-1B. Hard negatives are selected from the right end of the distribution, exhibiting the highest semantic similarity to the ground-truth. These samples create a more challenging CFT setting by forcing the model to distinguish between highly similar descriptions. TABLE I: Description of Notations Nota… view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Comparison of LLaMA-3.1-8B embedding spaces for CIC-AndMal-2020 dataset: pre-trained vs. contrastive fine-tuning with random and similarity-based hard negatives. Each color represents a malware family. (a) Pre-trained (b) Random Negatives (CFT) (c) Similarity-based Neg…

**Figure 5.** Figure 5: Comparison of LLaMA-3.1-8B embedding spaces for BODMAS dataset: pre-trained vs. contrastive fine-tuning with random and similarity-based hard negatives. Each color represents a malware family. on dynamic attributes (binary features) provides soft labels to guide the mu…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 14 canonical work pages

[7]

Supervised contrastive learning for pre-trained language model fine-tuning

Beliz Gunel, Jingfei Du, Alexis Conneau, and Ves Stoyanov. Supervised contrastive learning for pre-trained language model fine-tuning. arXiv preprint arXiv:2011.01403, 2020

arXiv 2011
[10]

Contrastive learning with hard negative samples

Joshua Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka. Contrastive learning with hard negative samples. arXiv preprint arXiv:2010.04592, 2020

arXiv 2010
[11]

Enhancing information maximization with distance-aware contrastive learning for source-free cross-domain few- shot learning

Huali Xu, Li Liu, Shuaifeng Zhi, Shaojing Fu, Zhuo Su, Ming-Ming Cheng, and Yongxiang Liu. Enhancing information maximization with distance-aware contrastive learning for source-free cross-domain few- shot learning. IEEE Transactions on Image Processing , 33:2058–2070, 2024

work page 2024
[1]

statista.com/statistics/1491093/new-malware-variants-detected- worldwide, 2025

Annual number of new malware variants detected worldwide from 2019 to 2023. statista.com/statistics/1491093/new-malware-variants-detected- worldwide, 2025

work page arXiv 2019
[2]

Llms are few-shot in-context low-resource language learners

Samuel Cahyawijaya, Holy Lovenia, and Pascale Fung. Llms are few-shot in-context low-resource language learners. arXiv preprint arXiv:2403.16512, 2024

arXiv 2024
[3]

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748–8763. PmLR, 2021

2021
[4]

unb.ca/cic/datasets/andmal2020.html, 2020

Cccs-cic-andmal-2020. unb.ca/cic/datasets/andmal2020.html, 2020

work page 2020
[5]

Bodmas: An open dataset for learning based temporal analysis of pe malware

Limin Yang, Arridhana Ciptadi, Ihar Laziuk, Ali Ahmadzadeh, and Gang Wang. Bodmas: An open dataset for learning based temporal analysis of pe malware. In 4th Deep Learning and Security Workshop , 2021

work page 2021

Show all 24 references

[6]

Model-agnostic meta-learning for fast adaptation of deep networks

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning , pages 1126–1135. PMLR, 2017

2017
[8]

Representation learning with contrastive predictive coding

Aaron van den Oord et al. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 , 2018

2018 arXiv
[9]

Super- vised contrastive learning

Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Super- vised contrastive learning. Advances in neural information processing systems, 33:18661–18673, 2020

2020
[12]

Llama: Open and efficient foundation language models

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie- Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 , 2023

2023 arXiv
[13]

Can gpt models follow cybersecurity analyst tasks? arXiv preprint arXiv:2303.06545 , 2023

Shaza Alkhatib et al. Can gpt models follow cybersecurity analyst tasks? arXiv preprint arXiv:2303.06545 , 2023

2023 arXiv
[14]

Multimodal deep learning framework for malware detection

Muhammad Shafiq, Zhaoyan Tian, et al. Multimodal deep learning framework for malware detection. IEEE Access, 8:79347–79361, 2020

2020
[15]

Multimodal malware detection using deep learning

Taejoon Kim et al. Multimodal malware detection using deep learning. IEEE Access, 8:180012–180022, 2020

2020
[16]

Multimodal detection of hateful memes by applying a vision-language pre-training model

Yuyang Chen and Feng Pan. Multimodal detection of hateful memes by applying a vision-language pre-training model. PLOS ONE , 17(9):e0274300, 2022

2022
[17]

Meta-learning for few-shot intrusion detection

Hongliang Yao et al. Meta-learning for few-shot intrusion detection. In IEEE Conference on Communications and Network Security , 2019

2019
[18]

Meta-learning for cybersecurity tasks

Yuyang Wang et al. Meta-learning for cybersecurity tasks. IEEE Transactions on Information Forensics and Security , 2021

2021
[19]

Distilling the knowledge in a neural network

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 , 2015

2015 arXiv
[20]

Knowledge distillation: A survey

Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: A survey. International Journal of Computer Vision, 129:1789–1819, 2021

2021
[21]

Mul- timodal machine learning: A survey and taxonomy

Tadas Baltru ˇsaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Mul- timodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence , 41(2):423–443, 2018

2018
[22]

A survey on deep learning for multimodal data fusion

Jing Gao, Peng Li, Zhikang Chen, and Jianlong Zhang. A survey on deep learning for multimodal data fusion. Neural Computing and Applications, 32(10):6253–6271, 2020

2020
[23]

Learning robust representations for multimodal data with knowledge distillation

Qi Guo, Xiaohui Wang, Jing Wu, Xiao Liu, Qiao Guan, and Jingdong Zhang. Learning robust representations for multimodal data with knowledge distillation. Information Fusion, 86:86–98, 2022

2022
[24]

H. S. Anderson and P. Roth. EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models. ArXiv e-prints , April 2018

2018

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [7]

Supervised contrastive learning for pre-trained language model fine-tuning

Beliz Gunel, Jingfei Du, Alexis Conneau, and Ves Stoyanov. Supervised contrastive learning for pre-trained language model fine-tuning. arXiv preprint arXiv:2011.01403, 2020

arXiv 2011

[2] [10]

Contrastive learning with hard negative samples

Joshua Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka. Contrastive learning with hard negative samples. arXiv preprint arXiv:2010.04592, 2020

arXiv 2010

[3] [11]

Enhancing information maximization with distance-aware contrastive learning for source-free cross-domain few- shot learning

Huali Xu, Li Liu, Shuaifeng Zhi, Shaojing Fu, Zhuo Su, Ming-Ming Cheng, and Yongxiang Liu. Enhancing information maximization with distance-aware contrastive learning for source-free cross-domain few- shot learning. IEEE Transactions on Image Processing , 33:2058–2070, 2024

work page 2024

[4] [1]

statista.com/statistics/1491093/new-malware-variants-detected- worldwide, 2025

Annual number of new malware variants detected worldwide from 2019 to 2023. statista.com/statistics/1491093/new-malware-variants-detected- worldwide, 2025

work page arXiv 2019

[5] [2]

Llms are few-shot in-context low-resource language learners

Samuel Cahyawijaya, Holy Lovenia, and Pascale Fung. Llms are few-shot in-context low-resource language learners. arXiv preprint arXiv:2403.16512, 2024

arXiv 2024

[6] [3]

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748–8763. PmLR, 2021

2021

[7] [4]

unb.ca/cic/datasets/andmal2020.html, 2020

Cccs-cic-andmal-2020. unb.ca/cic/datasets/andmal2020.html, 2020

work page 2020

[8] [5]

Bodmas: An open dataset for learning based temporal analysis of pe malware

Limin Yang, Arridhana Ciptadi, Ihar Laziuk, Ali Ahmadzadeh, and Gang Wang. Bodmas: An open dataset for learning based temporal analysis of pe malware. In 4th Deep Learning and Security Workshop , 2021

work page 2021