REVIEW 14 cited by
A Cookbook of Self-Supervised Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning. Yet, much like cooking, training SSL methods is a delicate art with a high barrier to entry. While many components are familiar, successfully training a SSL method involves a dizzying set of choices from the pretext tasks to training hyper-parameters. Our goal is to lower the barrier to entry into SSL research by laying the foundations and latest SSL recipes in the style of a cookbook. We hope to empower the curious researcher to navigate the terrain of methods, understand the role of the various knobs, and gain the know-how required to explore how delicious SSL can be.
Forward citations
Cited by 14 Pith papers
-
Self-Soupervision: Cooking Model Soups without Labels
Model soups work when ingredient models are trained with different self-supervised objectives or on unlabeled data, improving robustness and transfer without labels.
-
Next-Latent Prediction Transformers Learn Compact World Models
NextLat augments next-token prediction with latent next-state prediction, theoretically converging latents to belief states and showing empirical gains in world modeling, reasoning, planning, and faster inference via ...
-
Where Do Backdoors Live? A Component-Level Analysis of Backdoor Propagation in Speech Language Models
Backdoors propagate through SLM components with persistence or erasure depending on the targeted part, and poisoned samples are not directly separable from benign ones in shared multitask embeddings.
-
Multi-Class-Token Transformer for Multitask Self-supervised Music Information Retrieval
A single ViT with two specialized class tokens, one per self-supervised pretext task, learns both timbre-focused and harmony-focused music representations and outperforms a much larger MLM model on most MIR benchmarks.
-
XxaCT-NN: Structure Agnostic Multimodal Learning for Materials Science
Composition plus simulated XRD, fused by cross-attention with a new masked-XRD pretraining objective, reaches near-structure-based property accuracy on 5M materials.
-
Data Normalization Strategies for EEG Deep Learning
Window-level, per-channel normalization helps supervised EEG tasks, while minimal or cross-channel window normalization suits contrastive self-supervised learning on EEG.
-
scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Data
A benchmark of 19 self-supervised learning methods on 9 single-cell datasets shows generic SSL methods outperform specialized frameworks on multi-modal integration and cell typing, while masking is the most effective ...
-
Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions
CHARM is a 7M-parameter self-supervised embedding model for multivariate time series that uses channel descriptions to beat specialized baselines on forecasting, classification, and anomaly detection.
-
Chained Recursive Language Models for Multi-Iteration Reasoning
Chained fresh-root model calls with plain-text artifacts improve reported long-context reasoning accuracy over a single-call baseline, but the evidence lacks error bars and compute-matched comparison.
-
Contrastive Self-Supervised Network Intrusion Detection using Augmented Negative Pairs
CLAN clusters genuine benign network flows while repelling augmented copies, then classifies new flows by distance to the cluster centroid; on Lycos2017 it reports the highest mean AUROC among compared SSL and anomaly...
-
Data-Driven Self-Supervised Learning for the Discovery of Solution Singularity for Partial Differential Equations
A density-filtering plus polynomial-fit pipeline detects PDE solution singularities from unlabeled adaptive mesh nodes.
-
CITADEL: Continual Anomaly Detection for Enhanced Learning in IoT Intrusion Detection
CITADEL combines self-supervised masked autoencoders with KL-divergence-based memory selection and a hierarchical buffer to detect IoT intrusions without attack labels while retaining old knowledge.
-
AI-Driven Climate Policy Scenario Generation for Sub-Saharan Africa
A RAG pipeline using llama3.2-3B and UN COP documents generated 34 policy scenarios for Sub-Saharan Africa, 30 passed author validation, but automated evaluation showed mixed agreement with human judgment.
-
Bridging Brain with Foundation Models through Self-Supervised Learning
A PRISMA-based survey maps self-supervised learning techniques, brain foundation models, datasets, and evaluation protocols for EEG and related neural signals, including a skeptical review of EEG-to-text decoding.
Discussion (0). Continue with ORCID to comment.