REVIEW 5 cited by
Transferability in Deep Learning: A Survey
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The success of deep learning algorithms generally depends on large-scale data, while humans appear to have inherent ability of knowledge transfer, by recognizing and applying relevant knowledge from previous learning experiences when encountering and solving unseen tasks. Such an ability to acquire and reuse knowledge is known as transferability in deep learning. It has formed the long-term quest towards making deep learning as data-efficient as human learning, and has been motivating fruitful design of more powerful deep learning algorithms. We present this survey to connect different isolated areas in deep learning with their relation to transferability, and to provide a unified and complete view to investigating transferability through the whole lifecycle of deep learning. The survey elaborates the fundamental goals and challenges in parallel with the core principles and methods, covering recent cornerstones in deep architectures, pre-training, task adaptation and domain adaptation. This highlights unanswered questions on the appropriate objectives for learning transferable knowledge and for adapting the knowledge to new tasks and domains, avoiding catastrophic forgetting and negative transfer. Finally, we implement a benchmark and an open-source library, enabling a fair evaluation of deep learning methods in terms of transferability.
Forward citations
Cited by 5 Pith papers
-
Decomposing the Time Series Forecasting Pipeline: A Modular Approach for Time Series Representation, Information Extraction, and Projection
REP-Net, a modular pipeline of representation, memory, and projection modules, achieves competitive forecasting accuracy on seven multivariate benchmarks with lower computational cost.
-
Human Heterogeneity Invariant Stress Sensing
HHISS uses person-wise pruning intersection, plus continuous-label regularization, to achieve state-of-the-art out-of-distribution stress detection across seven wearable datasets.
-
Towards Realistic Hand-Object Interaction with Gravity-Field Based Diffusion Bridge
A gravity-field and diffusion-based optimization refines hand-object contacts to reduce interpenetration and gaps while LLM text prompts guide contact regions.
-
rETF-semiSL: Semi-Supervised Learning for Neural Collapse in Temporal Data
rETF-semiSL pre-trains encoders with a rotational equiangular tight frame classifier, center loss, and pseudo-labels, and reports gains in time series classification across LSTM, transformer, and state-space models.
-
Understanding Knowledge Transferability for Transfer Learning: A Survey
A survey that classifies transferability metrics by knowledge modality (dataset vs. model) and granularity (task vs. instance), with a theoretical primer and applications to eight learning paradigms.
Discussion (0). Continue with ORCID to comment.