Full-support von Mises-Fisher sampling satisfies a diversity condition allowing global contrastive loss minimizers to recover latent geometry up to orthogonal transformation, while restricted sampling permits non-orthogonal maps to achieve lower loss; a support-corrected InfoNCE is introduced.
hub Canonical reference
Improved Regularization of Convolutional Neural Networks with Cutout
Canonical reference. 88% of citing Pith papers cite this work as background.
55
Pith papers citing it
Background
88% of classified citations
abstract
Convolutional neural networks are capable of learning powerful representational spaces, which are necessary for tackling complex learning tasks. However, due to the model capacity required to capture such representations, they are often susceptible to overfitting and therefore require proper regularization in order to generalize well. In this paper, we show that the simple regularization technique of randomly masking out square regions of input during training, which we call cutout, can be used to improve the robustness and overall performance of convolutional neural networks. Not only is this method extremely easy to implement, but we also demonstrate that it can be used in conjunction with existing forms of data augmentation and other regularizers to further improve model performance. We evaluate this method by applying it to current state-of-the-art architectures on the CIFAR-10, CIFAR-100, and SVHN datasets, yielding new state-of-the-art results of 2.56%, 15.20%, and 1.30% test error respectively. Code is available at https://github.com/uoguelph-mlrg/Cutout
hub tools
citation-role summary
background 7
method 1
citation-polarity summary
representative citing papers
LLQR+SAM pairs a slow learned geometry preconditioner with fast SAM perturbations to amplify escape from locally sharp 'potholes' while stabilizing flat basins, producing consistent gains over SAM and LLQR alone.
BICL uses biased non-uniform transition matrices to generate constrained complementary labels, enabling effective learning and over sevenfold accuracy gains on many-class image datasets.
The test error of random-feature ridge regression with arbitrary data augmentation admits a closed-form asymptotic characterization in the proportional regime that depends only on population covariances and augmentation statistics.
SeBA is a joint-embedding framework that separates tabular data into two complementary views and aligns one view's representations to the nearest-neighbor structure of the other, improving feature-label relationships and achieving SOTA results in most benchmarks without relying on augmentations.
Steepest descent under divergence-induced quadratic models equals an LQR problem, enabling learning of diagonal or Kronecker-factored inverse preconditioners via a global layerwise objective for scalable geometry-aware training.
QB-LIF uses a trainable quantization scale for burst neurons in SNNs to raise accuracy at ultra-low latency on vision and event datasets while preserving neuromorphic hardware compatibility.
Unlearnable examples fail under pretraining-finetuning due to semantic filtering by frozen layers, but Shallow Semantic Camouflage restores effectiveness by confining perturbations to semantically valid subspaces.
SAM-family models split into occluder-aware types that avoid predicting into occluded regions and occluder-agnostic types that confidently segment hidden areas, shown via a new benchmark on polyp datasets.
PAR fine-tunes CLIP to remove backdoors from structured triggers while preserving standard performance, and works even with only synthetic image-text pairs.
SimCLR learns visual representations by contrasting augmented views of the same image and reaches 76.5% ImageNet top-1 accuracy with a linear classifier, matching a supervised ResNet-50.
AGVBench benchmarks 30 augmentation strategies for vein recognition and finds mixing methods improve accuracy but harm calibration and adversarial robustness.
FUSE creates full-spectrum unlearnable perturbations using random spectral masking during training and cross-band guidance to enforce consistency between frequency components.
A robotic system achieves the first autonomous clip positioning on a laparoscopic surgery phantom by segmenting colorless point clouds, using spline interpolation for targets, and reaching 0.75 mm localization precision at 95% success with 100% clip placement success after synthetic pre-training on
PEACH uses a novel spatio-temporal point cloud sequence encoder plus auxiliary supervision to enable zero-shot adaptation of graph network simulators to unseen physical properties, outperforming mesh-based baselines in simulation accuracy while being more deployable for real scenes.
Deep learning on information-rich scientific images collapses to one-dimensional predictions due to a mismatch between data priors and the model's simplicity bias, even after robustification techniques.
IonMorphNet is a ConvNeXt-based classifier trained on six spatial pattern classes from 53 MSI datasets that performs generalizable peak picking and improves mSCF1 by 7% over prior methods while also aiding tumor classification via ion selection.
PLAG boosts tabular anomaly detection by using pseudo-label-guided synthetic anomaly generation with a two-stage filter, achieving SOTA results and lifting F1 scores by 0.08-0.21 when added to existing detectors.
LPQLD reduces soft label storage in dataset distillation by 78-500x on ImageNet datasets via pruning with dynamic reuse and quantization with student-teacher alignment, while improving accuracy.
FireSenseNet dual-branch CNN with CAFIM cross-attention outperforms larger models on next-day wildfire spread prediction, reaching F1 of 0.4176 on the Google benchmark.
OASIC uses anomaly-based masking and severity estimation to select occlusion-matched models, improving AUC on occluded images by up to 23.7 points.
Semantic-aware random convolution and intensity-based source matching enable effective single-source domain generalization for medical image segmentation, outperforming prior methods and sometimes matching in-domain performance.
Masked Language Prompting masks selected words in reference captions and leverages LLMs to produce diverse, semantically coherent completions for style-consistent generative image augmentation without fine-tuning.
Orthogonal subspace decomposition via SVD on vision foundation model features preserves high-rank pre-trained knowledge by freezing principal components and adapting residuals, reducing overfitting for better generalization in AI-generated image detection.
citing papers explorer
-
The Loss Is Not Enough: Sampling Conditions and Inductive Bias in Contrastive Representation Learning
Full-support von Mises-Fisher sampling satisfies a diversity condition allowing global contrastive loss minimizers to recover latent geometry up to orthogonal transformation, while restricted sampling permits non-orthogonal maps to achieve lower loss; a support-corrected InfoNCE is introduced.
-
Navigating Potholes with Geometry-Aware Sharpness Minimization
LLQR+SAM pairs a slow learned geometry preconditioner with fast SAM perturbations to amplify escape from locally sharp 'potholes' while stabilizing flat basins, producing consistent gains over SAM and LLQR alone.
-
Embracing Biased Transition Matrices for Complementary-Label Learning with Many Classes
BICL uses biased non-uniform transition matrices to generate constrained complementary labels, enabling effective learning and over sevenfold accuracy gains on many-class image datasets.
-
Characterizing the Generalization Error of Random Feature Regression with Arbitrary Data-Augmentation
The test error of random-feature ridge regression with arbitrary data augmentation admits a closed-form asymptotic characterization in the proportional regime that depends only on population covariances and augmentation statistics.
-
SeBA: Semi-supervised few-shot learning via Separated-at-Birth Alignment for tabular data
SeBA is a joint-embedding framework that separates tabular data into two complementary views and aligns one view's representations to the nearest-neighbor structure of the other, improving feature-label relationships and achieving SOTA results in most benchmarks without relying on augmentations.
-
Layerwise LQR for Geometry-Aware Optimization of Deep Networks
Steepest descent under divergence-induced quadratic models equals an LQR problem, enabling learning of diagonal or Kronecker-factored inverse preconditioners via a global layerwise objective for scalable geometry-aware training.
-
QB-LIF: Learnable-Scale Quantized Burst Neurons for Efficient SNNs
QB-LIF uses a trainable quantization scale for burst neurons in SNNs to raise accuracy at ultra-low latency on vision and event datasets while preserving neuromorphic hardware compatibility.
-
Channel-Level Semantic Perturbations: Unlearnable Examples for Diverse Training Paradigms
Unlearnable examples fail under pretraining-finetuning due to semantic filtering by frozen layers, but Shallow Semantic Camouflage restores effectiveness by confining perturbations to semantically valid subspaces.
-
Seeing Through the Tool: A Controlled Benchmark for Occlusion Robustness in Foundation Segmentation Models
SAM-family models split into occluder-aware types that avoid predicting into occluded regions and occluder-agnostic types that confidently segment hidden areas, shown via a new benchmark on polyp datasets.
-
Perturb and Recover: Fine-tuning for Effective Backdoor Removal from CLIP
PAR fine-tunes CLIP to remove backdoors from structured triggers while preserving standard performance, and works even with only synthetic image-text pairs.
-
A Simple Framework for Contrastive Learning of Visual Representations
SimCLR learns visual representations by contrasting augmented views of the same image and reaches 76.5% ImageNet top-1 accuracy with a linear classifier, matching a supervised ResNet-50.
-
AGVBench: A Reliability-Oriented Benchmark of Data Augmentation for Vein Recognition
AGVBench benchmarks 30 augmentation strategies for vein recognition and finds mixing methods improve accuracy but harm calibration and adversarial robustness.
-
Full spectrum Unlearnable Examples via Spectral Equalization
FUSE creates full-spectrum unlearnable perturbations using random spectral masking during training and cross-band guidance to enforce consistency between frequency components.
-
Point Cloud Segmentation for Autonomous Clip Positioning in Laparoscopic Cholecystectomy on a Phantom
A robotic system achieves the first autonomous clip positioning on a laparoscopic surgery phantom by segmenting colorless point clouds, using spline interpolation for targets, and reaching 0.75 mm localization precision at 95% success with 100% clip placement success after synthetic pre-training on
-
Point Cloud Sequence Encoding for Material-conditioned Graph Network Simulators
PEACH uses a novel spatio-temporal point cloud sequence encoder plus auxiliary supervision to enable zero-shot adaptation of graph network simulators to unseen physical properties, outperforming mesh-based baselines in simulation accuracy while being more deployable for real scenes.
-
Anatomy of a failure: When, how, and why deep vision fails in scientific domains
Deep learning on information-rich scientific images collapses to one-dimensional predictions due to a mismatch between data priors and the model's simplicity bias, even after robustification techniques.
-
IonMorphNet: Generalizable Learning of Ion Image Morphologies for Peak Picking in Mass Spectrometry Imaging
IonMorphNet is a ConvNeXt-based classifier trained on six spatial pattern classes from 53 MSI datasets that performs generalizable peak picking and improves mSCF1 by 7% over prior methods while also aiding tumor classification via ion selection.
-
Enhancing Tabular Anomaly Detection via Pseudo-Label-Guided Generation
PLAG boosts tabular anomaly detection by using pseudo-label-guided synthetic anomaly generation with a two-stage filter, achieving SOTA results and lifting F1 scores by 0.08-0.21 when added to existing detectors.
-
Soft Label Pruning and Quantization for Large-Scale Dataset Distillation
LPQLD reduces soft label storage in dataset distillation by 78-500x on ImageNet datasets via pruning with dynamic reuse and quantization with student-teacher alignment, while improving accuracy.
-
FireSenseNet: A Dual-Branch CNN with Cross-Attentive Feature Interaction for Next-Day Wildfire Spread Prediction
FireSenseNet dual-branch CNN with CAFIM cross-attention outperforms larger models on next-day wildfire spread prediction, reaching F1 of 0.4176 on the Google benchmark.
-
OASIC: Occlusion-Agnostic and Severity-Informed Classification
OASIC uses anomaly-based masking and severity estimation to select occlusion-matched models, improving AUC on occluded images by up to 23.7 points.
-
Semantic-aware Random Convolution and Source Matching for Domain Generalization in Medical Image Segmentation
Semantic-aware random convolution and intensity-based source matching enable effective single-source domain generalization for medical image segmentation, outperforming prior methods and sometimes matching in-domain performance.
-
Masked Language Prompting for Generative Data Augmentation in Few-shot Fashion Style Recognition
Masked Language Prompting masks selected words in reference captions and leverages LLMs to produce diverse, semantically coherent completions for style-consistent generative image augmentation without fine-tuning.
-
Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection
Orthogonal subspace decomposition via SVD on vision foundation model features preserves high-rank pre-trained knowledge by freezing principal components and adapting residuals, reducing overfitting for better generalization in AI-generated image detection.
-
Decouple then Converge: Handling Unknown Unlabeled Distributions in Long-Tailed Semi-Supervised Learning
DeCon decouples LTSSL into head-class and tail-class branches that interact and converge, delivering SOTA accuracy on mismatched-distribution benchmarks and outperforming prior methods even on matched distributions.
-
Sharpness-Aware Minimization for Efficiently Improving Generalization
SAM solves a min-max problem to locate flat low-loss regions, improving generalization on CIFAR, ImageNet and label-noise tasks.
-
DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks
DropAttention regularizes attention weights in fully-connected self-attention networks to reduce overfitting and improve performance.
-
XferNAS: Transfer Neural Architecture Search
XferNAS transfers knowledge across neural architecture searches to reduce search time by a factor of 33 on CIFAR-10/100 while achieving new records of 1.99% and 14.06% error.
-
Learning Data Augmentation Strategies for Object Detection
Learned data augmentation policies optimized for object detection improve COCO mAP by more than 2.3 and transfer to other datasets and models.
-
InterCMDM: Block-Causal Diffusion for Autoregressive Human Interaction Generation
InterCMDM proposes a block-causal latent diffusion framework with dual-stream causal transformers and multi-task attention masks for autoregressive text-conditioned two-person interaction generation and reports SOTA results on InterHuman and Inter-X.
-
Controllable Histopathology Image Synthesis with Training-free Structural Initialization and Textural Modulation
CHIS steers pretrained diffusion models to generate histopathology images aligned with input structural masks via frequency-domain structural initialization and wavelet-based textural modulation without any training on annotated data.
-
Reconstructing Randomly Masked Spectra Helps DNNs Identify Discriminant Wavenumbers
TeaNet augments scarce spectroscopic data via masked spectrum reconstruction to train DNNs that outperform CNNs and better identify key wavenumbers.
-
ReSAGE-PAR: Representational Similarity Assessment for Generative Expansion in Pedestrian Attribute Recognition
ReSAGE-PAR adapts diffusion models with LoRA, scores generated images via vision-language prompts, and applies Bayesian classification to produce pseudo-labels, yielding up to 8.7% gains when used to expand PAR datasets.
-
Adaptive Sharpness-Aware Minimization with a Polyak-type Step size: A Theory-Grounded Scheduler
Proposes Polyak schedulers for SAM with convergence proofs in deterministic and stochastic settings and empirical results showing reduced tuning needs.
-
DAMEL: Dual-Axis Multi-Expert Learning for Class-Imbalanced Learning
DAMEL reduces both prediction bias and variance in class-imbalanced learning by concatenating multi-expert representations with an auxiliary balanced classifier and aggregating model weights across training epochs.
-
Dual-Prompt CLIP with Hybrid Visual Encoders for Occluded Person Re-Identification
DPL-ReID adds dual prompt learning, real-world occlusion augmentation, and weighted gated fusion to CLIP for state-of-the-art occluded person re-identification on benchmark datasets.
-
ZScribbleSeg: A comprehensive segmentation framework with modeling of efficient annotation and maximization of scribble supervision
ZScribbleSeg maximizes scribble supervision with efficient annotation forms, spatial regularization, and EM-estimated class ratios to deliver competitive performance on six medical segmentation tasks without full labels.
-
Accuracy Improvement of Semi-Supervised Segmentation Using Supervised ClassMix and Sup-Unsup Feature Discriminator
Supervised ClassMix and a Sup-Unsup Feature Discriminator yield an average 2.07% mIoU gain over standard semi-supervised methods on Chase and COVID-19 datasets.
-
Bi-Level Optimization for Single Domain Generalization
BiSDG applies bi-level optimization with surrogate domains and a domain prompt encoder to achieve state-of-the-art results in single domain generalization.
-
WRF4CIR: Weight-Regularized Fine-Tuning Network for Composed Image Retrieval
WRF4CIR uses weight-regularized fine-tuning with adversarial perturbations to mitigate overfitting in composed image retrieval and narrows the generalization gap on benchmarks.
-
Why Invariance is Not Enough for Biomedical Domain Generalization and How to Fix It
MaskGen improves domain generalization for biomedical image segmentation by using source intensities plus domain-stable foundation model representations with minimal added complexity.
-
YOLOv4: Optimal Speed and Accuracy of Object Detection
YOLOv4 achieves 43.5% AP (65.7% AP50) on MS COCO at ~65 FPS on Tesla V100 by integrating WRC, CSP, CmBN, SAT, Mish activation, Mosaic augmentation, DropBlock, and CIoU loss.
-
APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms
Presents APRIL-MedSeg, a modular YAML-configurable toolbox for 2D medical image segmentation integrating semi-supervised, domain adaptation, distillation, weakly supervised, text-guided, and foundation model paradigms with unified dataset and deployment interfaces.
-
Facial Expression Recognition in the Deep Learning Era: A Systematic Multi-Criteria Review of Methods, Models, Datasets, Performance, Challenges, and Future Research Directions
This survey organizes deep learning FER literature into five evolutionary phases and a seven-criteria taxonomy, compares datasets and performance, and outlines challenges.
-
Tiny Collaborative Inference for Occlusion-Robust Object Detection
Decision-level fusion with WBF outperforms feature-level fusion for occlusion-robust detection on ultra-low-end hardware, with gains up to +0.3827 mAP across three views and on-device execution on Coral boards.
-
Margin-Adaptive Confidence Ranking for Reliable LLM Judgement
Develops a margin-adaptive learned confidence estimator for LLMs with generalization guarantees to improve agreement rates with human judgments over heuristic baselines.
-
How Data Augmentation Shapes Neural Representations
Data augmentation produces well-behaved trajectories in shape-invariant representation space, with augmentation type steering distinct directions and geometry predicting ensembling gains.
-
AtteConDA: Attention-Based Conflict Suppression in Multi-Condition Diffusion Models and Synthetic Data Augmentation
AtteConDA adds attention-based conflict suppression to multi-condition diffusion models so that generated driving-scene images retain richer structural cues from the original annotations.
-
FGML-DG: Feynman-Inspired Cognitive Science Paradigm for Cross-Domain Medical Image Segmentation
FGML-DG applies Feynman-inspired principles of concept simplification, memory recall, and error-focused retraining within a meta-learning setup to enhance domain generalization for medical image segmentation.
-
Single-bit-per-weight deep convolutional neural networks without batch-normalization layers for embedded systems
Experiments show that shifted-ReLU layers can replace batch-normalization in single-bit-weight wide residual networks on CIFAR-10/100 and ImageNet without consistent accuracy penalty.