REVIEW 7 cited by
What Do Compressed Deep Neural Networks Forget?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Deep neural network pruning and quantization techniques have demonstrated it is possible to achieve high levels of compression with surprisingly little degradation to test set accuracy. However, this measure of performance conceals significant differences in how different classes and images are impacted by model compression techniques. We find that models with radically different numbers of weights have comparable top-line performance metrics but diverge considerably in behavior on a narrow subset of the dataset. This small subset of data points, which we term Pruning Identified Exemplars (PIEs) are systematically more impacted by the introduction of sparsity. Compression disproportionately impacts model performance on the underrepresented long-tail of the data distribution. PIEs over-index on atypical or noisy images that are far more challenging for both humans and algorithms to classify. Our work provides intuition into the role of capacity in deep neural networks and the trade-offs incurred by compression. An understanding of this disparate impact is critical given the widespread deployment of compressed models in the wild.
Forward citations
Cited by 7 Pith papers
-
QuantiBias: Benchmarking Quantization-Induced Bias in LLMs
Quantization leaves refusal and multiple-choice bias checks flat while open-ended stereotype endorsement remains high (~24–27% under an independent judge), a gap standard safety evaluations miss.
-
The Uneven Impact of Post-Training Quantization in Machine Translation
Across five LLMs and four quantization methods, 4-bit compression mostly preserves translation quality for high-resource languages, while 2-bit compression disproportionately degrades low-resource and Indic languages,...
-
Uncertainty-Driven Reliability: Selective Prediction and Trustworthy Deployment in Modern Machine Learning
A training-dynamics abstention method matches deep ensembles at a fraction of the training cost, and a five-term error budget explains why selective classifiers still fall short of the oracle.
-
SemSegBench & DetecBench: Benchmarking Reliability and Generalization Beyond Classification
A large-scale benchmark of 76 segmentation and 61 detection models shows that robustness to attacks and corruptions does not reliably track clean accuracy, and that transformer backbones generalize better under shift.
-
Explaining How Quantization Disparately Skews a Model
Quantization exacerbates accuracy disparity across groups via a cascade of weight, logit, and probability changes, and a combination of sampling, weighted loss, and mixed-precision training mitigates it.
-
Compressed Models are NOT Trust-equivalent to Their Large Counterparts
Compressed BERT models share at most 67% of their top decision features with BERT-base and show different calibration profiles even at similar accuracy, so accuracy parity does not ensure trust-equivalence.
-
Uncovering Critical Features for Deepfake Detection through the Lottery Ticket Hypothesis
Deepfake detectors contain lottery ticket subnetworks that preserve accuracy at high sparsity, and iterative magnitude pruning outperforms one-shot pruning in the experiments reported.
Discussion (0). Sign in to comment.