REVIEW 6 cited by
Ternary Weight Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
We present a memory and computation efficient ternary weight networks (TWNs) - with weights constrained to +1, 0 and -1. The Euclidian distance between full (float or double) precision weights and the ternary weights along with a scaling factor is minimized in training stage. Besides, a threshold-based ternary function is optimized to get an approximated solution which can be fast and easily computed. TWNs have shown better expressive abilities than binary precision counterparts. Meanwhile, TWNs achieve up to 16$\times$ model compression rate and need fewer multiplications compared with the float32 precision counterparts. Extensive experiments on MNIST, CIFAR-10, and ImageNet datasets show that the TWNs achieve much better result than the Binary-Weight-Networks (BWNs) and the classification performance on MNIST and CIFAR-10 is very close to the full precision networks. We also verify our method on object detection task and show that TWNs significantly outperforms BWN by more than 10\% mAP on PASCAL VOC dataset. The pytorch version of source code is available at: https://github.com/Thinklab-SJTU/twns.
Forward citations
Cited by 6 Pith papers
-
ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level
Expanding the inner rank of a ternary LLM weight factorization makes the quantization error monotonically decrease and lets effective bit-width approach bf16 arbitrarily closely.
-
Optimizing Binary and Ternary Neural Network Inference on RRAM Crossbars using CIM-Explorer
CIM-Explorer integrates a TVM-based compiler, multiple RRAM crossbar mappings, and simulators into a design-space exploration flow for binary and ternary neural networks.
-
Forget the Data and Fine-Tuning! Just Fold the Network to Compress
Model folding compresses a network by k-means clustering similar neurons across adjacent layers and repairing activation statistics without data (Fold-AR, Fold-DIR), surpassing prior data-free methods at high sparsity.
-
Multibit neural inference in a N-ary crossbar architecture
Simulation of 4-state MTJ crossbars achieves 94.48% MNIST accuracy for neural inference, close to 97.56% software baseline, with analysis showing quantization as primary error and an optimal number of states per cell.
-
Latent-Space Mean-Field Theory for Deep BitNet-like Training: Constrained Gradient Flows with Smooth Quantization and STE Limits
The paper's main mean-field theorem for BitNet-like training is not proven: it assumes key regularity and uses an invalid distributional limit step.
-
DFQ-ViT: Data-Free Quantization for Vision Transformers without Fine-tuning
DFQ-ViT combines easy-to-hard synthetic sample generation with activation correction to quantize vision transformers without data or fine-tuning, outperforming PSAQ-ViT and roughly matching real-data calibration.
Discussion (0). Continue with ORCID to comment.