FAR substitutes self-attention in pretrained DeiTs with multi-head bidirectional LSTMs via block-wise distillation and structured pruning to enable IMC-compatible inference with comparable accuracy and lower latency.
Demonstration of transformer-based albert model on a 14nm analog ai inference chip.Nature Communications, 16
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CV 1years
2025 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
FAR: Function-preserving Attention Replacement for IMC-friendly Inference
FAR substitutes self-attention in pretrained DeiTs with multi-head bidirectional LSTMs via block-wise distillation and structured pruning to enable IMC-compatible inference with comparable accuracy and lower latency.