REVIEW 2 cited by
Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems
read the original abstract
We propose a simplified model of attention which is applicable to feed-forward neural networks and demonstrate that the resulting model can solve the synthetic "addition" and "multiplication" long-term memory problems for sequence lengths which are both longer and more widely varying than the best published results for these tasks.
Forward citations
Cited by 2 Pith papers
-
Eigenvalues as a Metric for Memory Dynamics in Sequence Models
Eigenvalue spectra of attention and SSM dynamics show consistent signatures of memory retention and selective forgetting that align with task requirements.
-
An Attention Mechanism for Musical Instrument Recognition
An attention model improves multi-label instrument recognition accuracy on the weakly labeled OpenMIC dataset compared to baseline, RNN, and fully connected networks across 20 instruments.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.