REVIEW 1 cited by
Minimum Description Length Hopfield Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Associative memory architectures are designed for memorization but also offer, through their retrieval method, a form of generalization to unseen inputs: stored memories can be seen as prototypes from this point of view. Focusing on Modern Hopfield Networks (MHN), we show that a large memorization capacity undermines the generalization opportunity. We offer a solution to better optimize this tradeoff. It relies on Minimum Description Length (MDL) to determine during training which memories to store, as well as how many of them.
Forward citations
Cited by 1 Pith paper
-
A Minimum Description Length Approach to Regularization in Neural Networks
MDL-based regularization, which balances data fit with a network's encoding length, preserves perfect solutions on several formal-language tasks, while standard L1, L2, and no regularization degrade them.
Discussion (0). Continue with ORCID to comment.