Pith. sign in

REVIEW 2 cited by

Memory Mosaics

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.06394 v3 pith:LKRKVOLL submitted 2024-05-10 cs.LG cs.AIcs.NE

Jianyu Zhang , Niklas Nolte , Ranajoy Sadhukhan , Beidi Chen , Léon Bottou This is my paper · ORCID
classification cs.LGcs.AIcs.NE
keywords memorymosaicscapabilitiestransformersachieveassociativebettercomparatively
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Memory Mosaics are networks of associative memories working in concert to achieve a prediction task of interest. Like transformers, memory mosaics possess compositional capabilities and in-context learning capabilities. Unlike transformers, memory mosaics achieve these capabilities in comparatively transparent way ("predictive disentanglement"). We illustrate these capabilities on a toy example and also show that memory mosaics perform as well or better than transformers on medium-scale language modeling tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Inference-time sparse attention with asymmetric indexing

    cs.CL 2025-02 conditional novelty 6.0 of 10

    Saap approximates attention by clustering keys with k-means and learning a query classifier, reducing KV-cache lookups about 20x at 4-5% selectivity with small accuracy loss on several long-context benchmarks.

  2. Extending LLM Context via Associative Recurrent Memory

    cs.CL 2026-07 conditional novelty 5.0 of 10

    ARMT-augmented 1B-class LLMs, trained with continued pretraining, synthetic long data, curriculum, and selective memory layers, keep in-window quality while generalizing past 32k–65k tokens at constant memory and ~30%...

Pith tools