REVIEW 4 cited by
Compositional generalization in a deep seq2seq model by separating syntax and semantics
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Standard methods in deep learning for natural language processing fail to capture the compositional structure of human language that allows for systematic generalization outside of the training distribution. However, human learners readily generalize in this way, e.g. by applying known grammatical rules to novel words. Inspired by work in neuroscience suggesting separate brain systems for syntactic and semantic processing, we implement a modification to standard approaches in neural machine translation, imposing an analogous separation. The novel model, which we call Syntactic Attention, substantially outperforms standard methods in deep learning on the SCAN dataset, a compositional generalization task, without any hand-engineered features or additional supervision. Our work suggests that separating syntactic from semantic learning may be a useful heuristic for capturing compositional structure.
Forward citations
Cited by 4 Pith papers
-
GXJoin: Generalized Cell Transformations for Explainable Joinability
Adding relative indexes, reusable and optional rule parts, bidirectional search, and simplicity tie-breaking raises the coverage of the best discovered transformation by up to about 10% on the authors' benchmarks.
-
Compositional Generalization Across Distributional Shifts with Sparse Tree Operations
Sparse Differentiable Tree Machine represents trees as sparse coordinate lists, enabling efficient tree operations via bit-shifts and extending the Differentiable Tree Machine to sequence-to-sequence tasks with strong...
-
Cognitive Decision Routing in Large Language Models: When to Think Fast, When to Think Slow
A four-feature routing rule is claimed to cut token use by 34% and improve accuracy, but the paper provides no code, no data, and only hand-wavy details.
-
A Theoretical Analysis of Compositional Generalization in Neural Networks: A Necessary and Sufficient Condition
A network generalizes to unseen combinations exactly when its computation graph matches a graph that already solves the test set, its internal codes are unambiguous, and the codes carry no extra information.
Discussion (0). Continue with ORCID to comment.