RACE Attention is a strictly linear-time attention mechanism that approximates softmax attention outputs using Gaussian projections and soft LSH to enable training on contexts up to 12 million tokens.
In: Proceedings of the 2018 World Wide Web Conference
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
verdicts
UNVERDICTED 2representative citing papers
A higher-order network comparison of real Ile-de-France mobility traces against a synthetic simulator finds the simulator promising yet limited on path-level statistics.
citing papers explorer
-
RACE Attention: A Strictly Linear-Time Attention Layer for Training on Outrageously Large Contexts
RACE Attention is a strictly linear-time attention mechanism that approximates softmax attention outputs using Gaussian projections and soft LSH to enable training on contexts up to 12 million tokens.
-
Higher-order Network Analysis of Human Mobility Data
A higher-order network comparison of real Ile-de-France mobility traces against a synthetic simulator finds the simulator promising yet limited on path-level statistics.