GRAPE unifies RoPE and ALiBi as special cases of group actions on positions, providing a principled design space for positional encodings via SO(d) rotations and GL unipotent transformations.
In the canonical rank-1case of Lemma J.4 with∥A∥ 2 = 1, one has the sharper small-|s|behaviorσ max(I+sA) = 1 + |s| 2 +O(s 2)and σmin(I+sA) = 1− |s| 2 +O(s 2)
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Group Representational Position Encoding
GRAPE unifies RoPE and ALiBi as special cases of group actions on positions, providing a principled design space for positional encodings via SO(d) rotations and GL unipotent transformations.