Gunjan Dhanuka — Learning Notes
Search
Search
Dark mode
Light mode
Reader mode
Explorer
relative-position
4 items with this tag.
Sep 10, 2026
ALiBi (Attention with Linear Biases)
attention
positional-encoding
alibi
relative-position
length-extrapolation
long-context
transformers
Sep 10, 2026
Positional Encoding
attention
positional-encoding
sinusoidal
learned-embedding
relative-position
transformers
foundations
Sep 10, 2026
RoPE (Rotary Position Embedding)
attention
positional-encoding
rope
relative-position
long-context
rotation
transformers
Sep 10, 2026
Learn: Positional Information (Sinusoidal → RoPE → ALiBi)
attention
positional-encoding
sinusoidal
rope
alibi
relative-position
long-context
transformers