Gunjan Dhanuka — Learning Notes
Search
Search
Dark mode
Light mode
Reader mode
Explorer
linear-attention
7 items with this tag.
Sep 14, 2026
Hybrid Attention + SSM Architectures
attention
ssm
mamba
hybrid
linear-attention
delta-rule
kv-cache
long-context
nemotron
hymba
jamba
Sep 12, 2026
Delta Rule (DeltaNet)
attention
linear-attention
delta-rule
deltanet
fast-weights
associative-memory
error-correction
subquadratic
chunkwise
Sep 12, 2026
Gated DeltaNet (GDN)
attention
linear-attention
delta-rule
gated-deltanet
gdn
mamba2
data-dependent-decay
subquadratic
qwen
hybrid
Sep 12, 2026
KDA (Kimi Delta Attention)
attention
linear-attention
delta-rule
kda
kimi
moonshot
channel-wise-decay
hybrid
mla
long-context
kv-cache
Sep 12, 2026
Kimi K3: Open Frontier Intelligence
kimi
moonshot
kda
mla
delta-rule
linear-attention
hybrid
moe
long-context
nope
vision
Sep 10, 2026
Linear Attention
attention
linear-attention
kernel-trick
performers
linformer
recurrent
subquadratic
long-context
recall-gap
Sep 10, 2026
Learn: Linear & Sub-Quadratic Attention (& the SSM Turn)
attention
linear-attention
kernel-trick
performers
linformer
state-space-models
s4
mamba
ssm
subquadratic
long-context
hybrids
transformers