Gunjan Dhanuka — Learning Notes
Search
Search
Dark mode
Light mode
Reader mode
Explorer
hybrid
5 items with this tag.
Sep 14, 2026
Hybrid Attention + SSM Architectures
attention
ssm
mamba
hybrid
linear-attention
delta-rule
kv-cache
long-context
nemotron
hymba
jamba
Sep 12, 2026
Gated DeltaNet (GDN)
attention
linear-attention
delta-rule
gated-deltanet
gdn
mamba2
data-dependent-decay
subquadratic
qwen
hybrid
Sep 12, 2026
KDA (Kimi Delta Attention)
attention
linear-attention
delta-rule
kda
kimi
moonshot
channel-wise-decay
hybrid
mla
long-context
kv-cache
Sep 12, 2026
Kimi K3: Open Frontier Intelligence
kimi
moonshot
kda
mla
delta-rule
linear-attention
hybrid
moe
long-context
nope
vision
Sep 12, 2026
On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
qwen
alibaba
qsa
gated-deltanet
sparse-attention
hybrid
moe
muon
long-context
deepseek
n-gram