Gunjan Dhanuka — Learning Notes
Search
Search
Dark mode
Light mode
Reader mode
Explorer
mla
6 items with this tag.
Sep 14, 2026
DSA (DeepSeek Sparse Attention)
attention
sparse-attention
dsa
csa
deepseek
trainable-sparse
lightning-indexer
top-k
mla
long-context
kv-cache
Sep 12, 2026
KDA (Kimi Delta Attention)
attention
linear-attention
delta-rule
kda
kimi
moonshot
channel-wise-decay
hybrid
mla
long-context
kv-cache
Sep 12, 2026
Kimi K3: Open Frontier Intelligence
kimi
moonshot
kda
mla
delta-rule
linear-attention
hybrid
moe
long-context
nope
vision
Sep 11, 2026
Learn: SOTA Attention Synthesis (Sept 2026)
attention
sota
synthesis
frontier
rope
gqa
mla
flashattention
nsa
sparse-attention
ssm
hybrids
multimodal
capstone
transformers
Sep 10, 2026
MLA (Multi-head Latent Attention)
attention
kv-cache
mla
deepseek
low-rank
weight-absorption
decoupled-rope
inference
long-context
Sep 10, 2026
Learn: MQA → GQA → MLA (Shrinking the KV Cache)
attention
kv-cache
mqa
gqa
mla
deepseek
inference
long-context
transformers