Gunjan Dhanuka — Learning Notes
Search
Search
Dark mode
Light mode
Reader mode
Explorer
deepseek
7 items with this tag.
Sep 14, 2026
DSA (DeepSeek Sparse Attention)
attention
sparse-attention
dsa
csa
deepseek
trainable-sparse
lightning-indexer
top-k
mla
long-context
kv-cache
Sep 12, 2026
QSA (Qwen Sparse Attention)
attention
sparse-attention
qsa
qwen
trainable-sparse
block-selection
indexer
compression
long-context
deepseek
Sep 12, 2026
On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
qwen
alibaba
qsa
gated-deltanet
sparse-attention
hybrid
moe
muon
long-context
deepseek
n-gram
Sep 10, 2026
MLA (Multi-head Latent Attention)
attention
kv-cache
mla
deepseek
low-rank
weight-absorption
decoupled-rope
inference
long-context
Sep 10, 2026
Learn: MQA → GQA → MLA (Shrinking the KV Cache)
attention
kv-cache
mqa
gqa
mla
deepseek
inference
long-context
transformers
Sep 02, 2026
Expert Parallelism (EP)
gpu
model-parallelism
expert-parallelism
moe
all-to-all
load-balancing
deepseek
megatron
llm-systems
Sep 02, 2026
GRPO (Group Relative Policy Optimization)
grpo
rl
rlvr
deepseek
critic-free
dr-grpo
dapo
reasoning
rl-for-llms