Gunjan Dhanuka — Learning Notes
Search
Search
Dark mode
Light mode
Reader mode
Explorer
kv-cache
14 items with this tag.
Sep 14, 2026
DSA (DeepSeek Sparse Attention)
attention
sparse-attention
dsa
csa
deepseek
trainable-sparse
lightning-indexer
top-k
mla
long-context
kv-cache
Sep 14, 2026
Hybrid Attention + SSM Architectures
attention
ssm
mamba
hybrid
linear-attention
delta-rule
kv-cache
long-context
nemotron
hymba
jamba
Sep 14, 2026
Local/Global Interleaved Attention
attention
local-global
sliding-window
gemma
kv-cache
rope
long-context
soft-capping
gqa
Sep 12, 2026
KDA (Kimi Delta Attention)
attention
linear-attention
delta-rule
kda
kimi
moonshot
channel-wise-decay
hybrid
mla
long-context
kv-cache
Sep 11, 2026
Long Context
attention
long-context
rope
yarn
ring-attention
kv-cache
evaluation
niah
ruler
systems
Sep 10, 2026
Attention Sinks & StreamingLLM
attention
attention-sinks
streaming
streamingllm
kv-cache
softmax
inference
long-context
Sep 10, 2026
GQA / MQA (Grouped & Multi-Query Attention)
attention
kv-cache
gqa
mqa
inference
uptraining
llama
long-context
Sep 10, 2026
MLA (Multi-head Latent Attention)
attention
kv-cache
mla
deepseek
low-rank
weight-absorption
decoupled-rope
inference
long-context
Sep 10, 2026
Sliding-Window Attention
attention
sparse-attention
sliding-window
local-attention
mistral
long-context
receptive-field
kv-cache
Sep 10, 2026
Learn: MQA → GQA → MLA (Shrinking the KV Cache)
attention
kv-cache
mqa
gqa
mla
deepseek
inference
long-context
transformers
Sep 10, 2026
Learn: The Long-Context Race
attention
long-context
rope
position-interpolation
yarn
ntk-aware
ring-attention
kv-cache
evaluation
ruler
transformers
Sep 02, 2026
KV Cache
gpu
kv-cache
inference
serving
decode
memory-bound
gqa
mqa
llm-systems
Sep 02, 2026
PagedAttention
gpu
paged-attention
vllm
kv-cache
memory-management
fragmentation
prefix-sharing
llm-systems
Sep 02, 2026
Learn: LLM Inference & Serving Systems
gpu
inference
serving
kv-cache
paged-attention
vllm
speculative-decoding
quantization
prefill-decode