Gunjan Dhanuka — Learning Notes

mla

6 items with this tag.

  • Sep 14, 2026

    DSA (DeepSeek Sparse Attention)

    • attention
    • sparse-attention
    • dsa
    • csa
    • deepseek
    • trainable-sparse
    • lightning-indexer
    • top-k
    • mla
    • long-context
    • kv-cache
  • Sep 12, 2026

    KDA (Kimi Delta Attention)

    • attention
    • linear-attention
    • delta-rule
    • kda
    • kimi
    • moonshot
    • channel-wise-decay
    • hybrid
    • mla
    • long-context
    • kv-cache
  • Sep 12, 2026

    Kimi K3: Open Frontier Intelligence

    • kimi
    • moonshot
    • kda
    • mla
    • delta-rule
    • linear-attention
    • hybrid
    • moe
    • long-context
    • nope
    • vision
  • Sep 11, 2026

    Learn: SOTA Attention Synthesis (Sept 2026)

    • attention
    • sota
    • synthesis
    • frontier
    • rope
    • gqa
    • mla
    • flashattention
    • nsa
    • sparse-attention
    • ssm
    • hybrids
    • multimodal
    • capstone
    • transformers
  • Sep 10, 2026

    MLA (Multi-head Latent Attention)

    • attention
    • kv-cache
    • mla
    • deepseek
    • low-rank
    • weight-absorption
    • decoupled-rope
    • inference
    • long-context
  • Sep 10, 2026

    Learn: MQA → GQA → MLA (Shrinking the KV Cache)

    • attention
    • kv-cache
    • mqa
    • gqa
    • mla
    • deepseek
    • inference
    • long-context
    • transformers

Created with Quartz v5.0.0 © 2026

  • Personal site
  • Research
  • Field notes
  • Source