Gunjan Dhanuka — Learning Notes

gqa

5 items with this tag.

  • Sep 14, 2026

    Local/Global Interleaved Attention

    • attention
    • local-global
    • sliding-window
    • gemma
    • kv-cache
    • rope
    • long-context
    • soft-capping
    • gqa
  • Sep 11, 2026

    Learn: SOTA Attention Synthesis (Sept 2026)

    • attention
    • sota
    • synthesis
    • frontier
    • rope
    • gqa
    • mla
    • flashattention
    • nsa
    • sparse-attention
    • ssm
    • hybrids
    • multimodal
    • capstone
    • transformers
  • Sep 10, 2026

    GQA / MQA (Grouped & Multi-Query Attention)

    • attention
    • kv-cache
    • gqa
    • mqa
    • inference
    • uptraining
    • llama
    • long-context
  • Sep 10, 2026

    Learn: MQA → GQA → MLA (Shrinking the KV Cache)

    • attention
    • kv-cache
    • mqa
    • gqa
    • mla
    • deepseek
    • inference
    • long-context
    • transformers
  • Sep 02, 2026

    KV Cache

    • gpu
    • kv-cache
    • inference
    • serving
    • decode
    • memory-bound
    • gqa
    • mqa
    • llm-systems

Created with Quartz v5.0.0 © 2026

  • Personal site
  • Research
  • Field notes
  • Source