Gunjan Dhanuka — Learning Notes

linear-attention

7 items with this tag.

  • Sep 14, 2026

    Hybrid Attention + SSM Architectures

    • attention
    • ssm
    • mamba
    • hybrid
    • linear-attention
    • delta-rule
    • kv-cache
    • long-context
    • nemotron
    • hymba
    • jamba
  • Sep 12, 2026

    Delta Rule (DeltaNet)

    • attention
    • linear-attention
    • delta-rule
    • deltanet
    • fast-weights
    • associative-memory
    • error-correction
    • subquadratic
    • chunkwise
  • Sep 12, 2026

    Gated DeltaNet (GDN)

    • attention
    • linear-attention
    • delta-rule
    • gated-deltanet
    • gdn
    • mamba2
    • data-dependent-decay
    • subquadratic
    • qwen
    • hybrid
  • Sep 12, 2026

    KDA (Kimi Delta Attention)

    • attention
    • linear-attention
    • delta-rule
    • kda
    • kimi
    • moonshot
    • channel-wise-decay
    • hybrid
    • mla
    • long-context
    • kv-cache
  • Sep 12, 2026

    Kimi K3: Open Frontier Intelligence

    • kimi
    • moonshot
    • kda
    • mla
    • delta-rule
    • linear-attention
    • hybrid
    • moe
    • long-context
    • nope
    • vision
  • Sep 10, 2026

    Linear Attention

    • attention
    • linear-attention
    • kernel-trick
    • performers
    • linformer
    • recurrent
    • subquadratic
    • long-context
    • recall-gap
  • Sep 10, 2026

    Learn: Linear & Sub-Quadratic Attention (& the SSM Turn)

    • attention
    • linear-attention
    • kernel-trick
    • performers
    • linformer
    • state-space-models
    • s4
    • mamba
    • ssm
    • subquadratic
    • long-context
    • hybrids
    • transformers

Created with Quartz v5.0.0 © 2026

  • Personal site
  • Research
  • Field notes
  • Source