Gunjan Dhanuka — Learning Notes

hybrid

5 items with this tag.

  • Sep 14, 2026

    Hybrid Attention + SSM Architectures

    • attention
    • ssm
    • mamba
    • hybrid
    • linear-attention
    • delta-rule
    • kv-cache
    • long-context
    • nemotron
    • hymba
    • jamba
  • Sep 12, 2026

    Gated DeltaNet (GDN)

    • attention
    • linear-attention
    • delta-rule
    • gated-deltanet
    • gdn
    • mamba2
    • data-dependent-decay
    • subquadratic
    • qwen
    • hybrid
  • Sep 12, 2026

    KDA (Kimi Delta Attention)

    • attention
    • linear-attention
    • delta-rule
    • kda
    • kimi
    • moonshot
    • channel-wise-decay
    • hybrid
    • mla
    • long-context
    • kv-cache
  • Sep 12, 2026

    Kimi K3: Open Frontier Intelligence

    • kimi
    • moonshot
    • kda
    • mla
    • delta-rule
    • linear-attention
    • hybrid
    • moe
    • long-context
    • nope
    • vision
  • Sep 12, 2026

    On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability

    • qwen
    • alibaba
    • qsa
    • gated-deltanet
    • sparse-attention
    • hybrid
    • moe
    • muon
    • long-context
    • deepseek
    • n-gram

Created with Quartz v5.0.0 © 2026

  • Personal site
  • Research
  • Field notes
  • Source