moe

Sep 12, 2026

• kimi
• moonshot
• kda
• mla
• delta-rule
• linear-attention
• hybrid
• moe
• long-context
• nope
• vision

Sep 12, 2026

• qwen
• alibaba
• qsa
• gated-deltanet
• sparse-attention
• hybrid
• moe
• muon
• long-context
• deepseek
• n-gram

Sep 02, 2026

• gpu
• model-parallelism
• expert-parallelism
• moe
• all-to-all
• load-balancing
• deepseek
• megatron
• llm-systems

Sep 02, 2026

• gpu
• model-parallelism
• tensor-parallelism
• pipeline-parallelism
• sequence-parallelism
• expert-parallelism
• context-parallelism
• megatron
• moe
• distributed-training
• llm-systems

Sep 02, 2026

• gpu
• distributed-training
• 3d-parallelism
• fp8
• transformer-engine
• megatron
• torchtitan
• deepseek-v3
• llama-3
• mfu
• activation-recomputation
• fault-tolerance
• moe