concepts

Sep 14, 2026

• attention
• sparse-attention
• dsa
• csa
• deepseek
• trainable-sparse
• lightning-indexer
• top-k
• mla
• long-context
• kv-cache

Sep 14, 2026

• attention
• ssm
• mamba
• hybrid
• linear-attention
• delta-rule
• kv-cache
• long-context
• nemotron
• hymba
• jamba

Sep 14, 2026

• attention
• local-global
• sliding-window
• gemma
• kv-cache
• rope
• long-context
• soft-capping
• gqa

Sep 14, 2026

• attention
• sparse-attention
• moba
• moonshot
• kimi
• block-sparse
• top-k
• mixture-of-experts
• long-context
• trainable-sparse

Sep 12, 2026

• attention
• linear-attention
• delta-rule
• deltanet
• fast-weights
• associative-memory
• error-correction
• subquadratic
• chunkwise

Sep 12, 2026

• attention
• linear-attention
• delta-rule
• gated-deltanet
• gdn
• mamba2
• data-dependent-decay
• subquadratic
• qwen
• hybrid

Sep 12, 2026

• attention
• linear-attention
• delta-rule
• kda
• kimi
• moonshot
• channel-wise-decay
• hybrid
• mla
• long-context
• kv-cache

Sep 12, 2026

• attention
• sparse-attention
• qsa
• qwen
• trainable-sparse
• block-selection
• indexer
• compression
• long-context
• deepseek

Sep 11, 2026

• attention
• cross-attention
• self-attention
• multimodal
• vlm
• encoder-decoder
• fusion

Sep 11, 2026

• attention
• long-context
• rope
• yarn
• ring-attention
• kv-cache
• evaluation
• niah
• ruler
• systems

Sep 11, 2026

• attention
• cross-attention
• perceiver
• resampler
• q-former
• blip2
• flamingo
• token-compression
• multimodal

Sep 11, 2026

• attention
• ring-attention
• long-context
• context-parallelism
• sequence-sharding
• blockwise
• online-softmax
• distributed

Sep 11, 2026

• attention
• rope
• long-context
• position-interpolation
• ntk-aware
• yarn
• positional-generalization

Sep 11, 2026

• attention
• vision-transformer
• vit
• patch-embedding
• self-attention
• multimodal
• cls-token

Sep 11, 2026

• attention
• vlm
• multimodal
• fusion
• cross-attention
• flamingo
• llava
• taxonomy

Sep 10, 2026

• attention
• positional-encoding
• alibi
• relative-position
• length-extrapolation
• long-context
• transformers

Sep 10, 2026

• attention
• glossary
• reference

Sep 10, 2026

• attention
• attention-sinks
• streaming
• streamingllm
• kv-cache
• softmax
• inference
• long-context

Sep 10, 2026

• attention
• kv-cache
• gqa
• mqa
• inference
• uptraining
• llama
• long-context

Sep 10, 2026

• attention
• linear-attention
• kernel-trick
• performers
• linformer
• recurrent
• subquadratic
• long-context
• recall-gap

Sep 10, 2026

• mamba
• ssm
• selective-ssm
• s6
• state-space-models
• parallel-scan
• subquadratic
• long-context
• attention-alternative

Sep 10, 2026

• attention
• kv-cache
• mla
• deepseek
• low-rank
• weight-absorption
• decoupled-rope
• inference
• long-context

Sep 10, 2026

• attention
• positional-encoding
• sinusoidal
• learned-embedding
• relative-position
• transformers
• foundations

Sep 10, 2026

• attention
• positional-encoding
• rope
• relative-position
• long-context
• rotation
• transformers

Sep 10, 2026

• attention
• sparse-attention
• sliding-window
• local-attention
• mistral
• long-context
• receptive-field
• kv-cache

Sep 10, 2026

• attention
• sparse-attention
• longformer
• bigbird
• global-tokens
• dilation
• random-attention
• long-context

Sep 10, 2026

• ssm
• state-space-models
• s4
• hippo
• recurrence
• convolution
• subquadratic
• long-context
• attention-alternative

Sep 02, 2026

• gpu
• distributed-training
• 3d-parallelism
• tensor-parallelism
• pipeline-parallelism
• data-parallelism
• topology
• nvlink
• infiniband
• llm-systems

Sep 02, 2026

• reinforcement-learning
• actor-critic
• critic
• variance-reduction
• ppo
• rl-for-llms

Sep 02, 2026

• reinforcement-learning
• advantage
• variance-reduction
• policy-gradient
• rl-for-llms

Sep 02, 2026

• agentic-rl
• multi-turn
• tool-use
• credit-assignment
• environments
• rlvr
• echo-trap
• rl-for-llms

Sep 02, 2026

• gpu
• arithmetic-intensity
• roofline
• flops
• bandwidth
• memory-bound
• compute-bound
• llm-systems

Sep 02, 2026

• best-of-n
• rejection-sampling
• rft
• rest
• expert-iteration
• test-time-compute
• rl-for-llms

Sep 02, 2026

• bradley-terry
• preference-learning
• reward-model
• dpo
• rlhf
• rl-for-llms

Sep 02, 2026

• chain-of-thought
• reasoning
• rlvr
• grpo
• o1
• deepseek-r1
• test-time-compute
• rl-for-llms

Sep 02, 2026

• gpu
• distributed-training
• collective-communication
• all-reduce
• reduce-scatter
• all-gather
• all-to-all
• ring-all-reduce
• nccl

Sep 02, 2026

• constitutional-ai
• rlaif
• alignment
• harmlessness
• anthropic
• rlhf
• rl-for-llms

Sep 02, 2026

• gpu
• model-parallelism
• context-parallelism
• ring-attention
• long-context
• flash-attention
• activation-memory
• llm-systems

Sep 02, 2026

• gpu
• continuous-batching
• iteration-level-scheduling
• orca
• serving
• throughput
• llm-systems

Sep 02, 2026

• gpu
• cuda
• simt
• execution-model
• threads
• blocks
• shared-memory
• llm-systems

Sep 02, 2026

• gpu
• distributed-training
• data-parallelism
• all-reduce
• ddp
• gradient-accumulation
• llm-systems

Sep 02, 2026

• dpo
• preference-optimization
• rlhf
• bradley-terry
• kl-regularization
• alignment
• rl-for-llms

Sep 02, 2026

• gpu
• model-parallelism
• expert-parallelism
• moe
• all-to-all
• load-balancing
• deepseek
• megatron
• llm-systems

Sep 02, 2026

• gpu
• flashattention
• attention
• io-aware
• online-softmax
• kernel-fusion
• recomputation
• hopper
• blackwell
• llm-systems

Sep 02, 2026

• gpu
• mixed-precision
• fp8
• e4m3
• e5m2
• transformer-engine
• deepseek-v3
• quantization
• llm-systems

Sep 02, 2026

• gpu
• distributed-training
• fsdp
• fsdp2
• pytorch
• dtensor
• sharded-data-parallelism
• zero
• collectives

Sep 02, 2026

• gpu
• gemm
• matmul
• tensor-core
• arithmetic-intensity
• tiling
• transformer
• llm-systems

Sep 02, 2026

• reinforcement-learning
• gae
• advantage-estimation
• td-learning
• variance-reduction
• ppo
• rl-for-llms

Sep 02, 2026

• vision-encoder
• multimodal
• native-resolution
• high-resolution

Sep 02, 2026

• gpu
• memory-hierarchy
• registers
• shared-memory
• l2
• hbm
• bandwidth
• latency
• llm-systems

Sep 02, 2026

• grpo
• rl
• rlvr
• deepseek
• critic-free
• dr-grpo
• dapo
• reasoning
• rl-for-llms

Sep 02, 2026

• gpu
• io-aware
• memory-wall
• arithmetic-intensity
• co-design
• flashattention
• data-movement
• llm-systems

Sep 02, 2026

• gpu
• autotuning
• triton
• block-size
• num-warps
• num-stages
• occupancy
• performance
• llm-systems

Sep 02, 2026

• gpu
• kernel-fusion
• memory-wall
• arithmetic-intensity
• torch-compile
• triton
• llm-systems

Sep 02, 2026

• rlhf
• kl-divergence
• reward-hacking
• reference-policy
• ppo
• rl-for-llms

Sep 02, 2026

• gpu
• kv-cache
• inference
• serving
• decode
• memory-bound
• gqa
• mqa
• llm-systems

Sep 02, 2026

• reinforcement-learning
• mdp
• bellman
• rl-for-llms

Sep 02, 2026

• gpu
• distributed-training
• megatron
• tensor-parallelism
• pipeline-parallelism
• sequence-parallelism
• expert-parallelism
• framework
• nvidia
• llm-systems

Sep 02, 2026

• gpu
• coalescing
• hbm
• global-memory
• bandwidth
• sectors
• cache-line
• cuda
• llm-systems

Sep 02, 2026

• gpu
• memory-wall
• bandwidth
• hbm
• ridge-point
• scaling
• energy
• llm-systems

Sep 02, 2026

• gpu
• mfu
• hfu
• utilization
• 6nd
• training-efficiency
• scaling
• llm-systems

Sep 02, 2026

• gpu
• mixed-precision
• fp32
• bf16
• fp16
• fp8
• tf32
• loss-scaling
• master-weights
• llm-systems

Sep 02, 2026

• vision-encoder
• multimodal
• native-resolution
• vit
• kimi-vl
• moonshot-ai

Sep 02, 2026

• vision-transformer
• efficiency
• training
• variable-length
• flashattention

Sep 02, 2026

• gpu
• distributed-training
• nccl
• collective-communication
• all-reduce
• nvlink
• infiniband
• sharp
• topology-aware

Sep 02, 2026

• gpu
• occupancy
• latency-hiding
• registers
• shared-memory
• littles-law
• cuda
• llm-systems

Sep 02, 2026

• gpu
• online-softmax
• softmax
• flashattention
• streaming
• numerical-stability
• llm-systems

Sep 02, 2026

• gpu
• distributed-training
• memory
• optimizer-state
• adam
• mixed-precision
• zero
• fsdp

Sep 02, 2026

• gpu
• paged-attention
• vllm
• kv-cache
• memory-management
• fragmentation
• prefix-sharing
• llm-systems

Sep 02, 2026

• gpu
• model-parallelism
• pipeline-parallelism
• gpipe
• 1f1b
• pipedream
• bubble
• megatron
• llm-systems

Sep 02, 2026

• reinforcement-learning
• policy-gradient
• score-function
• rlhf
• rl-for-llms

Sep 02, 2026

• rl
• ppo
• trust-region
• policy-gradient
• rlhf
• importance-sampling
• rl-for-llms

Sep 02, 2026

• preference-optimization
• dpo
• ipo
• kto
• orpo
• simpo
• cpo
• alignment
• rl-for-llms

Sep 02, 2026

• gpu
• prefill
• decode
• inference
• ttft
• tpot
• compute-bound
• memory-bound
• llm-systems

Sep 02, 2026

• prm
• orm
• process-supervision
• reasoning
• reward-model
• rlvr
• math-shepherd
• rl-for-llms

Sep 02, 2026

• gpu
• quantization
• gptq
• awq
• smoothquant
• fp8
• kv-quant
• inference
• llm-systems

Sep 02, 2026

• gpu
• recomputation
• activation-checkpointing
• memory
• flashattention
• training
• mfu
• llm-systems

Sep 02, 2026

• gpu
• gemm
• register-blocking
• thread-coarsening
• outer-product
• warptiling
• data-reuse
• llm-systems

Sep 02, 2026

• reinforcement-learning
• policy-gradient
• reinforce
• monte-carlo
• rlhf
• rl-for-llms

Sep 02, 2026

• rlhf
• reward-model
• preference-learning
• bradley-terry
• reward-hacking
• rl-for-llms

Sep 02, 2026

• reward-overoptimization
• reward-hacking
• goodhart
• scaling-laws
• warm
• kl-regularization
• rl-for-llms

Sep 02, 2026

• rl-infra
• async-rl
• off-policy
• actor-learner
• importance-sampling
• verifier
• vllm
• rl-for-llms

Sep 02, 2026

• rlaif
• llm-as-a-judge
• constitutional-ai
• rlhf
• reward-model
• alignment
• rl-for-llms

Sep 02, 2026

• rlhf
• alignment
• reward-model
• ppo
• instructgpt
• llm-training
• rl-for-llms

Sep 02, 2026

• rlvr
• rl
• verifiable-rewards
• reasoning
• grpo
• reward-hacking
• deepseek-r1
• rl-for-llms

Sep 02, 2026

• gpu
• roofline
• arithmetic-intensity
• ridge-point
• memory-bound
• compute-bound
• performance-modeling
• llm-systems

Sep 02, 2026

• gpu
• model-parallelism
• sequence-parallelism
• tensor-parallelism
• activation-memory
• megatron
• llm-systems

Sep 02, 2026

• gpu
• shared-memory
• bank-conflicts
• smem
• padding
• transpose
• cuda
• llm-systems

Sep 02, 2026

• gpu
• gemm
• tiling
• shared-memory
• arithmetic-intensity
• data-reuse
• llm-systems

Sep 02, 2026

• gpu
• speculative-decoding
• draft-verify
• medusa
• eagle
• decode
• latency
• llm-systems

Sep 02, 2026

• gpu
• sm
• hardware
• scheduling
• warp-scheduler
• tensor-core
• occupancy
• llm-systems

Sep 02, 2026

• gpu
• tensor-core
• mma
• precision
• fp8
• bf16
• tf32
• matmul
• llm-systems

Sep 02, 2026

• gpu
• model-parallelism
• tensor-parallelism
• megatron
• all-reduce
• nvlink
• llm-systems

Sep 02, 2026

• gpu
• torch-compile
• torchinductor
• torchdynamo
• aotautograd
• triton
• kernel-fusion
• codegen
• llm-systems

Sep 02, 2026

• gpu
• triton
• kernels
• compiler
• mlir
• block-level
• autotuning
• torch-compile
• llm-systems

Sep 02, 2026

• rl
• trust-region
• trpo
• ppo
• natural-gradient
• kl-divergence
• rl-for-llms

Sep 02, 2026

• reinforcement-learning
• value-function
• bellman
• critic
• rl-for-llms

Sep 02, 2026

• gpu
• simt
• warp
• divergence
• predication
• lockstep
• cuda
• llm-systems

Sep 02, 2026

• gpu
• distributed-training
• zero
• deepspeed
• sharded-data-parallelism
• optimizer-state
• memory
• offloading