Gunjan Dhanuka — Learning Notes
Search
Search
Dark mode
Light mode
Reader mode
Explorer
Home
❯
concepts
concepts
52 items under this folder.
Sep 02, 2026
Actor-Critic
reinforcement-learning
actor-critic
critic
variance-reduction
ppo
rl-for-llms
Sep 02, 2026
Advantage Function
reinforcement-learning
advantage
variance-reduction
policy-gradient
rl-for-llms
Sep 02, 2026
Agentic RL (multi-turn, tool-use)
agentic-rl
multi-turn
tool-use
credit-assignment
environments
rlvr
echo-trap
rl-for-llms
Sep 02, 2026
Arithmetic Intensity
gpu
arithmetic-intensity
roofline
flops
bandwidth
memory-bound
compute-bound
llm-systems
Sep 02, 2026
Best-of-N Sampling (and RFT / ReST)
best-of-n
rejection-sampling
rft
rest
expert-iteration
test-time-compute
rl-for-llms
Sep 02, 2026
Bradley–Terry Model
bradley-terry
preference-learning
reward-model
dpo
rlhf
rl-for-llms
Sep 02, 2026
Chain-of-Thought RL (emergent reasoning)
chain-of-thought
reasoning
rlvr
grpo
o1
deepseek-r1
test-time-compute
rl-for-llms
Sep 02, 2026
Constitutional AI (CAI)
constitutional-ai
rlaif
alignment
harmlessness
anthropic
rlhf
rl-for-llms
Sep 02, 2026
CUDA Execution Model
gpu
cuda
simt
execution-model
threads
blocks
shared-memory
llm-systems
Sep 02, 2026
DPO (Direct Preference Optimization)
dpo
preference-optimization
rlhf
bradley-terry
kl-regularization
alignment
rl-for-llms
Sep 02, 2026
FlashAttention
gpu
flashattention
attention
io-aware
online-softmax
kernel-fusion
recomputation
hopper
blackwell
llm-systems
Sep 02, 2026
GEMM (General Matrix Multiply)
gpu
gemm
matmul
tensor-core
arithmetic-intensity
tiling
transformer
llm-systems
Sep 02, 2026
Generalized Advantage Estimation (GAE)
reinforcement-learning
gae
advantage-estimation
td-learning
variance-reduction
ppo
rl-for-llms
Sep 02, 2026
Global Native-Resolution Encoding (GNE)
vision-encoder
multimodal
native-resolution
high-resolution
Sep 02, 2026
GPU Memory Hierarchy
gpu
memory-hierarchy
registers
shared-memory
l2
hbm
bandwidth
latency
llm-systems
Sep 02, 2026
GRPO (Group Relative Policy Optimization)
grpo
rl
rlvr
deepseek
critic-free
dr-grpo
dapo
reasoning
rl-for-llms
Sep 02, 2026
IO-Aware Algorithms
gpu
io-aware
memory-wall
arithmetic-intensity
co-design
flashattention
data-movement
llm-systems
Sep 02, 2026
Kernel Autotuning
gpu
autotuning
triton
block-size
num-warps
num-stages
occupancy
performance
llm-systems
Sep 02, 2026
Kernel Fusion
gpu
kernel-fusion
memory-wall
arithmetic-intensity
torch-compile
triton
llm-systems
Sep 02, 2026
KL Regularization in RLHF
rlhf
kl-divergence
reward-hacking
reference-policy
ppo
rl-for-llms
Sep 02, 2026
Markov Decision Process
reinforcement-learning
mdp
bellman
rl-for-llms
Sep 02, 2026
Memory Coalescing
gpu
coalescing
hbm
global-memory
bandwidth
sectors
cache-line
cuda
llm-systems
Sep 02, 2026
Memory Wall
gpu
memory-wall
bandwidth
hbm
ridge-point
scaling
energy
llm-systems
Sep 02, 2026
MFU (Model FLOPs Utilization)
gpu
mfu
hfu
utilization
6nd
training-efficiency
scaling
llm-systems
Sep 02, 2026
Mixed-Precision Training
gpu
mixed-precision
fp32
bf16
fp16
fp8
tf32
loss-scaling
master-weights
llm-systems
Sep 02, 2026
MoonViT
vision-encoder
multimodal
native-resolution
vit
kimi-vl
moonshot-ai
Sep 02, 2026
NaViT Sequence Packing
vision-transformer
efficiency
training
variable-length
flashattention
Sep 02, 2026
Occupancy
gpu
occupancy
latency-hiding
registers
shared-memory
littles-law
cuda
llm-systems
Sep 02, 2026
Online Softmax
gpu
online-softmax
softmax
flashattention
streaming
numerical-stability
llm-systems
Sep 02, 2026
Policy Gradient Theorem
reinforcement-learning
policy-gradient
score-function
rlhf
rl-for-llms
Sep 02, 2026
PPO (Proximal Policy Optimization)
rl
ppo
trust-region
policy-gradient
rlhf
importance-sampling
rl-for-llms
Sep 02, 2026
Preference Optimization (RL-free family)
preference-optimization
dpo
ipo
kto
orpo
simpo
cpo
alignment
rl-for-llms
Sep 02, 2026
Process Reward Model (PRM)
prm
orm
process-supervision
reasoning
reward-model
rlvr
math-shepherd
rl-for-llms
Sep 02, 2026
Recomputation (Activation Checkpointing)
gpu
recomputation
activation-checkpointing
memory
flashattention
training
mfu
llm-systems
Sep 02, 2026
Register Blocking (Thread Coarsening)
gpu
gemm
register-blocking
thread-coarsening
outer-product
warptiling
data-reuse
llm-systems
Sep 02, 2026
REINFORCE
reinforcement-learning
policy-gradient
reinforce
monte-carlo
rlhf
rl-for-llms
Sep 02, 2026
Reward Model
rlhf
reward-model
preference-learning
bradley-terry
reward-hacking
rl-for-llms
Sep 02, 2026
Reward Overoptimization (Goodhart)
reward-overoptimization
reward-hacking
goodhart
scaling-laws
warm
kl-regularization
rl-for-llms
Sep 02, 2026
RL Post-Training Infrastructure
rl-infra
async-rl
off-policy
actor-learner
importance-sampling
verifier
vllm
rl-for-llms
Sep 02, 2026
RLAIF (RL from AI Feedback)
rlaif
llm-as-a-judge
constitutional-ai
rlhf
reward-model
alignment
rl-for-llms
Sep 02, 2026
RLHF (Reinforcement Learning from Human Feedback)
rlhf
alignment
reward-model
ppo
instructgpt
llm-training
rl-for-llms
Sep 02, 2026
RLVR (RL with Verifiable Rewards)
rlvr
rl
verifiable-rewards
reasoning
grpo
reward-hacking
deepseek-r1
rl-for-llms
Sep 02, 2026
Roofline Model
gpu
roofline
arithmetic-intensity
ridge-point
memory-bound
compute-bound
performance-modeling
llm-systems
Sep 02, 2026
Shared Memory Bank Conflicts
gpu
shared-memory
bank-conflicts
smem
padding
transpose
cuda
llm-systems
Sep 02, 2026
Shared-Memory Tiling (Blocking)
gpu
gemm
tiling
shared-memory
arithmetic-intensity
data-reuse
llm-systems
Sep 02, 2026
Streaming Multiprocessor (SM)
gpu
sm
hardware
scheduling
warp-scheduler
tensor-core
occupancy
llm-systems
Sep 02, 2026
Tensor Core
gpu
tensor-core
mma
precision
fp8
bf16
tf32
matmul
llm-systems
Sep 02, 2026
torch.compile (TorchDynamo / TorchInductor)
gpu
torch-compile
torchinductor
torchdynamo
aotautograd
triton
kernel-fusion
codegen
llm-systems
Sep 02, 2026
Triton
gpu
triton
kernels
compiler
mlir
block-level
autotuning
torch-compile
llm-systems
Sep 02, 2026
Trust Region
rl
trust-region
trpo
ppo
natural-gradient
kl-divergence
rl-for-llms
Sep 02, 2026
Value Function
reinforcement-learning
value-function
bellman
critic
rl-for-llms
Sep 02, 2026
Warp & SIMT
gpu
simt
warp
divergence
predication
lockstep
cuda
llm-systems