GPU MODE

What it is

GPU MODE (formerly CUDA MODE) is the leading open community for GPU/kernel programming — a weekly recorded lecture series, a very active Discord (~30k members), curated resources, and a competitive kernel leaderboard. Founded by Mark Saroufim and Andreas Köpf. It is widely cited as the single best community resource for the aspiring GPU engineer.

Key contributions / resources

• Lecture series — weekly recorded GPU-programming lectures (gpumode.com/lectures, YouTube “GPU MODE”), organized substantially around the PMPP textbook (Programming Massively Parallel Processors, 4th ed.).
• Discord — ~30k members; reading groups, help channels, and the hub where competitions run. Being useful here (answering questions, reviewing kernels) is a recognized route to referrals.
• resource-stream — a curated GitHub link repo of GPU-programming resources.
• reference-kernels + KernelBot — the official problem sets and reference kernels for the GPU MODE Leaderboard, run via the “KernelBot” Discord bot. Hosts sponsored competitions with real prize pools (AMD kernel/distributed sets, an NVIDIA Blackwell NVFP4 set) plus academic problems (PMPP practice, Stanford CS149, Princeton). A leaderboard placement is a dated, public, adversarial signal of skill.

Associated work

• The roadmap flags GPU MODE as a top learning resource and warns against “tutorial hell” — consuming lectures without shipping a benchmarked kernel: 13-gpu-engineer-roadmap (§6 practice ladder, §8 how to break in).
• Competitions/benchmarks context: kernelbench (a complementary source of well-specified kernel problems).
• Practice targets align with flash-attention and GEMM optimization work in the gpu-systems-for-llms curriculum.