TL;DR: A public, code-first curriculum walks builders from naive CUDA GEMM to Hopper/Blackwell kernels like FlashAttention, filling a gap in practical kernel-engineering material for AI developers.
Summary: An independent creator published the intro and roadmap video for a free 44-session GPU kernel engineering series, with code to be made public. The curriculum is organized around the roofline model — framing kernel performance as a data-movement problem bounded by min(peak compute, bandwidth × arithmetic intensity) — and uses FlashAttention as the worked example, tiling attention so the full N×N score matrix never hits HBM. Sessions progress from CUDA fundamentals and the memory hierarchy through optimized GEMM, tensor cores, profiling, PyTorch/Triton integration, then Hopper and Blackwell topics including TMA, WGMMA, CUTLASS, FP8/NVFP4 quantization, multi-GPU basics, FlashAttention 4, and DeepSeek's kernels; each session ends with a benchmark against a baseline.
Why it matters: Most ML engineers treat kernel optimization as a black box, and this series gives a structured, reproducible path to writing and profiling custom kernels on current NVIDIA hardware. Worth following if you work on inference performance, attention variants, or low-precision quantization — start with the roofline and GEMM sessions before the Hopper/Blackwell material.
Source: reddit