2. KernelOperator Implementation and Performance Optimization
Corpus alignment: distinguish operator libraries, kernel libraries, kernel languages, and compiler-owned execution paths. Kernel exercises do not imply that all vendor interfaces are public.
2.1Kernel Programming Models and Execution Fundamentals62.2Tensor Layout and Memory Access62.3From Naive GEMM to Tiled GEMM62.4Tensor Core GEMM: Matrix Instructions, Data Layout, and Cooperation Scope72.5Advanced GEMM Pipelining and Scheduling72.6Matrix Computation Across Shapes and Scenarios62.7Elementwise, Reduction, and Scan62.8Fusion and Memory-Traffic Elimination62.9Exact Attention Kernels62.10Decode Attention and KV Cache Kernels62.11Sparse and Irregular Kernels62.12MoE Kernels62.13Low-Precision and Quantized Kernels62.14Communication Kernels: Device-Initiated Transfer, Data Movement, and Compute Fusion72.15Key Kernels Beyond Language Models62.16GPU Kernel Languages, DSLs, and Operator/Kernel Libraries82.17NPU Kernel Programming: Tiles, Scratchpad, and Heterogeneous Engines82.18Spatial Dataflow Kernels: PEs, Codelets, and Data-Triggered Tasks82.19Profiling, Benchmarking, and Correctness72.20From Operator to Model: Registration, Porting, and Backend Integration7