GPU Kernel Languages, DSLs, and Operator/Kernel Libraries

SourcesNVIDIA layer mapping and AMD layer mapping; other original library topics retained.

2.16.1CUDA/HIP: Threads, Warp/Wavefront, and Device-Specific Interfaces#

2.16.2Triton: Tile Programming, Autotuning, and Backend Differences#

2.16.3CuTe/CUTLASS and CuTe DSL: Explicit Layout and Cooperative Scheduling#

2.16.4CUDA Tile IR/cuTile: The Boundary Between Tile-Level Abstraction and Low-Level Control#

2.16.5cuBLAS/cuBLASLt/cuDNN and CUB/Thrust/libcu++#

2.16.6rocBLAS/hipBLASLt/MIOpen and CK/CK-Tile/AITER/Tensile#

2.16.7TileLang, FlashInfer, Transformer Engine, and DeepGEMM#

2.16.8CPU Counterpart: SIMD/AMX Microkernels and oneDNN#