Sparse and Irregular Kernels

2.11.1COO, CSR, CSC, BSR, and Compressed Layouts#

2.11.2SpMV, SpMM, and SDDMM#

2.11.3Structured Sparsity, N:M Sparsity, and Sparse Tensor Core#

2.11.4Block-Sparse Attention and Dynamic Indexing#

2.11.5Sparse Selection, Gather/Scatter, and Load Imbalance#

2.11.6Sparsity Metadata Overhead and the Limits of Speedup#