NPU Kernel Programming: Tiles, Scratchpad, and Heterogeneous Engines

SourcesTPU, Neuron, Ascend and Cambricon corpus records.

2.17.1TPU Pallas: BlockSpec, Memory Space, and MXU/Vector Cooperation#

2.17.2Interfacing Pallas/Mosaic with Graph Compilation#

2.17.3AWS NKI: Language API, ISA Intrinsics, and SBUF/PSUM#

2.17.4Operator Division of Labor Across Neuron Tensor/Vector/Scalar/GPSIMD#

2.17.5Ascend C: Data Movement, Cube/Vector, and Pipeline Synchronization#

2.17.6Ascend TBE/TIK, CATLASS, and Device-Specific DSLs#

2.17.7Cambricon BANG C and Explicit Local-Memory Programming#

2.17.8Layout and Capacity Constraints of GEMM/Attention on Different NPUs#