Elementwise, Reduction, and Scan

2.7.1Vector Add and Elementwise Fusion#

2.7.2Softmax and Online Softmax#

2.7.3LayerNorm, RMSNorm, and Reduction Layout#

2.7.4Prefix Sum, Scan, and Histogram#

2.7.5Top-k, Argmax, Sort, and Sampling#

2.7.6Welford's Algorithm and Stable Reduction#