AI Architecture & Systems
1
Microarchitecture
2
Kernel
3
Compiler
4
Architecture
5
System
6
Algorithms
Kernel
/
§2.8
Fusion and Memory-Traffic Elimination
2.8.1
Bias/Activation/Residual Epilogue Fusion
#
2.8.2
QKV Projection and RoPE Fusion
#
2.8.3
Norm, Residual, and Quantization Fusion
#
2.8.4
Fused MLP, SwiGLU, and GeGLU
#
2.8.5
Fused Loss and Fused Optimizer
#
2.8.6
Launch Overhead, Memory-Traffic Savings, and Fusion Boundaries
#
← Previous
2.7 Elementwise, Reduction, and Scan
→ Next
2.9 Exact Attention Kernels