Model Workloads and Resource Profiles

5.4.1Training, Prefill, Decode, and Verification#

5.4.2Parameters, Gradients, Optimizer States, and Activations#

5.4.3KV Cache, Recurrent State, and Model State#

5.4.4Execution Profiles of Dense, MoE, Multimodal, and Diffusion Models#

5.4.5Compute/Memory/Communication/CPU Bottlenecks#

5.4.6Batch Size, Context Length, and Output Length Distribution#