Scheduling Heterogeneous Fleets and Specialized Resource Pools

5.36.1Phase Profiles of Training, Prefill, Decode, Draft/Verify, and Multimodal Stages#

5.36.2Unified Resource Pools versus Workload-Partitioned Specialized Pools#

5.36.3Scheduling Constraints from Model Compatibility, Precision Support, and Available Kernels#

5.36.4Capacity Fragmentation, Idle Devices, Queue Imbalance, and Resource Rebalancing#

5.36.5Data-Movement Cost of Migrating Weights, KV Cache, and Activations#

5.36.6Rack/Pod Placement and Cross-Network-Domain Communication#

5.36.7Throughput, Tail Latency, Cost, and Reuse of Deployed Hardware#