Execution Organization: SIMT, Heterogeneous Engines, and Spatial Execution

SourcesSurvey §3 and §3.1; Graphcore layer mapping and Cerebras layer mapping.

1.3.1Scalar, SIMD, SIMT, MIMD, and VLIW#

1.3.2SM, CU, Warp, Wavefront, and Thread Scheduling#

1.3.3Instruction Pipelines, Dependency Checking, and Scoreboarding#

1.3.4Latency Hiding, ILP, TLP, and Branch Divergence#

1.3.5Division of Labor Among Matrix, Vector, Scalar, and Special-Function Engines#

1.3.6PE-Local Execution, BSP, and Data-Triggered Execution#

1.3.7Dynamic Hardware Scheduling, Static Software Scheduling, and Their Combination#

1.3.8Local Systolic Dataflow versus Chip-Wide Spatial Dataflow#