Memory Hierarchy: From Hardware Cache to Software Scratchpad

SourcesSurvey §3.2 and Fig. 5; memory organization is a comparison axis, not a replacement for the four-category taxonomy.

4.11.1Fully Hardware-Managed, Hybrid, and Fully Software-Managed Memory#

4.11.2The CPU Cache Counterpart and the Gaudi Cache-Management Case#

4.11.3GPU: L1/Shared Memory/L2 and Dedicated Tensor Memory#

4.11.4NPU: Shared Scratchpad, Accumulator Buffer, and Explicit Prefetch#

4.11.5PE Array: Distributed Local SRAM and Memory Placement#

4.11.6HBM, GDDR, LPDDR, DDR, and SRAM-Centric Organizations#

4.11.7Determinism, QoS, Compilation Complexity, and Dynamic Access Patterns#