Execution Organization and Compute Units

A processing element (PE) is the smallest unit that does arithmetic: a multiplier, an adder, a little local state, and just enough control to know what to do next. Accelerators differ enormously at the chip level, but nearly all of them are built by replicating a PE thousands of times and then deciding how operands reach it.

That decision is where the designs diverge. A general-purpose machine hands every PE its operands from a centralized register file or SRAM, which is flexible but expensive: the same value gets read many times over. Systolic arrays take the opposite approach. Activations and weights are injected at the boundary of the array, and interior PEs receive their operands from neighbours and pass partial sums onward. This neighbour-to-neighbour reuse replaces a large fraction of costly memory accesses with short local transfers, which is why systolic arrays are so common in accelerators that are built primarily for matrix multiplication ^[1].

Not every accelerator has a matrix engine at all. Some rely entirely on SIMD or vector units, betting on memory bandwidth instead of operand reuse. Cerebras takes this route with an SRAM-centric wafer-scale design, which sustains near-peak performance even on the memory-bound BLAS-1 and BLAS-2 operations that fall far short of peak on HBM-based systems ^[2]. Processing-in-memory designs from Samsung and SK hynix make a similar bet, placing vector or SIMD units next to DRAM banks to get very high effective bandwidth ^{[3, 4]}.