AI Architecture & Systems

1Microarchitecture 2Kernel 3Compiler 4Architecture 5System 6Algorithms

Architecture

Overview
4.1Scope, Terminology, and Evidence Standards for Architectural Comparison
4.2AI Datacenter Context and Four Categories of Resource Pressure
4.3Core Taxonomy: Four Classes of AI Accelerator Architecture
4.4GPU: General-Purpose Parallel Execution and Specialized Matrix Computation
4.5NPU: Heterogeneous Compute Engines and Shared Local Memory
4.6Spatial Dataflow I: PE Arrays and Distributed Local Memory
4.7Spatial Dataflow II: Reconfigurable Architectures
4.8Spatial Dataflow III: Functional-Slice Streaming
4.9Compute-in-Memory: SRAM, DRAM, and the Location of Computation
4.10Compute Engines: Cross-Category Organization and Trade-offs
4.11Memory Hierarchy: From Hardware Cache to Software Scratchpad
4.12Programming Models: Understanding Hardware Constraints Through Software Layers
4.13Packaging, Chiplets, and Logic–Memory Integration
4.14Host, Remote Memory, and Memory-Centric Extensions
4.15Physical Deployment Hierarchy and Scale-Up/Scale-Out Domains
4.16Node-Scale Scale-Up Interconnect
4.17Rack-Scale Scale-Up Interconnect
4.18Pod-Scale Scale-Up Interconnect
4.19Scale-Out, Network Datapaths, and Optical Interconnect
4.20Topology Performance: Connectivity Is Not Effective Communication Performance
4.21Collective Semantics: Endpoint Data Transformation
4.22Collective Algorithms: Logical Communication Scheduling
4.23Collective–Topology Mapping and Communication Offload
4.24GPU Generational Evolution: Compute, Data Supply, and Cooperation Scope
4.25Specialized Accelerator Generations: Workload Positioning and Hardware–Software Co-Design
4.26Cross-Generation Comparison: Numerical Formats, Structured Sparsity, and Peak Conventions
4.27Cross-Generation Comparison: Memory Hierarchy and Scale-Up Fabric
4.28Power Delivery, Cooling, and Practically Operable Capacity
4.29Future Design Challenges: Generality versus Specialization
4.30Extensions and Contrast Cases: Photonics, Neuromorphic, and Limited-Disclosure Designs
4.31Architectural Evaluation and Full-Stack Case Organization

Architecture/§4.5

NPU: Heterogeneous Compute Engines and Shared Local Memory

SourcesSurvey §3 and Table 2; the final two entries broaden the case pool using the corpus.

4.5.1Google TPU: MXU, Vector/Scalar, and Generation-Specific Units#

4.5.2AWS Trainium/Inferentia: NeuronCore and Explicit Scratchpad#

4.5.3Huawei Ascend: Da Vinci, Cube/Vector/Scalar, and CANN#

4.5.4Intel Gaudi: Matrix Engine, TPC, and Network Integration#

4.5.5Microsoft Maia: Matrix/Vector and Cloud Deployment#

4.5.6Qualcomm Cloud AI: Matrix, Vector, and Scalar Engines#

4.5.7Cambricon MLU: Multi-Core Neural Processor and BANG C#

4.5.8Corpus Extensions: Alibaba T-Head, Kunlunxin, and Furiosa#

4.5.9Corpus Extensions: Sophgo, Vastai, Tecorigin, and Stream Computing#

← Previous4.4 GPU: General-Purpose Parallel Execution and Specialized Matrix Computation → Next4.6 Spatial Dataflow I: PE Arrays and Distributed Local Memory

On this page

4.5.1Google TPU: MXU, Vector/Scalar, and Generation-Specific Units
4.5.2AWS Trainium/Inferentia: NeuronCore and Explicit Scratchpad
4.5.3Huawei Ascend: Da Vinci, Cube/Vector/Scalar, and CANN
4.5.4Intel Gaudi: Matrix Engine, TPC, and Network Integration
4.5.5Microsoft Maia: Matrix/Vector and Cloud Deployment
4.5.6Qualcomm Cloud AI: Matrix, Vector, and Scalar Engines
4.5.7Cambricon MLU: Multi-Core Neural Processor and BANG C
4.5.8Corpus Extensions: Alibaba T-Head, Kunlunxin, and Furiosa
4.5.9Corpus Extensions: Sophgo, Vastai, Tecorigin, and Stream Computing