AI Architecture & Systems

AI Architecture and Systems Tutorial

A topic map from GEMM execution to datacenter-scale AI, centered on generality, specialization, and data movement.

Revision: 2026-09-16 · Curriculum status: planned topics

This outline connects six layers: Microarchitecture → Kernel → Compiler → Architecture → System → Algorithms. Hardware chapters follow the supplied survey, Balancing Generality and Specialization: A Survey on AI Datacenter Hardware Architecture, and the AI Datacenter Accelerator Research Corpus. Kernel, compiler, and runtime comparisons use the corpus’s per-chip software mappings. The broader systems and algorithms topics are carried forward from the original outline, with Awesome-ML-SYS-Tutorial as a systems reading index.

1. Microarchitecture
How does GEMM execute inside an accelerator?
PE, systolic array, memory hierarchy, dataflow, roofline, and operand delivery.
17 sections · 116 topics
2. Kernel
Operator Implementation and Performance Optimization
How are operators implemented efficiently?
Layout, tiling, pipelines, matrix instructions, libraries, and device-specific kernel programming.
20 sections · 131 topics
3. Compiler
Compilation and Program Mapping
How does a model become an executable device program?
Graph/IR lowering, scheduling, memory planning, placement/routing, and executable artifacts.
20 sections · 116 topics
4. Architecture
Accelerator Architecture and AI Datacenters
How are hardware resources organized?
Accelerator taxonomy, programming models, fabrics, generations, power/cooling, and design challenges.
31 sections · 226 topics
5. System
Training, Inference, and AI Infrastructure
How are workloads executed and operated?
Distributed training, serving, post-training, runtimes, heterogeneous fleets, and production infrastructure.
40 sections · 264 topics
6. Algorithms
Models, Training Methods, and Generation Algorithms
What computation does the model require?
Model structures, training objectives, generation, compression, and model-family evolution.
35 sections · 230 topics
Source map and scope

Survey. Yufeng Gu, Jiazhen Wang, and Reetuparna Das, September 2026, supplied 34-page draft. “Survey §…” and figure references refer to that attachment. Companion project page.

Corpus. Platform links are pinned to commit c49a55c6cdbb. Per-chip records distinguish confirmed, inferred, contested, nonpublic, historical, and announced information. Those qualifications remain relevant when developing the corresponding chapters.

Core and extensions. The survey’s core taxonomy is GPU, NPU, Spatial Dataflow, and Compute-in-Memory. Additional vendor cases, photonic/neuromorphic comparisons, and CXL/HBF topics are labeled extensions. Untagged foundations and the detailed systems/algorithms curriculum are retained topics, rather than findings attributed to the hardware survey.

Cross-layer boundaries. Numerical hardware support, quantized kernels, compiler transformations, deployment policy, and quantization algorithms are separate topics. Likewise, collective semantics, communication algorithms, and physical topology are separated. A physical server/rack/pod need not coincide with a scale-up domain.