5. SystemTraining, Inference, and AI Infrastructure

The original training/serving/RL scope is retained. Corpus-derived runtime boundaries and survey-derived fleet, communication, and power constraints are added as explicit cross-layer topics.

5.1AI Runtime and Operating System Fundamentals6 5.2The Execution Chain of Device Runtime, Driver, and Firmware8 5.3System Metrics, Queueing, and Capacity Planning7 5.4Model Workloads and Resource Profiles6 5.5Distributed Runtime and Collective Communication8 5.6Data Parallelism and Parameter Sharding6 5.7Tensor Parallelism6 5.8Pipeline Parallelism6 5.9Sequence and Context Parallelism6 5.10Expert Parallelism6 5.11Hybrid Parallelism and Automatic Planning7 5.12Training Memory Management6 5.13Training Data and Checkpoint Pipelines6 5.14Training Runtime and Numerical Stability6 5.15Elasticity and Fault Tolerance in Large-Scale Training6 5.16Distributed Training Frameworks and Accelerator Backends7 5.17The Full Lifecycle of an Inference Request6 5.18Batching and Online Scheduling7 5.19KV Cache Allocation, Addressing, and Lifecycle6 5.20Prefix Cache and Multi-Tier Caching7 5.21Disaggregated Serving7 5.22Routing and Distributed Serving Scheduling6 5.23System Design for MoE Serving6 5.24Deploying and Tuning Quantized Models6 5.25System Support for Sparse Models and Long Context6 5.26System Implementation of Speculative Decoding7 5.27Multi-Model, Multi-Tenant, and Adapter Serving6 5.28Serving Frameworks, Component Ecosystem, and Device Backends9 5.29RL Post-Training Infrastructure7 5.30Training–Inference Consistency and RL Correctness7 5.31RL Frameworks and Source-Code Studies7 5.32Multimodal and Real-Time Speech Serving6 5.33Diffusion and Non-Autoregressive Model Systems6 5.34RAG, Agents, and Compound AI Systems7 5.35Cluster Orchestration and Production Deployment6 5.36Scheduling Heterogeneous Fleets and Specialized Resource Pools7 5.37Power/Thermal-Aware System Operation7 5.38Security, Privacy, and Trustworthy AI Infrastructure6 5.39Observability, Benchmarking, and Troubleshooting8 5.40 Reading System Source Code and Cross-Layer End-to-End Practice8