AI Architecture & Systems
1
Microarchitecture
2
Kernel
3
Compiler
4
Architecture
5
System
6
Algorithms
Algorithms
/
§6.2
The Full Lifecycle of a Foundation Model
6.2.1
Pretraining, Continued Pretraining, and Mid-Training
#
6.2.2
Supervised Fine-Tuning and Instruction Tuning
#
6.2.3
Preference Optimization and Reinforcement Learning
#
6.2.4
Distillation, Compression, and Deployment
#
6.2.5
Inference, Test-Time Compute, and Continuous Evaluation
#
6.2.6
Scaling Laws and Compute-Optimal Training
#
← Previous
6.1 Deep Learning and Model Computation Fundamentals
→ Next
6.3 Tokenization and Input Representation