Deep Learning and Model Computation Fundamentals

6.1.1Tensor, Linear Layer, MLP, and Activation#

6.1.2Loss, Gradient, Backpropagation, and Optimizer#

6.1.3Batch, Sequence, Hidden Dimension, and Parameter Count#

6.1.4Computation Graphs, Automatic Differentiation, and Training/Inference Differences#

6.1.5Computational Complexity, Space Complexity, and Data Movement#

6.1.6Dense, Sparse, and Conditional Computation#