Training–Inference Consistency and RL Correctness

5.30.1Aligning Tokenizer, Chat Template, and Special Tokens#

5.30.2Log Probability, Masking, and Sequence Boundaries#

5.30.3Consistency Between the Sampling Distribution and the Training Objective#

5.30.4Numerical Differences Across Kernels, Precision, and Reduction Paths#

5.30.5Policy Version, Weight Staleness, and Off-Policy Drift#

5.30.6Importance Sampling and Mismatch Correction#

5.30.7Correctness Validation of Gradients, Samples, and Performance Optimizations#