Speculative Decoding

6.18.1The Draft–Verify Framework and Speculative Sampling#

6.18.2Acceptance/Rejection and Target-Distribution Preservation#

6.18.3Independent Draft Models and Self-Speculation#

6.18.4N-Gram, Prompt Lookup, and Retrieval-Based Drafting#

6.18.5The Relationship Between Multi-Token Prediction and Speculative Decoding#

6.18.6Medusa, Hydra, ReDrafter, and Multi-Head/Tree Drafting#

6.18.7EAGLE, EAGLE-2, and EAGLE-3#

6.18.8DFlash: Block-Diffusion Drafting#

6.18.9DSpark: Semi-Autoregressive Drafting and Confidence-Scheduled Verification#

6.18.10Acceptance Length, Draft Cost, Verification Cost, and the Limits of Speedup#