KV Cache Allocation, Addressing, and Lifecycle

5.19.1Contiguous, Paged, and Blocked KV Cache#

5.19.2Block Table, Memory Pool, and Dynamic Growth#

5.19.3Cache Organization for MHA/GQA/MLA#

5.19.4Prefix Sharing, Reference Counting, and Copy-on-Write#

5.19.5KV Cache Eviction, Compaction, and Fragmentation#

5.19.6Diagnosing Cache Hits, OOM, and Memory Budget#