Keep every request’s KV cache in small fixed-size blocks from one pool, with a block table per request, a free list, and reference counts so parallel samples share their prompt.
This interactive visualization needs JavaScript.