Requests of different lengths share the model’s batch. Static batching waits for the longest in each batch; continuous batching fills a free slot at the very next iteration.
This interactive visualization needs JavaScript.