Serving capacity planner

Plan the endpoint before setting its request limits.

Import the model or set its planning profile, then use maximum model length, sequences per data-parallel engine, VRAM per GPU, TP, and DP to make the cache boundary visible.

vLLM serving planner

Turn an inference fit into an endpoint boundary.

Separate tensor-parallel model capacity from data-parallel endpoint capacity before setting request limits.

How the estimates work →

Model configuration

Import a Hugging Face model profile.

Load the public model configuration to set parameters, attention shape, and available context in this plan.

Hugging Face model ID

Model parameters (B)

Weight format

KV Cache architecture

KV Cache precision

Maximum context (tokens)

Sequences per DP engine

VRAM per GPU (GB)

Tensor-parallel GPUs

Data-parallel engines

GPU memory utilization

Required per GPU——

KV Cache budget / engine——

Target cache pressure——

Endpoint capacity——

vLLM start shape

Start with explicit engine limits.

—

—

What it models

Loaded weights, runtime reserve, allocatable KV Cache, target model length, sequences per engine, and TP/DP topology.

Runtime handoff

Copy the starting vLLM flags, then observe cache pressure as real traffic reaches the endpoint.

Next learning path

Set max model length →