Serving capacity planner

Plan the endpoint before setting its request limits.

Import the model or set its planning profile, then use maximum model length, sequences per replica, VRAM per GPU, tensor parallelism, and independent replicas to make the cache boundary visible.

vLLM serving planner

Turn an inference fit into an endpoint boundary.

Separate tensor-parallel model capacity from independent endpoint replica capacity before setting request limits.

How the estimates work →

The values below are editable planning baselines. Change the few inputs you know; open Advanced assumptions only when they change your decision.

Model configuration

Import a Hugging Face model profile.

Load a public model revision to set its verified parameter total and decoder attention shape. The current workload context stays unchanged.

Hugging Face model ID

Current hardware

No hardware profile saved.

Save the GPU capacity you already have, then apply it across VRAM planning paths.

Profile nameInstalled GPUs

Model parameters (B)

Weight format

Maximum context (tokens)

Sequences per replica

VRAM per GPU (GB)

Advanced assumptionsAdjust architecture, cache format, parallel topology, and utilization after the request envelope is clear.

KV Cache architecture

KV Cache precision

Tensor-parallel GPUs

Independent replicas

GPU memory utilization

Required per GPU——

KV Cache budget / engine——

Target cache pressure——

Endpoint capacity——

Planned with margin—

Usable per GPU—

——
——
——
——
——

—

Pressure matrix

Context × simultaneous requests

Each cell shows required VRAM per GPU. The outlined cell is the current plan.

Requests ↓ / context →	—	—	—	—
—	——	——	——	——
—	——	——	——	——
—	——	——	——	——
—	——	——	——	——

Weight-format comparison

Keep the workload constant.

Weights change between rows; architecture, KV Cache pressure, runtime reserve, and GPU topology stay fixed.

Weight format	Weights / GPU	Required / GPU	After margin
—	—	—	—
—	—	—	—
—	—	—	—

Per-replica vLLM shape

Start each replica with explicit engine limits.

—

Next decision

Verify vLLM limits Pressure-test concurrency Diagnose an observed OOM

Plan tools

Keep this useful version close, shareable, and easy to revisit.

Reset inputs

Versioned runtime handoff

Per-replica starting arguments

vLLM v0.27.0 ↗Reviewed 2026-08-11

—

--gpu-memory-utilizationOfficial reference ↗: —Caps the fraction of GPU memory available to this vLLM instance.
--max-model-lenOfficial reference ↗: —Sets the combined prompt-and-output token boundary for the engine.
--max-num-seqsOfficial reference ↗: —Caps the sequences processed in one engine iteration.
--tensor-parallel-sizeOfficial reference ↗: —Sets the number of tensor-parallel groups used by the engine.
--kv-cache-dtypeOfficial reference ↗: —Selects the data type used to store the engine KV Cache.

Every emitted argument is constrained by the versioned vLLM CLI reference ↗.

What it models

Loaded weights, runtime reserve, allocatable KV Cache, target model length, sequences per replica, and the replica topology.

Runtime handoff

Copy the per-replica vLLM arguments, then observe cache pressure as real traffic reaches the endpoint.

Next learning path

Set max model length →