Methodology
Make each capacity boundary visible.
The planners use a shared model profile, then apply the memory and parallelism rules that matter for inference, vLLM serving, and adapter training.
Model → Workload → Hardware
One model baseline, six explicit intent paths
Local chat, coding, and RAG keep the inference memory boundary in view. API serving continues into per-replica vLLM limits. LoRA and QLoRA carry the same model architecture, workload length, and per-GPU capacity into adapter-training memory. The training intent explicitly selects its matching BF16/FP16 LoRA or 4-bit QLoRA base-weight path while leaving user-entered context and hardware values intact.
Imported model profile
Public config → parameters, layers, hidden size, and KV heads
A model ID import reads the public Hugging Face configuration and carries the architecture into the planner. The shared link retains the imported values, so a planning brief remains reproducible without a model lookup.
Inference and vLLM
Weights + request-driven KV Cache + runtime reserve
Inference capacity starts with loaded weights, then adds KV Cache from layers, KV heads, head dimension, context, precision, and active requests. Tensor parallelism divides one engine across GPUs. Independent serving replicas each keep their own weights and KV Cache, so the vLLM page separates per-replica limits from endpoint-wide capacity.
Versioned runtime export
Five generated flags, one pinned CLI contract
The serving planner emits only --gpu-memory-utilization, --max-model-len, --max-num-seqs, --tensor-parallel-size, --kv-cache-dtype. Each value is linked to the official vLLM v0.27.0 CLI reference ↗, and the copied brief carries the same release, source URL, and 2026-08-11 review date. The current release feed remains a separate maintenance trigger, so upstream movement prompts a contract review instead of silently changing generated output.
Decision-quality comparison
One calculation model, three views of the capacity boundary
The composition view separates per-GPU weights, KV Cache, runtime reserve, implementation margin, and capacity left after that margin. The context-by-request matrix recalculates the same inference model at nearby workload points. The weight-format table changes only the modeled weight format while retaining architecture, context, requests, cache precision, reserve, utilization, and GPU topology. Each view remains a directly comparable memory-capacity result.
Current hardware profile
One explicitly saved capacity record across every VRAM path
The local hardware profile stores a name, VRAM per GPU, installed GPU count, and usable-memory target on the visitor’s device. Applying it changes hardware capacity inputs while preserving the selected model, workload, context, requests, training shape, and topology choices.
OOM diagnostic order
Topology → weights → runtime base → KV Cache → margin
The diagnostic rebuilds the attempted inference shape with the canonical planner. It checks whether the requested engine topology exists, whether weights enter usable VRAM, whether weights plus runtime reserve fit, whether the requested context and simultaneous requests fit, and whether the implementation margin remains. Proposed corrections recalculate one changed input against the same model rather than applying a second memory formula.
LoRA and QLoRA
Frozen base + adapter target scope + optimizer + activations
The training planner derives adapter parameters from rank, hidden size, layers, and target scope. Sequence length and micro-batch drive the activation term. Gradient checkpointing uses a lower activation profile to represent the recomputation-for-memory tradeoff.
Training topology
Single GPU and DDP replicate; FSDP shards model state
Single-GPU and DDP plans assess the full training state per device. The FSDP plan shards base, adapter, and optimizer state across the selected GPU count while retaining the request-sized activation load on each device.