Training memory
Set sequence length and micro-batch before increasing VRAM
Adapter training can still exhaust a GPU when sequences or micro-batches grow. Gradient checkpointing exchanges some compute time for a lower activation-memory requirement.
Decision 01
Sequence length is a first-order memory lever
Activation memory grows with the batch and sequence dimensions, and long-context attention can add further pressure. Choose a sequence target that matches the training examples rather than the largest supported window.
Decision 02
Micro-batch and effective batch are separate choices
Lower the per-device micro-batch when memory is the constraint, then use gradient accumulation when you need a larger effective batch. This preserves the distinction between GPU fit and optimization schedule.
Decision 03
Use checkpointing after setting the workload
Gradient checkpointing saves fewer activations and recomputes them during backward passes. Enable it when the desired sequence and micro-batch do not leave enough activation headroom, then account for the additional compute time.