Start with a complete training step
Your first test should run through data loading, the forward pass, gradient computation, and the optimizer update. Measure this full cycle before increasing the batch. Weights are only part of the memory used: activations and training states must also fit within the 80 GB.
Keep a representative sequence of inputs, including the large examples. You'll avoid sizing the entire campaign based on a particularly easy first batch.
Mixed precision and controlled comparison
PyTorch lets you choose a precision suited to certain operations with autocast. Test this setting on your loss function and validation criteria. Enabling an option is not proof of numerical stability: keep a comparison run, metrics, and the exact parameters.
For multiple cards, distinguish per-GPU batch from global batch. The number of processes and gradient accumulation are part of the protocol, just like the learning rate.
SXM, PCIe, or more memory
The H100 PCIe keeps an 80 GB capacity and is worth comparing for primarily single-card work. For frequent GPU-to-GPU exchanges, measure the topology actually used by your application; the SXM name does not replace this check. Prefer the H200 if the identified problem is primarily a lack of memory per card.
Reserve a period that includes recovery
Choose 3 days to validate a loop and a checkpoint, 7 days for an ablation series, or 30 days for extended experiments. Include evaluation and export in that schedule, not just training epochs.
In the configurator, select the PyTorch or custom setup, your batches, and your duration. Create your account or log in before saving the order and choosing crypto payment. The "I've paid" button is used to report the transfer; payment tracking remains visible in your account.