Test the extensions, not just PyTorch
An environment that worked on an RTX 4090 should not be copied over without verification. The RTX 5090 belongs to another CUDA architecture. A precompiled extension may require an adapted version or a recompilation; simply importing it does not test the kernels it will run.
Prepare a short test for each important component: attention, quantization or a custom operator. Note the installed package and the operation actually executed, then freeze that set in the project folder.
Use 32 GB without counting only the weights
This capacity may offer useful headroom for a project constrained by 24 GB. To find out what it changes, replay your largest inputs at the same precision, then try a larger batch. Also keep a measurement after several requests to spot objects unintentionally retained in memory.
If you are exploring quantization, compare quality with a fixed validation set. Reducing memory usage alone does not explain whether the outputs remain suitable for your application.
Choose Blackwell for a measurable reason
The RTX 4090 remains an alternative if your Ada stack is stable and 24 GB is enough. The RTX 6000 Ada moves up to 48 GB when capacity is the first obstacle. The B200 batch addresses a different project: two 180 GB cards with an explicit multi-card strategy, not an automatic extension of the single-card program.
From qualification to campaign
Take 3 days to validate the dependencies and a complete scenario, 7 days to compare your configurations, or 30 days for a series of jobs after qualification. Distinguish initial compilation time from recurring processing in your reports.
Configure the duration, the batches and the desired preparation, then your contact details. Crypto payment does not require KYC identity documents. After the transfer on the selected network, "I've paid" reports your payment; tracking remains attached to the order.