Define the deliverable that justifies your rental
For file processing, define the volume to process, the output format, and the resume rule. A finished file must be recognizable without re-reading the entire run. For an interactive service, define the maximum request size, the acceptable latency, and the behavior when capacity is reached. The same model can require two very different organizations depending on these constraints.
Keep a representative sample with short, typical cases and cases close to your limits. In an embeddings pipeline, associate each vector with the identifier and version of its input. In text generation, record the generation settings used for evaluation. You must be able to explain a difference in results without immediately attributing it to the GPU.
Choose the memory for the entire inference workload
The size of the model's files does not describe all the memory used during inference. You must also account for temporary tensors, inputs, retained outputs, and, for the models concerned, the attention key-value cache. This cache can become significant when sequences grow longer or when multiple requests are processed together.
Start with a card whose memory allows your representative trial with a measured margin. Models of 24, 32, 48, 80 GB and beyond meet different needs; no capacity guarantees that a given model will fit with all its settings. Reducing precision or quantizing can change the footprint, but requires verifying software support and output quality on your own inputs.
Pick the GPU compatible with your software stack
List the inference engine, its specific operators, and the extensions you depend on before choosing the hardware. A PyTorch application can offer several execution paths, whereas a specialized extension supports only one. Verify the full chain on CUDA for NVIDIA or on ROCm for AMD, including model loading and its preprocessing.
Keep a minimal command that runs the pipeline all the way through to writing a result. Only then enable your optimizations one at a time. Each change of precision, compilation, or engine must preserve a comparable quality check. A successful load demonstrates that the weights are readable; it does not demonstrate that all the necessary computation paths work.
Set the batch according to your processing goal
Example method: build three groups of texts by length, then process each with a batch of 1, 2, and 4 inputs. These sizes serve as trial points, not a universal recommendation. For each combination, note the completed inputs, the total time, the maximum observed memory, and the errors. Stop the progression when a limit appears instead of hiding failures in an average.
For an interactive service, add the time spent in the queue. A larger batch can change the throughput and the latency felt by a request; a single average is not enough to choose. For offline processing, make sure that grouping does not mix up the order of the results. Ultimately choose a configuration that respects your quality criterion and your latency constraint.
Move to multiple GPUs if your application can distribute the work
If each instance of the model fits on one card, you can organize several workers that consume distinct partitions of the inputs. You must then coordinate identifiers, resumes, and output collection. If the model must be split across cards, use a parallelism strategy supported by your engine and verify its communication requirements.
Ordered lots describe the quantity of hardware, not the application batch or a merged memory space. For B200, one lot includes two cards; for the other offerings, one lot includes one card. Document in your file the number of workers planned, the share of inputs assigned to each, and how you will verify that a job is actually finished.
Choose a period that includes the checks and the export
For a first 3-day rental, set a limited goal: install, validate the pipeline, and produce a first usable result. A 7-day duration can be used to explore more variants; 30 days to repeat a process and consolidate its operation. These are ways to organize the work, not promises of execution time.
On exit, keep the weights or their version, the configuration, the quality checks, the measurements actually obtained, and the exported results. You choose your software and processes autonomously; Kernodeck does not inspect their content. Prepare your own access, backups, and the authorizations required to use the models and data.