PyTorch does not see the GPU: diagnose in the right order

The diagnostic path in four decisions

The goal is to find the first layer that fails, not to try several installations in a row. Keep the command you ran, the first error message and the result of each check. If you change Python, the PyTorch package and the batch size all at once, you will no longer know which change solved the problem.

The downloadable script applies this progression and produces a limited technical report. It does not run your model and does not modify your installation. Use it in the same environment as your project, otherwise you will be checking a different interpreter than the one used by the failing program.

Scroll the table to read all columns.

Stop at the first step that fails.
Check	If the check fails	What a successful check lets you do
1. Interpreter and import	Fix the Python being used or its PyTorch installation.	Read the version and backend of the package actually imported.
2. Backend and device	Examine the package, the driver, GPU exposure and permissions.	Request an allocation on the target GPU.
3. Small GPU computation	Keep the allocation, computation or synchronization error.	Move on to a reduced input of the application.
4. Representative application	Isolate weights, extension, format, memory or incorrect output.	Gradually increase the real workload.

1. Identify the Python actually being run

A terminal, a notebook and a service may use different interpreters. Print sys.executable in the context that launches the project, then check the version. The path helps spot a forgotten virtual environment or a notebook still on another kernel. Examine it on your machine; there is no need to publish your personal directory tree in a report.

Then use that same interpreter to query the packages. The command python -m pip show torch gives the PyTorch information associated with that Python. If import torch fails, the next step is to fix that installation: reducing the batch size or changing the model weights will not solve a missing module.

To run in the same environment as the project

python -c "import sys; print(sys.executable); print(sys.version)"
python -m pip show torch

2. Distinguish CUDA, ROCm and a package without GPU acceleration

Record torch.__version__, torch.version.cuda and torch.version.hip separately. Do not conclude "CPU package" from the None value of torch.version.cuda alone: PyTorch for ROCm uses HIP, reuses torch.cuda and also expects a device named cuda. Replacing that name with rocm or hip is not the fix to apply.

Then check torch.cuda.is_available() and torch.cuda.device_count(). These results describe what this Python environment can use at that moment. They do not replace the minimal computation. A system tool may see a card while the package, the driver accessible to the process, or its environment prevents PyTorch from using it.

Reading the indicators without loading a model

python -c "import torch; print(torch.__version__); print(torch.version.cuda); print(torch.version.hip); print(torch.cuda.is_available()); print(torch.cuda.device_count())"

3. Generate the report with the Kernodeck script

After downloading the file, place it in a working folder and run it with the project's Python. By default, it requires a GPU. CPU mode must be requested explicitly: its success verifies the CPU branch of the diagnostic and never turns an unavailable GPU into a validated GPU. The report is written to the terminal and, with --output, to a new JSON file. An existing file is never overwritten: choose another name for your next attempt.

The script allocates two 2 × 2 matrices in float32, checks their product, then a gradient, and synchronizes the GPU device. The expected loss is 196 for this fixed computation. This very short check loads no model weights and measures no throughput. It asks the backend for a small real computation, beyond a simple device detection.

The optional system check uses nvidia-smi when present. It reports only the NVIDIA driver version and the total memory visible to that tool; it is not an equivalent system check for ROCm. The computation timeout is 30 seconds by default and can range from 5 to 120 seconds. The system check has its own maximum timeout of 3 seconds.

GPU check and report saving

python kernodeck-diagnostic-v1.py --device-index 0 --timeout 30 --output diagnostic-gpu.json

CPU check deliberately kept separate

python kernodeck-diagnostic-v1.py --device cpu --output diagnostic-cpu.json

Add the optional NVIDIA system check

python kernodeck-diagnostic-v1.py --host-check --output diagnostic-gpu-systeme.json

4. Read the report and choose the next action

Start with status, code, exit_code, and stage. The runtime block identifies the Python version and the system family. The pytorch block distinguishes the imported package, its CUDA/HIP build versions, the declared backend, and the visible devices. The execution block indicates where the computation actually took place and whether the product and the gradient were verified.

In CPU mode, gpu_available and visible_device_count remain null: the script does not request the GPU driver status. This is neither a zero nor a failure. Also read execution.device: a package compiled for CUDA can very well run this check on CPU when explicitly requested.

The report contains a selection of technical data. It does not include environment variables, machine paths, session identifiers, a full package list, or the raw trace of an exception. The script does not send any report to Kernodeck. For a detailed error from your application, keep its trace in your workspace and remove secrets before sharing it.

Scroll the table to read all columns.

Useful codes for deciding what to do next; the full list accompanies the script.
Result	Meaning	Next action
GPU_CHECK_PASSED · 0	Product and gradient verified on the chosen GPU.	Move on to a small input from your application.
CPU_CHECK_PASSED · 0	Product and gradient verified on CPU only.	Do not draw conclusions about CUDA or ROCm.
TORCH_MISSING · 3 / TORCH_IMPORT_FAILED · 4	PyTorch missing from this Python, or import failing.	Check the interpreter, the package, and its dependencies.
GPU_BACKEND_ABSENT · 5	The package declares neither CUDA nor HIP.	Install the package suited to your environment.
GPU_UNAVAILABLE · 6 / DEVICE_INDEX_INVALID · 7	GPU unusable in this process, or index outside the visible devices.	Check the exposure of the cards, the driver, and the requested index.
CHECK_FAILED · 8 / OUT_OF_MEMORY or RUNTIME_ERROR · 9	Failure of the fixed computation, of the allocation, or of a backend operation.	Read the flagged stage before launching the full model.
TIMEOUT · 10 / WORKER_FAILED · 11	Check stopped by the timeout, or with no usable report.	Treat the check as a failure; examine the environment.
OUTPUT_WRITE_FAILED · 12	The report was not saved to the requested destination.	Use a new, accessible file name.

5. From the small computation to your application

Before launching, have a reproducible command, an identified model, a small dataset and an accessible output directory. Choose an input that preserves the important characteristics of the final job: text length, image dimensions, audio format or required fields. An artificially short input can hide the problem you are trying to observe.

Write a concrete success criterion. For an embedding computation, each input identifier must return a vector of the expected dimension, with finite values. For training, a step must produce a usable loss, update the intended parameters and allow a checkpoint. The process exit code complements these checks; it does not replace them.

Add markers before and after reading parameters, importing libraries, loading weights, preparing data, transferring it, computing and writing. Give each run an identifier and keep the associated parameters. A "model loaded" message must correspond to a completed event, not merely an intention to load.

Log the shapes, types and devices of the relevant tensors without copying the entire dataset. A summary such as "input: 8 sequences, max length 512, device cuda:0" helps compare two runs. These numbers describe an example log here, not a universal configuration. Avoid placing access tokens or sensitive input content in these messages.

6. Fix the error at the right layer

If the small computation passes but the weights cannot be found, check their path, format and access permissions. If an extension fails to import, verify its compatibility with the PyTorch package and the project's backend. A successful diagnostic does not qualify every extension in the application. Resume from the first step that fails instead of changing several dependencies at once.

A device error can come from an input left on the CPU while the model is on the GPU. A type error can come from a partial conversion or an operator incompatible with the chosen precision. Keep the first complete message and its trace. Change one assumption at a time, then rerun the minimal input before reintroducing the final volume.

7. If the model starts and then exceeds memory

Identify whether the overflow occurs during weight loading, the first computation or after several iterations. These moments point to different causes: model too large, large activations or generation cache, accumulation of retained tensors. Record torch.cuda.memory_allocated() and torch.cuda.memory_reserved() at the same steps. The first tracks tensor allocations; the second covers memory managed by the allocator.

torch.cuda.empty_cache() can release unused cache, but it does not remove tensors that are still referenced. So inspect output lists, loss histories and objects holding a computation graph. Then reduce the batch or input length to isolate the determining factor. Switching cards becomes an informed decision once you know the phase that overflows and the margin actually needed.

8. Measure computation without forgetting asynchrony

GPU operations can be asynchronous relative to the Python program. A timer placed around a call may therefore mainly measure the dispatch of the work. For a diagnostic measurement, synchronize the GPU at the boundaries of the observed segment, or use suitable events. This synchronization changes the execution flow: keep this instrumentation separate from the normal operation of your application.

Build a simple example with three segments: input preparation, computation, output writing. For the GPU segment, call torch.cuda.synchronize(), capture time.perf_counter(), run the computation, synchronize again, then compute the difference. Keep the first pass and the following ones separate. A load or an initialization must not disappear into an average presented as the full response time.

PyTorch: synchronization and GPU computation timing

9. Check outputs and keep a reusable diagnostic

For standard inference, model.eval() sets the behavior of the relevant modules, while torch.inference_mode() disables the tracking needed for gradients. These two settings serve different purposes. Use the second when the tensors produced must not later participate in a computation with gradients. Evaluating a model during training requires explicitly restoring the right mode before resuming.

Now compare the outputs against the prepared contract: number of results, matching identifiers, dimensions, finite values and the appropriate business metric. If you increase the batch, verify this correspondence again. If you add GPUs, check the distribution of inputs and the collection of outputs. Rental batches refer to cards ordered; batch refers to examples processed together by your program.

The result of this method is a small folder: command, versions, parameters, minimal input, last successful step, first error, memory observations and the output obtained. If the run works, keep this folder as a point of comparison before increasing the load. If the run fails, it lets you reproduce the problem without starting the whole investigation over.

Before a long run, also perform a clean stop and resume on this small set of inputs. Verify that outputs already written are neither lost nor counted twice. Once these checks pass, gradually increase a single axis — batch, length, concurrency or number of processes — and record the limit observed. You get a measured operating range for your application, rather than a guess based on the GPU name.

The evidence provided and its limits

The downloadable examples come from real checks performed on September 24, 2026. Both runs with PyTorch use Windows, Python 3.14.6 and PyTorch 2.11.0+cu128. The GPU check uses CUDA, on an NVIDIA GeForce RTX 5070; the CPU check explicitly requests the CPU. This test hardware is not presented as a Kernodeck offering. No ROCm computation was run for this evidence.

A successful small computation shows that an allocation and computation path works on the chosen device. It measures neither the speed of your model, nor the memory required by its largest inputs, nor its compatibility with a particular extension. The report also does not certify a multi-card topology. Move on to the representative test before deciding to increase the load or the rental.

For a CUDA application, compare an NVIDIA spec sheet against your memory and library requirements; for a ROCm stack, review the conditions for the MI300X. The linked spec sheets are options to qualify for your project, not the list of hardware used in the evidence. Include the initial check and export time in your 3, 7 or 30-day period.

Scroll the table to read all columns.

Observed results of script v1.0.0; no performance figures.
Real check	Observed result	Scope
Explicit CPU · Python 3.14.6 / PyTorch 2.11.0+cu128	CPU_CHECK_PASSED; product and gradient exact; loss 196.	The fixed computation works on CPU.
CUDA · RTX 5070 / CUDA package 12.8	GPU_CHECK_PASSED; product and gradient exact; loss 196.	The fixed computation works on this card in this environment.
PyTorch missing · Python 3.12.14	TORCH_MISSING; exit code 3.	The missing module produces an explicit failure.
GPU made invisible to the check process	GPU_UNAVAILABLE; exit code 6.	The script does not silently substitute the CPU for the GPU.