Swap CUDA Toolkit Versions on Windows

Here I will do a quick run down on how to swap CUDA versions.

For ease, I will be demonstrating switching from CUDA 11.6 to CUDA 11.3, the same methods apply to other versions.

Step 0: Check CUDA Version

Check what version of CUDA you have. You can enter this to any command prompt (cmd, anaconda, etc)

nvcc --version

if you get something like this:
```
'nvcc' is not recognized as an internal or external command,
 operable program or batch file.
```
This means you don't have any CUDA installed. You can download your desired CUDA Toolkit version here (everything default would be fine)

A quick rule of thumb:
- NVIDIA GPU >= 30 series --> CUDA 11.0+
- NVIDIA GPU < 30 series --> CUDA 10.2 (CUDA 10.0 & 10.1 kinda outdated, use 10.2 unless specified)
You can also check your GPU compatibility here for NVIDIA GPU < 30 series. If your GPU has CC >= 3.7, then it supports PyTorch.

If you just freshly downloaded CUDA, then you would not need to proceed in the following steps, because you would have the CUDA version you want. You can do another quick check with nvcc --version to check your version in any command prompt.
if you get something like this:

Then it means you have CUDA installed. And in my case, it's CUDA 11.6. I will be swapping to CUDA 11.3 in the following steps.

Open up your environment variables. You can search "env" in the search tab, it should look something like this.

Then open it. Then click "Environment Variables"

Then it should open up a winodw like this

Double check on CUDA_PATH and this window should pop up

Then enter the target version of your CUDA there. In my case it's changing 11.6 to 11.3

Press ok and proceed next step.

Scroll down and find Path, double click to open

You should see your current version on the very top. You going have to move your desired version to the very top

So it should look like this after moving

Press ok and you may now close all the windows for environment variables & system properties.

Close the last command prompt, and open a new one. Enter the following command:

nvcc --version

If it outputs your desired version, then you have succeed in swapping CUDA version.