Dao-AILab/flash-attention

Error Installing FlashAttention on Windows 11 with CUDA 11.8 - "CUDA_HOME environment variable is not set"

Mr-Natural opened this issue · 6 comments

Mr-Natural commented

I am encountering an error while attempting to install the flash-attn library on my Windows 11 machine with CUDA 11.8. Despite having the nvcc compiler and CUDA properly installed and accessible, the installation fails due to a supposed missing CUDA_HOME environment variable.

Setup Information:
Operating System: Windows 11 (version 10.0.22631.3593)
CUDA Version: 11.8
PyTorch Version: 2.3.0+cpu
NVIDIA Driver Version: 522.06

Steps to Reproduce:

  1. Set CUDA_HOME environment variable:
    setx CUDA_HOME "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8"
    setx PATH "%CUDA_HOME%\bin;%PATH%"
    setx PATH "%CUDA_HOME%\lib64;%PATH%"

  2. Verify CUDA_HOME and nvcc
    echo %CUDA_HOME%
    nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:41:10_Pacific_Daylight_Time_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

  1. Create and activate a new Conda environment:
    conda create -n phivision_clean python=3.9
    conda activate phivision_clean

  2. Install prerequisites:
    pip install packaging ninja
    pip cache purge
    set MAX_JOBS=4

  3. Attempt to install FlashAttention
    pip install flash-attn --no-build-isolation

Error Log:
Collecting flash-attn
Downloading flash_attn-2.5.9.post1.tar.gz (2.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 5.7 MB/s eta 0:00:00
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [22 lines of output]
fatal: not a git repository (or any of the parent directories): .git
C:\Users\allan\AppData\Local\Temp\pip-install-9pn3ykxp\flash-attn_1cfbe659b3424b648b092c7a32f1614f\setup.py:78: UserWarning: flash_attn was requested, but nvcc was not found. Are you sure your environment has nvcc available? If you're installing within a container from https://hub.docker.com/r/pytorch/pytorch, only images whose names contain 'devel' will provide nvcc.
warnings.warn(
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "C:\Users\allan\AppData\Local\Temp\pip-install-9pn3ykxp\flash-attn_1cfbe659b3424b648b092c7a32f1614f\setup.py", line 134, in
CUDAExtension(
File "C:\Users\allan\miniconda3\envs\phivision\Lib\site-packages\torch\utils\cpp_extension.py", line 1077, in CUDAExtension
library_dirs += library_paths(cuda=True)
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\allan\miniconda3\envs\phivision\Lib\site-packages\torch\utils\cpp_extension.py", line 1211, in library_paths
paths.append(_join_cuda_home(lib_dir))
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\allan\miniconda3\envs\phivision\Lib\site-packages\torch\utils\cpp_extension.py", line 2419, in _join_cuda_home
raise OSError('CUDA_HOME environment variable is not set. '
OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.

  torch.__version__  = 2.3.0+cpu


  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Additional Information:

nvcc output:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:41:10_Pacific_Daylight_Time_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

nvidia-smi output:
Sat Jun 8 21:58:06 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 522.06 Driver Version: 522.06 CUDA Version: 11.8 |
|-------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... WDDM | 00000000:01:00.0 On | N/A |
| 31% 35C P8 13W / 175W | 1125MiB / 8192MiB | 2% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

Notes:
Despite setting CUDA_HOME and confirming nvcc is available, the installation script does not seem to recognize the CUDA_HOME variable.
The error might also be linked to residual files or incorrect environment setup from previous installation attempts.
Thank you for your assistance.

Tellterubouzu commented

This may be unrelated, but is your torch the cpu version instead of the cuda?

Mr-Natural commented

Thanks for the question. The version I am using is GPU enabled one. Details: torch.version = 2.3.1+cu118

  • 👍1
bdashore3 commented

FA2 does not support cuda 11.8 on windows. You'd need to update to cuda 12 and install a prebuilt wheel from my fork https://github.com/bdashore3/flash-attention/releases

  • 👍1
Mr-Natural commented

Thank you for taking the time to respond. I'll follow your guidance and all going to plan things will be operational soon!

  • 👍1