A little confused.
TeaCult opened this issue · 3 comments
TeaCult commented
I am usually using containers and pyenv and no conda environment.
It claims there are mismatch. Is runtime libraries enough, should I remove cuda from base install ?
How can I remedy this ? Thank you.
(vicuna-matata) [gediz@ggVicuna GPTQ-for-LLaMa]$ conda list |grep -E 'cuda|nvidia'
cuda-cccl 11.7.58 hc415cf5_0 nvidia/label/cuda-11.7.0
cuda-command-line-tools 11.7.0 0 nvidia/label/cuda-11.7.0
cuda-compiler 11.7.0 0 nvidia/label/cuda-11.7.0
cuda-cudart 11.7.60 h9538e0e_0 nvidia/label/cuda-11.7.0
cuda-cudart-dev 11.7.60 h6a7c232_0 nvidia/label/cuda-11.7.0
cuda-cuobjdump 11.7.50 h28cc80a_0 nvidia/label/cuda-11.7.0
cuda-cupti 11.7.50 hb6f9eaf_0 nvidia/label/cuda-11.7.0
cuda-cuxxfilt 11.7.50 hb365495_0 nvidia/label/cuda-11.7.0
cuda-documentation 11.7.50 0 nvidia/label/cuda-11.7.0
cuda-driver-dev 11.7.60 0 nvidia/label/cuda-11.7.0
cuda-gdb 11.7.50 h4a0ac72_0 nvidia/label/cuda-11.7.0
cuda-libraries 11.7.0 0 nvidia/label/cuda-11.7.0
cuda-libraries-dev 11.7.0 0 nvidia/label/cuda-11.7.0
cuda-memcheck 11.7.50 hc446b2b_0 nvidia/label/cuda-11.7.0
cuda-nsight 11.7.50 0 nvidia/label/cuda-11.7.0
cuda-nsight-compute 11.7.0 0 nvidia/label/cuda-11.7.0
cuda-nvcc 11.7.64 0 nvidia/label/cuda-11.7.0
cuda-nvdisasm 11.7.50 h5bd0695_0 nvidia/label/cuda-11.7.0
cuda-nvml-dev 11.7.50 h3af1343_0 nvidia/label/cuda-11.7.0
cuda-nvprof 11.7.50 h7a2404d_0 nvidia/label/cuda-11.7.0
cuda-nvprune 11.7.50 h7add7b4_0 nvidia/label/cuda-11.7.0
cuda-nvrtc 11.7.50 hd0285e0_0 nvidia/label/cuda-11.7.0
cuda-nvrtc-dev 11.7.50 heada363_0 nvidia/label/cuda-11.7.0
cuda-nvtx 11.7.50 h05b0816_0 nvidia/label/cuda-11.7.0
cuda-nvvp 11.7.50 hd2289d5_0 nvidia/label/cuda-11.7.0
cuda-runtime 11.7.0 0 nvidia/label/cuda-11.7.0
cuda-sanitizer-api 11.7.50 hb424887_0 nvidia/label/cuda-11.7.0
cuda-toolkit 11.7.0 0 nvidia/label/cuda-11.7.0
cuda-tools 11.7.0 0 nvidia/label/cuda-11.7.0
cuda-visual-tools 11.7.0 0 nvidia/label/cuda-11.7.0
gds-tools 1.3.0.44 0 nvidia/label/cuda-11.7.0
libcublas 11.10.1.25 he442b6f_0 nvidia/label/cuda-11.7.0
libcublas-dev 11.10.1.25 h0c8ac2b_0 nvidia/label/cuda-11.7.0
libcufft 10.7.2.50 h80a1efe_0 nvidia/label/cuda-11.7.0
libcufft-dev 10.7.2.50 h59a5ac8_0 nvidia/label/cuda-11.7.0
libcufile 1.3.0.44 0 nvidia/label/cuda-11.7.0
libcufile-dev 1.3.0.44 0 nvidia/label/cuda-11.7.0
libcurand 10.2.10.50 heec50f7_0 nvidia/label/cuda-11.7.0
libcurand-dev 10.2.10.50 hd49a9cd_0 nvidia/label/cuda-11.7.0
libcusolver 11.3.5.50 hcab339c_0 nvidia/label/cuda-11.7.0
libcusolver-dev 11.3.5.50 hc6eba6f_0 nvidia/label/cuda-11.7.0
libcusparse 11.7.3.50 h6aaafad_0 nvidia/label/cuda-11.7.0
libcusparse-dev 11.7.3.50 hc644b96_0 nvidia/label/cuda-11.7.0
libnpp 11.7.3.21 h3effbd9_0 nvidia/label/cuda-11.7.0
libnpp-dev 11.7.3.21 hb6476a9_0 nvidia/label/cuda-11.7.0
libnvjpeg 11.7.2.34 hfe236c7_0 nvidia/label/cuda-11.7.0
libnvjpeg-dev 11.7.2.34 h2e48410_0 nvidia/label/cuda-11.7.0
nsight-compute 2022.2.0.13 0 nvidia/label/cuda-11.7.0
pytorch 2.0.1 py3.11_cuda11.7_cudnn8.5.0_0 pytorch
pytorch-cuda 11.7 h778d358_5 pytorch
pytorch-mutex 1.0 cuda pytorch
(vicuna-matata) [gediz@ggVicuna GPTQ-for-LLaMa]$ sudo pacman -Q |grep -E "nvidia|cuda"
cuda 12.1.1-3
nvidia 530.41.03-15
nvidia-utils 530.41.03-1
opencl-nvidia 530.41.03-1
(vicuna-matata) [gediz@ggVicuna GPTQ-for-LLaMa]$ uname -a
Linux ggVicuna 6.3.5-arch1-1 #1 SMP PREEMPT_DYNAMIC Tue, 30 May 2023 13:44:01 +0000 x86_64 GNU/Linux
(vicuna-matata) [gediz@ggVicuna GPTQ-for-LLaMa]$ whereis nvcc
nvcc: /opt/cuda/bin/nvcc /home/gediz/miniconda3/envs/vicuna-matata/bin/nvcc
(vicuna-matata) [gediz@ggVicuna GPTQ-for-LLaMa]$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_May__3_18:49:52_PDT_2022
Cuda compilation tools, release 11.7, V11.7.64
Build cuda_11.7.r11.7/compiler.31294372_0
(vicuna-matata) [gediz@ggVicuna GPTQ-for-LLaMa]$ nvidia-smi
Sun Jun 4 02:08:02 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 530.41.03 Driver Version: 530.41.03 CUDA Version: 12.1 |
|-----------------------------------------+----------------------+----------------------+
uname -a
Error:
(vicuna-matata) [gediz@ggVicuna GPTQ-for-LLaMa]$ python setup_cuda.py install
running install
/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` directly.
Instead, use pypa/build, pypa/installer, pypa/build or
other standards-based tools.
See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
********************************************************************************
!!
self.initialize_options()
/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` and ``easy_install``.
Instead, use pypa/build, pypa/installer, pypa/build or
other standards-based tools.
See https://github.com/pypa/setuptools/issues/917 for details.
********************************************************************************
!!
self.initialize_options()
running bdist_egg
running egg_info
writing quant_cuda.egg-info/PKG-INFO
writing dependency_links to quant_cuda.egg-info/dependency_links.txt
writing top-level names to quant_cuda.egg-info/top_level.txt
/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
warnings.warn(msg.format('we could not find ninja.'))
reading manifest file 'quant_cuda.egg-info/SOURCES.txt'
writing manifest file 'quant_cuda.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
Traceback (most recent call last):
File "/home/gediz/text-generation-webui/repositories/GPTQ-for-LLaMa/setup_cuda.py", line 4, in <module>
setup(
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/command/install.py", line 80, in run
self.do_egg_install()
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/command/install.py", line 129, in do_egg_install
self.run_command('bdist_egg')
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/command/bdist_egg.py", line 164, in run
cmd = self.call_command('install_lib', warn_dir=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
self.run_command(cmdname)
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/command/install_lib.py", line 111, in build
self.run_command('build_ext')
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 84, in run
_build_ext.run(self)
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
self.build_extensions()
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 499, in build_extensions
_check_cuda_version(compiler_name, compiler_version)
File "/home/gediz/miniconda3/envs/vicuna-matata/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 387, in _check_cuda_version
raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
RuntimeError:
The detected CUDA version (12.1) mismatches the version that was used to compile
PyTorch (11.7). Please make sure to use the same CUDA versions.
fredi-python commented
You're not using llama.cpp, right?
TeaCult commented
I am following the guide. Do I have to externally use llama.cpp somewhere.
I expected that gradio based text-... ui to work out of the box.
…On Sun, Jun 4, 2023, 1:12 PM fredithefish ***@***.***> wrote:
You're not using llama.cpp, right?
—
Reply to this email directly, view it on GitHub
<#9 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEB5YX2XBWCE5KHTQNE4FCLXJRNPPANCNFSM6AAAAAAYZPKXNU>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
fredi-python commented
I don't know how up to date the code of the text generation webui is, but I think in this case it's more of an issue for the textgenwebui then for this guide