In R, tensorflow does not find TensorRT and does not see the GPU; in Python, it does!

Question

In R, tensorflow does not find TensorRT and does not see the GPU; in Python, it does!

Closed this issue a year ago · 16 comments

First, I want to thank the RStudio/Posit people for all the wonderful packages.

After a lot of work, I managed to set up tensorflow 2.13.0 on my Acer Nitro 5 (Ubuntu 22.04) with GPU support (NVidia GP107M [GeForce GTX 1050 Ti Mobile]).

When I run the following from the bash shell (with my conda "r-reticulate" environment activated)

conda activate r-reticulate
python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

tensorflow detects my GPU without problem:

2023-08-24 14:22:14.775474: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-24 14:22:16.309070: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-08-24 14:22:16.335162: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-08-24 14:22:16.335752: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

Now, when I go to RStudio and run the following on the console (with the same conda "r-reticulate" environment activated)

library(tensorflow)
tf$config$list_physical_devices("GPU")

then tensorflow cannot find the TensorRT library and does not detect the GPU:

2023-08-24 14:41:12.929659: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-24 14:41:13.491739: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2023-08-24 14:41:14.683245: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-08-24 14:41:14.711384: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1960] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
list()

I have no idea why the GPU is detected from Python, but not from R, when I am using the same conda environment.

I tried running it from the Python REPL in RStudio, but still no GPU detected:

> reticulate::repl_python()
Python 3.10.4 (/ssd/miniconda/envs/r-reticulate/bin/python)
Reticulate 1.31.0.9000 REPL -- A Python interpreter in R.
Enter 'exit' or 'quit' to exit the REPL and return to R.
>>> import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))
[]

As additional info, here is the output from install_tensorflow(method="conda", envname = 'r-reticulate', pip_ignore_installed = TRUE):

+ . /ssd/miniconda/bin/activate
+ conda activate 'r-reticulate'
+ '/ssd/miniconda/envs/r-reticulate/bin/python' -m pip install --upgrade --no-user --ignore-installed 'tensorflow==2.13.*' 'nvidia-cudnn-cu11==8.6.*'
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting tensorflow==2.13.*
  Obtaining dependency information for tensorflow==2.13.* from https://files.pythonhosted.org/packages/5a/f2/5c2f878c62c8b79c629b11b33516bb55054d7677eba6f56f3a20296b56bd/tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.4 kB)
Collecting nvidia-cudnn-cu11==8.6.*
  Downloading nvidia_cudnn_cu11-8.6.0.163-py3-none-manylinux1_x86_64.whl (715.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 715.7/715.7 MB 5.1 MB/s eta 0:00:00
Collecting absl-py>=1.0.0 (from tensorflow==2.13.*)
  Downloading absl_py-1.4.0-py3-none-any.whl (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.5/126.5 kB 4.1 MB/s eta 0:00:00
Collecting astunparse>=1.6.0 (from tensorflow==2.13.*)
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting flatbuffers>=23.1.21 (from tensorflow==2.13.*)
  Obtaining dependency information for flatbuffers>=23.1.21 from https://files.pythonhosted.org/packages/6f/12/d5c79ee252793ffe845d58a913197bfa02ae9a0b5c9bc3dc4b58d477b9e7/flatbuffers-23.5.26-py2.py3-none-any.whl.metadata
  Downloading flatbuffers-23.5.26-py2.py3-none-any.whl.metadata (850 bytes)
Collecting gast<=0.4.0,>=0.2.1 (from tensorflow==2.13.*)
  Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting google-pasta>=0.1.1 (from tensorflow==2.13.*)
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.5/57.5 kB 2.4 MB/s eta 0:00:00
Collecting grpcio<2.0,>=1.24.3 (from tensorflow==2.13.*)
  Obtaining dependency information for grpcio<2.0,>=1.24.3 from https://files.pythonhosted.org/packages/28/fa/c38a010d3fffcac07ef121abb34eb2c3db0876df74267ce5bde13c3a6ed7/grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
Collecting h5py>=2.9.0 (from tensorflow==2.13.*)
  Obtaining dependency information for h5py>=2.9.0 from https://files.pythonhosted.org/packages/0d/7a/e55589e4093cca1934db5e99644c1c2424a9b3aac104b7f6176605a5eeb7/h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB)
Collecting keras<2.14,>=2.13.1 (from tensorflow==2.13.*)
  Obtaining dependency information for keras<2.14,>=2.13.1 from https://files.pythonhosted.org/packages/2e/f3/19da7511b45e80216cbbd9467137b2d28919c58ba1ccb971435cb631e470/keras-2.13.1-py3-none-any.whl.metadata
  Downloading keras-2.13.1-py3-none-any.whl.metadata (2.4 kB)
Collecting libclang>=13.0.0 (from tensorflow==2.13.*)
  Obtaining dependency information for libclang>=13.0.0 from https://files.pythonhosted.org/packages/ea/df/55525e489c43f9dbb6c8ea27d8a567b3dcd18a22f3c45483055f5ca6611d/libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl.metadata
  Downloading libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl.metadata (5.2 kB)
Collecting numpy<=1.24.3,>=1.22 (from tensorflow==2.13.*)
  Downloading numpy-1.24.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.3/17.3 MB 6.8 MB/s eta 0:00:00
Collecting opt-einsum>=2.3.2 (from tensorflow==2.13.*)
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 kB 3.0 MB/s eta 0:00:00
Collecting packaging (from tensorflow==2.13.*)
  Downloading packaging-23.1-py3-none-any.whl (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 2.0 MB/s eta 0:00:00
Collecting protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 (from tensorflow==2.13.*)
  Obtaining dependency information for protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 from https://files.pythonhosted.org/packages/4c/87/59648989ad7f5ba6fe3c7f8abc555183f28559b6f6cd14ad17a3f0d3094f/protobuf-4.24.1-cp37-abi3-manylinux2014_x86_64.whl.metadata
  Downloading protobuf-4.24.1-cp37-abi3-manylinux2014_x86_64.whl.metadata (540 bytes)
Collecting setuptools (from tensorflow==2.13.*)
  Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/4f/ab/0bcfebdfc3bfa8554b2b2c97a555569c4c1ebc74ea288741ea8326c51906/setuptools-68.1.2-py3-none-any.whl.metadata
  Downloading setuptools-68.1.2-py3-none-any.whl.metadata (6.2 kB)
Collecting six>=1.12.0 (from tensorflow==2.13.*)
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting tensorboard<2.14,>=2.13 (from tensorflow==2.13.*)
  Downloading tensorboard-2.13.0-py3-none-any.whl (5.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 MB 7.0 MB/s eta 0:00:00
Collecting tensorflow-estimator<2.14,>=2.13.0 (from tensorflow==2.13.*)
  Obtaining dependency information for tensorflow-estimator<2.14,>=2.13.0 from https://files.pythonhosted.org/packages/72/5c/c318268d96791c6222ad7df1651bbd1b2409139afeb6f468c0f327177016/tensorflow_estimator-2.13.0-py2.py3-none-any.whl.metadata
  Downloading tensorflow_estimator-2.13.0-py2.py3-none-any.whl.metadata (1.3 kB)
Collecting termcolor>=1.1.0 (from tensorflow==2.13.*)
  Downloading termcolor-2.3.0-py3-none-any.whl (6.9 kB)
Collecting typing-extensions<4.6.0,>=3.6.6 (from tensorflow==2.13.*)
  Downloading typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Collecting wrapt>=1.11.0 (from tensorflow==2.13.*)
  Downloading wrapt-1.15.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.4/78.4 kB 4.1 MB/s eta 0:00:00
Collecting tensorflow-io-gcs-filesystem>=0.23.1 (from tensorflow==2.13.*)
  Obtaining dependency information for tensorflow-io-gcs-filesystem>=0.23.1 from https://files.pythonhosted.org/packages/6a/48/3cdab86db01701ea043de445f3af1c3e539835781e57d803d2a87f256475/tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata
  Downloading tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (14 kB)
Collecting nvidia-cublas-cu11 (from nvidia-cudnn-cu11==8.6.*)
  Downloading https://developer.download.nvidia.com/compute/redist/nvidia-cublas-cu11/nvidia-cublas-cu11-2022.4.8.tar.gz (16 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting wheel<1.0,>=0.23.0 (from astunparse>=1.6.0->tensorflow==2.13.*)
  Obtaining dependency information for wheel<1.0,>=0.23.0 from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl.metadata
  Downloading wheel-0.41.2-py3-none-any.whl.metadata (2.2 kB)
Collecting google-auth<3,>=1.6.3 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for google-auth<3,>=1.6.3 from https://files.pythonhosted.org/packages/9c/8d/bff87fc722553a5691d8514da5523c23547f3894189ba03b57592e37bdc2/google_auth-2.22.0-py2.py3-none-any.whl.metadata
  Downloading google_auth-2.22.0-py2.py3-none-any.whl.metadata (4.2 kB)
Collecting google-auth-oauthlib<1.1,>=0.5 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading google_auth_oauthlib-1.0.0-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for markdown>=2.6.8 from https://files.pythonhosted.org/packages/1a/b5/228c1cdcfe138f1a8e01ab1b54284c8b83735476cb22b6ba251656ed13ad/Markdown-3.4.4-py3-none-any.whl.metadata
  Downloading Markdown-3.4.4-py3-none-any.whl.metadata (6.9 kB)
Collecting requests<3,>=2.21.0 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for requests<3,>=2.21.0 from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
  Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting tensorboard-data-server<0.8.0,>=0.7.0 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for tensorboard-data-server<0.8.0,>=0.7.0 from https://files.pythonhosted.org/packages/02/52/fb9e51fba47951aabd7a6b25e41d73eae94208ccf62d886168096941a781/tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata
  Downloading tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (1.1 kB)
Collecting werkzeug>=1.0.1 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for werkzeug>=1.0.1 from https://files.pythonhosted.org/packages/9b/59/a7c32e3d8d0e546a206e0552a2c04444544f15c1da4a01df8938d20c6ffc/werkzeug-2.3.7-py3-none-any.whl.metadata
  Downloading werkzeug-2.3.7-py3-none-any.whl.metadata (4.1 kB)
Collecting nvidia-cublas-cu117 (from nvidia-cublas-cu11->nvidia-cudnn-cu11==8.6.*)
  Downloading https://developer.download.nvidia.com/compute/redist/nvidia-cublas-cu117/nvidia_cublas_cu117-11.10.1.25-py3-none-manylinux1_x86_64.whl (333.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 333.1/333.1 MB 1.1 MB/s eta 0:00:00
Collecting cachetools<6.0,>=2.0.0 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for cachetools<6.0,>=2.0.0 from https://files.pythonhosted.org/packages/a9/c9/c8a7710f2cedcb1db9224fdd4d8307c9e48cbddc46c18b515fefc0f1abbe/cachetools-5.3.1-py3-none-any.whl.metadata
  Downloading cachetools-5.3.1-py3-none-any.whl.metadata (5.2 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 kB 5.4 MB/s eta 0:00:00
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
  Downloading urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.4/48.4 kB 178.8 MB/s eta 0:00:00
Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting charset-normalizer<4,>=2 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/a4/65/057bf29660aae6ade0816457f8db4e749e5c0bfa2366eb5f67db9912fa4c/charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading idna-3.4-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 11.0 MB/s eta 0:00:00
Collecting certifi>=2017.4.17 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
  Downloading certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting MarkupSafe>=2.1.1 (from werkzeug>=1.0.1->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for MarkupSafe>=2.1.1 from https://files.pythonhosted.org/packages/12/b3/d9ed2c0971e1435b8a62354b18d3060b66c8cb1d368399ec0b9baa7c0ee5/MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting pyasn1<0.6.0,>=0.4.6 (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 83.9/83.9 kB 2.4 MB/s eta 0:00:00
Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 10.1 MB/s eta 0:00:00
Downloading tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (524.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 524.1/524.1 MB 13.6 MB/s eta 0:00:00
Downloading flatbuffers-23.5.26-py2.py3-none-any.whl (26 kB)
Downloading grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.3/5.3 MB 9.3 MB/s eta 0:00:00
Downloading h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.8/4.8 MB 9.6 MB/s eta 0:00:00
Downloading keras-2.13.1-py3-none-any.whl (1.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 8.9 MB/s eta 0:00:00
Downloading libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl (22.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 22.9/22.9 MB 11.9 MB/s eta 0:00:00
Downloading protobuf-4.24.1-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 311.4/311.4 kB 12.3 MB/s eta 0:00:00
Downloading setuptools-68.1.2-py3-none-any.whl (805 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 805.1/805.1 kB 11.6 MB/s eta 0:00:00
Downloading tensorflow_estimator-2.13.0-py2.py3-none-any.whl (440 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 440.8/440.8 kB 11.4 MB/s eta 0:00:00
Downloading tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 8.7 MB/s eta 0:00:00
Downloading google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.8/181.8 kB 9.9 MB/s eta 0:00:00
Downloading Markdown-3.4.4-py3-none-any.whl (94 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.2/94.2 kB 13.1 MB/s eta 0:00:00
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 5.3 MB/s eta 0:00:00
Downloading tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl (6.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 10.3 MB/s eta 0:00:00
Downloading werkzeug-2.3.7-py3-none-any.whl (242 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 242.2/242.2 kB 7.4 MB/s eta 0:00:00
Downloading wheel-0.41.2-py3-none-any.whl (64 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.8/64.8 kB 6.3 MB/s eta 0:00:00
Downloading cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Downloading certifi-2023.7.22-py3-none-any.whl (158 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 kB 14.2 MB/s eta 0:00:00
Downloading charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (201 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 201.8/201.8 kB 14.6 MB/s eta 0:00:00
Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Downloading urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.1/143.1 kB 13.2 MB/s eta 0:00:00
Building wheels for collected packages: nvidia-cublas-cu11
  Building wheel for nvidia-cublas-cu11 (setup.py): started
  Building wheel for nvidia-cublas-cu11 (setup.py): finished with status 'done'
  Created wheel for nvidia-cublas-cu11: filename=nvidia_cublas_cu11-2022.4.8-py3-none-any.whl size=15605 sha256=b2903bb173e823ac84b1815a94f97ed41204efc5a34e783a69c4ea4ba5873f58
  Stored in directory: /tmp/pip-ephem-wheel-cache-ed2h1zws/wheels/8f/ed/66/aa1caefa04698673b59e1799fface640a8f6e840e59b9c27e3
Successfully built nvidia-cublas-cu11
Installing collected packages: libclang, flatbuffers, wrapt, wheel, urllib3, typing-extensions, termcolor, tensorflow-io-gcs-filesystem, tensorflow-estimator, tensorboard-data-server, six, setuptools, pyasn1, protobuf, packaging, oauthlib, numpy, MarkupSafe, markdown, keras, idna, grpcio, gast, charset-normalizer, certifi, cachetools, absl-py, werkzeug, rsa, requests, pyasn1-modules, opt-einsum, nvidia-cublas-cu117, h5py, google-pasta, astunparse, requests-oauthlib, nvidia-cublas-cu11, google-auth, nvidia-cudnn-cu11, google-auth-oauthlib, tensorboard, tensorflow
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow-metadata 1.14.0 requires protobuf<4.21,>=3.20.3, but you have protobuf 4.24.1 which is incompatible.
Successfully installed MarkupSafe-2.1.1 absl-py-1.4.0 astunparse-1.6.3 cachetools-5.3.1 certifi-2023.7.22 charset-normalizer-2.0.4 flatbuffers-23.5.26 gast-0.4.0 google-auth-2.22.0 google-auth-oauthlib-1.0.0 google-pasta-0.2.0 grpcio-1.57.0 h5py-3.9.0 idna-3.4 keras-2.13.1 libclang-16.0.6 markdown-3.4.4 numpy-1.24.3 nvidia-cublas-cu11-11.11.3.6 nvidia-cublas-cu117-11.10.1.25 nvidia-cudnn-cu11-8.6.0.163 oauthlib-3.2.2 opt-einsum-3.3.0 packaging-21.3 protobuf-3.20.3 pyasn1-0.5.0 pyasn1-modules-0.3.0 requests-2.31.0 requests-oauthlib-1.3.1 rsa-4.9 setuptools-68.0.0 six-1.16.0 tensorboard-2.13.0 tensorboard-data-server-0.7.1 tensorflow-2.13.0 tensorflow-estimator-2.13.0 tensorflow-io-gcs-filesystem-0.33.0 termcolor-2.3.0 typing-extensions-4.5.0 urllib3-1.26.16 werkzeug-2.3.7 wheel-0.37.1 wrapt-1.15.0

Installation complete.

Thanks

Answer 1 · 2023-08-24T18:04:56.000Z

Can you please try with the virtual environment method?

tensorflow::install_tensorflow()
tensorflow::tf$config$list_physical_devices()

Answer 2 · 2023-08-24T21:24:32.000Z

Can you please try with the virtual environment method?
tensorflow::install_tensorflow()
tensorflow::tf$config$list_physical_devices()

I used tensorflow::install_tensorflow(method = 'virtualenv').

It still complains about not finding TensorRT, but now, instead of returning an empty list, it returns the CPU (not the NVidia GPU):

> tensorflow::tf$config$list_physical_devices()
2023-08-24 18:20:46.966749: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-24 18:20:55.640925: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2023-08-24 18:21:04.848258: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-08-24 18:21:05.175078: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1960] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
[[1]]
PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')

Edit:

Now I realize tensorflow::tf$config$list_physical_devices() asks for ALL devices, not only GPUs.

So the answer is, no progress. The GPU still goes undetected.

Answer 3 · 2023-08-24T23:31:53.000Z

Can you please post the full output from running tensorflow::install_tensorflow()? Also, sessionInfo().

Answer 4 · 2023-08-25T19:07:34.000Z

tensorflow::tf$config$list_physical_devices()

Here goes:

Before installation:

> sessionInfo()
R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.10.0 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: America/Sao_Paulo
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] R6_2.5.1               shrtcts_0.1.2          xfun_0.40              Matrix_1.6-0           lattice_0.21-8         reticulate_1.31.0.9000
 [7] magrittr_2.0.3         glue_1.6.2             stringr_1.5.0          knitr_1.43             png_0.1-8              lifecycle_1.0.3       
[13] xml2_1.3.5             cli_3.6.1              grid_4.3.1             vctrs_0.6.3            compiler_4.3.1         purrr_1.0.2           
[19] rstudioapi_0.15.0      tools_4.3.1            Rcpp_1.0.11            jsonlite_1.8.7         rlang_1.1.1            roxygen2_7.2.3        
[25] stringi_1.7.12        
>

Output of tensorflow::install_tensorflow():

> tensorflow::install_tensorflow(pip_ignore_installed = TRUE)
Virtual environment 'r-tensorflow' removed.
Using Python: /usr/bin/python3.10
Creating virtual environment 'r-tensorflow' ... 
+ /usr/bin/python3.10 -m virtualenv -p '/usr/bin/python3.10' /home/fnaufel/.virtualenvs/r-tensorflow
created virtual environment CPython3.10.12.final.0-64 in 2139ms
  creator CPython3Posix(dest=/home/fnaufel/.virtualenvs/r-tensorflow, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/fnaufel/.local/share/virtualenv)
    added seed packages: pip==22.0.2, setuptools==59.6.0, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
Done!
Installing packages: pip, wheel, setuptools
+ /home/fnaufel/.virtualenvs/r-tensorflow/bin/python -m pip install --upgrade --no-user pip wheel setuptools
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in /home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages (22.0.2)
Collecting pip
  Downloading pip-23.2.1-py3-none-any.whl (2.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 8.5 MB/s eta 0:00:00
Requirement already satisfied: wheel in /home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages (0.37.1)
Collecting wheel
  Downloading wheel-0.41.2-py3-none-any.whl (64 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.8/64.8 KB 3.2 MB/s eta 0:00:00
Requirement already satisfied: setuptools in /home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages (59.6.0)
Collecting setuptools
  Downloading setuptools-68.1.2-py3-none-any.whl (805 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 805.1/805.1 KB 11.2 MB/s eta 0:00:00
Installing collected packages: wheel, setuptools, pip
  Attempting uninstall: wheel
    Found existing installation: wheel 0.37.1
    Uninstalling wheel-0.37.1:
      Successfully uninstalled wheel-0.37.1
  Attempting uninstall: setuptools
    Found existing installation: setuptools 59.6.0
    Uninstalling setuptools-59.6.0:
      Successfully uninstalled setuptools-59.6.0
  Attempting uninstall: pip
    Found existing installation: pip 22.0.2
    Uninstalling pip-22.0.2:
      Successfully uninstalled pip-22.0.2
Successfully installed pip-23.2.1 setuptools-68.1.2 wheel-0.41.2
Virtual environment 'r-tensorflow' successfully created.
Using virtual environment 'r-tensorflow' ...
+ /home/fnaufel/.virtualenvs/r-tensorflow/bin/python -m pip install --upgrade --no-user --ignore-installed 'tensorflow==2.13.*' 'nvidia-cudnn-cu11==8.6.*'
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting tensorflow==2.13.*
  Obtaining dependency information for tensorflow==2.13.* from https://files.pythonhosted.org/packages/5a/f2/5c2f878c62c8b79c629b11b33516bb55054d7677eba6f56f3a20296b56bd/tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.4 kB)
Collecting nvidia-cudnn-cu11==8.6.*
  Downloading nvidia_cudnn_cu11-8.6.0.163-py3-none-manylinux1_x86_64.whl (715.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 715.7/715.7 MB 8.7 MB/s eta 0:00:00
Collecting absl-py>=1.0.0 (from tensorflow==2.13.*)
  Downloading absl_py-1.4.0-py3-none-any.whl (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.5/126.5 kB 10.8 MB/s eta 0:00:00
Collecting astunparse>=1.6.0 (from tensorflow==2.13.*)
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting flatbuffers>=23.1.21 (from tensorflow==2.13.*)
  Obtaining dependency information for flatbuffers>=23.1.21 from https://files.pythonhosted.org/packages/6f/12/d5c79ee252793ffe845d58a913197bfa02ae9a0b5c9bc3dc4b58d477b9e7/flatbuffers-23.5.26-py2.py3-none-any.whl.metadata
  Downloading flatbuffers-23.5.26-py2.py3-none-any.whl.metadata (850 bytes)
Collecting gast<=0.4.0,>=0.2.1 (from tensorflow==2.13.*)
  Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting google-pasta>=0.1.1 (from tensorflow==2.13.*)
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.5/57.5 kB 3.9 MB/s eta 0:00:00
Collecting grpcio<2.0,>=1.24.3 (from tensorflow==2.13.*)
  Obtaining dependency information for grpcio<2.0,>=1.24.3 from https://files.pythonhosted.org/packages/28/fa/c38a010d3fffcac07ef121abb34eb2c3db0876df74267ce5bde13c3a6ed7/grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
Collecting h5py>=2.9.0 (from tensorflow==2.13.*)
  Obtaining dependency information for h5py>=2.9.0 from https://files.pythonhosted.org/packages/0d/7a/e55589e4093cca1934db5e99644c1c2424a9b3aac104b7f6176605a5eeb7/h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB)
Collecting keras<2.14,>=2.13.1 (from tensorflow==2.13.*)
  Obtaining dependency information for keras<2.14,>=2.13.1 from https://files.pythonhosted.org/packages/2e/f3/19da7511b45e80216cbbd9467137b2d28919c58ba1ccb971435cb631e470/keras-2.13.1-py3-none-any.whl.metadata
  Downloading keras-2.13.1-py3-none-any.whl.metadata (2.4 kB)
Collecting libclang>=13.0.0 (from tensorflow==2.13.*)
  Obtaining dependency information for libclang>=13.0.0 from https://files.pythonhosted.org/packages/ea/df/55525e489c43f9dbb6c8ea27d8a567b3dcd18a22f3c45483055f5ca6611d/libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl.metadata
  Downloading libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl.metadata (5.2 kB)
Collecting numpy<=1.24.3,>=1.22 (from tensorflow==2.13.*)
  Downloading numpy-1.24.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.3/17.3 MB 11.2 MB/s eta 0:00:00
Collecting opt-einsum>=2.3.2 (from tensorflow==2.13.*)
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 kB 5.0 MB/s eta 0:00:00
Collecting packaging (from tensorflow==2.13.*)
  Downloading packaging-23.1-py3-none-any.whl (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 2.5 MB/s eta 0:00:00
Collecting protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 (from tensorflow==2.13.*)
  Obtaining dependency information for protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 from https://files.pythonhosted.org/packages/4c/87/59648989ad7f5ba6fe3c7f8abc555183f28559b6f6cd14ad17a3f0d3094f/protobuf-4.24.1-cp37-abi3-manylinux2014_x86_64.whl.metadata
  Downloading protobuf-4.24.1-cp37-abi3-manylinux2014_x86_64.whl.metadata (540 bytes)
Collecting setuptools (from tensorflow==2.13.*)
  Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/4f/ab/0bcfebdfc3bfa8554b2b2c97a555569c4c1ebc74ea288741ea8326c51906/setuptools-68.1.2-py3-none-any.whl.metadata
  Downloading setuptools-68.1.2-py3-none-any.whl.metadata (6.2 kB)
Collecting six>=1.12.0 (from tensorflow==2.13.*)
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting tensorboard<2.14,>=2.13 (from tensorflow==2.13.*)
  Downloading tensorboard-2.13.0-py3-none-any.whl (5.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 MB 8.7 MB/s eta 0:00:00
Collecting tensorflow-estimator<2.14,>=2.13.0 (from tensorflow==2.13.*)
  Obtaining dependency information for tensorflow-estimator<2.14,>=2.13.0 from https://files.pythonhosted.org/packages/72/5c/c318268d96791c6222ad7df1651bbd1b2409139afeb6f468c0f327177016/tensorflow_estimator-2.13.0-py2.py3-none-any.whl.metadata
  Downloading tensorflow_estimator-2.13.0-py2.py3-none-any.whl.metadata (1.3 kB)
Collecting termcolor>=1.1.0 (from tensorflow==2.13.*)
  Downloading termcolor-2.3.0-py3-none-any.whl (6.9 kB)
Collecting typing-extensions<4.6.0,>=3.6.6 (from tensorflow==2.13.*)
  Downloading typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Collecting wrapt>=1.11.0 (from tensorflow==2.13.*)
  Downloading wrapt-1.15.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.4/78.4 kB 2.0 MB/s eta 0:00:00
Collecting tensorflow-io-gcs-filesystem>=0.23.1 (from tensorflow==2.13.*)
  Obtaining dependency information for tensorflow-io-gcs-filesystem>=0.23.1 from https://files.pythonhosted.org/packages/6a/48/3cdab86db01701ea043de445f3af1c3e539835781e57d803d2a87f256475/tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata
  Downloading tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (14 kB)
Collecting nvidia-cublas-cu11 (from nvidia-cudnn-cu11==8.6.*)
  Downloading https://developer.download.nvidia.com/compute/redist/nvidia-cublas-cu11/nvidia-cublas-cu11-2022.4.8.tar.gz (16 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting wheel<1.0,>=0.23.0 (from astunparse>=1.6.0->tensorflow==2.13.*)
  Obtaining dependency information for wheel<1.0,>=0.23.0 from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl.metadata
  Downloading wheel-0.41.2-py3-none-any.whl.metadata (2.2 kB)
Collecting google-auth<3,>=1.6.3 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for google-auth<3,>=1.6.3 from https://files.pythonhosted.org/packages/9c/8d/bff87fc722553a5691d8514da5523c23547f3894189ba03b57592e37bdc2/google_auth-2.22.0-py2.py3-none-any.whl.metadata
  Downloading google_auth-2.22.0-py2.py3-none-any.whl.metadata (4.2 kB)
Collecting google-auth-oauthlib<1.1,>=0.5 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading google_auth_oauthlib-1.0.0-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for markdown>=2.6.8 from https://files.pythonhosted.org/packages/1a/b5/228c1cdcfe138f1a8e01ab1b54284c8b83735476cb22b6ba251656ed13ad/Markdown-3.4.4-py3-none-any.whl.metadata
  Downloading Markdown-3.4.4-py3-none-any.whl.metadata (6.9 kB)
Collecting requests<3,>=2.21.0 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for requests<3,>=2.21.0 from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
  Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting tensorboard-data-server<0.8.0,>=0.7.0 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for tensorboard-data-server<0.8.0,>=0.7.0 from https://files.pythonhosted.org/packages/02/52/fb9e51fba47951aabd7a6b25e41d73eae94208ccf62d886168096941a781/tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata
  Downloading tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (1.1 kB)
Collecting werkzeug>=1.0.1 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for werkzeug>=1.0.1 from https://files.pythonhosted.org/packages/9b/59/a7c32e3d8d0e546a206e0552a2c04444544f15c1da4a01df8938d20c6ffc/werkzeug-2.3.7-py3-none-any.whl.metadata
  Downloading werkzeug-2.3.7-py3-none-any.whl.metadata (4.1 kB)
Collecting nvidia-cublas-cu117 (from nvidia-cublas-cu11->nvidia-cudnn-cu11==8.6.*)
  Downloading https://developer.download.nvidia.com/compute/redist/nvidia-cublas-cu117/nvidia_cublas_cu117-11.10.1.25-py3-none-manylinux1_x86_64.whl (333.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 333.1/333.1 MB 9.4 MB/s eta 0:00:00
Collecting cachetools<6.0,>=2.0.0 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for cachetools<6.0,>=2.0.0 from https://files.pythonhosted.org/packages/a9/c9/c8a7710f2cedcb1db9224fdd4d8307c9e48cbddc46c18b515fefc0f1abbe/cachetools-5.3.1-py3-none-any.whl.metadata
  Downloading cachetools-5.3.1-py3-none-any.whl.metadata (5.2 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 kB 4.1 MB/s eta 0:00:00
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
  Downloading urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.4/48.4 kB 6.0 MB/s eta 0:00:00
Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting charset-normalizer<4,>=2 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/a4/65/057bf29660aae6ade0816457f8db4e749e5c0bfa2366eb5f67db9912fa4c/charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading idna-3.4-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 10.4 MB/s eta 0:00:00
Collecting certifi>=2017.4.17 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
  Downloading certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting MarkupSafe>=2.1.1 (from werkzeug>=1.0.1->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for MarkupSafe>=2.1.1 from https://files.pythonhosted.org/packages/12/b3/d9ed2c0971e1435b8a62354b18d3060b66c8cb1d368399ec0b9baa7c0ee5/MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting pyasn1<0.6.0,>=0.4.6 (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 83.9/83.9 kB 214.7 MB/s eta 0:00:00
Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 4.7 MB/s eta 0:00:00
Downloading tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (524.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 524.1/524.1 MB 5.5 MB/s eta 0:00:00
Downloading flatbuffers-23.5.26-py2.py3-none-any.whl (26 kB)
Downloading grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.3/5.3 MB 2.7 MB/s eta 0:00:00
Downloading h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.8/4.8 MB 13.1 MB/s eta 0:00:00
Downloading keras-2.13.1-py3-none-any.whl (1.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 12.4 MB/s eta 0:00:00
Downloading libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl (22.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 22.9/22.9 MB 14.0 MB/s eta 0:00:00
Downloading protobuf-4.24.1-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 311.4/311.4 kB 22.7 MB/s eta 0:00:00
Downloading setuptools-68.1.2-py3-none-any.whl (805 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 805.1/805.1 kB 13.0 MB/s eta 0:00:00
Downloading tensorflow_estimator-2.13.0-py2.py3-none-any.whl (440 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 440.8/440.8 kB 12.5 MB/s eta 0:00:00
Downloading tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 13.3 MB/s eta 0:00:00
Downloading google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.8/181.8 kB 19.7 MB/s eta 0:00:00
Downloading Markdown-3.4.4-py3-none-any.whl (94 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.2/94.2 kB 5.8 MB/s eta 0:00:00
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 29.1 MB/s eta 0:00:00
Downloading tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl (6.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 13.7 MB/s eta 0:00:00
Downloading werkzeug-2.3.7-py3-none-any.whl (242 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 242.2/242.2 kB 10.7 MB/s eta 0:00:00
Downloading wheel-0.41.2-py3-none-any.whl (64 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.8/64.8 kB 3.9 MB/s eta 0:00:00
Downloading cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Downloading certifi-2023.7.22-py3-none-any.whl (158 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 kB 12.5 MB/s eta 0:00:00
Downloading charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (201 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 201.8/201.8 kB 12.2 MB/s eta 0:00:00
Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Downloading urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.1/143.1 kB 9.6 MB/s eta 0:00:00
Building wheels for collected packages: nvidia-cublas-cu11
  Building wheel for nvidia-cublas-cu11 (setup.py): started
  Building wheel for nvidia-cublas-cu11 (setup.py): finished with status 'done'
  Created wheel for nvidia-cublas-cu11: filename=nvidia_cublas_cu11-2022.4.8-py3-none-any.whl size=15604 sha256=56a0a0bc0d018d79b4b30f09551d714ff6daa319b41c4a94e7eec39e13daecc8
  Stored in directory: /tmp/pip-ephem-wheel-cache-dv_77eme/wheels/8f/ed/66/aa1caefa04698673b59e1799fface640a8f6e840e59b9c27e3
Successfully built nvidia-cublas-cu11
Installing collected packages: libclang, flatbuffers, wrapt, wheel, urllib3, typing-extensions, termcolor, tensorflow-io-gcs-filesystem, tensorflow-estimator, tensorboard-data-server, six, setuptools, pyasn1, protobuf, packaging, oauthlib, numpy, MarkupSafe, markdown, keras, idna, grpcio, gast, charset-normalizer, certifi, cachetools, absl-py, werkzeug, rsa, requests, pyasn1-modules, opt-einsum, nvidia-cublas-cu117, h5py, google-pasta, astunparse, requests-oauthlib, nvidia-cublas-cu11, google-auth, nvidia-cudnn-cu11, google-auth-oauthlib, tensorboard, tensorflow
Successfully installed MarkupSafe-2.1.3 absl-py-1.4.0 astunparse-1.6.3 cachetools-5.3.1 certifi-2023.7.22 charset-normalizer-3.2.0 flatbuffers-23.5.26 gast-0.4.0 google-auth-2.22.0 google-auth-oauthlib-1.0.0 google-pasta-0.2.0 grpcio-1.57.0 h5py-3.9.0 idna-3.4 keras-2.13.1 libclang-16.0.6 markdown-3.4.4 numpy-1.24.3 nvidia-cublas-cu11-2022.4.8 nvidia-cublas-cu117-11.10.1.25 nvidia-cudnn-cu11-8.6.0.163 oauthlib-3.2.2 opt-einsum-3.3.0 packaging-23.1 protobuf-4.24.1 pyasn1-0.5.0 pyasn1-modules-0.3.0 requests-2.31.0 requests-oauthlib-1.3.1 rsa-4.9 setuptools-68.1.2 six-1.16.0 tensorboard-2.13.0 tensorboard-data-server-0.7.1 tensorflow-2.13.0 tensorflow-estimator-2.13.0 tensorflow-io-gcs-filesystem-0.33.0 termcolor-2.3.0 typing-extensions-4.5.0 urllib3-1.26.16 werkzeug-2.3.7 wheel-0.41.2 wrapt-1.15.0
Error in system2(python, c("-c", shQuote("import nvidia.cudnn;print(nvidia.cudnn.__file__)")),  : 
  error in running command
>

Answer 5 · 2023-08-25T19:45:51.000Z

It looks like the nvidia-cudnn-cu11==8.6.* pip package was successfully installed, but then attempting to import the nvidia.cudnn python module raised an error. What is the output from:

system2('~/.virtualenvs/r-tensorflow/bin/python', c("-c", 
  shQuote("import nvidia.cudnn;print(nvidia.cudnn.__file__)")

Answer 6 · 2023-08-25T21:28:15.000Z

It looks like the nvidia-cudnn-cu11==8.6.* pip package was successfully installed, but then attempting to import the nvidia.cudnn python module raised an error. What is the output from:
system2('~/.virtualenvs/r-tensorflow/bin/python', c("-c", 
  shQuote("import nvidia.cudnn;print(nvidia.cudnn.__file__)")

The error was because python was not quoted in the call to system2.

When I run it, the output is

system2('/home/fnaufel/.virtualenvs/r-tensorflow/bin/python', c("-c", shQuote("import nvidia.cudnn; print(nvidia.cudnn.__file__)")))
/home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/nvidia/cudnn/__init__.py

When I activate the r-tensorflow virtualenv and run pip list, I get

Package                      Version
---------------------------- ----------
absl-py                      1.4.0
astunparse                   1.6.3
cachetools                   5.3.1
certifi                      2023.7.22
charset-normalizer           3.2.0
flatbuffers                  23.5.26
gast                         0.4.0
google-auth                  2.22.0
google-auth-oauthlib         1.0.0
google-pasta                 0.2.0
grpcio                       1.57.0
h5py                         3.9.0
idna                         3.4
keras                        2.13.1
libclang                     16.0.6
Markdown                     3.4.4
MarkupSafe                   2.1.3
numpy                        1.24.3
nvidia-cublas-cu11           2022.4.8
nvidia-cublas-cu117          11.10.1.25
nvidia-cudnn-cu11            8.6.0.163
oauthlib                     3.2.2
opt-einsum                   3.3.0
packaging                    23.1
pip                          23.2.1
protobuf                     4.24.1
pyasn1                       0.5.0
pyasn1-modules               0.3.0
requests                     2.31.0
requests-oauthlib            1.3.1
rsa                          4.9
setuptools                   68.1.2
six                          1.16.0
tensorboard                  2.13.0
tensorboard-data-server      0.7.1
tensorflow                   2.13.0
tensorflow-estimator         2.13.0
tensorflow-io-gcs-filesystem 0.33.0
termcolor                    2.3.0
typing_extensions            4.5.0
urllib3                      1.26.16
Werkzeug                     2.3.7
wheel                        0.41.2
wrapt                        1.15.0

Then, from bash, I installed tensorrt:

pip install tensorrt
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting tensorrt
  Downloading tensorrt-8.6.1.post1.tar.gz (18 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: tensorrt
  Building wheel for tensorrt (setup.py) ... done
  Created wheel for tensorrt: filename=tensorrt-8.6.1.post1-py2.py3-none-any.whl size=17281 sha256=0c1d825da95ec5f8573bae7fb03223f2cd8f3dfd1217e1fe7cb0c33ce53b1e90
  Stored in directory: /tmp/pip-ephem-wheel-cache-ntcqq205/wheels/f4/c8/0e/b79b08e45752491b9acfdbd69e8a609e8b2ed7640dda5a3e59
Successfully built tensorrt
Installing collected packages: tensorrt
Successfully installed tensorrt-8.6.1.post1

Now pip list outputs

Package                      Version
---------------------------- -----------
absl-py                      1.4.0
astunparse                   1.6.3
cachetools                   5.3.1
certifi                      2023.7.22
charset-normalizer           3.2.0
flatbuffers                  23.5.26
gast                         0.4.0
google-auth                  2.22.0
google-auth-oauthlib         1.0.0
google-pasta                 0.2.0
grpcio                       1.57.0
h5py                         3.9.0
idna                         3.4
keras                        2.13.1
libclang                     16.0.6
Markdown                     3.4.4
MarkupSafe                   2.1.3
numpy                        1.24.3
nvidia-cublas-cu11           2022.4.8
nvidia-cublas-cu117          11.10.1.25
nvidia-cublas-cu12           12.2.4.5
nvidia-cuda-nvrtc-cu12       12.2.128
nvidia-cuda-runtime-cu12     12.2.128
nvidia-cudnn-cu11            8.6.0.163
nvidia-cudnn-cu12            8.9.4.25
oauthlib                     3.2.2
opt-einsum                   3.3.0
packaging                    23.1
pip                          23.2.1
protobuf                     4.24.1
pyasn1                       0.5.0
pyasn1-modules               0.3.0
requests                     2.31.0
requests-oauthlib            1.3.1
rsa                          4.9
setuptools                   68.1.2
six                          1.16.0
tensorboard                  2.13.0
tensorboard-data-server      0.7.1
tensorflow                   2.13.0
tensorflow-estimator         2.13.0
tensorflow-io-gcs-filesystem 0.33.0
tensorrt                     8.6.1.post1
tensorrt-bindings            8.6.1
tensorrt-libs                8.6.1
termcolor                    2.3.0
typing_extensions            4.5.0
urllib3                      1.26.16
Werkzeug                     2.3.7
wheel                        0.41.2
wrapt                        1.15.0

Now the tensorrt packages have been installed, but the problem persists:

> tensorflow::tf$config$list_physical_devices()
2023-08-25 18:56:44.214799: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-25 18:56:44.873787: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2023-08-25 18:56:45.669909: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-08-25 18:56:45.699589: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1960] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
[[1]]
PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')

while from the bash shell, from the same virtualenv:

python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

2023-08-25 18:50:25.306409: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-25 18:50:27.417430: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-08-25 18:50:27.486107: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-08-25 18:50:27.486910: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

I also searched for tensorrt packages from the Ubuntu repositories (I thought they might be used when I ran python from bash, but not from R). There are three packages, but none are installed in my machine:

tensorrt-dev/unknown 8.6.1.6-1+cuda12.0 amd64
tensorrt-libs/unknown 8.6.1.6-1+cuda12.0 amd64
tensorrt/unknown 8.6.1.6-1+cuda12.0 amd64

Answer 7 · 2023-08-28T17:47:59.000Z

Thanks for reporting! Can you please try with the development version of Tensorflow and see if the issue is fixed?

remotes::install_github("rstudio/tensorflow")
tensorflow::install_tensorflow()

Sidenote: the tensorrt package you are installing with pip is incompatible with the tensorflow pip package. The TensorFlow wheels provided require cuda 11, while the tensorrt pip package requires cuda 12. You either need to download a tensorrt package that is built for cuda 11 (available behind an NVIDIA account login with an EULA acceptance), or build tensorflow from source, linking to cuda 12.

It's worth noting that tensorrt is completely optional, and unlikely to help much unless you are heavily optimizing for inference only workflows.

Answer 8 · 2023-08-30T16:01:36.000Z

Thanks for reporting! Can you please try with the development version of Tensorflow and see if the issue is fixed?
remotes::install_github("rstudio/tensorflow")
tensorflow::install_tensorflow()

Good news! Now the GPU is detected by tf in R!

BUT NOT if I install using the conda environment, as shown below.

Using CONDA

remotes::install_github("rstudio/tensorflow")
tensorflow::install_tensorflow('conda', envname = 'r-reticulate', pip_ignore_installed = TRUE)

> tensorflow::install_tensorflow('conda', envname = 'r-reticulate', pip_ignore_installed = TRUE)
+ . /ssd/miniconda/bin/activate
+ conda activate 'r-reticulate'
+ '/ssd/miniconda/envs/r-reticulate/bin/python' -m pip install --upgrade --no-user --ignore-installed 'tensorflow==2.13.*' 'nvidia-cudnn-cu11==8.6.*'
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting tensorflow==2.13.*
  Obtaining dependency information for tensorflow==2.13.* from https://files.pythonhosted.org/packages/5a/f2/5c2f878c62c8b79c629b11b33516bb55054d7677eba6f56f3a20296b56bd/tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.4 kB)
Collecting nvidia-cudnn-cu11==8.6.*
  Downloading nvidia_cudnn_cu11-8.6.0.163-py3-none-manylinux1_x86_64.whl (715.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 715.7/715.7 MB 8.8 MB/s eta 0:00:00
Collecting absl-py>=1.0.0 (from tensorflow==2.13.*)
  Downloading absl_py-1.4.0-py3-none-any.whl (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.5/126.5 kB 7.3 MB/s eta 0:00:00
Collecting astunparse>=1.6.0 (from tensorflow==2.13.*)
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting flatbuffers>=23.1.21 (from tensorflow==2.13.*)
  Obtaining dependency information for flatbuffers>=23.1.21 from https://files.pythonhosted.org/packages/6f/12/d5c79ee252793ffe845d58a913197bfa02ae9a0b5c9bc3dc4b58d477b9e7/flatbuffers-23.5.26-py2.py3-none-any.whl.metadata
  Downloading flatbuffers-23.5.26-py2.py3-none-any.whl.metadata (850 bytes)
Collecting gast<=0.4.0,>=0.2.1 (from tensorflow==2.13.*)
  Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting google-pasta>=0.1.1 (from tensorflow==2.13.*)
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.5/57.5 kB 11.2 MB/s eta 0:00:00
Collecting grpcio<2.0,>=1.24.3 (from tensorflow==2.13.*)
  Obtaining dependency information for grpcio<2.0,>=1.24.3 from https://files.pythonhosted.org/packages/28/fa/c38a010d3fffcac07ef121abb34eb2c3db0876df74267ce5bde13c3a6ed7/grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
Collecting h5py>=2.9.0 (from tensorflow==2.13.*)
  Obtaining dependency information for h5py>=2.9.0 from https://files.pythonhosted.org/packages/0d/7a/e55589e4093cca1934db5e99644c1c2424a9b3aac104b7f6176605a5eeb7/h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB)
Collecting keras<2.14,>=2.13.1 (from tensorflow==2.13.*)
  Obtaining dependency information for keras<2.14,>=2.13.1 from https://files.pythonhosted.org/packages/2e/f3/19da7511b45e80216cbbd9467137b2d28919c58ba1ccb971435cb631e470/keras-2.13.1-py3-none-any.whl.metadata
  Downloading keras-2.13.1-py3-none-any.whl.metadata (2.4 kB)
Collecting libclang>=13.0.0 (from tensorflow==2.13.*)
  Obtaining dependency information for libclang>=13.0.0 from https://files.pythonhosted.org/packages/ea/df/55525e489c43f9dbb6c8ea27d8a567b3dcd18a22f3c45483055f5ca6611d/libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl.metadata
  Downloading libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl.metadata (5.2 kB)
Collecting numpy<=1.24.3,>=1.22 (from tensorflow==2.13.*)
  Downloading numpy-1.24.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.3/17.3 MB 11.7 MB/s eta 0:00:00
Collecting opt-einsum>=2.3.2 (from tensorflow==2.13.*)
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 kB 2.8 MB/s eta 0:00:00
Collecting packaging (from tensorflow==2.13.*)
  Downloading packaging-23.1-py3-none-any.whl (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 2.6 MB/s eta 0:00:00
Collecting protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 (from tensorflow==2.13.*)
  Obtaining dependency information for protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 from https://files.pythonhosted.org/packages/f3/d8/fb02c40aa129c385430d177e4d9fa0160cb89be29305c8760861e538a2e4/protobuf-4.24.2-cp37-abi3-manylinux2014_x86_64.whl.metadata
  Downloading protobuf-4.24.2-cp37-abi3-manylinux2014_x86_64.whl.metadata (540 bytes)
Collecting setuptools (from tensorflow==2.13.*)
  Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/4f/ab/0bcfebdfc3bfa8554b2b2c97a555569c4c1ebc74ea288741ea8326c51906/setuptools-68.1.2-py3-none-any.whl.metadata
  Downloading setuptools-68.1.2-py3-none-any.whl.metadata (6.2 kB)
Collecting six>=1.12.0 (from tensorflow==2.13.*)
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting tensorboard<2.14,>=2.13 (from tensorflow==2.13.*)
  Downloading tensorboard-2.13.0-py3-none-any.whl (5.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 MB 10.9 MB/s eta 0:00:00
Collecting tensorflow-estimator<2.14,>=2.13.0 (from tensorflow==2.13.*)
  Obtaining dependency information for tensorflow-estimator<2.14,>=2.13.0 from https://files.pythonhosted.org/packages/72/5c/c318268d96791c6222ad7df1651bbd1b2409139afeb6f468c0f327177016/tensorflow_estimator-2.13.0-py2.py3-none-any.whl.metadata
  Downloading tensorflow_estimator-2.13.0-py2.py3-none-any.whl.metadata (1.3 kB)
Collecting termcolor>=1.1.0 (from tensorflow==2.13.*)
  Downloading termcolor-2.3.0-py3-none-any.whl (6.9 kB)
Collecting typing-extensions<4.6.0,>=3.6.6 (from tensorflow==2.13.*)
  Downloading typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Collecting wrapt>=1.11.0 (from tensorflow==2.13.*)
  Downloading wrapt-1.15.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.4/78.4 kB 14.5 MB/s eta 0:00:00
Collecting tensorflow-io-gcs-filesystem>=0.23.1 (from tensorflow==2.13.*)
  Obtaining dependency information for tensorflow-io-gcs-filesystem>=0.23.1 from https://files.pythonhosted.org/packages/6a/48/3cdab86db01701ea043de445f3af1c3e539835781e57d803d2a87f256475/tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata
  Downloading tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (14 kB)
Collecting nvidia-cublas-cu11 (from nvidia-cudnn-cu11==8.6.*)
  Downloading https://developer.download.nvidia.com/compute/redist/nvidia-cublas-cu11/nvidia-cublas-cu11-2022.4.8.tar.gz (16 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting wheel<1.0,>=0.23.0 (from astunparse>=1.6.0->tensorflow==2.13.*)
  Obtaining dependency information for wheel<1.0,>=0.23.0 from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl.metadata
  Downloading wheel-0.41.2-py3-none-any.whl.metadata (2.2 kB)
Collecting google-auth<3,>=1.6.3 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for google-auth<3,>=1.6.3 from https://files.pythonhosted.org/packages/9c/8d/bff87fc722553a5691d8514da5523c23547f3894189ba03b57592e37bdc2/google_auth-2.22.0-py2.py3-none-any.whl.metadata
  Downloading google_auth-2.22.0-py2.py3-none-any.whl.metadata (4.2 kB)
Collecting google-auth-oauthlib<1.1,>=0.5 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading google_auth_oauthlib-1.0.0-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for markdown>=2.6.8 from https://files.pythonhosted.org/packages/1a/b5/228c1cdcfe138f1a8e01ab1b54284c8b83735476cb22b6ba251656ed13ad/Markdown-3.4.4-py3-none-any.whl.metadata
  Downloading Markdown-3.4.4-py3-none-any.whl.metadata (6.9 kB)
Collecting requests<3,>=2.21.0 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for requests<3,>=2.21.0 from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
  Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting tensorboard-data-server<0.8.0,>=0.7.0 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for tensorboard-data-server<0.8.0,>=0.7.0 from https://files.pythonhosted.org/packages/02/52/fb9e51fba47951aabd7a6b25e41d73eae94208ccf62d886168096941a781/tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata
  Downloading tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (1.1 kB)
Collecting werkzeug>=1.0.1 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for werkzeug>=1.0.1 from https://files.pythonhosted.org/packages/9b/59/a7c32e3d8d0e546a206e0552a2c04444544f15c1da4a01df8938d20c6ffc/werkzeug-2.3.7-py3-none-any.whl.metadata
  Downloading werkzeug-2.3.7-py3-none-any.whl.metadata (4.1 kB)
Collecting nvidia-cublas-cu117 (from nvidia-cublas-cu11->nvidia-cudnn-cu11==8.6.*)
  Downloading https://developer.download.nvidia.com/compute/redist/nvidia-cublas-cu117/nvidia_cublas_cu117-11.10.1.25-py3-none-manylinux1_x86_64.whl (333.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 333.1/333.1 MB 15.6 MB/s eta 0:00:00
Collecting cachetools<6.0,>=2.0.0 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for cachetools<6.0,>=2.0.0 from https://files.pythonhosted.org/packages/a9/c9/c8a7710f2cedcb1db9224fdd4d8307c9e48cbddc46c18b515fefc0f1abbe/cachetools-5.3.1-py3-none-any.whl.metadata
  Downloading cachetools-5.3.1-py3-none-any.whl.metadata (5.2 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 kB 9.5 MB/s eta 0:00:00
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
  Downloading urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.4/48.4 kB 3.6 MB/s eta 0:00:00
Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting charset-normalizer<4,>=2 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/a4/65/057bf29660aae6ade0816457f8db4e749e5c0bfa2366eb5f67db9912fa4c/charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading idna-3.4-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 4.6 MB/s eta 0:00:00
Collecting certifi>=2017.4.17 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
  Downloading certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting MarkupSafe>=2.1.1 (from werkzeug>=1.0.1->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for MarkupSafe>=2.1.1 from https://files.pythonhosted.org/packages/12/b3/d9ed2c0971e1435b8a62354b18d3060b66c8cb1d368399ec0b9baa7c0ee5/MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting pyasn1<0.6.0,>=0.4.6 (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 83.9/83.9 kB 10.3 MB/s eta 0:00:00
Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 11.6 MB/s eta 0:00:00
Downloading tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (524.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 524.1/524.1 MB 4.6 MB/s eta 0:00:00
Downloading flatbuffers-23.5.26-py2.py3-none-any.whl (26 kB)
Downloading grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.3/5.3 MB 13.6 MB/s eta 0:00:00
Downloading h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.8/4.8 MB 11.6 MB/s eta 0:00:00
Downloading keras-2.13.1-py3-none-any.whl (1.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 3.9 MB/s eta 0:00:00
Downloading libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl (22.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 22.9/22.9 MB 8.2 MB/s eta 0:00:00
Downloading protobuf-4.24.2-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 311.4/311.4 kB 5.0 MB/s eta 0:00:00
Downloading setuptools-68.1.2-py3-none-any.whl (805 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 805.1/805.1 kB 5.6 MB/s eta 0:00:00
Downloading tensorflow_estimator-2.13.0-py2.py3-none-any.whl (440 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 440.8/440.8 kB 6.6 MB/s eta 0:00:00
Downloading tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 8.1 MB/s eta 0:00:00
Downloading google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.8/181.8 kB 3.3 MB/s eta 0:00:00
Downloading Markdown-3.4.4-py3-none-any.whl (94 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.2/94.2 kB 3.9 MB/s eta 0:00:00
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 4.1 MB/s eta 0:00:00
Downloading tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl (6.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 9.5 MB/s eta 0:00:00
Downloading werkzeug-2.3.7-py3-none-any.whl (242 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 242.2/242.2 kB 7.5 MB/s eta 0:00:00
Downloading wheel-0.41.2-py3-none-any.whl (64 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.8/64.8 kB 2.4 MB/s eta 0:00:00
Downloading cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Downloading certifi-2023.7.22-py3-none-any.whl (158 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 kB 5.8 MB/s eta 0:00:00
Downloading charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (201 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 201.8/201.8 kB 6.4 MB/s eta 0:00:00
Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Downloading urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.1/143.1 kB 6.4 MB/s eta 0:00:00
Building wheels for collected packages: nvidia-cublas-cu11
  Building wheel for nvidia-cublas-cu11 (setup.py): started
  Building wheel for nvidia-cublas-cu11 (setup.py): finished with status 'done'
  Created wheel for nvidia-cublas-cu11: filename=nvidia_cublas_cu11-2022.4.8-py3-none-any.whl size=15604 sha256=8bc9579fa78bd624d2f4cdab78264747877b66b9b34b1bf4ee8def814dc739bb
  Stored in directory: /tmp/pip-ephem-wheel-cache-ov3ls0cr/wheels/8f/ed/66/aa1caefa04698673b59e1799fface640a8f6e840e59b9c27e3
Successfully built nvidia-cublas-cu11
Installing collected packages: libclang, flatbuffers, wrapt, wheel, urllib3, typing-extensions, termcolor, tensorflow-io-gcs-filesystem, tensorflow-estimator, tensorboard-data-server, six, setuptools, pyasn1, protobuf, packaging, oauthlib, numpy, MarkupSafe, markdown, keras, idna, grpcio, gast, charset-normalizer, certifi, cachetools, absl-py, werkzeug, rsa, requests, pyasn1-modules, opt-einsum, nvidia-cublas-cu117, h5py, google-pasta, astunparse, requests-oauthlib, nvidia-cublas-cu11, google-auth, nvidia-cudnn-cu11, google-auth-oauthlib, tensorboard, tensorflow
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow-metadata 1.14.0 requires protobuf<4.21,>=3.20.3, but you have protobuf 4.24.2 which is incompatible.
Successfully installed MarkupSafe-2.1.1 absl-py-1.4.0 astunparse-1.6.3 cachetools-5.3.1 certifi-2023.7.22 charset-normalizer-2.0.4 flatbuffers-23.5.26 gast-0.4.0 google-auth-2.22.0 google-auth-oauthlib-1.0.0 google-pasta-0.2.0 grpcio-1.57.0 h5py-3.9.0 idna-3.4 keras-2.13.1 libclang-16.0.6 markdown-3.4.4 numpy-1.24.3 nvidia-cublas-cu11-11.11.3.6 nvidia-cublas-cu117-11.10.1.25 nvidia-cudnn-cu11-8.6.0.163 oauthlib-3.2.2 opt-einsum-3.3.0 packaging-21.3 protobuf-3.20.3 pyasn1-0.5.0 pyasn1-modules-0.3.0 requests-2.31.0 requests-oauthlib-1.3.1 rsa-4.9 setuptools-68.0.0 six-1.16.0 tensorboard-2.13.0 tensorboard-data-server-0.7.1 tensorflow-2.13.0 tensorflow-estimator-2.13.0 tensorflow-io-gcs-filesystem-0.33.0 termcolor-2.3.0 typing-extensions-4.5.0 urllib3-1.26.16 werkzeug-2.3.7 wheel-0.37.1 wrapt-1.15.0

Installation complete.

> tensorflow::tf$config$list_physical_devices()
2023-08-30 12:11:52.630944: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-30 12:11:58.663029: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2023-08-30 12:12:04.424608: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-08-30 12:12:04.600133: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1960] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
[[1]]
PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'

Using VENV

remotes::install_github("rstudio/tensorflow")
tensorflow::install_tensorflow('virtualenv', pip_ignore_installed = TRUE)

> tensorflow::install_tensorflow('virtualenv', pip_ignore_installed = TRUE)
Virtual environment 'r-tensorflow' removed.
Using Python: /usr/bin/python3.10
Creating virtual environment 'r-tensorflow' ... 
+ /usr/bin/python3.10 -m virtualenv -p '/usr/bin/python3.10' /home/fnaufel/.virtualenvs/r-tensorflow
created virtual environment CPython3.10.12.final.0-64 in 2124ms
  creator CPython3Posix(dest=/home/fnaufel/.virtualenvs/r-tensorflow, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/fnaufel/.local/share/virtualenv)
    added seed packages: pip==22.0.2, setuptools==59.6.0, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
Done!
Installing packages: pip, wheel, setuptools
+ /home/fnaufel/.virtualenvs/r-tensorflow/bin/python -m pip install --upgrade --no-user pip wheel setuptools
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in /home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages (22.0.2)
Collecting pip
  Downloading pip-23.2.1-py3-none-any.whl (2.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 9.8 MB/s eta 0:00:00
Requirement already satisfied: wheel in /home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages (0.37.1)
Collecting wheel
  Downloading wheel-0.41.2-py3-none-any.whl (64 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.8/64.8 KB 3.7 MB/s eta 0:00:00
Requirement already satisfied: setuptools in /home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages (59.6.0)
Collecting setuptools
  Downloading setuptools-68.1.2-py3-none-any.whl (805 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 805.1/805.1 KB 11.0 MB/s eta 0:00:00
Installing collected packages: wheel, setuptools, pip
  Attempting uninstall: wheel
    Found existing installation: wheel 0.37.1
    Uninstalling wheel-0.37.1:
      Successfully uninstalled wheel-0.37.1
  Attempting uninstall: setuptools
    Found existing installation: setuptools 59.6.0
    Uninstalling setuptools-59.6.0:
      Successfully uninstalled setuptools-59.6.0
  Attempting uninstall: pip
    Found existing installation: pip 22.0.2
    Uninstalling pip-22.0.2:
      Successfully uninstalled pip-22.0.2
Successfully installed pip-23.2.1 setuptools-68.1.2 wheel-0.41.2
Virtual environment 'r-tensorflow' successfully created.
Using virtual environment 'r-tensorflow' ...
+ /home/fnaufel/.virtualenvs/r-tensorflow/bin/python -m pip install --upgrade --no-user --ignore-installed 'tensorflow==2.13.*' 'nvidia-cudnn-cu11==8.6.*'
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting tensorflow==2.13.*
  Obtaining dependency information for tensorflow==2.13.* from https://files.pythonhosted.org/packages/5a/f2/5c2f878c62c8b79c629b11b33516bb55054d7677eba6f56f3a20296b56bd/tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.4 kB)
Collecting nvidia-cudnn-cu11==8.6.*
  Downloading nvidia_cudnn_cu11-8.6.0.163-py3-none-manylinux1_x86_64.whl (715.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 715.7/715.7 MB 6.7 MB/s eta 0:00:00
Collecting absl-py>=1.0.0 (from tensorflow==2.13.*)
  Downloading absl_py-1.4.0-py3-none-any.whl (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.5/126.5 kB 5.7 MB/s eta 0:00:00
Collecting astunparse>=1.6.0 (from tensorflow==2.13.*)
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting flatbuffers>=23.1.21 (from tensorflow==2.13.*)
  Obtaining dependency information for flatbuffers>=23.1.21 from https://files.pythonhosted.org/packages/6f/12/d5c79ee252793ffe845d58a913197bfa02ae9a0b5c9bc3dc4b58d477b9e7/flatbuffers-23.5.26-py2.py3-none-any.whl.metadata
  Downloading flatbuffers-23.5.26-py2.py3-none-any.whl.metadata (850 bytes)
Collecting gast<=0.4.0,>=0.2.1 (from tensorflow==2.13.*)
  Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting google-pasta>=0.1.1 (from tensorflow==2.13.*)
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.5/57.5 kB 3.6 MB/s eta 0:00:00
Collecting grpcio<2.0,>=1.24.3 (from tensorflow==2.13.*)
  Obtaining dependency information for grpcio<2.0,>=1.24.3 from https://files.pythonhosted.org/packages/28/fa/c38a010d3fffcac07ef121abb34eb2c3db0876df74267ce5bde13c3a6ed7/grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
Collecting h5py>=2.9.0 (from tensorflow==2.13.*)
  Obtaining dependency information for h5py>=2.9.0 from https://files.pythonhosted.org/packages/0d/7a/e55589e4093cca1934db5e99644c1c2424a9b3aac104b7f6176605a5eeb7/h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB)
Collecting keras<2.14,>=2.13.1 (from tensorflow==2.13.*)
  Obtaining dependency information for keras<2.14,>=2.13.1 from https://files.pythonhosted.org/packages/2e/f3/19da7511b45e80216cbbd9467137b2d28919c58ba1ccb971435cb631e470/keras-2.13.1-py3-none-any.whl.metadata
  Downloading keras-2.13.1-py3-none-any.whl.metadata (2.4 kB)
Collecting libclang>=13.0.0 (from tensorflow==2.13.*)
  Obtaining dependency information for libclang>=13.0.0 from https://files.pythonhosted.org/packages/ea/df/55525e489c43f9dbb6c8ea27d8a567b3dcd18a22f3c45483055f5ca6611d/libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl.metadata
  Downloading libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl.metadata (5.2 kB)
Collecting numpy<=1.24.3,>=1.22 (from tensorflow==2.13.*)
  Downloading numpy-1.24.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.3/17.3 MB 10.0 MB/s eta 0:00:00
Collecting opt-einsum>=2.3.2 (from tensorflow==2.13.*)
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 kB 16.1 MB/s eta 0:00:00
Collecting packaging (from tensorflow==2.13.*)
  Downloading packaging-23.1-py3-none-any.whl (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 2.3 MB/s eta 0:00:00
Collecting protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 (from tensorflow==2.13.*)
  Obtaining dependency information for protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 from https://files.pythonhosted.org/packages/f3/d8/fb02c40aa129c385430d177e4d9fa0160cb89be29305c8760861e538a2e4/protobuf-4.24.2-cp37-abi3-manylinux2014_x86_64.whl.metadata
  Downloading protobuf-4.24.2-cp37-abi3-manylinux2014_x86_64.whl.metadata (540 bytes)
Collecting setuptools (from tensorflow==2.13.*)
  Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/4f/ab/0bcfebdfc3bfa8554b2b2c97a555569c4c1ebc74ea288741ea8326c51906/setuptools-68.1.2-py3-none-any.whl.metadata
  Downloading setuptools-68.1.2-py3-none-any.whl.metadata (6.2 kB)
Collecting six>=1.12.0 (from tensorflow==2.13.*)
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting tensorboard<2.14,>=2.13 (from tensorflow==2.13.*)
  Downloading tensorboard-2.13.0-py3-none-any.whl (5.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.6/5.6 MB 11.1 MB/s eta 0:00:00
Collecting tensorflow-estimator<2.14,>=2.13.0 (from tensorflow==2.13.*)
  Obtaining dependency information for tensorflow-estimator<2.14,>=2.13.0 from https://files.pythonhosted.org/packages/72/5c/c318268d96791c6222ad7df1651bbd1b2409139afeb6f468c0f327177016/tensorflow_estimator-2.13.0-py2.py3-none-any.whl.metadata
  Downloading tensorflow_estimator-2.13.0-py2.py3-none-any.whl.metadata (1.3 kB)
Collecting termcolor>=1.1.0 (from tensorflow==2.13.*)
  Downloading termcolor-2.3.0-py3-none-any.whl (6.9 kB)
Collecting typing-extensions<4.6.0,>=3.6.6 (from tensorflow==2.13.*)
  Downloading typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Collecting wrapt>=1.11.0 (from tensorflow==2.13.*)
  Downloading wrapt-1.15.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.4/78.4 kB 7.3 MB/s eta 0:00:00
Collecting tensorflow-io-gcs-filesystem>=0.23.1 (from tensorflow==2.13.*)
  Obtaining dependency information for tensorflow-io-gcs-filesystem>=0.23.1 from https://files.pythonhosted.org/packages/6a/48/3cdab86db01701ea043de445f3af1c3e539835781e57d803d2a87f256475/tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata
  Downloading tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (14 kB)
Collecting nvidia-cublas-cu11 (from nvidia-cudnn-cu11==8.6.*)
  Downloading https://developer.download.nvidia.com/compute/redist/nvidia-cublas-cu11/nvidia-cublas-cu11-2022.4.8.tar.gz (16 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting wheel<1.0,>=0.23.0 (from astunparse>=1.6.0->tensorflow==2.13.*)
  Obtaining dependency information for wheel<1.0,>=0.23.0 from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl.metadata
  Downloading wheel-0.41.2-py3-none-any.whl.metadata (2.2 kB)
Collecting google-auth<3,>=1.6.3 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for google-auth<3,>=1.6.3 from https://files.pythonhosted.org/packages/9c/8d/bff87fc722553a5691d8514da5523c23547f3894189ba03b57592e37bdc2/google_auth-2.22.0-py2.py3-none-any.whl.metadata
  Downloading google_auth-2.22.0-py2.py3-none-any.whl.metadata (4.2 kB)
Collecting google-auth-oauthlib<1.1,>=0.5 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading google_auth_oauthlib-1.0.0-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for markdown>=2.6.8 from https://files.pythonhosted.org/packages/1a/b5/228c1cdcfe138f1a8e01ab1b54284c8b83735476cb22b6ba251656ed13ad/Markdown-3.4.4-py3-none-any.whl.metadata
  Downloading Markdown-3.4.4-py3-none-any.whl.metadata (6.9 kB)
Collecting requests<3,>=2.21.0 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for requests<3,>=2.21.0 from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
  Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting tensorboard-data-server<0.8.0,>=0.7.0 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for tensorboard-data-server<0.8.0,>=0.7.0 from https://files.pythonhosted.org/packages/02/52/fb9e51fba47951aabd7a6b25e41d73eae94208ccf62d886168096941a781/tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata
  Downloading tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (1.1 kB)
Collecting werkzeug>=1.0.1 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for werkzeug>=1.0.1 from https://files.pythonhosted.org/packages/9b/59/a7c32e3d8d0e546a206e0552a2c04444544f15c1da4a01df8938d20c6ffc/werkzeug-2.3.7-py3-none-any.whl.metadata
  Downloading werkzeug-2.3.7-py3-none-any.whl.metadata (4.1 kB)
Collecting nvidia-cublas-cu117 (from nvidia-cublas-cu11->nvidia-cudnn-cu11==8.6.*)
  Downloading https://developer.download.nvidia.com/compute/redist/nvidia-cublas-cu117/nvidia_cublas_cu117-11.10.1.25-py3-none-manylinux1_x86_64.whl (333.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 333.1/333.1 MB 3.6 MB/s eta 0:00:00
Collecting cachetools<6.0,>=2.0.0 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for cachetools<6.0,>=2.0.0 from https://files.pythonhosted.org/packages/a9/c9/c8a7710f2cedcb1db9224fdd4d8307c9e48cbddc46c18b515fefc0f1abbe/cachetools-5.3.1-py3-none-any.whl.metadata
  Downloading cachetools-5.3.1-py3-none-any.whl.metadata (5.2 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 kB 7.7 MB/s eta 0:00:00
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
  Downloading urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.4/48.4 kB 2.1 MB/s eta 0:00:00
Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting charset-normalizer<4,>=2 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/a4/65/057bf29660aae6ade0816457f8db4e749e5c0bfa2366eb5f67db9912fa4c/charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading idna-3.4-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 3.8 MB/s eta 0:00:00
Collecting certifi>=2017.4.17 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
  Downloading certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting MarkupSafe>=2.1.1 (from werkzeug>=1.0.1->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for MarkupSafe>=2.1.1 from https://files.pythonhosted.org/packages/12/b3/d9ed2c0971e1435b8a62354b18d3060b66c8cb1d368399ec0b9baa7c0ee5/MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting pyasn1<0.6.0,>=0.4.6 (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 83.9/83.9 kB 5.6 MB/s eta 0:00:00
Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 7.4 MB/s eta 0:00:00
Downloading tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (524.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 524.1/524.1 MB 11.7 MB/s eta 0:00:00
Downloading flatbuffers-23.5.26-py2.py3-none-any.whl (26 kB)
Downloading grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.3/5.3 MB 9.8 MB/s eta 0:00:00
Downloading h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.8/4.8 MB 13.3 MB/s eta 0:00:00
Downloading keras-2.13.1-py3-none-any.whl (1.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 13.8 MB/s eta 0:00:00
Downloading libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl (22.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 22.9/22.9 MB 6.6 MB/s eta 0:00:00
Downloading protobuf-4.24.2-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 311.4/311.4 kB 8.0 MB/s eta 0:00:00
Downloading setuptools-68.1.2-py3-none-any.whl (805 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 805.1/805.1 kB 4.9 MB/s eta 0:00:00
Downloading tensorflow_estimator-2.13.0-py2.py3-none-any.whl (440 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 440.8/440.8 kB 4.1 MB/s eta 0:00:00
Downloading tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 8.3 MB/s eta 0:00:00
Downloading google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.8/181.8 kB 8.1 MB/s eta 0:00:00
Downloading Markdown-3.4.4-py3-none-any.whl (94 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.2/94.2 kB 4.9 MB/s eta 0:00:00
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 4.5 MB/s eta 0:00:00
Downloading tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl (6.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 7.4 MB/s eta 0:00:00
Downloading werkzeug-2.3.7-py3-none-any.whl (242 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 242.2/242.2 kB 5.5 MB/s eta 0:00:00
Downloading wheel-0.41.2-py3-none-any.whl (64 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.8/64.8 kB 3.9 MB/s eta 0:00:00
Downloading cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Downloading certifi-2023.7.22-py3-none-any.whl (158 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 kB 5.0 MB/s eta 0:00:00
Downloading charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (201 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 201.8/201.8 kB 7.6 MB/s eta 0:00:00
Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Downloading urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.1/143.1 kB 9.2 MB/s eta 0:00:00
Building wheels for collected packages: nvidia-cublas-cu11
  Building wheel for nvidia-cublas-cu11 (setup.py): started
  Building wheel for nvidia-cublas-cu11 (setup.py): finished with status 'done'
  Created wheel for nvidia-cublas-cu11: filename=nvidia_cublas_cu11-2022.4.8-py3-none-any.whl size=15604 sha256=36063794f0a6278589f63e75314f446de5c2afc5123c68996e95ac6bcb94305a
  Stored in directory: /tmp/pip-ephem-wheel-cache-kcew7di7/wheels/8f/ed/66/aa1caefa04698673b59e1799fface640a8f6e840e59b9c27e3
Successfully built nvidia-cublas-cu11
Installing collected packages: libclang, flatbuffers, wrapt, wheel, urllib3, typing-extensions, termcolor, tensorflow-io-gcs-filesystem, tensorflow-estimator, tensorboard-data-server, six, setuptools, pyasn1, protobuf, packaging, oauthlib, numpy, MarkupSafe, markdown, keras, idna, grpcio, gast, charset-normalizer, certifi, cachetools, absl-py, werkzeug, rsa, requests, pyasn1-modules, opt-einsum, nvidia-cublas-cu117, h5py, google-pasta, astunparse, requests-oauthlib, nvidia-cublas-cu11, google-auth, nvidia-cudnn-cu11, google-auth-oauthlib, tensorboard, tensorflow
Successfully installed MarkupSafe-2.1.3 absl-py-1.4.0 astunparse-1.6.3 cachetools-5.3.1 certifi-2023.7.22 charset-normalizer-3.2.0 flatbuffers-23.5.26 gast-0.4.0 google-auth-2.22.0 google-auth-oauthlib-1.0.0 google-pasta-0.2.0 grpcio-1.57.0 h5py-3.9.0 idna-3.4 keras-2.13.1 libclang-16.0.6 markdown-3.4.4 numpy-1.24.3 nvidia-cublas-cu11-2022.4.8 nvidia-cublas-cu117-11.10.1.25 nvidia-cudnn-cu11-8.6.0.163 oauthlib-3.2.2 opt-einsum-3.3.0 packaging-23.1 protobuf-4.24.2 pyasn1-0.5.0 pyasn1-modules-0.3.0 requests-2.31.0 requests-oauthlib-1.3.1 rsa-4.9 setuptools-68.1.2 six-1.16.0 tensorboard-2.13.0 tensorboard-data-server-0.7.1 tensorflow-2.13.0 tensorflow-estimator-2.13.0 tensorflow-io-gcs-filesystem-0.33.0 termcolor-2.3.0 typing-extensions-4.5.0 urllib3-1.26.16 werkzeug-2.3.7 wheel-0.41.2 wrapt-1.15.0
creating symlinks:
- '/home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/tensorflow/libcudnn_adv_infer.so.8' -> '../nvidia/cudnn/lib/libcudnn_adv_infer.so.8'
- '/home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/tensorflow/libcudnn_adv_train.so.8' -> '../nvidia/cudnn/lib/libcudnn_adv_train.so.8'
- '/home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/tensorflow/libcudnn_cnn_infer.so.8' -> '../nvidia/cudnn/lib/libcudnn_cnn_infer.so.8'
- '/home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/tensorflow/libcudnn_cnn_train.so.8' -> '../nvidia/cudnn/lib/libcudnn_cnn_train.so.8'
- '/home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/tensorflow/libcudnn_ops_infer.so.8' -> '../nvidia/cudnn/lib/libcudnn_ops_infer.so.8'
- '/home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/tensorflow/libcudnn_ops_train.so.8' -> '../nvidia/cudnn/lib/libcudnn_ops_train.so.8'
- '/home/fnaufel/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/tensorflow/libcudnn.so.8' -> '../nvidia/cudnn/lib/libcudnn.so.8'
- To install the necessary drivers to enable GPU usage, run the following terminal commands:
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda-toolkit-11-8

Installation complete.


Restarting R session...

> tensorflow::tf$config$list_physical_devices()
2023-08-30 12:22:17.601779: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-30 12:22:18.737339: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2023-08-30 12:22:20.087699: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-08-30 12:22:20.142510: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-08-30 12:22:20.143090: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
[[1]]
PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')

[[2]]
PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')

Fitting a model from RStudio

historico1 <- rede1 %>% fit(
  dados_treino, 
  metas_treino, 
  epochs = n_epochs, 
  batch_size = tamanho_batch
)

Epoch 1/100
2023-08-30 12:25:26.843757: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f873c002ea0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2023-08-30 12:25:26.843791: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce GTX 1050 Ti, Compute Capability 6.1
2023-08-30 12:25:26.852162: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:255] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2023-08-30 12:25:27.131104: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:432] Loaded cuDNN version 8600
2023-08-30 12:25:27.201429: I tensorflow/tsl/platform/default/subprocess.cc:304] Start cannot spawn child process: No such file or directory
2023-08-30 12:25:27.355272: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.

Sidenote: the tensorrt package you are installing with pip is incompatible with the tensorflow pip package. The TensorFlow wheels provided require cuda 11, while the tensorrt pip package requires cuda 12. You either need to download a tensorrt package that is built for cuda 11 (available behind an NVIDIA account login with an EULA acceptance), or build tensorflow from source, linking to cuda 12.

It's worth noting that tensorrt is completely optional, and unlikely to help much unless you are heavily optimizing for inference only workflows.

I see.

But then, when I try to call tf from R after installing via conda like so

> tensorflow::tf$config$list_physical_devices()

(...)

2023-08-30 12:12:04.600133: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1960] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
[[1]]
PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'

which missing libraries (tensorflow/core/common_runtime/gpu/gpu_device.cc:1960] Cannot dlopen some GPU libraries.) is it referring to? I had assumed tensorRT, but your comments imply it's something else.

Indeed, when running the same command from R using the virtualenv tf install, the same warning about tensorRT is emitted, but the GPU is detected now, which confirms that tensorRT was not necessary, as you said.

At the end of the CONDA install, there is a complaint about protobuf:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow-metadata 1.14.0 requires protobuf<4.21,>=3.20.3, but you have protobuf 4.24.2 which is incompatible.

The complaint does not show up in the VIRTUALENV install, but then when I go check which version of protobuf is installed in the virtual envirnoment, I get:

❯ pip list protobuf | grep protobuf
protobuf                     4.24.2

which is exactly I get in the conda r-reticulate environment too!

Anyways...

I want to thank you for your help in solving my problem.

RStudio and Posit are the best!

Answer 9 · 2023-09-01T09:16:08.000Z

I'm trying to resolve the same issue, unfortunately I've not been able to fix it using the same approach so far.
I've got an added layer of 'fun' where I'm running RStudio-server on WSL because I'm on a Windows 10 system.

Python can see the GPU but R using the same environment can't.
Both installing as a venv or using conda I still seem to have the same issue.

Do you have a walkthrough on how you got yours installed and working I feel I may be missing an something?

Answer 10 · 2023-09-03T16:12:39.000Z

I'm trying to resolve the same issue, unfortunately I've not been able to fix it using the same approach so far. I've got an added layer of 'fun' where I'm running RStudio-server on WSL because I'm on a Windows 10 system.

Python can see the GPU but R using the same environment can't. Both installing as a venv or using conda I still seem to have the same issue.

Do you have a walkthrough on how you got yours installed and working I feel I may be missing an something?

What worked for me was to install tf from the github repo, running the following code from the R console in RStudio:

remotes::install_github("rstudio/tensorflow")
tensorflow::install_tensorflow('virtualenv', pip_ignore_installed = TRUE)

Before I did this, I was in the same situation as you: the GPU was detected by Python but not by R; that's why I am not including anything I did before this.

Plus, the code in the repo was changed around August 28; I'm supposing they implemented a fix on or before that date.

After running the code above, I tested with (also from the RStudio R console)

tensorflow::tf$config$list_physical_devices()

and got

2023-08-30 12:22:17.601779: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-30 12:22:18.737339: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2023-08-30 12:22:20.087699: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-08-30 12:22:20.142510: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2023-08-30 12:22:20.143090: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:995] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
[[1]]
PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')

[[2]]
PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')

Answer 11 · 2023-09-04T06:02:19.000Z

I'm getting different issues with installing the venv.

Running

remotes::install_github("rstudio/tensorflow")
tensorflow::install_tensorflow(method = 'virtualenv')

seemed to install just fine and returned

`Virtual environment 'r-tensorflow' removed.
Using Python: /home/ember/.pyenv/versions/3.10.13/bin/python3.10
Creating virtual environment 'r-tensorflow' ... 
+ /home/ember/.pyenv/versions/3.10.13/bin/python3.10 -m venv /home/ember/.virtualenvs/r-tensorflow
Done!
Installing packages: pip, wheel, setuptools
+ /home/ember/.virtualenvs/r-tensorflow/bin/python -m pip install --upgrade --no-user pip wheel setuptools
Requirement already satisfied: pip in /home/ember/.virtualenvs/r-tensorflow/lib/python3.10/site-packages (23.0.1)
Collecting pip
  Using cached pip-23.2.1-py3-none-any.whl (2.1 MB)
Collecting wheel
  Using cached wheel-0.41.2-py3-none-any.whl (64 kB)
Requirement already satisfied: setuptools in /home/ember/.virtualenvs/r-tensorflow/lib/python3.10/site-packages (65.5.0)
Collecting setuptools
  Using cached setuptools-68.1.2-py3-none-any.whl (805 kB)
Installing collected packages: wheel, setuptools, pip
  Attempting uninstall: setuptools
    Found existing installation: setuptools 65.5.0
    Uninstalling setuptools-65.5.0:
      Successfully uninstalled setuptools-65.5.0
  Attempting uninstall: pip
    Found existing installation: pip 23.0.1
    Uninstalling pip-23.0.1:
      Successfully uninstalled pip-23.0.1
Successfully installed pip-23.2.1 setuptools-68.1.2 wheel-0.41.2
Virtual environment 'r-tensorflow' successfully created.
Using virtual environment 'r-tensorflow' ...
+ /home/ember/.virtualenvs/r-tensorflow/bin/python -m pip install --upgrade --no-user 'tensorflow==2.13.*'
Collecting tensorflow==2.13.*
  Obtaining dependency information for tensorflow==2.13.* from https://files.pythonhosted.org/packages/5a/f2/5c2f878c62c8b79c629b11b33516bb55054d7677eba6f56f3a20296b56bd/tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Using cached tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.4 kB)
Collecting absl-py>=1.0.0 (from tensorflow==2.13.*)
  Using cached absl_py-1.4.0-py3-none-any.whl (126 kB)
Collecting astunparse>=1.6.0 (from tensorflow==2.13.*)
  Using cached astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting flatbuffers>=23.1.21 (from tensorflow==2.13.*)
  Obtaining dependency information for flatbuffers>=23.1.21 from https://files.pythonhosted.org/packages/6f/12/d5c79ee252793ffe845d58a913197bfa02ae9a0b5c9bc3dc4b58d477b9e7/flatbuffers-23.5.26-py2.py3-none-any.whl.metadata
  Using cached flatbuffers-23.5.26-py2.py3-none-any.whl.metadata (850 bytes)
Collecting gast<=0.4.0,>=0.2.1 (from tensorflow==2.13.*)
  Using cached gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting google-pasta>=0.1.1 (from tensorflow==2.13.*)
  Using cached google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting grpcio<2.0,>=1.24.3 (from tensorflow==2.13.*)
  Obtaining dependency information for grpcio<2.0,>=1.24.3 from https://files.pythonhosted.org/packages/28/fa/c38a010d3fffcac07ef121abb34eb2c3db0876df74267ce5bde13c3a6ed7/grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Using cached grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
Collecting h5py>=2.9.0 (from tensorflow==2.13.*)
  Obtaining dependency information for h5py>=2.9.0 from https://files.pythonhosted.org/packages/0d/7a/e55589e4093cca1934db5e99644c1c2424a9b3aac104b7f6176605a5eeb7/h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Using cached h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB)
Collecting keras<2.14,>=2.13.1 (from tensorflow==2.13.*)
  Obtaining dependency information for keras<2.14,>=2.13.1 from https://files.pythonhosted.org/packages/2e/f3/19da7511b45e80216cbbd9467137b2d28919c58ba1ccb971435cb631e470/keras-2.13.1-py3-none-any.whl.metadata
  Using cached keras-2.13.1-py3-none-any.whl.metadata (2.4 kB)
Collecting libclang>=13.0.0 (from tensorflow==2.13.*)
  Obtaining dependency information for libclang>=13.0.0 from https://files.pythonhosted.org/packages/ea/df/55525e489c43f9dbb6c8ea27d8a567b3dcd18a22f3c45483055f5ca6611d/libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl.metadata
  Using cached libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl.metadata (5.2 kB)
Collecting numpy<=1.24.3,>=1.22 (from tensorflow==2.13.*)
  Using cached numpy-1.24.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Collecting opt-einsum>=2.3.2 (from tensorflow==2.13.*)
  Using cached opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Collecting packaging (from tensorflow==2.13.*)
  Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 (from tensorflow==2.13.*)
  Obtaining dependency information for protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 from https://files.pythonhosted.org/packages/f3/d8/fb02c40aa129c385430d177e4d9fa0160cb89be29305c8760861e538a2e4/protobuf-4.24.2-cp37-abi3-manylinux2014_x86_64.whl.metadata
  Using cached protobuf-4.24.2-cp37-abi3-manylinux2014_x86_64.whl.metadata (540 bytes)
Requirement already satisfied: setuptools in /home/ember/.virtualenvs/r-tensorflow/lib/python3.10/site-packages (from tensorflow==2.13.*) (68.1.2)
Collecting six>=1.12.0 (from tensorflow==2.13.*)
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting tensorboard<2.14,>=2.13 (from tensorflow==2.13.*)
  Using cached tensorboard-2.13.0-py3-none-any.whl (5.6 MB)
Collecting tensorflow-estimator<2.14,>=2.13.0 (from tensorflow==2.13.*)
  Obtaining dependency information for tensorflow-estimator<2.14,>=2.13.0 from https://files.pythonhosted.org/packages/72/5c/c318268d96791c6222ad7df1651bbd1b2409139afeb6f468c0f327177016/tensorflow_estimator-2.13.0-py2.py3-none-any.whl.metadata
  Using cached tensorflow_estimator-2.13.0-py2.py3-none-any.whl.metadata (1.3 kB)
Collecting termcolor>=1.1.0 (from tensorflow==2.13.*)
  Using cached termcolor-2.3.0-py3-none-any.whl (6.9 kB)
Collecting typing-extensions<4.6.0,>=3.6.6 (from tensorflow==2.13.*)
  Using cached typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Collecting wrapt>=1.11.0 (from tensorflow==2.13.*)
  Using cached wrapt-1.15.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (78 kB)
Collecting tensorflow-io-gcs-filesystem>=0.23.1 (from tensorflow==2.13.*)
  Obtaining dependency information for tensorflow-io-gcs-filesystem>=0.23.1 from https://files.pythonhosted.org/packages/6a/48/3cdab86db01701ea043de445f3af1c3e539835781e57d803d2a87f256475/tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata
  Using cached tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (14 kB)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /home/ember/.virtualenvs/r-tensorflow/lib/python3.10/site-packages (from astunparse>=1.6.0->tensorflow==2.13.*) (0.41.2)
Collecting google-auth<3,>=1.6.3 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for google-auth<3,>=1.6.3 from https://files.pythonhosted.org/packages/9c/8d/bff87fc722553a5691d8514da5523c23547f3894189ba03b57592e37bdc2/google_auth-2.22.0-py2.py3-none-any.whl.metadata
  Using cached google_auth-2.22.0-py2.py3-none-any.whl.metadata (4.2 kB)
Collecting google-auth-oauthlib<1.1,>=0.5 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Using cached google_auth_oauthlib-1.0.0-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for markdown>=2.6.8 from https://files.pythonhosted.org/packages/1a/b5/228c1cdcfe138f1a8e01ab1b54284c8b83735476cb22b6ba251656ed13ad/Markdown-3.4.4-py3-none-any.whl.metadata
  Using cached Markdown-3.4.4-py3-none-any.whl.metadata (6.9 kB)
Collecting requests<3,>=2.21.0 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for requests<3,>=2.21.0 from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
  Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting tensorboard-data-server<0.8.0,>=0.7.0 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for tensorboard-data-server<0.8.0,>=0.7.0 from https://files.pythonhosted.org/packages/02/52/fb9e51fba47951aabd7a6b25e41d73eae94208ccf62d886168096941a781/tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata
  Using cached tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (1.1 kB)
Collecting werkzeug>=1.0.1 (from tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for werkzeug>=1.0.1 from https://files.pythonhosted.org/packages/9b/59/a7c32e3d8d0e546a206e0552a2c04444544f15c1da4a01df8938d20c6ffc/werkzeug-2.3.7-py3-none-any.whl.metadata
  Using cached werkzeug-2.3.7-py3-none-any.whl.metadata (4.1 kB)
Collecting cachetools<6.0,>=2.0.0 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for cachetools<6.0,>=2.0.0 from https://files.pythonhosted.org/packages/a9/c9/c8a7710f2cedcb1db9224fdd4d8307c9e48cbddc46c18b515fefc0f1abbe/cachetools-5.3.1-py3-none-any.whl.metadata
  Using cached cachetools-5.3.1-py3-none-any.whl.metadata (5.2 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for urllib3<2.0 from https://files.pythonhosted.org/packages/c5/05/c214b32d21c0b465506f95c4f28ccbcba15022e000b043b72b3df7728471/urllib3-1.26.16-py2.py3-none-any.whl.metadata
  Using cached urllib3-1.26.16-py2.py3-none-any.whl.metadata (48 kB)
Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting charset-normalizer<4,>=2 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/a4/65/057bf29660aae6ade0816457f8db4e749e5c0bfa2366eb5f67db9912fa4c/charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Using cached charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17 (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
  Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting MarkupSafe>=2.1.1 (from werkzeug>=1.0.1->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Obtaining dependency information for MarkupSafe>=2.1.1 from https://files.pythonhosted.org/packages/12/b3/d9ed2c0971e1435b8a62354b18d3060b66c8cb1d368399ec0b9baa7c0ee5/MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Using cached MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting pyasn1<0.6.0,>=0.4.6 (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow==2.13.*)
  Using cached oauthlib-3.2.2-py3-none-any.whl (151 kB)
Using cached tensorflow-2.13.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (524.1 MB)
Using cached flatbuffers-23.5.26-py2.py3-none-any.whl (26 kB)
Using cached grpcio-1.57.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
Using cached h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)
Using cached keras-2.13.1-py3-none-any.whl (1.7 MB)
Using cached libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl (22.9 MB)
Using cached protobuf-4.24.2-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
Using cached tensorflow_estimator-2.13.0-py2.py3-none-any.whl (440 kB)
Using cached tensorflow_io_gcs_filesystem-0.33.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.4 MB)
Using cached google_auth-2.22.0-py2.py3-none-any.whl (181 kB)
Using cached Markdown-3.4.4-py3-none-any.whl (94 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached tensorboard_data_server-0.7.1-py3-none-manylinux2014_x86_64.whl (6.6 MB)
Using cached werkzeug-2.3.7-py3-none-any.whl (242 kB)
Using cached cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (201 kB)
Using cached MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Installing collected packages: libclang, flatbuffers, wrapt, urllib3, typing-extensions, termcolor, tensorflow-io-gcs-filesystem, tensorflow-estimator, tensorboard-data-server, six, pyasn1, protobuf, packaging, oauthlib, numpy, MarkupSafe, markdown, keras, idna, grpcio, gast, charset-normalizer, certifi, cachetools, absl-py, werkzeug, rsa, requests, pyasn1-modules, opt-einsum, h5py, google-pasta, astunparse, requests-oauthlib, google-auth, google-auth-oauthlib, tensorboard, tensorflow
Successfully installed MarkupSafe-2.1.3 absl-py-1.4.0 astunparse-1.6.3 cachetools-5.3.1 certifi-2023.7.22 charset-normalizer-3.2.0 flatbuffers-23.5.26 gast-0.4.0 google-auth-2.22.0 google-auth-oauthlib-1.0.0 google-pasta-0.2.0 grpcio-1.57.0 h5py-3.9.0 idna-3.4 keras-2.13.1 libclang-16.0.6 markdown-3.4.4 numpy-1.24.3 oauthlib-3.2.2 opt-einsum-3.3.0 packaging-23.1 protobuf-4.24.2 pyasn1-0.5.0 pyasn1-modules-0.3.0 requests-2.31.0 requests-oauthlib-1.3.1 rsa-4.9 six-1.16.0 tensorboard-2.13.0 tensorboard-data-server-0.7.1 tensorflow-2.13.0 tensorflow-estimator-2.13.0 tensorflow-io-gcs-filesystem-0.33.0 termcolor-2.3.0 typing-extensions-4.5.0 urllib3-1.26.16 werkzeug-2.3.7 wrapt-1.15.0

Installation complete.

But when testing it for attached devices

tensorflow::tf$config$list_physical_devices()

I now get

Error: Valid installation of TensorFlow not found.

Python environments searched for 'tensorflow' package:
 /home/ember/.pyenv/versions/3.10.13/bin/python3.10

Python exception encountered:
 Traceback (most recent call last):
  File "/home/ember/R/x86_64-pc-linux-gnu-library/4.1/reticulate/python/rpytools/loader.py", line 119, in _find_and_load_hook
    return _run_hook(name, _hook)
  File "/home/ember/R/x86_64-pc-linux-gnu-library/4.1/reticulate/python/rpytools/loader.py", line 93, in _run_hook
    module = hook()
  File "/home/ember/R/x86_64-pc-linux-gnu-library/4.1/reticulate/python/rpytools/loader.py", line 117, in _hook
    return _find_and_load(name, import_)
  File "/home/ember/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/tensorflow/__init__.py", line 38, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/home/ember/R/x86_64-pc-linux-gnu-library/4.1/reticulate/python/rpytools/loader.py", line 119, in _find_and_load_hook
    return

Answer 12 · 2023-09-04T13:43:16.000Z

Resolved that issue, hadn't installed python properly for use with reticulate and tensorflow.

But even with that resolved, using

remotes::install_github("rstudio/tensorflow")
tensorflow::install_tensorflow(method = 'virtualenv')

tensorflow::tf$config$list_physical_devices()

still didn't give access to the gpu

2023-09-04 23:39:02.113896: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-09-04 23:39:02.695433: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2023-09-04 23:39:03.520732: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2023-09-04 23:39:03.541021: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1960] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
[[1]]
PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')

Answer 13 · 2023-09-05T12:48:12.000Z

Is cuda 11.8 installed on your machine?

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda-toolkit-11-8

Note with the upcoming 2.14 release, installing cuda system wide will no longer be necessary.

Answer 14 · 2023-09-05T14:31:02.000Z

Yeah CUDA 11.8 with cuDNN 8.6.
I installed them on both the windows and on wsl.
Oooh excited to hear about the 2.14 release!

I was able to detect the GPU with the Conda environment, just not within R using:

tensorflow::use_condaenv(condaenv = "tf")

I was hoping to find a way to get that to work. But it may be worth the wait for 2.14, looks to be really soon.

Answer 15 · 2023-09-05T15:17:19.000Z

The upstream 2.14 release should be out tomorrow: https://discuss.tensorflow.org/t/upcoming-tensorflow-2-14-branch-cut-release-dates/18220

You can try the 2.14 release candidate today with:

reticulate::virtualenv_create("r-tensorflow", force = TRUE, version = ">=3.9,<3.11",
                              packages = 'tensorflow[and-cuda]==2.14.0rc1')
# restart R session
tensorflow::tf_config() # should indicate TF version 2.14

Answer 16 · 2023-09-29T14:48:50.000Z

With the current CRAN release install_tensorflow() now install TF v2.14