Upgrade cuda and cudnn versions in Docker image?
deep-diver opened this issue · 9 comments
From TensorFlow 2.12, the recommended versions of cuda and cudnn are 11.8 and 8.6. However, it seems that the latest tfx docker image only supports 11.3 and 8.2. I think this legacy versions led some significant problems when I used KerasNLP as follows(in Trainer component):
Node: 'StatefulPartitionedCall'
RET_CHECK failure (tensorflow/compiler/xla/service/gpu/gpu_compiler.cc:618) dnn != nullptr
[[{{node StatefulPartitionedCall}}]] [Op:__inference_train_function_241897]
Any plan to update them? or any guides for people to update them?
@deep-diver, Thank you for brining this up. We will discuss this internally and update this thread.
@singhniraj08 Thank you! Really looking forward to the updates :)
We internally discussed this issue and this is pending because of Python 3.10 support which is in progress. The image will be upgraded with updated dependencies once python 3.10 support is introduced.
We will update this thread once python 3.10 support is introduced and CUDA, CUDNN dependencies are updated.
Thank you.
@singhniraj08 thanks for the updates
Can you share the ETA roughly?
TFX 1.14.0 has been released, and it is now supporting Python 3.10.
@briron thanks!
How about cuda depencies?
Latest TFX docker image supports CUDA 11.8 and CUDNN 8.9. Please try the latest docker image and let us know if you face any issues. Thank you!
This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.
This issue was closed due to lack of activity after being marked stale for past 7 days.