loosening google-cloud-* package constraints for TFX 1.13
jeongukjae opened this issue · 10 comments
- Have I specified the code to reproduce the issue (Yes, No):
- Environment in which the code is executed (e.g., Local(Linux/MacOS/Windows),
Interactive Notebook, Google Cloud, etc): - TensorFlow version: 2.12
- TFX Version: 1.13
- Python version: 3.9
- Python dependencies (from
pip freezeoutput):
Describe the current behavior
https://github.com/tensorflow/tensorflow/blob/r2.12/tensorflow/tools/pip_package/setup.py#L100-L107
TF 2.12 can accept protobuf>=3.20.3,
but the TFX comment says
https://github.com/tensorflow/tfx/blob/r1.13.0/tfx/dependencies.py#L83-L90
So, it seems reasonable to loosen google-cloud-* and google-api-core package constraints to use more updated versions.
Can you try to update TFX to 1.14.0 release? google-cloud-* package constraints have been updated to support updated package versions. Thank you!
@singhniraj08 Sadly, I'm using TFX with Vertex AI, so cannot update it. 1.13 release is the latest TFX version that Vertex AI supports.
https://cloud.google.com/vertex-ai/docs/supported-frameworks-list#supported-tfx-sdk
Updating google-cloud-* package constraints were planned to be done after TF 2.12 release but not sure why it couldn't make to TFX 1.13 release. I tried to checkout the 1.13 release in my local and bumped up the google-cloud-* package constraints but could install it because of dependency errors.
I guess you have to wait till TFX 1.14 support is introduced in Vertex AI to use the updated google-cloud packages. Or can you raise a feature request for TFX 1.14 SDK on Vertex AI side? Meanwhile I will try to check internally, if this is something which can be done on our end.
@jeongukjae @singhniraj08 Hi, I raised a feature request with GCP support case for below context.
As you can see in r1.13.0/tfx/dependencies.py#L89, the google-cloud-aiplatform>=1.6.2,<1.18 constraint makes it difficult to utilize the latest features of aiplatform in TFX 1.13.0.
In TFX 1.14.0, this constraint is loosened to google-cloud-aiplatform>=1.6.2,<2, but currently Vertex AI supports TFX versions up to 1.13.0.
I found this #6375 issue. As mentioned in the issue, there are conflicting version requirements for TFX and TF, so I think it can be solved together if TFX 1.14.0 is supported.
@singhniraj08 It looks like it's possible to loosen them, and I tested it. Successfully installed them, so I wrote the PR for this issue: #6384
Can you review it and consider the TFX 1.13.1 release with that?
@jeongukjae, Let me check internally if this can make a TFX 1.13.1 release. Thank you for the contributions.
Sorry for my late response. But looks like the docsite itself is just outdated. Does the VAI genuinely not support TFX 1.14.0? I tried the tutorial, and looks like it works with the latest version TFX v1.14.0.
I appreciate if you confirm from your side again. If it is okay, then I will update the doc accordingly.
Thanks for your consideration.
@lego0901 I asked it to GCP Support Team. I heard they are going to support 1.14.0 officially soon and internal test is ongoing for now.
Sorry for my too-late response.. Just updating the branch does not impact to the SDK release or docker images, so I just recompiled and reuploaded the newly-built docker image (from v1.14.0) to the dockerhub with "latest" tag. (i.e., tensorflow/tfx:latest) Thank you.