cloudfoundry-community/cf-python-client

Cannot push vendorized 1.16.0 version

51acorsi opened this issue · 14 comments

Hello,

We are getting the following error while trying to cf push a vendored tar.gz file of the 1.16.0 version:

          Processing ./vendor/cloudfoundry-client-1.16.0.tar.gz
              Preparing wheel metadata: started
              Preparing wheel metadata: finished with status 'error'
              ERROR: Command errored out with exit status 1:
               command: /tmp/contents051346007/deps/0/bin/python /tmp/contents051346007/deps/0/python/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpn3bbyhzm
                   cwd: /tmp/pip-install-m0fzbeh1/cloudfoundry-client
              Complete output (13 lines):
              running dist_info
              creating /tmp/pip-modern-metadata-u1vp_m5i/cloudfoundry_client.egg-info
              writing /tmp/pip-modern-metadata-u1vp_m5i/cloudfoundry_client.egg-info/PKG-INFO
              writing dependency_links to /tmp/pip-modern-metadata-u1vp_m5i/cloudfoundry_client.egg-info/dependency_links.txt
              writing entry points to /tmp/pip-modern-metadata-u1vp_m5i/cloudfoundry_client.egg-info/entry_points.txt
              writing requirements to /tmp/pip-modern-metadata-u1vp_m5i/cloudfoundry_client.egg-info/requires.txt
              writing top-level names to /tmp/pip-modern-metadata-u1vp_m5i/cloudfoundry_client.egg-info/top_level.txt
              writing manifest file '/tmp/pip-modern-metadata-u1vp_m5i/cloudfoundry_client.egg-info/SOURCES.txt'
              reading manifest file '/tmp/pip-modern-metadata-u1vp_m5i/cloudfoundry_client.egg-info/SOURCES.txt'
              reading manifest template 'MANIFEST.in'
              writing manifest file '/tmp/pip-modern-metadata-u1vp_m5i/cloudfoundry_client.egg-info/SOURCES.txt'
              creating '/tmp/pip-modern-metadata-u1vp_m5i/cloudfoundry_client.dist-info'
              error: invalid command 'bdist_wheel'
              ----------------------------------------
          ERROR: Command errored out with exit status 1: /tmp/contents051346007/deps/0/bin/python /tmp/contents051346007/deps/0/python/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpn3bbyhzm Check the logs for full command output.

Version 1.15.0 would work fine with the same instructions.

Steps to reproduce:

  • Add cloudfoundry-client==1.16.0 as dependency
  • Vendorize with pip download -r requirements.txt --no-binary=:none: -d vendor
  • Try to cf push and the error should appear

Further information:
Buildpack: Python Buildpack version 1.7.29

K0nne commented

Hello!

I use the same steps and can reproduce this.

@51acorsi by "Version 1.15.0 would work fine with the same instructions.", you mean that previous version 1.15.0 worked fine?

Yes, I can confirm that both versions v1.14.1 and v1.15.0 works fine with the very same above instructions.

Update: I have found a workaround by using wheel on the tar.gz, which is what I believe CF is attempting to do but fails since the buildpack does not have wheel installed.

This is the command that must be prompted before the cf push:

pip wheel ./vendor/cloudfoundry-client-1.16.0.tar.gz -w ./vendor

It will create the whl file for the library inside the vendor folder, which will be the one used by the buildpack.

K0nne commented

Thanks for the workaround! I will try this tomorrow.

Loooking at the diff between 1.15.0 1.16.0, despite all code formatting, I spotted the diff in requirements.txt

-aiohttp==3.6.2
+aiohttp==3.7.4
 protobuf==3.6.1
 oauth2-client==1.2.1
 websocket-client==0.54.0
-PyYAML==5.3.1
+PyYAML==5.4.1
+requests~=2.25.1
+setuptools~=49.2.1
+polling2==0.4.6

I am not sure I can have access to a valid cloudfoundry instance currently.
Looking at some solution around the net for the error message advise to upgrade bwheel or
I guess that putting setuptools~=49.2.1 is not a good idea as it is not a dependency of the library. I've just pushed a beta version of the 1.1.16 that is available: 1.16.1b0. It is the version corresponding to the changes in the pull request.

Other option will be to add setup_requires=['wheel'] in setup.py if it does not work.

I got the same issue with the beta version:

Processing ./vendor/cloudfoundry-client-1.16.1b0.tar.gz
              Preparing wheel metadata: started
              Preparing wheel metadata: finished with status 'error'
              ERROR: Command errored out with exit status 1:
               command: /tmp/contents450101200/deps/0/bin/python /tmp/contents450101200/deps/0/python/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp_96nmzcf
                   cwd: /tmp/pip-install-b8fqg_jx/cloudfoundry-client
              Complete output (13 lines):
              running dist_info
              creating /tmp/pip-modern-metadata-sa51qv2c/cloudfoundry_client.egg-info
              writing /tmp/pip-modern-metadata-sa51qv2c/cloudfoundry_client.egg-info/PKG-INFO
              writing dependency_links to /tmp/pip-modern-metadata-sa51qv2c/cloudfoundry_client.egg-info/dependency_links.txt
              writing entry points to /tmp/pip-modern-metadata-sa51qv2c/cloudfoundry_client.egg-info/entry_points.txt
              writing requirements to /tmp/pip-modern-metadata-sa51qv2c/cloudfoundry_client.egg-info/requires.txt
              writing top-level names to /tmp/pip-modern-metadata-sa51qv2c/cloudfoundry_client.egg-info/top_level.txt
              writing manifest file '/tmp/pip-modern-metadata-sa51qv2c/cloudfoundry_client.egg-info/SOURCES.txt'
              reading manifest file '/tmp/pip-modern-metadata-sa51qv2c/cloudfoundry_client.egg-info/SOURCES.txt'
              reading manifest template 'MANIFEST.in'
              writing manifest file '/tmp/pip-modern-metadata-sa51qv2c/cloudfoundry_client.egg-info/SOURCES.txt'
              creating '/tmp/pip-modern-metadata-sa51qv2c/cloudfoundry_client.dist-info'
              error: invalid command 'bdist_wheel'
              ----------------------------------------
          ERROR: Command errored out with exit status 1: /tmp/contents450101200/deps/0/bin/python /tmp/contents450101200/deps/0/python/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp_96nmzcf Check the logs for full command output.

Is it possible to provide not only the tar.gz but also the whl file in pypi? I figure that would fix the issue.

Other option will be to add setup_requires=['wheel'] in setup.py if it does not work.

I tested and this also doesn't work. It looks like CF ignores setup.py when you have a vendor folder.

@51acorsi just released a 1.16.1b1with both sdist and bdist_wheel

@antechrestos it worked now.
The download command now added the .whl file instead of the tar.gz and the push went on smoothly.
If that is a possible solution, adding the additional distribution will make it work.

Thank you

@antechrestos Do you plan to make both dist as official or should we try another solution?

@51acorsi sorry for the late reply : yeah if that solves the issue. I have not much time to find the reason of this sudden problem 😩

I am fine with the dist solution. You may close the issue after it. Will you change it for the current version or create a new patch?

I have

  • released 1.16.0 with bdist_wheel
  • made a patch 1.16.1 (removing the setuptoolsdependency) with both sdistand bdist_wheel
    I will let you close the issue once you see that one of the released package is convenient .

Have a nice weekend.

Issue is solved from version 1.16.0, where you can already see the whl distribution. Closing the issue.