youtype/mypy_boto3_builder

Large speed regression in pip install

rpmcginty opened this issue · 6 comments

rpmcginty commented

Describe the bug
We are seeing a sudden slowdown when installing boto3-stubs[...] using pip with python 3.8 from around Jan 11, 2023.

Though we pin the boto3-stubs to a specific version, all of the extras that are specified (essential, ssm, stepfunctions etc.) are bound only by whatever is specified in boto3-stubs release. Because constraints for the various service-specific generated packages (mypy_boto_*) are fairly relaxed, pip relies on backtracking to resolve the most suitable versions for each package to make everyone happy (see this). This has been the case always.

Luckily we have comparative logs from gh action builds capturing this slowdown. Between the two logs, nothing in our code base or other dependencies change.

First Workflow (FAST)

INFO: pip is looking at multiple versions of mypy-extensions to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of mypy-boto3-sts to determine which version is compatible with other requirements. This could take a while.
Collecting mypy-boto3-sts>=1.20.0
  Downloading mypy_boto3_sts-1.26.0.post1-py3-none-any.whl (18 kB)
INFO: pip is looking at multiple versions of mypy-boto3-stepfunctions to determine which version is compatible with other requirements. This could take a while.
Collecting mypy-boto3-stepfunctions>=1.20.0
  Downloading mypy_boto3_stepfunctions-1.26.13.post16-py3-none-any.whl (29 kB)

Second Workflow (SLOW)

INFO: pip is looking at multiple versions of mypy-extensions to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of mypy-boto3-sts to determine which version is compatible with other requirements. This could take a while.
Collecting mypy-boto3-sts>=1.20.0
  Downloading mypy_boto3_sts-1.26.0.post1-py3-none-any.whl (18 kB)
  Downloading mypy_boto3_sts-1.26.0-py3-none-any.whl (18 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
  Downloading mypy_boto3_sts-1.25.0-py3-none-any.whl (18 kB)
  Downloading mypy_boto3_sts-1.24.36.post1-py3-none-any.whl (18 kB)
  Downloading mypy_boto3_sts-1.24.0-py3-none-any.whl (17 kB)
  Downloading mypy_boto3_sts-1.23.1-py3-none-any.whl (17 kB)
  Downloading mypy_boto3_sts-1.23.0.post1-py3-none-any.whl (17 kB)
INFO: pip is looking at multiple versions of mypy-boto3-sts to determine which version is compatible with other requirements. This could take a while.
  Downloading mypy_boto3_sts-1.23.0-py3-none-any.whl (17 kB)
  Downloading mypy_boto3_sts-1.22.8-py3-none-any.whl (17 kB)
  Downloading mypy_boto3_sts-1.22.0.post1-py3-none-any.whl (17 kB)
  Downloading mypy_boto3_sts-1.22.0-py3-none-any.whl (17 kB)
  Downloading mypy_boto3_sts-1.21.34-py3-none-any.whl (17 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
  Downloading mypy_boto3_sts-1.21.31-py3-none-any.whl (17 kB)
  Downloading mypy_boto3_sts-1.21.30-py3-none-any.whl (17 kB)
  Downloading mypy_boto3_sts-1.21.27-py3-none-any.whl (17 kB)
  Downloading mypy_boto3_sts-1.21.23.post1-py3-none-any.whl (17 kB)
  Downloading mypy_boto3_sts-1.21.23-py3-none-any.whl (17 kB)
  Downloading mypy_boto3_sts-1.21.13-py3-none-any.whl (17 kB)
  Downloading mypy_boto3_sts-1.21.0-py3-none-any.whl (17 kB)
INFO: pip is looking at multiple versions of mypy-boto3-stepfunctions to determine which version is compatible with other requirements. This could take a while.
Collecting mypy-boto3-stepfunctions>=1.20.0
  Downloading mypy_boto3_stepfunctions-1.26.13.post16-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post15-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post14-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post13-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post12-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post11-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post10-py3-none-any.whl (29 kB)
INFO: pip is looking at multiple versions of mypy-boto3-stepfunctions to determine which version is compatible with other requirements. This could take a while.
  Downloading mypy_boto3_stepfunctions-1.26.13.post9-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post8-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post7-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post6-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post5-py3-none-any.whl (29 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
  Downloading mypy_boto3_stepfunctions-1.26.13.post4-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post3-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post2-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13.post1-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.13-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.0.post1-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.26.0-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.25.0-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.24.36.post1-py3-none-any.whl (29 kB)
  Downloading mypy_boto3_stepfunctions-1.24.0-py3-none-any.whl (28 kB)
  Downloading mypy_boto3_stepfunctions-1.23.0.post1-py3-none-any.whl (28 kB)
  Downloading mypy_boto3_stepfunctions-1.23.0-py3-none-any.whl (28 kB)
  Downloading mypy_boto3_stepfunctions-1.22.8-py3-none-any.whl (28 kB)
  Downloading mypy_boto3_stepfunctions-1.22.0.post1-py3-none-any.whl (28 kB)
  Downloading mypy_boto3_stepfunctions-1.22.0-py3-none-any.whl (28 kB)
  Downloading mypy_boto3_stepfunctions-1.21.34-py3-none-any.whl (28 kB)
  Downloading mypy_boto3_stepfunctions-1.21.31-py3-none-any.whl (28 kB)
  Downloading mypy_boto3_stepfunctions-1.21.30-py3-none-any.whl (28 kB)
  Downloading mypy_boto3_stepfunctions-1.21.27-py3-none-any.whl (27 kB)
  Downloading mypy_boto3_stepfunctions-1.21.23.post1-py3-none-any.whl (27 kB)
  Downloading mypy_boto3_stepfunctions-1.21.23-py3-none-any.whl (26 kB)
  Downloading mypy_boto3_stepfunctions-1.21.0-py3-none-any.whl (27 kB)
  Downloading mypy_boto3_stepfunctions-1.20.49-py3-none-any.whl (26 kB)

It seems that now the available releases for previous versions are all available for download for the pip dependency resolver.

To Reproduce
I've scoped down one of our packages to show the problem

  1. setup
mkdir /tmp/mypy_boto3_issues
cd /tmp/mypy_boto3_issues
# Use python 3.9 / 3.8
python -m venv .venv && source .venv/bin/activate

cat <<EOF > setup.cfg
[metadata]
name = mypy_boto3_issues
version = 0.0.1

[options]
platforms = any
include_package_data = True
python_requires = >=3.8
setup_requires = 
    pytest-runner

install_requires =
    boto3~=1.26.49
    marshmallow==3.14.1
    dataclasses-json==0.5.7
    pydantic==1.9.0
    pytz
    pyyaml==6.0
    typing_inspect
    requests

[options.extras_require]
dev = 
    black==22.3
    boto3-stubs[boto3,athena,apigateway,batch,ecr,ecs,efs,essential,sns,ssm,sts,stepfunctions]
    coverage[toml]~=7.0.4
    flake8==4.0.1
    isort==5.10
    moto[all]
    mypy~=0.960
    pytest==6.2.5
    pytest-cov
    tox>=3.4.0

EOF

cat <<EOF > setup.py
import setuptools

setuptools.setup()

EOF
  1. install
pip install ".[dev]" --force-reinstall --no-cache-dir
rpmcginty commented

It should be noted that this happens even if you pin the version of boto3-stubs[...] == 1.xx.

The runtime seems to be inconsistent between identical installs.

rpmcginty commented

@vemel sorry to target you, but do you have any ideas for why it is taking so long to install? I was wondering if this could be somehow related to this release: https://github.com/youtype/mypy_boto3_builder/releases/tag/7.12.0 (although even previous versions of boto3 like 1.20.23 are also affected by this)

vemel commented

Thank you for the report!

No, this is not related to the latest release, but it is for sure related to builder 6.x+, because I no longer rebuild services that were not changed in the current boto3/botocore release. I will take a look if this can be optimized somehow.

vemel commented

Unfortunately, I could not find a good solution to speed up boto3-stubs[boto3] installation. However, if you synchronize boto3-stubs and boto3 version manually, pip installs packages way faster. So, consider removing boto3 extra from boto3-stubs.

  • 👍1
vemel commented

You might want to remove boto3 extra from boto3-stubs dev dependency. THis might be the reason pip is slow.

rpmcginty commented

You might want to remove boto3 extra from boto3-stubs dev dependency. THis might be the reason pip is slow.

This suggestion worked! thanks for looking into it. You might want to make a note about this in documentation if others run into this issue as well.