Upgrade jupyterhub-singleuser custom image deps so it can build w/ Python 3.11
Closed this issue · 11 comments
After upgrading JupyterHub, we need to upgrade our custom docker image to incorporate the latest jupyterhub library. It appears that this dependency comes is as part of the jupyter/datascience-notebook base image we use. In #3350 I upgraded it from :python-3.9 to the latest :python-3.11.6 since the :python-3.9 tag hasn't been updated in over 2 years, but now the image fails to build:
=> [15/23] COPY ./pyproject.toml /reqs/pyproject.toml 0.2s
=> [16/23] COPY ./poetry.lock /reqs/poetry.lock 0.0s
=> [17/23] RUN poetry config virtualenvs.create false 0.7s
=> ERROR [18/23] RUN cd /reqs && poetry install --with=shared_utils --with=portfolio 0.8s
------
> [18/23] RUN cd /reqs && poetry install --with=shared_utils --with=portfolio:
0.600 The currently activated Python version 3.11.6 is not supported by the project (~3.9).
0.600 Trying to find and use a compatible version.
0.691
0.691 Poetry was unable to find a compatible version. If you have one, you can explicitly use it via the "env use" command.
------
Dockerfile:45
--------------------
43 | COPY ./poetry.lock /reqs/poetry.lock
44 | RUN poetry config virtualenvs.create false
45 | >>> RUN cd /reqs && poetry install --with=shared_utils --with=portfolio
46 | RUN poetry config virtualenvs.create true
47 |
--------------------
ERROR: failed to solve: process "/bin/bash -o pipefail -c cd /reqs && poetry install --with=shared_utils --with=portfolio" did not complete successfully: exit code: 1To reproduce locally, check out the latest main branch and:
cd images/jupyter-singleuser/Dockerfile
docker build .It appears that some poetry work will need to be done to upgrade dependencies to be compatible with Python 3.11
@themightychris have you tried updating the Python version in the pyproject.toml file? That might be enough to fix, but there may also be additional package version changes needed with the Python upgrade.
@mjumbewu good catch, after changing that line to python = "~3.11" I get:
@themightychris Ooh, looks like the calitp-map-utils package is pinned to Python 3.9.x. Will have to take a look at that.
Yep, it's pinned. That will have to be updated and pushed up to PyPI.
We'll run into the same issue for calitp-data-analysis.
@mjumbewu yep just spotted that, already on it :-)
Yea i'm testing out different combos - building it on my local mac using docker.
Right now i'm running into a dask 2022 not working with python 3.11. Also dask seems to be both in the calitp-data-analysis package as well as the base jupyter-single-user-image. I'm not sure why.
Tiffany says dask wasn't working when it was just in the image, "dependencies needed weren't getting installed / weren't passing tests," they needed to install it to the package
I propopse we remove it from the caltip-data-analysis package and just trying to have the requirements in the jupyter-single-user-image for now. The higher dask version. Then assuming we get this image working, can test out dask, and install the packages back in and test later.
OOf except calitp-data-analysis doesn't like that:
https://github.com/cal-itp/data-infra/actions/runs/9230746834/job/25399500299?pr=3359#step:6:176