Binder environment solve failing
kafitzgerald opened this issue · 2 comments
kafitzgerald commented
First, thanks so much for all of the great resources you've put together!
In trying to launch the tutorial on the cloud (using the launch binder link), I noticed the environment creation on Binder seems to be failing (see details below). It looks like the [file for the specific version of the] package of interest is marked broken on conda-forge.
Waiting for build to start...
Picked Git content provider.
Cloning into '/tmp/repo2dockerw2qrr8o4'...
HEAD is now at 4e29164 [pre-commit.ci] pre-commit autoupdate (#253)
Building conda environment for python=3.11
Using CondaBuildPack builder
Step 1/50 : FROM docker.io/library/buildpack-deps:jammy
---> b6235acc32ce
Step 2/50 : ENV DEBIAN_FRONTEND=noninteractive
---> Using cache
---> 755cd1b1e980
Step 3/50 : RUN apt-get -qq update && apt-get -qq install --yes --no-install-recommends locales > /dev/null && apt-get -qq purge && apt-get -qq clean && rm -rf /var/lib/apt/lists/*
---> Using cache
---> de9f0e4a05cd
Step 4/50 : RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen && locale-gen
---> Using cache
---> 92f56f191f0c
Step 5/50 : ENV LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 LANGUAGE=en_US.UTF-8
---> Using cache
---> 183e604467e3
Step 6/50 : ENV SHELL=/bin/bash
---> Using cache
---> 29d870378c0e
Step 7/50 : ARG NB_USER
---> Using cache
---> 3294b005ebab
Step 8/50 : ARG NB_UID
---> Using cache
---> fb48a5f7c4f6
Step 9/50 : ENV USER=${NB_USER} HOME=/home/${NB_USER}
---> Using cache
---> 6f69538c7a7f
Step 10/50 : RUN groupadd --gid ${NB_UID} ${NB_USER} && useradd --comment "Default user" --create-home --gid ${NB_UID} --no-log-init --shell /bin/bash --uid ${NB_UID} ${NB_USER}
---> Using cache
---> eda0cac86361
Step 11/50 : RUN apt-get -qq update && apt-get -qq install --yes --no-install-recommends gettext-base less unzip > /dev/null && apt-get -qq purge && apt-get -qq clean && rm -rf /var/lib/apt/lists/*
---> Using cache
---> 1ff96132f912
Step 12/50 : EXPOSE 8888
---> Using cache
---> 5a4faedfdb99
Step 13/50 : ENV APP_BASE=/srv
---> Using cache
---> 99cded126c26
Step 14/50 : ENV CONDA_DIR=${APP_BASE}/conda
---> Using cache
---> 2cdab4c7820c
Step 15/50 : ENV NB_PYTHON_PREFIX=${CONDA_DIR}/envs/notebook
---> Using cache
---> 1fe020df2668
Step 16/50 : ENV NPM_DIR=${APP_BASE}/npm
---> Using cache
---> 860da8dad357
Step 17/50 : ENV NPM_CONFIG_GLOBALCONFIG=${NPM_DIR}/npmrc
---> Using cache
---> 98eea01df937
Step 18/50 : ENV NB_ENVIRONMENT_FILE=/tmp/env/environment.lock
---> Using cache
---> 8cbdc0f7937a
Step 19/50 : ENV MAMBA_ROOT_PREFIX=${CONDA_DIR}
---> Using cache
---> 73d9bc52ee1b
Step 20/50 : ENV MAMBA_EXE=${CONDA_DIR}/bin/mamba
---> Using cache
---> 02bf661ccfc9
Step 21/50 : ENV CONDA_PLATFORM=linux-64
---> Using cache
---> e84d65f49470
Step 22/50 : ENV KERNEL_PYTHON_PREFIX=${NB_PYTHON_PREFIX}
---> Using cache
---> f3920c4a7f2a
Step 23/50 : ENV PATH=${NB_PYTHON_PREFIX}/bin:${CONDA_DIR}/bin:${NPM_DIR}/bin:${PATH}
---> Using cache
---> b4e06e806525
Step 24/50 : COPY --chown=1000:1000 build_script_files/-2fopt-2fvenv-2flib-2fpython3-2e11-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2factivate-2dconda-2esh-e70a7b /etc/profile.d/activate-conda.sh
---> Using cache
---> 7de2973d9eb0
Step 25/50 : COPY --chown=1000:1000 build_script_files/-2fopt-2fvenv-2flib-2fpython3-2e11-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2fenvironment-2epy-2d3-2e11-2dlinux-2d64-2elock-4045c1 /tmp/env/environment.lock
---> Using cache
---> d5e0338e1261
Step 26/50 : COPY --chown=1000:1000 build_script_files/-2fopt-2fvenv-2flib-2fpython3-2e11-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2finstall-2dbase-2denv-2ebash-6a6072 /tmp/install-base-env.bash
---> Using cache
---> 8b966e71692c
Step 27/50 : RUN TIMEFORMAT='time: %3R' bash -c 'time /tmp/install-base-env.bash' && rm -rf /tmp/install-base-env.bash /tmp/env
---> Using cache
---> c02f05a98b6e
Step 28/50 : RUN mkdir -p ${NPM_DIR} && chown -R ${NB_USER}:${NB_USER} ${NPM_DIR}
---> Using cache
---> c4693ba673c1
Step 29/50 : USER root
---> Using cache
---> 9ef8ec836ad9
Step 30/50 : ARG REPO_DIR=${HOME}
---> Using cache
---> 9bdf2484928b
Step 31/50 : ENV REPO_DIR=${REPO_DIR}
---> Using cache
---> d27859b6f602
Step 32/50 : RUN if [ ! -d "${REPO_DIR}" ]; then /usr/bin/install -o ${NB_USER} -g ${NB_USER} -d "${REPO_DIR}"; fi
---> Using cache
---> 1046374f9614
Step 33/50 : WORKDIR ${REPO_DIR}
---> Using cache
---> b45884b871ce
Step 34/50 : RUN chown ${NB_USER}:${NB_USER} ${REPO_DIR}
---> Using cache
---> b830b434c664
Step 35/50 : ENV PATH=${HOME}/.local/bin:${REPO_DIR}/.local/bin:${PATH}
---> Using cache
---> 74f4cdfb8892
Step 36/50 : ENV CONDA_DEFAULT_ENV=${KERNEL_PYTHON_PREFIX}
---> Using cache
---> 12e1a5c324a3
Step 37/50 : COPY --chown=1000:1000 src/.binder/environment.yml ${REPO_DIR}/.binder/environment.yml
---> Using cache
---> ef3ef735f463
Step 38/50 : USER ${NB_USER}
---> Using cache
---> 61bad9173e36
Step 39/50 : RUN TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file ".binder/environment.yml" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} '
---> Running in 292066c511c8
Channels:
- conda-forge
- defaults
Platform: linux-64
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed
PackagesNotFoundError: The following packages are not available from current channels:
- rdma-core==50.0=hd3aeb46_0
Current channels:
- https://conda.anaconda.org/conda-forge/linux-64
- https://repo.anaconda.com/pkgs/main/linux-64
- https://repo.anaconda.com/pkgs/r/linux-64
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
time: 60.367
---> Removed intermediate container 9d18d00af3eb
The command '/bin/sh -c TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_Pment.yml" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} '' retur
scottyhq commented
PackagesNotFoundError: The following packages are not available from current channels:
- rdma-core==50.0=hd3aeb46_0
Thanks for pointing this out @kafitzgerald. Looks like this rdma package is a transitive dependency of libarrow (from mamba repoquery whoneeds rdma-core)
Hopefully just bumping to more recent versions of pyarrow fixes this, until then it seems running via codespaces still works b/c we use a lockfile for that and so it isn't aware of packages being later labeled as "broken" https://github.com/xarray-contrib/xarray-tutorial?tab=readme-ov-file#github-codespaces
scottyhq commented
huh... turns out repo2docker (& therefore mybinder.org doesn't support python 3.12 yet)
jupyterhub/repo2docker#1225