anaconda/nb_conda_kernels

Compatibility with Mamba & Micromamba

epassaro opened this issue · 9 comments

epassaro commented

Hi! I noticed this extension does not work with micromamba.

My workflow consists of defining alias conda=micromamba and suits almost all my needs.

After looking at manager.py I noticed that the output of micromamba info --json is quite different than the expected one (a lot of different and missing keys).

So my questions are:

  1. Is there a known workaround to make this extension work with micromamba?
  2. Should I write a patch for this?

Thanks

ken-lauer commented

I have a proof-of-concept here that seems to be working in my deployment, at least: https://github.com/ken-lauer/nb_conda_kernels/tree/enh_micromamba
I expect there would be a fair amount more work to make it acceptable to the maintainers - were micromamba support a thing they would even consider, that is.

mcg1969 commented

You might be surprised to hear this but I have no problem with the idea of making this work with micromamba. But I do think it is essential that it be well tested.

  • 👍2
AniAggarwal commented

Haven't done much development on neither conda nor mamba but I'm happy to help test or contribute; lmk if you think there is something I can help with.

mcg1969 commented

Here's what I think would be required.

  • In testbed/build.sh, you would need to make sure that the root conda environment has mamba in it.
  • The code would need to be modified, likely following the pattern @ken-lauer has already done, but with the ability to configure whether conda or mamba is preferred. Obviously, if only one is installed, fine, go with that; but if both are installed, it would need to be selectable—if for no other reason than to facilitate testing.
  • The tests in the tests section would need to be modified to be able to run either with conda or with mamba, selectably.
  • 👍1
mcg1969 commented

Keep in mind that nb_conda_kernels deliberately does not assume that it lives in the same environment as conda; instead, it relies on calling it via a subprocess. This assumption needs to be preserved for mamba as well. That way, you can run Jupyter (lab|notebook) from a child environment and it still works properly.

genghisun commented

Any progress?

I tried @ken-lauer 's repo but it didn't work, I don't know if I'm doing something wrong. Here are my steps:

# in micromamba env with jupyter
> git clone https://github.com/ken-lauer/nb_conda_kernels.git
> cd nb_conda_kernels/
> git checkout enh_micromamba
> pip install -e .
> python -m nb_conda_kernels.install --enable
Enabling nb_conda_kernels...
CONDA_PREFIX: /path/to/micromamba/envs/jupyter
Status: enabled
> jupyter lab
[E 2024-05-16 14:33:09.968 ServerApp] [nb_conda_kernels] couldn't call conda:
    [Errno 2] No such file or directory: 'conda'
[I 2024-05-16 14:33:10.004 ServerApp] [nb_conda_kernels] enabled, 0 kernels found
# and the kernels of other micromamba environments doesn't show up in jupyter lab webui

> python -m nb_conda_kernels list
[ListKernelSpecs] ERROR | [nb_conda_kernels] couldn't call conda:
[Errno 2] No such file or directory: 'conda'
[ListKernelSpecs] [nb_conda_kernels] enabled, 0 kernels found
Available kernels:
  python3    /path/to/micromamba/envs/jupyter/share/jupyter/kernels/python3
> jupyter kernelspec list
Available kernels:
  python3    /path/to/micromamba/envs/jupyter/share/jupyter/kernels/python3
mcg1969 commented

I am not seeing any work being done on this. External contribution will be required. If nb_conda_kernels is essential for you, you'll need to use conda (with the libmamba solver, of course)!

ken-lauer commented

@genghisun I have conda aliased to micromamba for users on our machines as they're used to typing it. The branch linked above, for better or worse, assumes the same is true. A soft-link in your PATH of conda -> micromamba or just having the binary named conda should work.

I'm well aware that the above wouldn't be acceptable for inclusion into anaconda/nb_conda_kernels - but it's still for my own purposes at the moment. It is working well enough on my HPC cluster but is untested elsewhere.

  • 🚀1
genghisun commented

@ken-lauer Thanks! I made it after ln -s micromamba .local/bin/conda

  • 👍1