Python kernels in the system prefix without an absolute interpretor path fail to launch if started by `jupyter_client` in a virtual environment
joouha opened this issue · 4 comments
I've received several reports from users where a Python kernel fails to launch when started by a Jupyter client running in a virtual environment (e.g. joouha/euporie#25, joouha/euporie#75), and I've tracked down the cause to these lines of code in jupyter_client
.
Let's say I have ipykernel
installed in my system Python environment, and I want to run this kernel from a client (which uses jupyter_client
) installed in a virtual environment:
$ cat /usr/share/jupyter/kernels/python3/kernel.json
{
"argv": [
"python",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"display_name": "Python 3 (ipykernel)",
"language": "python",
"metadata": {
"debugger": true
}
}
$ python -m venv venv
$ ./venv/bin/pip install jupyter-console
$ ./venv/bin/pip uninstall ipykernel
$ ./venv/bin/jupyter-console --kernel=python3
/home/josiah/venv/bin/python: No module named ipykernel_launcher
jupyter_client
replaces the python
command in the kernelspec argv
line with the path to the interpreter on which jupyter_client
is running. This is implemented here:
I'm really not sure this is expected or desirable behaviour. There is a comment stating the following:
if the current process is in an env and has been launched by abspath without activating the env, python on PATH may not be sys.executable, but it should be.
but I don't think this assumption is correct, and I don't understand what problem was this originally implemented to solve.
This is also relevant:
When the ipykernel
wheel is built, the packaged kernelspec has just python
as the first argument of the kernel launch command.
If the user installs the kernel themselves (i.e. python -m ipykernel install --user
, the kernelspec is created with an absolute interpreter path.
You have to add envs manually like
{
"argv": [
"python",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"env":{
"PATH": PATH after activate venv.
},
"display_name": "Python 3.10 (ddip)",
"language": "python",
"metadata": {
"debugger": true
}
}
Thanks for your reply, but I think you have misunderstood this issue I described:
I'm writing a new Jupyter client which I want users to run from a virtual environment (using pipx
). My client does not depend on ipykernel
.
Typically the user will have ipykernel
(and a corresponding kernelspec) installed in their system prefix.
When the user tries to launch a kernel from the system kernelspec it fails, because jupyter_client
incorrectly edits the kernelspec, substituting the python
argv command with the absolute path of the interpreter in the client's venv.
Since ipykernel
is not installed in the pipx
venv, launching the kernel fails.
I think in the case of a non-absolute python
command, cmd[0]
should be set to sys._base_executable
, unless the client is running in the same prefix that the connection_file
is located, in which case cmd[0]
could reasonably be set to sys.executable
.