unable to interrupt cell execution with a kernel from an external environment

[evfro]

Description

This is a cross-issue, originally reported at jupyterlab/jupyterlab#8388. The steps to reproduce are also fully described there.

In short, when nb_conda_kernels is enabled, it's not possible to interrupt execution of cells from within the JupyterLab's interface if the kernel is not from the same env where JupyterLab is installed.

I'm duplicating it here after a short discussion in #155. In addition to it, below is log from terminal, corresponding to the steps that reproduce the issue.

Logs

Launching external ipython kernel from the dropdown menu activated by nb_conda_kernels:

[I 10:16:56.868 LabApp] Kernel started: b7b29d3f-b1f0-4121-8428-3ea5723c832f, name: conda-env-no_jlab-py
Active code page: 65001
CONDA_PREFIX=C:\Users\user\Anaconda3\envs\jupyterlab
[I 10:17:55.156 LabApp] Kernel restarted: b7b29d3f-b1f0-4121-8428-3ea5723c832f
Active code page: 65001
CONDA_PREFIX=C:\Users\user\Anaconda3\envs\jupyterlab

Executing a for loop in a cell and trying to interrupt it by hitting stop button 2 times:

[I 10:18:06.940 LabApp] Kernel interrupted: b7b29d3f-b1f0-4121-8428-3ea5723c832f
[I 10:18:07.794 LabApp] Kernel interrupted: b7b29d3f-b1f0-4121-8428-3ea5723c832f

It had no effect, the loop continued to run and got fully exhausted with no interruption, no KeyboardInterrupt message appeared.

Now switching to the original kernel the JupyterLab is from:

[I 10:18:54.722 LabApp] Kernel shutdown: b7b29d3f-b1f0-4121-8428-3ea5723c832f
[I 10:18:54.976 LabApp] Kernel started: d43ff21e-ae02-4cf0-b6ba-c72be23b4cfc, name: conda-env-jupyterlab-py

Again executing the for loop and hitting the stop button right after it started:

[I 10:19:01.181 LabApp] Kernel interrupted: d43ff21e-ae02-4cf0-b6ba-c72be23b4cfc

This time cell execution interrupts immediately as expected. I can see the KeyboardInterrupt message.

Analysis

There seem to be a discrepancy between conda prefix and actually used kernel, which is from a different environment. Not sure if it matters, though.

[evfro]

Thanks to the deep analysis performed by Kevin Bates in jupyter/notebook#5985, it was identified that the problem is likely to stem from the nb_conda_kernel.runner script, which doesn't ensure process inheritance

leading to both interrupt_events and signals not flowing to the "child" process

I hope, this information will help to make progress on fixing the issue.

[kevin-bates]

Thanks to the deep analysis

Let's call it more of a hunch.

[mcg1969]

I know it has been three years but I thought I'd try this now, with JupyterLab 4.1.2, and I am able to interrupt a kernel running in another conda environment.

[mcg1969]

I will close this as stale but if we can get a reproducible test case we can re-open