jupyter-server/kernel_gateway

Jupyter Notebook: 404 GET /api/kernels

mikechen66 opened this issue · 11 comments

Dear Sir or Madam:

I have a problem in running MNIST test code during running jupyter notebook. It shows the 404 GET /api/kernels. My dev environment is Ubuntu 16.04 LTS, Nvidia GeForce RTX 2060, tensorflor-gpu 13.1 with Anaconda. How can I solve the problem.

(tf-gpu) mike@mc:~$ jupyter notebook
[I 00:38:50.314 NotebookApp] Serving notebooks from local directory: /home/mike
[I 00:38:50.314 NotebookApp] The Jupyter Notebook is running at:
[I 00:38:50.314 NotebookApp] http://localhost:8888/?token=1bcc6ed52aee1e2b251fd6314c1ee501bccede316083d41a
[I 00:38:50.314 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 00:38:50.337 NotebookApp]

To access the notebook, open this file in a browser:
    file:///run/user/1000/jupyter/nbserver-7125-open.html
Or copy and paste one of these URLs:
    http://localhost:8888/?token=1bcc6ed52aee1e2b251fd6314c1ee501bccede316083d41a

[W 00:38:54.181 NotebookApp] 404 GET /api/kernels/8007b48e-6f37-45ce-91ed-7f30d4adc7ef/channels?session_id=ac64b3156ece4d858308fcf21dac720f (127.0.0.1): Kernel does not exist: 8007b48e-6f37-45ce-91ed-7f30d4adc7ef
[W 00:38:54.185 NotebookApp] 404 GET /api/kernels/8007b48e-6f37-45ce-91ed-7f30d4adc7ef/channels?session_id=ac64b3156ece4d858308fcf21dac720f (127.0.0.1) 9.40ms referer=None
[I 00:38:54.360 NotebookApp] Creating new notebook in
[I 00:38:54.820 NotebookApp] Kernel started: 4b53921e-3519-4ff1-bcc0-1921b1624d97
[I 00:38:56.895 NotebookApp] Adapting to protocol v5.1 for kernel 4b53921e-3519-4ff1-bcc0-1921b1624d97
[W 00:39:10.205 NotebookApp] Replacing stale connection: 8007b48e-6f37-45ce-91ed-7f30d4adc7ef:ac64b3156ece4d858308fcf21dac720f
[W 00:40:02.222 NotebookApp] Replacing stale connection: 8007b48e-6f37-45ce-91ed-7f30d4adc7ef:ac64b3156ece4d858308fcf21dac720f
[I 00:40:54.822 NotebookApp] Saving file at /Untitled7.ipynb

Thanks in advance,

Mike

All signs above indicate that you're not actually using Jupyter Kernel Gateway here, otherwise, we'd see information related to the override of various handlers that get produced by the activation of the NB2KG server extension.

That said, the 404's - as presented above - can occur if you kept a browser session running, but restarted the Notebook server. In this case, your browser session has used a kernel with id 8007b48e-6f37-45ce-91ed-7f30d4adc7ef and was trying to connect with it. But because the Notebook server has been restarted, it doesn't know anything about that particular kernel-id and issues the 404 response.

It also looks like a new kernel was started (with id 4b53921e-3519-4ff1-bcc0-1921b1624d97) which is probably functional.

Does that scenario fit your situation?

[I 09:24:52.865 NotebookApp] 302 GET / (127.0.0.1) 0.91ms
[E 09:24:52.941 NotebookApp] Could not open static file ''
[I 09:24:52.953 NotebookApp] 302 GET / (127.0.0.1) 0.38ms
[W 09:24:52.995 NotebookApp] 404 GET /static/components/react/react-dom.production.min.js (127.0.0.1) 9.38ms referer=http://localhost:8081/tree?
[W 09:24:53.107 NotebookApp] 404 GET /static/components/react/react-dom.production.min.js (127.0.0.1) 2.23ms referer=http://localhost:8081/tree?
[W 09:24:53.409 NotebookApp] 404 GET /static/components/react/react-dom.production.min.js (127.0.0.1) 2.27ms referer=http://localhost:8081/tree?
^C[I 09:25:27.368 NotebookApp] interrupted
Serving notebooks from local directory: /mnt/disk1/sarvagya
0 active kernels
The Jupyter Notebook is running at:
http://localhost:8081/?token=d6202868a7caa3a42e4aeb38b6e5e62ab534e267a6970cf2
 or http://127.0.0.1:8081/?token=d6202868a7caa3a42e4aeb38b6e5e62ab534e267a6970cf2
Shutdown this notebook server (y/[n])? y

I am kinda getting the same error. The only difference is, I am using ssh to run jupyter notebook on another machine but the browser is on my local machine.

Can you help me?

@Flock1 This repository is for Jupyter Kernel Gateway, not for Jupyter Notebook.

I'm closing this issue as off-topic.

So how do you fix the issue?

All signs above indicate that you're not actually using Jupyter Kernel Gateway here, otherwise, we'd see information related to the override of various handlers that get produced by the activation of the NB2KG server extension.

That said, the 404's - as presented above - can occur if you kept a browser session running, but restarted the Notebook server. In this case, your browser session has used a kernel with id 8007b48e-6f37-45ce-91ed-7f30d4adc7ef and was trying to connect with it. But because the Notebook server has been restarted, it doesn't know anything about that particular kernel-id and issues the 404 response.

It also looks like a new kernel was started (with id 4b53921e-3519-4ff1-bcc0-1921b1624d97) which is probably functional.

Does that scenario fit your situation?
It fits my scenario precisely, but I don't know what to do in order to fix it?

It fits my scenario precisely, but I don't know what to do in order to fix it?

You can't really. If the Notebook server was shutdown "gracefully" (via double ctrl-C or kill <pid>), then all running kernels are stopped. If the server crashed (or kill -9 <pid> was used), then the kernel process is likely still running (depending on the kernel). In either case, the browser is attempting to talk to a (restarted) Notebook server that has no knowledge of how to contact the kernel process (if it still exists).

Some applications provide the ability to persist information relative to started kernels such that in the event the server has crashed, another server instance will attempt to reestablish contact with the kernel - assuming that instance has access to the persisted state and the kernel is still running.

Hi Kevin,

I have the same error message sometimes. Kernels sometimes die on their own.

So what you are really suggesting is that the Notebook Server must be restarted in order to start a new kernel.

This means that the "Restart" command on the "Kernels" menu is only for restarting an instantiated kernel, i.e., one that was created and exists, but may also have been "Interrupted" by the user. It will not launch a new kernel on the server if it does not already exist in the session.

What it means is that the server must be restarted completely to get a new "session kernel" running.

And finally after the last kernel is stopped in a session, and new kernels can be started if the last one is killed or dies for some reason.

Is that correct?

So what you are really suggesting is that the Notebook Server must be restarted in order to start a new kernel.

No. I'm not. A new kernel can always be started (assuming configuration is correct, etc.). In some cases, like when a server is ungracefully torn down, the client may still reference a kernel id that the restarted server has no idea about. That's when messages like the following appear...

[W 00:38:54.181 NotebookApp] 404 GET /api/kernels/8007b48e-6f37-45ce-91ed-7f30d4adc7ef/channels?session_id=ac64b3156ece4d858308fcf21dac720f (127.0.0.1): Kernel does not exist: 8007b48e-6f37-45ce-91ed-7f30d4adc7ef

This means that the "Restart" command on the "Kernels" menu is only for restarting an instantiated kernel, i.e., one that was created and exists, but may also have been "Interrupted" by the user. It will not launch a new kernel on the server if it does not already exist in the session.

Yes, this is correct. "Restart" corresponds to the kernel that is associated with the active notebook, just like "interrupt". "Restart" will stop the current kernel process associated with that "session" and start a new kernel process, also associated with that "session" and identified by the same kernel id.

A new kernel (and "session") is created when opening a different notebook.

hello 你们解决了吗

所以你真正的建议是笔记本服务器必须重新启动才能启动新内核。

不,我不是。总是可以启动一个新内核(假设配置正确等)。在某些情况下,例如当服务器被不正常地拆除时,客户端可能仍然引用重新启动的服务器不知道的内核 ID。那时会出现如下消息...

[W 00:38:54.181 NotebookApp] 404 GET /api/kernels/8007b48e-6f37-45ce-91ed-7f30d4adc7ef/channels?session_id=ac64b3156ece4d858308fcf21dac720f (127.0.0.1): Kernel does not exist: 8007b48e-6f37-45ce-91ed-7f30d4adc7ef

这意味着“内核”菜单上的“重新启动”命令仅用于重新启动实例化的内核,即已创建并存在但也可能已被用户“中断”的内核。如果会话中不存在新内核,它将不会在服务器上启动它。

是的,这是正确的。“重启”对应于与活动笔记本关联的内核,就像“中断”一样。“重新启动”将停止与该“会话”关联的当前内核进程并启动一个新的内核进程,该进程也与该“会话”关联并由相同的内核 ID 标识。

打开不同的笔记本时会创建一个新内核(和“会话”)。

hello、 Have you already solved it

It looks like you quoted my previous response. If you're seeing this, did you try restarting the notebook server that is configured to use kernel gateway? It thinks there's a kernel active within the Kernel Gateway process, yet the gateway has no knowledge of that kernel. This usually indicates the kernel was not gracefully terminated within the gateway server and that "ungraceful termination" was not communicated back to the user-facing notebook server.