python.exe constant full CPU usage after opening terminal
zcattacz opened this issue · 8 comments
Hi, it's exciting to see terminal works on windows now. But I am experiencing constant cpu spike.
After started Jupyter-lab with jupyter lab --core-mode
4 new process spawned showed up in process explorer as usual
jupyter.exe
\-python.exe
|-jupyter-lab.exe
|-python.exe with listening port
after opening a terminal in the jupyter lab web portal.
one of the jupyter-lab python process is giving constant full cpu utilization,
though the terminal is totally idle.
jupyter.exe
\-python.exe
|-jupyter-lab.exe
|-python.exe <---- constant 33% cpu usage.
|- winpty-agent.exe
|- conhost.exe
|- powershell.exe
Is there anything I can try to solve this ?
BTW, the box is a Windows2012 R2 (VMWare VM), with CPU 3 core.
I have cygwin installed. Can I configure jupyter terminal to use Cygwin's bash instead ?
Hello, same here, hence I confirm the CPU issue. I would love to learn how can I solve this.
Update, this has nothing to do with selected shell, I tried both bash and powershell on Win10 and Win2012R2Server and both exhibit same CPU draining issue.
If this is helpful here's some stack trace from python.exe process bound to winpty-agent.exe.
ntdll.dll!RtlSetLastWin32Error+0x4
ucrtbase.dll!errno+0x7f
python36.dll!Py_FatalError+0x124f4
_socket.pyd+0x1722
_socket.pyd+0x38fd
python36.dll!PyCFunction_FastCallDict+0x182
python36.dll!PyObject_CallFunctionObjArgs+0x38b
python36.dll!PyEval_EvalFrameDefault+0x3d7
python36.dll!Py_CheckFunctionResult+0x1f4
python36.dll!PySet_Contains+0x697
python36.dll!PyEval_EvalFrameDefault+0x2ba8
python36.dll!PyObject_CallFunctionObjArgs+0x54f
python36.dll!PyEval_EvalFrameDefault+0x3d7
python36.dll!PyObject_CallFunctionObjArgs+0x54f
python36.dll!PyEval_EvalFrameDefault+0x3d7
python36.dll!PyFunction_FastCallDict+0xd3
python36.dll!PyObject_IsInstance+0x631
python36.dll!PyObject_Call+0x5e
python36.dll!PyOS_SigintEvent+0x135a
python36.dll!PyThread_start_new_thread+0x186
ucrtbase.dll!o__strtoui64+0x59
KERNEL32.DLL!BaseThreadInitThunk+0x14
ntdll.dll!RtlUserThreadStart+0x21
To me it looks like that it does not "sleep" waiting for input, but instead runs in some error loop, draining CPU.
Addressed in andfoy/pywinpty#94.
@blink1073 Please, don't get me wrong, I have full respect for your effort, but I think that proposed patch is a "workaround" rather than proper solution.
I believe, the proper approach would using WaitForMultipleObjects on both console stdout and process HANDLEs, and simply not use CPU at all if nothing happens. (Kind equivalent for Unix'es poll/epoll)
Current time.sleep approach in pywinpy.ptyprocess.PtyProcess._read_in_thread seem to be too much hackish. It simply trades few CPU cycles for read throughput, but CPU is still waken up constantly (10 times a sec), moreover sleep seems to be enforced even when there's a data in the buffer.
@blink1073 I tried to cook WaitForMultipleObjects patch, but it seems waiting is not supported for named-piped on Windows when using non-overlapped I/O.
Therefore I pushed andfoy/pywinpty#95 that adds blocking parameter (False by default) to pywinpy.winpty_wrapper.PTY.read and makes PtyProcess to use blocking call explicitly in the thread. This way we use absolutely no CPU is nothing is happening.
I suspect you have made it non-blocking in the first place in order to observe status of the console process (pid). But is it really necessary? Isn't winpty closing stdout pipe one process exits?
This is going to be part of pywinpty release v0.5.2, to be released during this week