GitSquared/edex-ui

Child process issue with v2.2.6 on Windows

handslive opened this issue · 11 comments

handslive commented

Technical information

Using version:

  • master (running from GitHub-published source code, currently v2.2.7-pre)
  • latest (latest release, currently v2.2.6)
  • vX.X.X (specify other version)

Running on:

  • Linux
  • Windows
  • macOS

How comfortable you are with your system and/or IT in general:

  • I'm kind of lost, honestly
  • I know what's up, I could help you run some commands or checks
  • My machine is fully under my control, tell me what you need
  • I attended Defcon last year

Problem

When running the updated version of the application for the first time, my CPU gradually rose to 100% usage and my physical memory use also rose to 100% usage and eventually pushed everything else out to swap. After running for about 30 or 40 minutes, the system became unusable and even stopping the application did not resolve the issue. After several minutes, I powered down because the disk was going constantly, and I couldn't move the mouse or even bring Task Manager up.

I ran the resource monitor for a bit the next couple of times, being careful to shut the application down before things got too crazy. I'm seeing a lot of cmd.exe, conhost.exe, and WMIC.exe running and in the process of terminating. The list of these processes was constantly growing, but the majority of them were in the "Terminated" state. It even took some time to remove them all after shutting down the application. It seems that these are the source of the CPU and memory issue. There's no one process eating all the resources, but a cloud of small processes is taking it over even though they're short-lived.

I've used earlier versions going back to v2.2.2 and this is my first time observing this behaviour. Possibly, it was simply never bad enough to kill my system before.

wrac4242 commented

could you try running from source, to see if it is a build related issue

GitSquared commented

What kind of hardware are you running? CPU, eventually gpu specs?

handslive commented

@wrac4242 attempting to run from source is kicking out a bunch of errors for me, starting with:
innerError Error: Cannot find module '../build/Debug/conpty.node'
Require stack:

  • E:\eDEX-UI\edex-ui\src\node_modules\node-pty\lib\windowsPtyAgent.js
  • E:\eDEX-UI\edex-ui\src\node_modules\node-pty\lib\windowsTerminal.js
  • E:\eDEX-UI\edex-ui\src\node_modules\node-pty\lib\index.js
  • E:\eDEX-UI\edex-ui\src\classes\terminal.class.js
  • E:\eDEX-UI\edex-ui\src_boot.js
  • E:\eDEX-UI\edex-ui\node_modules\electron\dist\resources\default_app.asar\main.js
  • at Module._resolveFilename (internal/modules/cjs/loader.js:972:15)
    at Function.n._resolveFilename (electron/js2c/browser_init.js:249:921)
    at Module._load (internal/modules/cjs/loader.js:848:27)
    at Function.f._load (electron/js2c/asar_bundle.js:5:12738)
    at Module.require (internal/modules/cjs/loader.js:1032:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at new WindowsPtyAgent (E:\eDEX-UI\edex-ui\src\node_modules\node-pty\lib\windowsPtyAgent.js:40:40)
    at new WindowsTerminal (E:\eDEX-UI\edex-ui\src\node_modules\node-pty\lib\windowsTerminal.js:51:24)
    at Object.spawn (E:\eDEX-UI\edex-ui\src\node_modules\node-pty\lib\index.js:29:12)
    at new Terminal (E:\eDEX-UI\edex-ui\src\classes\terminal.class.js:405:33) {
    code: 'MODULE_NOT_FOUND',
    requireStack: [
    'E:\eDEX-UI\edex-ui\src\node_modules\node-pty\lib\windowsPtyAgent.js',
    'E:\eDEX-UI\edex-ui\src\node_modules\node-pty\lib\windowsTerminal.js',
    'E:\eDEX-UI\edex-ui\src\node_modules\node-pty\lib\index.js',
    'E:\eDEX-UI\edex-ui\src\classes\terminal.class.js',
    'E:\eDEX-UI\edex-ui\src\_boot.js',
    'E:\eDEX-UI\edex-ui\node_modules\electron\dist\resources\default_app.asar\main.js',
    undefined
    ]

Running 'npm run install-windows' seemed to go fine, so I'm not sure what happened here.

@GitSquared Intel i5-4690, 16GB RAM, GeForce GTX 760 (a bit dated, the box is 6 years old). OS is on a 120GB SSD, majority of personal files on a 1TB HDD.

wrac4242 commented

which order did you run the source commands in? was it run install windows, then running it. Because it has a bunch of issues with missing modules

handslive commented

I followed the instructions off the main page:

  1. git clone https://github.com/GitSquared/edex-ui
  2. npm run-install windows
  3. npm start

Step 2 pulled down and installed a lot of build tools I didn't have and I'm afraid I didn't save an output log of that process. It didn't cause any errors that I noticed. In the event that the build did something unexpected, what would you suggest I uninstall/remove to get back to a pre-built state and try again?

wrac4242 commented

yea try deleting it then re downloading it, but if that doesn't work possibly try adding the npm install step between 1 and 2

handslive commented

So, I don't know if I simply missed this last time, but npm run install-windows does finish with an error. Running npm install beforehand doesn't seem to have any effect.

E:\eDEX-UI\edex-ui>npm run install-windows
(node:27664) ExperimentalWarning: The fs.promises API is experimental

edex-ui@2.2.6 preinstall-windows
npm install --global --production windows-build-tools && npm install --global node-gyp && setx PYTHON "%USERPROFILE%.windows-build-tools\python27\python.exe"

(node:22516) ExperimentalWarning: The fs.promises API is experimental
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated request@2.88.2: request has been deprecated, see request/request#3142

changed 147 packages, and audited 148 packages in 17s

6 packages are looking for funding
run npm fund for details

3 low severity vulnerabilities

To address all issues, run:
npm audit fix

Run npm audit for details.
(node:13892) ExperimentalWarning: The fs.promises API is experimental
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated request@2.88.2: request has been deprecated, see request/request#3142

changed 97 packages, and audited 98 packages in 3s

3 packages are looking for funding
run npm fund for details

found 0 vulnerabilities

SUCCESS: Specified value was saved.

edex-ui@2.2.6 install-windows
npm install && cd src && npm install && ..\node_modules.bin\electron-rebuild -f -w node-pty && cd ..

(node:26576) ExperimentalWarning: The fs.promises API is experimental

up to date, audited 288 packages in 964ms

19 packages are looking for funding
run npm fund for details

found 0 vulnerabilities
(node:16108) ExperimentalWarning: The fs.promises API is experimental

added 105 packages, and audited 106 packages in 50s

2 packages are looking for funding
run npm fund for details

found 0 vulnerabilities
'..\node_modules.bin\electron-rebuild' is not recognized as an internal or external command,
operable program or batch file.

There is no electron-rebuild file in node_modules.bin. There are 3 electron files of different types (shell, cmd, ps1) and 3 electron-builder files of different types (shell, cmd, ps1), but no electron-rebuild. Does this mean I need to attempt to install electron separately perhaps? I had an old copy on here a year ago or so, but uninstalled it.

  • 👍1
GitSquared commented

The electron-rebuild / node-pty couple seems to be broken in many places with the latest release, causing issues with installing and building. I unfortunately don't have a solution for now.

Re: og issue, there has always been this cloud of short-lived processes on Windows [1] and your hardware seems capable enough so it's weird that it has only recently become a problem. Are you using other programs which might starve edex from ram/cpu cycles?

[1]: We use a lib called systeminformation for fetching, well, information about your system. It abstracts away from this project the complexity of fetching info in different operating systems, and standardizing it to a common data model.

Now on unix systems most of the info is stored on a sysfs, which is basically a virtual directory with files we can read (very efficient), but on Windows its stored in registers that can only be accessed using utility processes like wmic.exe, which consumes ram, cpu cycles and thread management work time everytime we want to see how hot your CPU is.

I've just submitted them an idea for improving this: sebhildebrandt/systeminformation#502

handslive commented

I understand the approach you've used, but there's no denying that something changed in how this was handled between 2.2.5 and 2.2.6. I'm attaching 3 graphs from Performance Monitor. The first one shows v2.2.5 after running for about 5 minutes. The number of threads has climbed an additional 2K or so and the number of processes has hardly moved over that 5 minute period, sticking pretty close to 3K processes total.

Perfmon-eDEX-UI-v2-2-5

The second one shows v2.2.6 after running for about 5 minutes. The number of threads on the system is getting close to 60K and the number of processes is at about 6K and climbing. System performance is already affected. It took several seconds for the snip & sketch app to let me select the screen and grab this image.

Perfmon-eDEX-UI-v2-2-6

This shows the dumping of resources after I shut v2.2.6 down and things returned to normal.

Perfmon-eDEX-UI-shutdown

I'm not sure what's changed, but hopefully, this highlights that the difference is significant.

  • 👍2
mkcoldwolf commented

same issue on windows here, 16G memory is going 100% inside 5mins
some more observation is that the time used for opening a file (aka settings.json) takes much longer than reasonable time
and the network traffic and cpu graph obviously want to say goodbye to us (its become more far with time)
image
image
image

We use a lib called systeminformation for fetching, well, information about your system

dont know is this related or its caused by the app freezing

GitSquared commented

Added some throttling of WMIC calls which at least on my test machine (i5-8250U, win10) helps keep temperatures and cpu usage to a more acceptable level: 5bf9018

Will do a patch release soon