High memory usage of built package (3x that of source)

Question

High memory usage of built package (3x that of source)

HaveAGitGat opened this issue a year ago · 18 comments

HaveAGitGat commented a year ago

What version of pkg are you using?

5.8.1

What version of Node.js are you using?

18.16.1

What operating system are you using?

Ubuntu 20.04.6 LTS, Windows 11 22621.1848

What CPU architecture are you using?

x86_64

What Node versions, OSs and CPU architectures are you building for?

node18-linux-x64

Describe the Bug

Hi,

I've been using pkg to to build for node18-linux-x64 on Ubuntu 20.04.6 LTS and have noticed that memory usage is quite a bit higher than when running from source. For example when running ps aux:

Execution Mode	User	PID	%CPU	%MEM	VSZ	RSS	TTY	STAT	START	TIME	COMMAND
Running from source	h	258809	16.4	1.6	11675900	128308	pts/5	Sl+	08:31	0:03	node --max-old-space-size=8192 ./srcjs/main.js
Running binary	h	246012	2.3	5.3	11918036	426292	pts/5	Sl+	08:22	0:08	./build/Tdarr_Server

VSZ: The Virtual Memory Size. It includes all memory that the process can access, including memory that is swapped out, memory that is allocated, but not used, and memory that is from shared libraries.
RSS: The Resident Set Size. It shows the portion of a process’s memory that is held in RAM. This should be less than or equal to the VSZ.

As can be seen, the RSS memory increases from 128308 to 426292.

Build command is:

pkg -t node18 --no-bytecode --output build/Tdarr_Server .

Is this expected due to how pkg operates? Seems to be fine on Windows (binary RAM usage around 120MB).

Thanks

Expected Behavior

Binary should use similar amount of RAM as when running from source.

To Reproduce

Just wondering if this is expected behavior atm, if not then can try create steps to reproduce.

Answer 1 · 2023-07-05T14:21:55.000Z

This happens to me too, did you tried to donwgrade pkg to see if this only happens starting from a certain version?

Answer 2 · 2023-07-06T07:20:57.000Z

Seems 5.7.0 is about 25% lower but still much higher than from source:

pkg Version	Node Version	RSS
5.8.1	18	367640
5.8.1	18 (Bytecode)	362540
5.8.1	14	320248
5.8.0	18	370268
5.8.0	14	319360
5.7.0	18	287072
5.7.0	14	313964
5.6.0	16	395632

Answer 3 · 2023-07-06T07:34:35.000Z

Could you try to use flame and/or heapprofiler to track down the issue?

We need to compare the binary heap with the non-bionary one and check where all that memory goes...

Answer 4 · 2023-07-06T07:36:55.000Z

cc @baparham any clue what could cause this?

Answer 5 · 2023-07-07T07:14:43.000Z

Thanks I'll try using those. Do you have a guide to using those tools with the pkg binary? Had a look but couldn't find anything.

Answer 6 · 2023-07-07T07:58:05.000Z

Here are the files (uglified source which is also in the pkg) if you'd like to try:

mkdir tdarr_test && cd tdarr_test
mkdir tdarr_binary tdarr_source

# start pkg
wget -P ./tdarr_binary https://tdarrs.s3.us-west-000.backblazeb2.com/dev/versions/2.00.21/linux_x64/Tdarr_Server_2023_07_07T09_11_11z.zip
unzip ./tdarr_binary/Tdarr_Server_2023_07_07T09_11_11z.zip -d ./tdarr_binary/
./tdarr_binary/Tdarr_Server

# start source
wget -P ./tdarr_source https://tdarrs.s3.us-west-000.backblazeb2.com/dev/versions/2.00.21/linux_x64/Tdarr_Server_2023_07_07T09_11_11z_source.zip
unzip ./tdarr_source/Tdarr_Server_2023_07_07T09_11_11z_source.zip -d ./tdarr_source/
node ./tdarr_source/srcug/main.js

Notably on the first start of binary it settles at about 260MB RAM, but on restart uses 3-400MB.

Whereas source uses 130MB on both first start and restart.

Answer 7 · 2023-07-07T08:16:21.000Z

I never tried but those are the tools I use to debug my nodejs applications. I tried with the packaged binaries but seems them are not working, I also checked if there is a way to create a pkg with --inspect node arg but seems not so easy (need to re-create a custom node patch with inspect enabled as this is disabled by default in patches and build a new package using it).

The alternative would be to add some checks inside the code that prints stats to console that helps to debug where memory is used

Answer 8 · 2023-07-07T19:29:53.000Z

Ok I’ll dig into it a bit more. Interestingly I’ve redone the dev Tdarr_Server docker image to not use pkg and a user has said their RAM usage has gone from 1.15GB to 163MB so yeah definitely something weird going on.

Answer 9 · 2023-07-08T05:18:46.000Z

@HaveAGitGat is your application using .node compiled modules? Like serialport for example or sharp

Answer 10 · 2023-07-12T06:21:33.000Z

@HaveAGitGat Could we create a repro of this with a very simple nodejs package? Like an express server maybe or else

Answer 11 · 2023-07-12T18:41:32.000Z

@robertsLando not sure about the .node modules, certainly not anything I’ve done directly but maybe some dependencies have that. Will look to see if I can do a simpler recreation 👌

Answer 12 · 2023-07-13T11:58:28.000Z

I tried here but without success for now: https://github.com/robertsLando/pkg-bug

Both executable use the same memory for now...

Answer 13 · 2023-07-17T13:40:26.000Z

@HaveAGitGat I successfully managed to create an heap dump using https://www.npmjs.com/package/heapdump

I created the two dumps of the packaged and not packaged code and I'm tring to figure out if I understand where the memory has gone...

Answer 14 · 2023-07-18T08:01:55.000Z

@robertsLando thanks yeah I pretty much need to go through the deps one by one so might take some time

Answer 15 · 2023-07-21T22:06:02.000Z

Will reopen when I have more info

Answer 16 · 2023-07-24T07:24:41.000Z

Keep this open for now

Answer 17 · 2023-10-23T00:21:22.000Z

This issue is stale because it has been open 90 days with no activity. Remove the stale label or comment or this will be closed in 5 days. To ignore this issue entirely you can add the no-stale label

Answer 18 · 2023-10-29T00:22:41.000Z

This issue is now closed due to inactivity, you can of course reopen or reference this issue if you see fit.