typegoose/mongodb-memory-server

postinstallHelper always installs to $HOME/.cache instead of node_modules/.cache

Simon-Hayden-iteratec opened this issue · 6 comments

Simon-Hayden-iteratec commented

Versions

  • NodeJS: v20.3.1
  • mongodb-memory-server: 9.1.1
  • mongodb(the binary version): 6.0.9 (but should not be relevant)
  • mongodb(the js package): 5.9.1 (but should not be relevant)
  • mongoose: 7.6.3 (but should not be relevant)
  • system: Fedora Linux 39

package: mongo-memory-server

What is the Problem?

It looks like the local flag of the postInstallEnsureBinary is ignored. The postinstall script of the mongo-memory-server package will always install the binary to $HOME/.cache/mongodb-binaries .

Code Example

I tried running this in the shell to simulate the postinstall script:

node node_modules/mongodb-memory-server/postinstall.js

The output is always:

Mongodb-Memory-Server* checking MongoDB binaries
Mongodb-Memory-Server* found binary: "/home/<username>/.cache/mongodb-binaries/mongod-x64-fedora-6.0.9"

I already tried to attach a debugger to node (via node --inspect-brk node_modules/mongodb-memory-server/postinstall.js) and noticed that the path is picked, because resolveConfig(ResolveConfigVariables.PREFER_GLOBAL_PATH) will always return the default case, which is 'true'.

I don't know if this is intentional, but it contradicts the package description of mongodb-memory-server on npm:

Main default package which downloads mongod binary to ./node_modules/.cache directory on package install.

The reason why this is an issue is that the CI pipelines may only cache the node_modules folder, thinking that this will also include the MongoDB binaries for mongodb-memory-server. But it looks like this is not the case.

I also don't think this is a local-only issue, because I checked the output of env and there are MONGO* variables set (or anything else that could hint at the issue). We also don't have any config in the package.json or similar that I know of.

hasezoey commented

i assume you use package mongodb-memory-server and not the -global one.

according to the logs provided, everything is fine; it will NOT download a binary, if it can find a binary existing (even if it is in the global space):

Mongodb-Memory-Server* checking MongoDB binaries
Mongodb-Memory-Server* found binary: "/home/<username>/.cache/mongodb-binaries/mongod-x64-fedora-6.0.9"

means that it has already found the binary in the global space and so does not need to download it again

I already tried to attach a debugger to node (via node --inspect-brk node_modules/mongodb-memory-server/postinstall.js) and noticed that the path is picked, because resolveConfig(ResolveConfigVariables.PREFER_GLOBAL_PATH) will always return the default case, which is 'true'.

thanks, this does look to be the case, i will need to investigate further

hasezoey commented

i have investigated further, and yes it does seem to always use the home dir; if my skimming of git logs are anything to go by, then this issue seems to have existed for at least 3 years without notice (since 7.0.0)

this will be fixed with 9f523de (included in 9.1.2); this will not affect any binaries already installed

Simon-Hayden-iteratec commented

Thanks a lot for your super fast reply and fix 😄

i assume you use package mongodb-memory-server and not the -global one.

Yes, that is correct.

Mongodb-Memory-Server* checking MongoDB binaries
Mongodb-Memory-Server* found binary: "/home//.cache/mongodb-binaries/mongod-x64-fedora-6.0.9"

means that it has already found the binary in the global space and so does not need to download it again

Sorry, in hindsight the logs aren't very useful. Because what you don't see, is that the binary gets downloaded to the $HOME/.config folder. But because the download progress message is gone when they are finished, they don't appear in the final log.

What I also forgot to mention is, that I of course also always cleared the binary/-ies in $HOME/.config/mongodb-binaries/ before I re-executed the node_modules/mongodb-memory-server/postinstall.js script.

I will try the new version and get back to you if this fixes the issue I had!

Simon-Hayden-iteratec commented

Just tested and it correctly installs to node_modules/.cache now 🥳

Just one final question though: Should this be considered a breaking change?
Based on that a lot of companies most likely will already have set-up their pipeline in a way to cache the $HOME/.cache folder, there is a chance that pipelines will fail because the download at runtime takes longer than the test timeout permits (this was our issue). Granted, this is only an issue if they have been caching $HOME/.cache but haven't cached the node_modules folder - which is unlikely but could happen for whatever reason.

hasezoey commented

Just one final question though: Should this be considered a breaking change?

i dont really consider this one, because the documented behavior is what it now got changed to and it was this pre-7.0.0, and if the old behavior is still wanted, the package can be replaced with the -global one or set config option PREFER_GLOBAL_PATH (which will overwrite what the postinstall sets);
though you may have a point, i will consider such a change to maybe at least be put into a minor version instead of patch

  • 👍1