mscdex/mmmagic

Error: "Could not load any valid magic files"

srlowe opened this issue · 8 comments

srlowe commented

We have user reports of failures on Fedora 30 and Debian 9.11 with Error: "Could not load any valid magic files".

Are there some extra dependencies that need to be installed for some distributions?

mscdex commented

The only "dependency" is that any custom (including OS-provided) magic database file be compatible with the version of libmagic included in mmmagic. If the built-in magic database file is used, there should be no issue.

You might also check that there isn't something like a MAGIC environment variable set which could override the magic database file being used.

srlowe commented

I see thanks. Looks like we have a bit of investigating to do then. Is there some way of forcing it to use the built-in db, and ignore any system ones?

mscdex commented

Do you have a way to reliably reproduce the issue that I can use on my end?

srlowe commented

Not yet, but hopefully will shortly and will post here. Thanks.

srlowe commented

@mscdex So after further investigation it seems that the issue is with our build servers or process. It's only the pre-built binaries that are suffering from this problem. We are looking into it now.

srlowe commented

@mscdex Just a short update. We've narrowed it down to the pkg package, which we are using to bundle the project into an executable. Even though the magic.mgc file is declared as an asset in the config, mmmagic does not seem to be able to see it. We're currently investigating why and looking for a workaround.

creativecoder commented

@srlowe I'm having the same problem, the my pkg bundled executable doesn't seem to be able to see the magic.node file (which I've copied into the same directory as the executable), and so I get an error, "Could not load any valid magic files."

Did you ever find a solution?

traviscollins commented

This happens because PKG creates an "snapshot" file system in memory. The NodeJS runtime inside pkg is modified to read correctly from this in memory file system. But it has no control over the C code that the magic library uses to read files. Therefore the Magic library looks in paths that do not exist in the normal file system.

According to the Magic docs, the mmmagic project can fix this by first reading the magicFile property into a buffer. Then pass that buffer as the first argument to the Magic call.

I got around this by including the node_modules/mmmagic/magic/magic.mgc file next to the compiled PKG output. Then do this...

    const pathToMagicFile = path.join(path.dirname(process.execPath), "node_modules/mmmagic/magic/magic.mgc");

    const [magicMimeResults, mimeForkStream] = await streamMmmagic.promise(fileStream, {
        magicFile: pathToMagicFile
    });