[QUESTION] pacote and npm-registry-fetch/make-fetch-happen both cache tarballs.
everett1992 opened this issue · 1 comments
What / Why
I'm debugging tar extract errors that occur when installing packages from a private registry. I'm not sure if the issue is in npm or the server but I've been reading npm code to find out. I noticed that both pacote and make-fetch-happen cache tarball response bodies.
- pacote caches the body in istream
- while npm-registry-fetch passes cache options to make-fetch-happen
pacote seems to know that because _cacheFetches returns false but _cacheFetches isn't used?
Is this intentional, if so why, if not would you accept a PR to skip make-fetch-happen's cache?
Just in case you are interested here's the error I'm debugging.
npm will sporadically error while extracting package tarballs from our private registry.
npm WARN tar zlib: invalid code lengths set
npm WARN tar TAR_ENTRY_INVALID checksum failure
npm WARN tar TAR_ENTRY_INVALID invalid base256 encoding
npm WARN tarball tarball data for eslint@7.32.0 (sha512-VHZ8gX+EDfz+97jGcgyGCyRia/dPOd6Xh9yPv8Bl1+SoaIwD+a/vlrOmGRUyOYu7MwUhc7CxqeaDZU13S4+EpA==) seems to be corrupted. Trying again.
Files downloaded from the registry seem fine
curl -sS $(npm info lodash@4.17.19 dist.tarball) | shasum # correct
curl -sS $(npm info lodash@4.17.19 dist.tarball) | tar xzv # correct
I wrote pacote's tarball stream to a file and the file is corrupted.
# Added here https://github.com/npm/pacote/blob/main/lib/fetcher.js#L418
tarball.pipe(createWriteStream('/tmp/tarball')
I've been unable to reproduce the issue if I use mitmproxy which makes me think it's a http protocol or timing issue.
npm config set proxy http:localhost:8080
cacheFetches is used.
Line 215 in 80cce46