Not all packages are getting cached

Question

Not all packages are getting cached

tonyjchong opened this issue 10 years ago · 6 comments

I think I'm missing something but none of my packages that are listed in the dependencies section of my package.json are getting cached, but modules in my devDependencies are, along with some other modules I do not have listed. I'm guessing the latter is because those modules have dependencies of their own.

Thanks.

Answer 1 · 2014-10-22T18:34:43.000Z

Did you run rm -rf node_modules and npm cache clean? npm does it's own caching as well locally so it may not be requesting all the packages from npm_lazy.

npm_lazy will only cache packages that are requested from it, and it will still check for a newer version of the metadata on demand.

To debug this type of problem you'll need to provide a more detailed trace: incl. two things - the commands you ran and the log messages produced by npm_lazy.

Answer 2 · 2014-10-22T20:31:03.000Z

Thanks for responding. Here are some more details:

I have npm_lazy running on a Jenkins CI machine since all my code building and deployments happen from there. I wipe out my workspace with every new build so the node_modules directory doesn't exist yet, if this is what you are referring to. I just ran npm cache clean as the same user that does the builds and still not everything gets cached.

Here is some more information about my setup.

The registry is setup using the .npmrc file in my $HOME directory.
npm config list

; cli configs
registry = "http://localhost:8081/"

; userconfig /mnt2/jenkins/.npmrc
registry = "http://localhost:8081/"

; node bin location = /usr/local/bin/node
; cwd = /mnt2/jenkins/.npm
; HOME = /mnt2/jenkins
; 'npm config ls -l' to show all defaults.

All the jenkins job does is do a git clone, cd into that directory, and npm install.

Here is an excerpt from the log.

app debug cache get https://registry.npmjs.org/mocha/-/mocha-1.14.0.tgz
app debug [OK] Reusing cached result for https://registry.npmjs.org/mocha
app debug cache get https://registry.npmjs.org/should/-/should-2.0.2.tgz
app debug [GET] https://registry.npmjs.org/should/-/should-2.0.2.tgz
app debug cache get https://registry.npmjs.org/nock/-/nock-0.22.1.tgz
app debug [OK] Reusing cached result for https://registry.npmjs.org/nock
app debug cache get https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.2.11.tgz
app debug [OK] Reusing cached result for https://registry.npmjs.org/iconv-lite
app debug [OK] Reusing cached result for https://registry.npmjs.org/should
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/should/-/should-2.0.2.tgz should-2.0.2.tgz /mnt2/npm_cache/phzurv68ytn2vs4i
app debug [OK] Reusing cached result for https://registry.npmjs.org/propagate
app debug cache get https://registry.npmjs.org/propagate/-/propagate-0.2.2.tgz
app debug [GET] https://registry.npmjs.org/propagate/-/propagate-0.2.2.tgz
app debug [OK] Reusing cached result for https://registry.npmjs.org/propagate
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/propagate/-/propagate-0.2.2.tgz propagate-0.2.2.tgz /mnt2/npm_cache/rt9vyfjromoswcdi
app debug [OK] Reusing cached result for https://registry.npmjs.org/debug
app debug [OK] Reusing cached result for https://registry.npmjs.org/mkdirp
app debug [OK] Reusing cached result for https://registry.npmjs.org/glob
app debug [OK] Reusing cached result for https://registry.npmjs.org/commander
app debug [OK] Reusing cached result for https://registry.npmjs.org/growl
app debug [OK] Reusing cached result for https://registry.npmjs.org/jade
app debug [OK] Reusing cached result for https://registry.npmjs.org/diff
app debug cache get https://registry.npmjs.org/mkdirp/-/mkdirp-0.3.5.tgz
app debug [GET] https://registry.npmjs.org/mkdirp/-/mkdirp-0.3.5.tgz
app debug cache get https://registry.npmjs.org/commander/-/commander-2.0.0.tgz
app debug [OK] Reusing cached result for https://registry.npmjs.org/commander
app debug cache get https://registry.npmjs.org/debug/-/debug-2.1.0.tgz
app debug [GET] https://registry.npmjs.org/debug/-/debug-2.1.0.tgz
app debug cache get https://registry.npmjs.org/jade/-/jade-0.26.3.tgz
app debug [GET] https://registry.npmjs.org/jade/-/jade-0.26.3.tgz
app debug cache get https://registry.npmjs.org/growl/-/growl-1.7.0.tgz
app debug [GET] https://registry.npmjs.org/growl/-/growl-1.7.0.tgz
app debug cache get https://registry.npmjs.org/glob/-/glob-3.2.3.tgz
app debug [GET] https://registry.npmjs.org/glob/-/glob-3.2.3.tgz
app debug [OK] Reusing cached result for https://registry.npmjs.org/glob
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/glob/-/glob-3.2.3.tgz glob-3.2.3.tgz /mnt2/npm_cache/y80wo9o0d3gqr529
app debug [OK] Reusing cached result for https://registry.npmjs.org/debug
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/debug/-/debug-2.1.0.tgz debug-2.1.0.tgz /mnt2/npm_cache/zd24ydld4inhr529
app debug [OK] Reusing cached result for https://registry.npmjs.org/growl
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/growl/-/growl-1.7.0.tgz growl-1.7.0.tgz /mnt2/npm_cache/stl5pvhcvglgcik9
app debug cache get https://registry.npmjs.org/diff/-/diff-1.0.7.tgz
app debug [OK] Reusing cached result for https://registry.npmjs.org/diff
app debug [OK] Reusing cached result for https://registry.npmjs.org/mkdirp
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/mkdirp/-/mkdirp-0.3.5.tgz mkdirp-0.3.5.tgz /mnt2/npm_cache/y453xhkx15j0pb9
app debug [OK] Reusing cached result for https://registry.npmjs.org/jade
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/jade/-/jade-0.26.3.tgz jade-0.26.3.tgz /mnt2/npm_cache/up4gyvj31l2l0udi
app debug [OK] Reusing cached result for https://registry.npmjs.org/ms
app debug cache get https://registry.npmjs.org/ms/-/ms-0.6.2.tgz
app debug [GET] https://registry.npmjs.org/ms/-/ms-0.6.2.tgz
app debug [OK] Reusing cached result for https://registry.npmjs.org/ms
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/ms/-/ms-0.6.2.tgz ms-0.6.2.tgz /mnt2/npm_cache/jfd7kmvldvcjif6r
app debug [OK] Reusing cached result for https://registry.npmjs.org/inherits
app debug [OK] Reusing cached result for https://registry.npmjs.org/graceful-fs
app debug [OK] Reusing cached result for https://registry.npmjs.org/minimatch
app debug cache get https://registry.npmjs.org/inherits/-/inherits-2.0.1.tgz
app debug [GET] https://registry.npmjs.org/inherits/-/inherits-2.0.1.tgz
app debug cache get https://registry.npmjs.org/graceful-fs/-/graceful-fs-2.0.3.tgz
app debug [GET] https://registry.npmjs.org/graceful-fs/-/graceful-fs-2.0.3.tgz
app debug cache get https://registry.npmjs.org/minimatch/-/minimatch-0.2.14.tgz
app debug [GET] https://registry.npmjs.org/minimatch/-/minimatch-0.2.14.tgz
app debug [OK] Reusing cached result for https://registry.npmjs.org/graceful-fs
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/graceful-fs/-/graceful-fs-2.0.3.tgz graceful-fs-2.0.3.tgz /mnt2/npm_cache/40lxixikw3c5l8fr
app debug [OK] Reusing cached result for https://registry.npmjs.org/minimatch
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/minimatch/-/minimatch-0.2.14.tgz minimatch-0.2.14.tgz /mnt2/npm_cache/2bknlciw2ohfflxr
app debug [OK] Reusing cached result for https://registry.npmjs.org/inherits
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/inherits/-/inherits-2.0.1.tgz inherits-2.0.1.tgz /mnt2/npm_cache/0el7p4vqw3eah5mi
app debug [OK] Reusing cached result for https://registry.npmjs.org/mkdirp
app debug [OK] Reusing cached result for https://registry.npmjs.org/commander
app debug cache get https://registry.npmjs.org/mkdirp/-/mkdirp-0.3.0.tgz
app debug [GET] https://registry.npmjs.org/mkdirp/-/mkdirp-0.3.0.tgz
app debug cache get https://registry.npmjs.org/commander/-/commander-0.6.1.tgz
app debug [GET] https://registry.npmjs.org/commander/-/commander-0.6.1.tgz
app debug [OK] Reusing cached result for https://registry.npmjs.org/mkdirp
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/mkdirp/-/mkdirp-0.3.0.tgz mkdirp-0.3.0.tgz /mnt2/npm_cache/1b43m8krtqpvi
app debug [OK] Reusing cached result for https://registry.npmjs.org/commander
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/commander/-/commander-0.6.1.tgz commander-0.6.1.tgz /mnt2/npm_cache/yadfh1acykd6xbt9
app debug [OK] Reusing cached result for https://registry.npmjs.org/lru-cache
app debug [OK] Reusing cached result for https://registry.npmjs.org/sigmund
app debug cache get https://registry.npmjs.org/lru-cache/-/lru-cache-2.5.0.tgz
app debug [GET] https://registry.npmjs.org/lru-cache/-/lru-cache-2.5.0.tgz
app debug cache get https://registry.npmjs.org/sigmund/-/sigmund-1.0.0.tgz
app debug [GET] https://registry.npmjs.org/sigmund/-/sigmund-1.0.0.tgz
app debug [OK] Reusing cached result for https://registry.npmjs.org/sigmund
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/sigmund/-/sigmund-1.0.0.tgz sigmund-1.0.0.tgz /mnt2/npm_cache/yge0wfn268orms4i
app debug [OK] Reusing cached result for https://registry.npmjs.org/lru-cache
app debug [done][SHASUM OK] added to cache https://registry.npmjs.org/lru-cache/-/lru-cache-2.5.0.tgz lru-cache-2.5.0.tgz /mnt2/npm_cache/p277zdoy3crnqaor
app debug [OK] Reusing cached result for https://registry.npmjs.org/mocha
app debug [OK] Reusing cached result for https://registry.npmjs.org/should
app debug [OK] Reusing cached result for https://registry.npmjs.org/nock
app debug [OK] Reusing cached result for https://registry.npmjs.org/iconv-lite
app debug [OK] Reusing cached result for https://registry.npmjs.org/propagate
app debug [OK] Reusing cached result for https://registry.npmjs.org/debug
app debug [OK] Reusing cached result for https://registry.npmjs.org/mkdirp
app debug [OK] Reusing cached result for https://registry.npmjs.org/glob
app debug [OK] Reusing cached result for https://registry.npmjs.org/commander
app debug [OK] Reusing cached result for https://registry.npmjs.org/growl
app debug [OK] Reusing cached result for https://registry.npmjs.org/jade
app debug [OK] Reusing cached result for https://registry.npmjs.org/diff
app debug [OK] Reusing cached result for https://registry.npmjs.org/ms
app debug [OK] Reusing cached result for https://registry.npmjs.org/inherits
app debug [OK] Reusing cached result for https://registry.npmjs.org/minimatch
app debug [OK] Reusing cached result for https://registry.npmjs.org/graceful-fs
app debug [OK] Reusing cached result for https://registry.npmjs.org/lru-cache
app debug [OK] Reusing cached result for https://registry.npmjs.org/sigmund
app debug [OK] Reusing cached result for https://registry.npmjs.org/commander
app debug [OK] Reusing cached result for https://registry.npmjs.org/mkdirp

During my npm install, I can see some packages getting pulled from localhost, and some from the npm registry.

npm http 200 https://registry.npmjs.org/cssom/-/cssom-0.2.5.tgz
npm http GET http://localhost:8081/commander/2.0.0
npm http GET http://localhost:8081/growl
npm http GET http://localhost:8081/jade/0.26.3
npm http GET http://localhost:8081/diff/1.0.7
npm http GET http://localhost:8081/debug
npm http GET http://localhost:8081/mkdirp/0.3.5
npm http GET http://localhost:8081/glob/3.2.3
npm http 200 http://localhost:8081/commander/2.0.0
npm http 200 http://localhost:8081/growl
npm http 200 http://localhost:8081/jade/0.26.3
npm http 200 http://localhost:8081/diff/1.0.7
npm http 200 http://localhost:8081/debug
npm http 200 http://localhost:8081/glob/3.2.3
npm http 200 http://localhost:8081/mkdirp/0.3.5
npm http GET http://localhost:8081/commander/-/commander-2.0.0.tgz
npm http GET http://localhost:8081/jade/-/jade-0.26.3.tgz
npm http GET http://localhost:8081/mkdirp/-/mkdirp-0.3.5.tgz
npm http GET http://localhost:8081/growl/-/growl-1.7.0.tgz
npm http GET http://localhost:8081/debug/-/debug-2.1.0.tgz
npm http 200 http://localhost:8081/commander/-/commander-2.0.0.tgz
npm http GET http://localhost:8081/diff/-/diff-1.0.7.tgz
npm http GET http://localhost:8081/glob/-/glob-3.2.3.tgz
npm http 200 http://localhost:8081/jade/-/jade-0.26.3.tgz
npm http 200 http://localhost:8081/mkdirp/-/mkdirp-0.3.5.tgz
npm http 200 http://localhost:8081/growl/-/growl-1.7.0.tgz
npm http 200 http://localhost:8081/debug/-/debug-2.1.0.tgz
npm http 200 http://localhost:8081/glob/-/glob-3.2.3.tgz
npm http 200 http://localhost:8081/diff/-/diff-1.0.7.tgz
npm http GET https://registry.npmjs.org/ms/-/ms-0.6.2.tgz
npm http 200 https://registry.npmjs.org/ms/-/ms-0.6.2.tgz
npm http GET https://registry.npmjs.org/bson/-/bson-0.2.2.tgz
npm http GET https://registry.npmjs.org/kerberos/-/kerberos-0.0.3.tgz

Answer 3 · 2014-10-25T05:28:50.000Z

You don't happen to have a npm shrinkwrap file at all do you?

Also would be good to know what the package.json looks like to see where some of the dependencies are coming from.

Answer 4 · 2015-04-14T20:53:19.000Z

Was there ever a resolution for this?

Answer 5 · 2015-04-14T23:51:25.000Z

I'm happy to investigate if I get an example package.json file so we can look at reproducing the issue. I suspect it's specific to some particular urls being loaded that are not yet catered for.

Answer 6 · 2016-05-04T05:30:45.000Z

closing to clean up the issues list a bit; this looks like it may have been either old cached files but difficult to confirm.