Cache in git workflow not speeding up subsequent runs
aeppacher opened this issue · 4 comments
Our linting on our project is taking upwards of 4-5 minutes to run in git workflow. I thought a good optimization would be to cache the xo cache per branch and update it incrementally. That way subsequent commits will only have their linter run against that branches diff, the same way you would have it for local development.
My solution was to save the node_modules/.cache/xo-linter folder at the end of every successful run, and to restore it per branch on every subsequent run. This works fine, but the linting doesn't seem to be faster which makes me think the cache is somehow being invalidated.
I looked through the source to try and find how the cache invalidation works but I can only see code about creating a config which has the cache folder and passing that to eslint. Is this something that is caused by eslint, or any tips on how I might achieve this?
Caching in xo comes fully from eslint - and by default invalidates caches by file metadata. So copying/moving the cache between runs likely will invalidate it every time. See:
https://github.com/eslint/eslint/blob/a1f7ad77e2/lib/cli-engine/lint-result-cache.js
https://www.npmjs.com/package/file-entry-cache
Perhaps something like lint-staged could help you speed things up.
Or if that is too simple, perhaps we could expose cache options to cache with file content strategy from eslint which would probably resolve your use case, but I would leave that up to @sindresorhus - PRs are probably welcome.
Yes, we could support a cacheStrategy option.
https://eslint.org/docs/user-guide/command-line-interface#--cache-strategy
@sindresorhus that sounds like exactly what I want. Would be a great enhancement for remote compilation users as well that might also see invalidation of meta data between runs.
Just to add some findings: lint-staged by default does some git stashing stuff for backups. It seems like this changes the metadata for files and defeats the ESLint cache.
To make lint-staged work with the cache you need to run it with --no-stash. This makes runs (after the first) much faster.
I'd rather run XO/ESLint with --cache-strategy content and keep the lint-staged backup logic intact. (you could argue this is an issue on their end as well but ... 🤷♂️ )