Cache out of date
Closed this issue · 9 comments
The previous fix to emmalloc trim resulted in code cache needing to be cleared.
Can I run the Tag release and update Changelog action to do that? Just put current SHA of main branch as the Changeset there?
We just releases 4.0.14 two days ago, but if you want to go another release.
We don't normally trigger a release just because we change something in one of the system libraries (which always requires a cache clear). For git users we instead normally just rely on an "if in doubt clear the cache and try again" approach.
The Tag release and update Changelog action is designed to be triggered at the end of the chain of events the make up a release. Specifically it gets triggered automatically from the emsdk change that tags a release, which in turn is automatically triggered when the emscripten-releases builders have all finished building a release.
The release is initially triggered by a human running https://chromium.googlesource.com/emscripten-releases/+/refs/heads/main/src/create_release_candidate.py
I see.. back in the day we would bump the version number whenever a cache clear was needed. (that was the idea of tying cache clear to the versioning)
On my CI I have made the bots test each commit on an existing cache, so they would rely on that being up to date. (it saves a nice amount of electricity to not have to rebuild the system libs on every commit)
If you are testing each commit the safest thing to do to clear the cache each time.
If we bumped the version every time we touched any header or source that gets built into a system library that would result in a lot of version bumps a believe. We don't currently do that.
Maybe we could take the last modified timestamp of the system/ directory as part of the cache hash? Then if something is changed there as part of a git pull, then it'd trigger a cache clear as well?
I suppose you could design a system that forces the cache to be cleaned if any file in the system/ directory was newer than the oldest file in the cache? But you would need to remember that changes to the compile driver itself can change compile and link flags so any change to tools/ would probably have to trigger it too.
Hmm I'd imagine tools/system_libs.py would cover that well.
Rerunning test suite on my Linux bot takes ~30 minutes on a warm cache, and close to 50 minutes on a clean cache. It is quite a big difference, and a lot (majority) of our PRs that land are not touching the system libraries.
On circielci we currently build all the system libraries from scratch each time in the build-linux builder. Other builders than re-use those libraries to avoid building them themselves.
If build-linux step could be a no-op in some large fraction of PR that would be amazing, but we would need to figure out to persist the cache between runs.
Yeah, I'm not thinking about CircleCI in this case, but my own CI runtimes. Naturally on CI that always operates on a clean slate this won't be a problem/can't take benefit.