consider fetching service worker scripts with no-cache by default

Question

consider fetching service worker scripts with no-cache by default

wanderview opened this issue 9 years ago · 37 comments

Lately I've found myself recommending to people that they use Cache-control: no-cache or max-age: 0 on their service worker script resources. It seems best practice in order for service worker updates to be detected as devs expect.

Should Update algorithm just fetch the service worker script with Request.cache='no-cache' by default? One could argue that an "update check" is semantically similar to pressing the refresh button.

Edit: Removed the erroneous statement about the firefox http cache.

Answer 1 · 2016-06-06T13:35:54.000Z

This could be tweak for production environments by adding an option to the registration:

navigator.serviceWorker.register('worker.js', { cache: <policy> });

Answer 2 · 2016-07-25T13:25:42.000Z

I'm pretty open to this, it does seem like a gotcha. I use & recommend max-age=0 on all service workers. An option to re-enable the current behaviour would be nice too.

The only reason that it feels "wrong" is because avoiding the cache is the exception, but it does feel like the pragmatic option in this case.

Interested to know the opinion of those doing a lot of SW development. +@naskooskov @n8schloss @bmaurer.

Answer 3 · 2016-07-25T17:53:20.000Z

This would be really useful for companies providing services via service workers. We are deploying SW onto clients who can host scripts, but don't have a straight-forward way to set response headers.

Answer 4 · 2016-07-25T18:18:55.000Z

So, for us the current behavior is actually kind of useful. When we push out new code for the site we'll slowly ramp things up and some Facebook servers will be on the newest version while some will be on the old version. In the middle of the code push for any individual request it's just as likely that someone will hit new code or that they'll hit old code. For service workers this means that during code push we often see service workers constantly in the upgrade state. We have some mechanisms to ensure that we don't install an old version of the service worker, however we did still enter the install state more than is ideal. To solve this we actually cache the service worker script for an hour, this allows a level of stability that would otherwise be hard to achieve.

Answer 5 · 2016-07-25T18:20:23.000Z

(that being said, I do agree that defaulting to no cache with an option to re-enable the current behavior is probably a better developer experience)

Answer 6 · 2016-07-26T17:14:31.000Z

Coming from same perspective as @muralisr (we provide service workers to our clients and we are not in direct control), we find that they usually cache the service worker script similar to other JS files on their server, which tends to have a long cache time if they are doing things correctly. They should ideally add a separate server rule making the service workers script cache for a lesser time which they typically do not end up doing.

I like @delapuente's way of declaring the cache time during the service worker registration process.

Answer 7 · 2016-07-27T16:32:37.000Z

Pre F2F notes: do we want to make this change? We shouldn't do it without offering an API to stick with the current behaviour. Should these options be part of the registration call, or in the activate event?

Answer 8 · 2016-07-28T20:28:19.000Z

F2F:

It'd be great to make max-age opt in, but need to speak to big SW users first
Feels like this should be a setting on the install event

Answer 9 · 2016-07-28T20:32:03.000Z

We should do the above for all imported scripts & modules

Answer 10 · 2016-07-28T20:33:33.000Z

Even in the opt-in cache case, we still do the 24hr limit

Answer 11 · 2016-08-01T17:31:41.000Z

Gecko bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1290944

Answer 12 · 2016-08-03T16:19:09.000Z

At the moment third party servers can use max-age to reduce server load, are we kinda going against them here by making this opt-in by a different origin?

Answer 13 · 2016-08-03T16:56:30.000Z

Regarding the opt-in: the concurrent fetch feature will sit on the registration object, so maybe this should too.

Answer 14 · 2016-09-05T18:05:15.000Z

I just ran in to this. There definitely needs to be a way to force updates for fetching the SW script. I have to reiterate that as middleware/framework developers we are always developing for servers we don't control, so tweaking HTTP headers is simply not possible. Note this is also the case if you don't control the host (e.g. Dropbox public folder, Neocities, Github pages etc).

I just tried testing on a server which defaults all .js files to cache with a max-age of 50 days, presumably set up to use versioned URLs. Of course there's no sensible way to use versioned URLs on the sw.js script if the main page is cached by the SW itself, so I kept getting the stale sw.js returned from cache with no apparent workaround other than to change the server configuration, wait 24 hours or click the manual "update" button in Chrome devtools. In our case none of these are good options for our framework. I don't mind which way you choose to default this, but I think it's essential to have some way of saying "bypass cache for SW updates".

Answer 15 · 2016-09-20T09:38:29.000Z

Resolved:

navigator.serviceWorker.register('/sw.js', {
  useCache: true
});

Answer 16 · 2016-09-20T09:43:30.000Z

Just realised it's pretty similar to my first comment.

Answer 17 · 2016-10-02T07:15:04.000Z

Applause for this decision!!

Answer 18 · 2016-11-14T19:51:01.000Z

I'm reviewing gecko code to implement this and came up with a couple questions:

I assume this does not change Step 5.3 of Register which short-circuits the registration if its a duplicate of the same script already installed. Correct?
Should we expose a getter on the registration so someone can inspect the value of the useCache attribute?
Should we allow the useCache value to be changed in the install or activate events?

@jakearchibald, what do you think?

Answer 19 · 2016-11-29T14:44:55.000Z

I asked Jake about the short-circuit in step 5.3. He suggests the short-circuit should only happen if the useCache value is the same:

9:31 AM JakeA: so we are implementing the "use no-cache by default on updates" thing in gecko and a question arose
9:32 AM the issue defines an API like register(scriptURL, { useCache: true })
9:32 AM the spec also short circuits registration if the scriptURL and scope are the same as an existing registration
9:33 AM our question is, should that short circuit still happen even if the "useCache" value has changed?
9:33 AM or should there be another way for a service worker to change its "useCache" value?
9:33 AM JakeA: ^^^
9:34 AM I'm inclined to implement without any ability to change useCache to start, but thought I would get your opinion
9:34 AM wanderview: changing the value of useCache in .register feels like the right way to update this value
9:35 AM wanderview: whether it causes an immediate refetch of the SW (like changing the url does) is another question
9:35 AM JakeA: so the bypass check should be "if script URL, scope, and useCache value are identical, then bypass" ?
9:35 AM wanderview: yeah
9:36 AM ok
9:36 AM JakeA: it would be easiest if it just acted the same as if a script URL changed
9:36 AM wanderview: that works for me then

Answer 20 · 2016-11-29T14:46:58.000Z

Also, I agree we should have registration.useCache - a getter which reflects the current value.

Answer 21 · 2016-12-13T04:33:15.000Z

I've heard some concerns about effect on queries per second (QPS) for big sites. There are three proposed changes that will affect QPS:

whether imported scripts are included in the byte-to-byte comparison (#839)
whether to skip the HTTP cache by default for the main script (#893)
whether to skip the HTTP cache by default for imported scripts (#893)

I'm not sure we've discussed the impact on making import scripts both involved in the byte-to-byte check (1.) and making it no-cache by default (3.) particularly when importScripts is for a cross-origin resource. Jake seems to have raised the concern in the Aug 4 comment #893 (comment) but there are no replies here about it, although the F2F did get consensus to go ahead with (3.).

I guess it's unclear whether the impact of both (1.) and (3.) have been considered together yet. Have they?

Edit: Expanded "QPS".

Answer 22 · 2016-12-13T05:20:21.000Z

I think, during the f2f, we thought #893 (comment) is what devs would expect. (i.e. defaulting to "no-cache".) But with the QPS concern and both (1) and (3) considered together, I suggest we discuss and settle on the right default behavior this time before it's too late.

Answer 23 · 2016-12-13T14:27:49.000Z

Sorry, can you expand "QPS"? I'm not familiar with that term.

Note, browsers are starting to provide ways for script library hosts to indicate that no-cache checks should always validate. In firefox we support cache-control:immutable. I believe chrome does a magic heuristic around max-age being over a threshold. People hosting 3rd party libraries can use those to avoid increased server traffic from this change.

Answer 24 · 2016-12-13T14:32:20.000Z

I think it's "Query Per Second" used to mean server loads.

Answer 25 · 2016-12-13T15:12:16.000Z

Interesting. So, cache-control:immutable prevents a request with "no-cache" from sending a conditional revalidation in the first place. But it seems this can work only when a server provides this hint. In the GitHub hosting case, isn't setting this header impossible? That considered, providing opt-in to "no-cache" seems to satisfy both requirements.

Answer 26 · 2016-12-13T15:24:44.000Z

@jungkees I don't understand why a github user cares about 304s here. Github as a service allows arbitrary content to be hosted, so I'm not sure how they can have any expectation of a certain QPS to scripts.

Also, I would contend someone running a small site on github.io are exactly the sort of developers more likely to be surprised when their service workers don't update as expected because they get default caching they don't want.

I feel pretty strongly that no-cache should be the default. This is based on helping numerous developers work through this issue. I feel like its the root cause of at least half the problems people bring me. Making it default cache is not going to help here.

Answer 27 · 2016-12-13T15:41:05.000Z

I'm not sure how they can have any expectation of a certain QPS to scripts.

I thought of a case where "hosting 3rd party libraries" is done in github or some similar environment where server-side setup is not accessible to devs. Let me know if that doesn't make sense.

I feel pretty strongly that no-cache should be the default. This is based on helping numerous developers work through this issue. I feel like its the root cause of at least half the problems people bring me. Making it default cache is not going to help here.

No opposition. Just wanted to check if we can have a good compromise considering the end-to-end picture. But if opt-in to "no-cache" doesn't help much, I'd strongly support your idea.

Answer 28 · 2016-12-19T08:47:59.000Z

Chromium tracking bug: https://bugs.chromium.org/p/chromium/issues/detail?id=675540

Answer 29 · 2017-01-09T18:14:28.000Z

FYI, we've implemented this and landed it in FF53 (includes WPT tests):

https://bugzilla.mozilla.org/show_bug.cgi?id=1290944#c65

We are defaulting to no-cache right now with the useCache option available to opt into http cache.

This will ship in our release channel in April. If there is serious concern that we should switch the default please let us know before that point.

Answer 30 · 2017-02-03T16:01:00.000Z

I am in a situation where I want to build an immutable JavaScript secure loader. The secure loader only loads JS files with a suitable cryptographic signature. The 24 hours cache limit prevents me from building such a secure loader that persists longer than 24 hours. (The idea is that the secure loader would persist even if the servers that delivers index.html is compromised.)

Have you considered removing the 24 hours cache limit if explicitly opted-in with the useCache option?

Answer 31 · 2017-02-03T19:58:53.000Z

@JustinDrake Your use-case has been raised in some other issues as well. The short story is that the use-case is at odds with the intent of ServiceWorkers and the web security model. If you can create a benevolent SW that can never be removed so it can survive when the server is compromised, then an attacker can construct a nefarious SW that can never be removed the one time the server is compromised. And browsers can't distinguish between benevolent and nefarious, just whether the TLS cert is valid.

I think many of us are interested in this specific use-case, but SW is not going to be the solution for that on its own. The most practical solution at this time is to use the increasingly cross-browser WebExtensions efforts that are already built on a packaged/release model and APIs like webRequest that lets you intercept requests to specific sites and validate/enforce their contents, etc. It's also possible spec efforts like https://w3ctag.github.io/packaging-on-the-web/ may bear fruit at some point.

Answer 32 · 2017-02-07T02:01:03.000Z

Dear @jakearchibald need a little help here.
I have a service worker https://www.tavest.com/service-worker-tavest-test.js
Why somehow the service-worker-file is being cached by chrome? So when I update the service worker, the update didn't take place.

I've been also set this up in my .htaccess to make the service-worker not being cached, yet not successful.

  <filesMatch "^(service-worker)\.js$">
    FileETag None
    <ifModule mod_headers.c>
       Header unset ETag
       Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate"
       Header set Pragma "no-cache"
       Header set Expires "Wed, 11 Jan 1984 05:00:00 GMT"
    </ifModule>
  </filesMatch>

Thank you

Best regards,

Albert

Answer 33 · 2017-03-30T23:12:47.000Z

@albertsylvester if this is still a problem, can you post it to stackoverflow? I (or someone else) will answer it there.

Answer 34 · 2017-04-04T08:07:43.000Z

The F2F resolution was useCache would be an enum with the default to cache import scripts but not cache the top-level script. See issue #1104

Answer 35 · 2017-12-11T07:26:25.000Z

We settled on https://w3c.github.io/ServiceWorker/#dom-serviceworkerregistration-updateviacache defaulting to "imports" and have tests: https://github.com/w3c/web-platform-tests/blob/master/service-workers/service-worker/registration-updateviacache.https.html. Closing.

If I missed anything, please reopen.

Answer 36 · 2018-11-21T19:21:54.000Z

What does useCache mean exactly, to cache the service worker file or not? please explain

Answer 37 · 2019-04-10T09:55:28.000Z

What does useCache mean exactly, to cache the service worker file or not? please explain

@kgrosvenor I suppose by default browsers now use Cache-control: no-cache. That means, browsers will try to update sw.js by asking server (etag or max-age). If you want to get old behavior, you can set { useCache: true }. With it, browser will respect your Cache-control headers. But still forcibly update sw.js if more than 24 hours have passed.