Change the update frequency to 30 minutes
fmarier opened this issue · 5 comments
Using a simple tool, I can see that the update frequency is still set at 45 minutes, which means that clients will often encounter stale data, even if they update regularly.
We should change this to 30 minutes to match what Google emits.
This blocks https://bugzilla.mozilla.org/show_bug.cgi?id=1032393.
Poked at this. The jenkins task that generates the lists is set to run every 30 minutes, but was actually running every 45. No idea why. I've bumped the frequency up to 15 minutes for now and will keep an eye on the job.
@ckolos There's another component that probably needs updating (unless you've done that one too): the polling frequency that the server sends to the client in the response to the downloads
endpoint.
I think that's this param in the shavar.ini (not shown in the shavar.ini in this repo)
[shavar]
default_proto_ver = 2.0
lists_served = list1
list2
list3
list4
lists_root = xxx
default_interval = xxx
In stage this is set to:
default_interval = 1800
In current prod, this parameter appears instead
refresh_check_interval = 3600
and default_interval
doesn't exist
I can confirm that it's currently configured properly on tracking.stage.mozaws.net
:
$ ./download-list.py | head
n:1800
i:mozpub-track-digest256
a:1439927165:32:52416
and I think that @rtilder was saying that 0.6.4 will bring default_interval = 1800
to production.
Given the operational cost of having so many clients poll the server, we'll keep mozpub-track{,white}-digest256
in urlclassifier.disallow_completions
for the foreseeable future so we don't need the 30 minute interval anymore.