Version should use the version sent by the AS and not a timestamp
Closed this issue · 15 comments
Or at least, that was what we agreed. If not, we should discuss.
Reference: https://github.com/jrconlin/pushgo/blob/master/src/mozilla.org/simplepush/handlers.go#L62
While the full spec does note this, unfortunately, it is not possible to provide this sort of function at this time.
The problem is that in order to scale, you have to scale back information storage as much as possible. In this case, the problem with using the version being passed from the source is that there are a set of additional dependencies that go with it. To follow spec, I would need to store a near permanent record for a UAID.ChannelID to indicate the last value that we received (in order to assure that the version is always greater), which means that I'd need to track all channels assigned to a given UAID, etc. The current implementation does not do that. All storage is temporary, including UAIDs and their associated channels.
Using an arbitrary, albeit incrementing, value allows me to ensure that the client still receives a value greater than the last (meaning no changes are required to the client), while also ensuring that the system remains data free. I'll also note that we agreed that the content of the notification was not meant to be meaningful.
So version number is totally meaningless for apps or AS. We talked about using version numbers as a way for apps to ask their servers: "i want version 5" or "i want the differences between the previous saved 3 and the new 5". Whatever they decide to implement.
With this, we can drop any payload on PUT message (it behaves as a simple ping), but clients and servers should have much more logic to know what they should do: they should implement their own versioning or whatever.
If that is what we want (and we don't agree), we could have implemented something much more easy on our end (no database, just memcache or a redis that do not need to be persistent).
I'm a bit curious how this is different. Do you plan on having app servers repeatedly send the same update notification to applications? As it stands, if an app gets a version of 846754855 and a later version of 846755855, it should understand that it wants all information between those two updates. I'll also note that an app could easily store it's last update time itself, and query against that rather than relying on a third party system to be that source of truth.
The difference between a timestamp generated by the Push Server (PS) and a version generated by the AS, is that if an application receives a version=7, it can query to its server: "hey, give me the version 7 you want me to have!" or "hey, I had version 4, and I now want until version 7, give me the diff!", because the PS knows what "version=7" means because it sent that version to the app (thought the PS). Less AS logic.
With a timestamp, apps just receive a "number", bigger. Just a ping to say "you have something new". In your current implementation, apps cannot rely on it to ask their servers about what that version means (because that timestamp "version" is not reported back when the PUT is made), but "I had an update[, and this is the number]". AS should have more logic about what that means, or apps to have an internal version number. Or both. Or nothing. It depends on the app and the service.
Indeed, I think both things are totally correct, but using the version sent by the AS, could be used to map a version<-->data on the server, and now it is impossible with this, because the timestamp is not reported back to the AS (well, it is possible but with more logic on the app and on the AS).
So the question is: should we say to devs that the version number they are sending is reported back to the application or it is not mandatory for the PS to follow that rule?
(Because if not, we simplify the PUT, there will be no body).
If you don't allow server (AS) to select which version he wants to send ... why we're sending a version (generated at PNS) to the client? ... it's unusefull !
What you're developing here is not a PUSH server ... you're creating a PING or WakeUp server ...
So with this idea it isn't needed to send any version to the channel ... only told the phone agent "please wakeup channel XXXX" but with any version on any place (less checks, less cpu)
But, what we defined some months ago was sending a version number DECIDED by the AS since the developer CAN decide if he wants to map it to a REAL version of his own objects or he wants only wakeup the channel (sending timestamp) but this is a DEVELOPER decission !
As you noted, handling additional data for the transaction raises the cost per transaction. The original system was meant to be "data free", and was intended to be simply a wake service as you mentioned. If this is a message delivery system, then why not place additional data inside of the message other than just a number?
The decision was made to make this implementation data free specifically so we can avoid doing any substantial data storage. Reliable high read/write data storage for hundreds of millions of users is prohibitive for this project, even when using a limited data record.
The choice must be either:
a) Allow the version number to be passed from the AppServer to the client without performing any form of check on the version value. (e.g. A lower version number may result in a notification event since the previous version history may have elapsed and been garbage collected.)
or
b) Report an arbitrary, increasing value as the "version" to the App (optionally returned as the body of an Application Server "PUT" request), so that the remote app may be woken and instructed that action is requested by the remote server.
(edit: fixed auto-numbering helpfulness)
@jrconlin will a/b both result in the same amount of network activity for the push server, and shift responsibility of version checking to the client? Because the client already does version checking, so options (a) would be acceptable to the client.
@jrconlin of course we should check that the version is a int64, just not store it.
@nikhilm Yep, same amount of traffic, and yes, I'd limit to positive int64 values.
Hey guys, a little bit of ideas here:
- Sending useless data in a wired and fixed connection may not have any performance/battery impact, but it does on mobile. Think that 3G (with an open websocket) need to change from idle state to a high state just to send a data that will be discarded by the device. I think checks should be done on server.
- What we decided is not a "data free", but a "payload free" (no useful data through the connection), which is different. A "version=5" does not mean anything for a eavesdropper, but a "{ mail: { from: "me", to: "you", subject: "hello", body: "I am the NSA" } }" does.
@jrconlin as a historical fact: In telefónica, we began to develop the push service with payload as we think (and still think! :D) that it is the best that suits app developers. When we enter the review on Bugzilla, Justin Lebar said that he does not like the protocol, as data was no encrypted and can be read on the server.
Then @dougt and @nikhilm started the SimplePush (based on Thialfi ideas), and we decided that we needed to align on that and not work in parallel. We decided:
- Go for a thialfi-like system: version numbers instead of payload
- Using websockets instead of long-polling
- Using UDP for phones inside a reliable mobile network to lower the number of opened connections and the battery usage on phones and GGSN (endpoints between internet an private cellular networks, really expensive).
- Server should accept version numbers and pass to the client only if it is greater than previous one.
- A recovery protocol in case the server crashes and cannot restore saved data (channelIDs, clients and versions).
we should be passing through the version number.
Yep, I concur. Amongst other things because that lets the AS know that it doesn't matter if he sends one notification, or two, or 30 with version number = 1, the client will connect at most once.
And why would a AS want to send the same version number several times? Because 'guaranteed delivery' means 'eventual delivery' and if I've sent a version to a client but haven't heard from him yet Thialfi makes it safe to just send the same version again (since if the client already got it it won't do anything). If we don't process the AS provided version numbers then three push with the same AS provided version number will result on three connections from the client to the server (and the client not being able to assign any kind of semantic to the version number, like how old my data is).
Let's not get confused between the semantics of the protocol and the cost of shipping data (in terms of battery and everything).
If we decide to transmit through the version that the AS sends, it is perfectly fine. The client end will check that for the particular channel the version it received from PS is greater than AS, so the semantics are not affected as far as the app is concerned.
The concern that @jrconlin has is 'version number storage on PS is expensive' which is a acceptable problem. The solution is to always pass along the ping, even if the AS sends the same version number two times. Once again, lets be clear this does not affect correctness.
"Sending useless data in a wired and fixed connection may not have any performance/battery impact, but it does on mobile. Think that 3G (with an open websocket) need to change from idle state to a high state just to send a data that will be discarded by the device. I think checks should be done on server."
Now @willyaranda and @AntonioMA have a valid argument that this affects battery life on the device, but again, this is implementation specific. On 3G the device will connect to the TEF server. It is upto that PS to decide whether to pass on a version to the device or not, and whether to do a server side test of the version or not.
So, Mozilla's server need not store earlier versions, but will instead just forward pings as they come in.
Telefonica's server is free to store the last sent version and compare the new one and decide to ping only if its greater.
Three pushes from AS->PS on the Mozilla server will lead to 3 wakeups on the device, but the client has code to not wakeup the app when the same version is received, so the app can attach semantic meaning to the version.
Three pushes from AS->PS on the Tef server will lead to 1 wakeup on the device, which is also perfectly in line with the protocol.
My point being, the server is free to do checks as an optimization, but not required to, since the client does it anyway.
Closing older issues