Adding a Put to memory only call

[mdasilvapereira]

I'm really enjoying goon, but something I think it needs is a .PutMemoryOnly (which puts to cache and memcache, excluding datastore).

I see a big need for this since I have applications doing a lot of .Puts to maintain a last_seen timestamp (which I'm more than happy to only write to dbstore every 30/60 seconds - However I require it to still be available "if" possible via cache or memcache).

Currently my only option is to maintain my own cache separately, so I'm caching the same thing twice :/

[xStrom]

Am I understanding correctly, that you want a function that would purposely make the cache out-of-sync from the datastore? That is, would this PutMemoryOnly function take a regular entity, which would share the id (and cache) with the one used by Get?

I myself have solved this problem by using a simple memcache value that I then save via cron. Also, it's not caching the same thing twice, since this manual memcache value isn't the same that is in the datastore (and thus in goon's cache).

[mdasilvapereira]

I think you understand what I'm saying, so basically I want goon to store cache/memcache the entity in its formatting (so on a goon.Get I have the cache/memcache version) but not always put it to the dbstore. (so yes essentially mis-syncing for the purpose of saving on datastore .puts).

I could manually get around this by using my own memcache key for the entity and only putting when I feel I want to now store the data more persistently, but I'd have more caching within my application would be nice if goon was able to take this load off me. I'm currently doing 1 Million puts a day, for little or no work just to maintain a last_seen of a user. (which can be every second). So doing a put only every second or 3rd request would make a massive difference on dbstore put and response time.

[xStrom]

Keeping the goon cache in-sync is something that I have invested quite a bit of work into. I view it as a core design goal even. Thus I also view the idea of adding such a public API function as a conflict of interest.

That said, the problem you are having is real and I agree that the optimal solution is not always writing directly to the datastore. Like I said in my previous post, I have usually solved this by using a lightweight memcache value for the frequently updated properties that don't need guaranteed persistence. I guess we can call this model "Eventual consistency, usually".

However, if you really want to to piggyback on the goon infrastructure, then you could add a new simple public method for your own copy of goon, which implements the internal putMemcache method. It would be exactly what you're asking here. Do keep in mind though that the internal API is definitely not guaranteed to be stable between goon versions, not even in functionality. Another thing to keep in mind is that time.Time is only stored with microsecond precision to the datastore. However if you write a more precise time using putMemcache then you can end up with a cache that is not strictly equal to the datastore, even if the source time is the same.