Cloud NDB - Lock deletion after transaction not flushed

[justinkwaugh]

Cloud NDB v1.8.0

This is a minor one, I just noticed it while investigating other things, and is I believe present in the legacy NDB as well. Simply put, after a transaction finishes, a set of cache deletes are added to the event loop to delete the locked keys from the write. They are not flushed though, and while this is not a major issue as things like the top level context ending or additional ndb operations will cause them to be executed, the time frame of doing so obviously can be quite delayed, up to the ttl provided for the lock value (32s). Perhaps a bunch of network operations are happening after a transaction, or any other processing that could take quite some time, or the process is killed or whatnot. It seems to me that these lock values should be as short lived as possible and so they should be flushed immediately. The relevant code is in _transaction.py:

        tx_context._clear_global_cache()
        for callback in on_commit_callbacks:
            callback()

        raise tasklets.Return(result)

perhaps something like this would work (I'm not super familiar with the internals of the library yet)

        tx_context._clear_global_cache()
        tx_context.flush()
        for callback in on_commit_callbacks:
            callback()

        raise tasklets.Return(result)

[chrisrossi]

It looks with all the churn in this space, the code this issue is referring to has actually changed a couple of times since the issue was created. Looking at the current state of things, locks are cleared via "transaction_complete" callbacks that are called, and run synchronously, at the end of a transaction. I think we can just go ahead and close this.