@tasklet and @transactional interact funnily

[GoogleCodeExporter]

Not sure what to do about this, perhaps document it?

(1) @transactional @tasklet is synchronous despite the @tasklet decorator:

@ndb.transactional
@ndb.tasklet
def foo(id):
  assert ndb.in_transaction()
  e = yield Employee.get_by_id_async(id)
  raise Return(e)

Despite being decorated with @tasklet, this is a *synchronous* function; 
calling foo(id) returns an Employee instance or None, not a Future.  This is 
because @transactional calls transaction(func), and transaction() is 
synchronous; there is special code in Context.transaction() that recognizes 
when the callback is a tasklet and then waits for it.

(2) @tasklet @transactional does not run the generator in a transaction:

@ndb.tasklet
@ndb.transactional
def foo(id):
  assert ndb.in_transaction()
  e = yield Employee.get_by_id_async(id)
  raise Return(e)

This returns a Future, but when asking for its result, fails the assert!  The 
reason is that transactional passes the "bare" generator function as the 
callback to transaction(), and Context.transaction() *does not* recognize this; 
it calls the function which returns a generator object (an iterator) but does 
not execute any of the function's body; transaction() doesn't do anything with 
that generator but just returns it after executing an empty transaction.  The 
result of all this gets passed into the tasklet wrapper, which does recognize 
the generator and handles it properly, but at this point the transaction is 
gone.

Suggestion: I think we should just document (1), but maybe we can fix (2) by 
recognizing the "bare" generator at some point (maybe inside 
Context.transaction()) and do something with it?  It would be nice to be able 
to declare a transactional tasklet, i.e. something that behaves like a tasklet 
(returns a Future) and runs transactionally.  Right now if you wanted that 
you'd have to cobble it together using transaction() rather than @transactional.

Original issue reported on code.google.com by gvanrossum@gmail.com on 3 Jul 2012 at 8:49

[GoogleCodeExporter]

It looks like:
@ndb.tasklet
@ndb.transactional
@ndb.tasklet
def foo(id):
   ...

works?  This could easily put into a new decoration: @ndb.transactional_tasklet 
and @ndb.non_transactional_tasklet .

Original comment by jim@twist.com on 3 Jan 2013 at 7:36

[GoogleCodeExporter]

fyi, rafe has suggested some pretty slick code in the google group: 
https://groups.google.com/forum/?fromgroups=#!topic/appengine-ndb-discuss/XBMMmA
661a8

Original comment by faction.gregory on 12 Mar 2013 at 5:42

[GoogleCodeExporter]

rafek's code (slightly tweaked):

def transactional_tasklet(*ctx_args, **ctx_kwargs):
  def decorator(m):
    m_tasklet = ndb.tasklet(m)

    @ndb.tasklet
    def wrapper(*args, **kwargs):
      @ndb.tasklet
      def tx():
        value = yield m_tasklet(*args, **kwargs)
        raise ndb.Return(value)

      value = yield ndb.transaction_async(tx, *ctx_args, **ctx_kwargs)
      raise ndb.Return(value)
    return wrapper
  return decorator


@transactional_tasklet()
def fn():
  assert ndb.in_transaction()
  yield ndb.sleep(0.2)
  raise ndb.Return('hi')

print fn().get_result()  # Prints 'hi'

Original comment by arful...@google.com on 16 Mar 2013 at 5:45

[GoogleCodeExporter]

Took me awhile, but after looking at this thoroughly is seems that the 
async-ness of the function being decorated is entirely decoupled from the 
async-ness of the transaction itself (and there is no clean way to couple 
them). For example it makes sense for a sync function to become async when 
wrapped in a transaction, as the transaction itself has multiple points at 
which it is blocked by IO.

My solution is to add two wrappers:
@transactional_async - the wrapped function always returns a future
@transactonal_tasklet - helper decorator that is equivalent to 
@transactiona_async @tasklet

Original comment by arful...@google.com on 16 Jul 2013 at 6:16

[GoogleCodeExporter]

Fix will be in 1.8.4

Original comment by arful...@google.com on 19 Jul 2013 at 10:25

• Changed state: Fixed