aio-libs/aioodbc

SQLAlchemy integration

Gr1N opened this issue · 11 comments

Gr1N commented

Hi,

What about SQLAlchemy integration like in aiopg? Do you have any plans? Or it's already supported but missed in docs?

Sorry, not supported for now and I do not have plan to add, may be in future we will have async ORM that unifies all those libraries. PR is welcome.

I am planning on adding support for using SQLAlchemy Core as SQL query builder, like aiopg and aiomysql have. Looking at the code, it seems the primary parts missing are the create_engine method, Engine class, and SAConnection class. Can you think of any pitfalls I am likely to run into if we go down this path?

Should be easy to do just copy sa module from aiopg or aiomysql and update dialect and other parameters.

@jettify @vproman a pitfall that i encountered when replicating sa functionality:
The asynchronous file descriptor such as available in aiopg, which is supported by psycopg fileno under the hood, Is Not supported by in pyodbc because of this limitation

@jettify Do you have any suggestions for a possible workaround, in order to add SA support?

This is not an issue here at all, since we offload blocking operations into threads, and SQLA integration does not depend on any file descriptors.

@jettify thanks for your response. That confirms i was headed down a wrong path, in trying to replicate the functionality without using the offloading.

This is really the key finding for me! Could you please elaborate on "we offload blocking operations into threads"

Is this something you can achieve using ThreadPoolExecutor? It seems its achieved using run_in_executor on the event loop. Do you mind pointing to a simple Example of how to offload blocking operations into threads?

I think i found an example i was looking for here: https://github.com/calebmadrigal/asyncio-examples/blob/master/run_in_executor.py

Any more suggestions are welcomed! Thanks a bunch

This is pretty much only one call

def _execute(self, func, *args, **kwargs):
# execute function with args and kwargs in thread pool
func = partial(func, *args, **kwargs)
future = self._loop.run_in_executor(self._executor, func)
return future

we do it here. Official docs:
https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.AbstractEventLoop.run_in_executor

As for SQLA you may want to check https://github.com/aio-libs/aiopg/tree/master/aiopg/sa it should be pretty straightforward to adapt.

Looked into sqlalchemy code there are pyodbc dialects only for: mssql, mysql, sybase.

What is the status of this @jettify @vproman, has idea been dropped?

No plans to work on SQLA integration for now.