eerimoq/asyncbg

Asyncio background tasks

Asyncio background tasks

Asyncio background tasks in Python 3.7 and later.

Run CPU intensive long running tasks without blocking the asyncio loop, implemented as a lightweight asyncio layer on top of the multiprocessing module.

Alternatively run tasks in other threads.

Project homepage: https://github.com/eerimoq/asyncbg

Documentation: https://asyncbg.readthedocs.org/en/latest

Installation

pip install asyncbg

Examples

There are more examples in the examples folder.

Call

Call work(a, b) in another process. The script output is Result: 9.

import asyncio
import asyncbg

def work(a, b):
    return a + b

async def main():
    result = await asyncbg.call(work, 4, 5)
    print(f'Result: {result}')

asyncio.run(main())

Process pool

Create a process pool with two workers, and call work() three times in it (up to two callbacks called in parallel).

import asyncio
import asyncbg

def work():
    pass

async def main():
    with asyncbg.ProcessPoolExecutor(max_workers=2) as pool:
        await asyncio.gather(pool.call(work),
                             pool.call(work),
                             pool.call(work))

asyncio.run(main())

Call thread

Call work(a, b) in another thread. The script output is Result: 9.

import asyncio
import asyncbg

def work(a, b):
    return a + b

async def main():
    result = await asyncbg.call_thread(work, 4, 5)
    print(f'Result: {result}')

asyncio.run(main())

Thread pool

Create a thread pool with two workers, and call work() three times in it (up to two callbacks called in parallel).

import asyncio
import asyncbg

def work():
    pass

async def main():
    with asyncbg.ThreadPoolExecutor(max_workers=2) as pool:
        await asyncio.gather(pool.call(work),
                             pool.call(work),
                             pool.call(work))

asyncio.run(main())