rworker
Celery worker for R tasks.
Motivation
The main motivation for this package was the need for executing long running R functions triggered by Celery (asynchronous task queue package for Python).
Usage
Start consuming tasks from R
library(rworker)
library(magrittr)
# Broker url
redis_url <- 'redis://localhost:6379'
# Instantiate Rworker object
consumer <- rworker(name='celery', workers=2, queue=redis_url, backend=redis_url)
# Register tasks
(function(){
Sys.sleep(5)
}) %>% consumer$task(name='long_running_task')
# Start consuming messages
consumer$consume()
The rworker
function returns a Rworker
object. This object is responsible for registering tasks, listening for messages coming from the message queue and triggering tasks execution on background processes
Send tasks from Python
from celery import Celery
worker = Celery('app', broker="redis://localhost:6379/0", backend="redis://localhost:6379/0")
worker.send_task('long_running_task')
Tutorial
You can find more information about usage here.