/rq-gevent-worker

A worker using gevent for rq

Primary LanguagePython

rq-gevent-worker

Implement a new worker based on gevent

Downloads

##Install this fork

pip install --upgrade -e git+https://github.com/relic-/rq-gevent-worker.git@master#egg=rq-gevent-worker

##Install

$ pip install rq-gevent-worker

##Usage

$ rqgeventworker -h

$ export PYTHONPATH=<your project import path>:$PYTHONPATH; rqgeventworker

##Test

$ pip install -r requirements.txt

$ py.test tests

##Under The Hood TODO

##TODO

  • Add a command line option to specify gevent pool size

##Note

###Crash Official Worker use os.fork() to spawn a child process to execute a job, so if the job cause the process crash, the worker process is still alive.

When using gevent, we use the same process to execute job, the job may cause the whole worker process crash.

###Why not rqworker -w <geventworker> Because we need gevent monkey patch at the start of the process, rqworker import many modules before importing geventworker, so it will cause geventworker not work normally.

##Declaration

Most of the code is from lechup and jhorman,