Blackpiper

📒 Index

About
Development
- Pre-Requisites
- Development Environment
Credit/Acknowledgment
License

🔰 About

A simple app that use Django and Redis together to queueing jobs and processing them in the background. This project uses django-rq which is a simple app that allows you to confiure your queues in django. django-rq internally uses RQ.

In layman language RQ is an alternative of celery. There are many ways to run async tasks in python. Most of the companies uses celery, since it's pretty famous also. But, here we are, now looking at an alternative of celery which is pretty easy to setup as well RQ.

Difference between RQ and Celery

Let's go through a couple of difference between celery and RQ :

RQis known for it's simplicity whereas Celery is known for it robustness.
RQ uses redis as a message broker, whereas Celery suports both redis and rabbitMQ. Celery has clearly won here.

There are other differences as well, why don't you explore those on your own :)

IMHO, celery is awesome ;)

🔧 Development

📓 Pre-Requisites

Knowledge of Django and zeal to learn :)

🔩 Development Environment

1. Make a virtual environment and activate

python3 -m venv .venv
source ./.venv/bin/activate

2. Clone the repo and install requirements

git clone https://github.com/nightwarriorftw/blackpiper.git
pip install -r requirements.txt

3. Setup Redis

RQ uses redis as a message broker. So, it's important to have redis installed on your system.

In case you don't have redis installed on your system. You can follow this to install redis on your system.

4. Setup Environment variables

Create a .env file in the root folder of the project.

Add the following variables in your .env file :

SECRET_KEY="<your django secret key>"
DEBUG=TRUE
RQ_QUEUE_HOST="localhost"
RQ_QUEUE_PORT=6379
RQ_QUEUE_DB=0
RQ_QUEUE_DEFAULT_TIMEOUT=360
RQ_QUEUE_PASSWORD="<your redis password>"

5. Redis Configuration

Add the following in your project's settings.py

# RQ Configurations
RQ_QUEUES = {
    "default": 
        {
            "HOST": env("RQ_QUEUE_HOST"), 
            "PORT": env("RQ_QUEUE_PORT"), 
            "DB": env("RQ_QUEUE_DB"), 
            "DEFAULT_TIMEOUT": env("RQ_QUEUE_DEFAULT_TIMEOUT"),
            "PASSWORD": env("RQ_QUEUE_PASSWORD")
        },
}

# RQ Logging Configuration
LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "handlers": {"console": {"class": "logging.StreamHandler",},},
    "root": {"handlers": ["console"], "level": "DEBUG",},
}

You can change your logging configuration depending upon your usecase. Like, there are hell lot of options you can find here.

6. Makemigrations, migrate and run the server

python manage.py makemigrations
python manage.py migrate

6. Run server

python manage.py runserver

7. Start RQ worker

Open another new terminal and run the following command

python manage.py rqworker default

The above command will start rq default worker. There are lot of ways you can configure your rq workers. Change your RQ configuration depending upon your usecase. Refer, django-rq docs for this :)

8. Test the real

Open the shell
```
python manage.py shell
```
Run hello method to test if everything is working on not
```
from piper.tasks import hello

hello.delay()
```

You will find the call will return a reference of async method. You can check your terminal/console to validate if your setup is complete.

Do read the django-rq and RQ docs. It's pretty awesome :)

🌟 Credit/Acknowledgment

Aman Verma

Github: nightwarriorftw
Linkedin: nightwarriorftw
Twitter: nightwarriorftw

Credits goes to me

🔒 License

LICENSE