/redis-resp

Make the Redis Mass Insertion by using the REdis Serialization Protocol (RESP) simple.

Primary LanguagePythonMIT LicenseMIT

redis-resp

Build Status PyPI PyPI GitHub license

Make the Redis Mass Insertion by using the REdis Serialization Protocol (RESP) simple.

Installation

pip install resp

Usage

python \
    -m resp \                           # the installed module 'resp' 
    -i /path/to/dump.txt \              # the path to the input file
    -d ',' \                            # the delimiter (default: ',')
    -r 'RPUSH {0} {1} | SET {0} {2}' \  # the Redis command(s)
        | redis-cli --pipe

If the order of the data isn't relevant, you can split the origin file in multiple chunks and pipe them to the script concurrently:

cat /path/to/dump.txt | parallel -j4 --block 1M -I ,, --pipe 'python -m resp -d "," -r "SET {0} {1}" -p' | redis-cli --pipe

If parallel isn't already installed on your system, you can install it with sudo apt-get install parallel on Linux or brew install parallel on macOS.

API

python -m resp [-h] -r REDIS [-i INPUT] [-d DELIMITER] [-p]
python -m resp [--help] --redis REDIS [--input INPUT] [--delimiter DELIMITER] [--pipe]

Development

Environment

Install the required environment modules by executing the script environment.sh:

conda env create -n resp python=2 -f environment.yml
source activate resp

Testing

python -m unittest discover -vp '*Test.py'

Questions?

Don't be shy and feel free to contact me by e-mail or on Twitter.

License

The module is Open Source Software released under the MIT license.