prozzie: A Shell repository from Chorro

Main entry point for the data plane of Wizzie Data Platform.

Under the hoods, prozzie is just a docker-compose file that provides you the basics for sending the data events to WDP: authentication, encryption, homogenization and a flexible kafka buffer for back-pressure and local data persistence.

It provides out-of-the-box support for json over kafka, http POSTs, and mqtt, and it supports others such netflow, snmp and json over mqtt with a small configuration. Please see Supported Protocols for a more exhaustive list.

Basic usage

Sending data

You can send raw data using curl or kafka protocol, and prozzie will manage to send the data to WDP. After that, you will be able to see it in the analytic platform.

Here you can send data using curl:

$ curl -d '{"test":1,"timestamp":1518086046}' \
localhost:7980/v1/data/testtopic

You can batch JSONs and prozzie will split:

$ curl -d \
'{"test":1,"timestamp":1518086046}{"test":1,"timestamp":1518086047}' \
localhost:7980/v1/data/testtopic

And using a kafka producer (for example, prozzie built in one):

$ prozzie kafka produce testtopic

You can check that prozzie kafka receives message:

$ prozzie kafka consume testtopic

And check the result on WDP. (Note: as a simpler alternative to WDP, we use an example echo server that will receive JSON messages in the demo).

Installation

Clone the repo and execute the setups/linux_setup.sh script that will guide you through the entire installation. You can also use the next command if you don't want to waste your time messing with git you can run next command as root user or sudo command:

sudo bash -c "$(curl -L --header 'Accept: application/vnd.github.v3.raw' 'https://api.github.com/repos/wizzie-io/prozzie/contents/setups/linux_setup.sh?ref=0.7.0')"

You will be asked for a prozzie installation path. After installation, you can start and stop the prozzie using bin/start-prozzie.sh and bin/stop-prozzie.sh under that installation path.

After that, you will be prompted for the different protocols configurations prozzie supports.

Since all prozzie is contained in a docker compose, you can use docker-compose start and docker-compose stop in the prozzie folder to start and stop the prozzie, and docker-compose down for delete all created containers.

Operation

After installation, you can use prozzie config command to (re)configure different prozzie components.

Syslog and mqtt protocols are handled via kafka connect, so you need to call prozzie kcli component to handle them.

In order to operate on created compose, you can use prozzie compose command, and it will forward arguments with proper compose file and configurations. You can start, stop, create or destroy prozzie compose with shortcuts prozzie start, prozzie stop, prozzie up and prozzie down shortcuts, and arguments will be forwarded to these commands.

Supported Protocols

LICENSE