kafka_dummy_agent

The kafka dummy agent includes a dummy producer and a dummy consumer to generate and receive customizable dummy data streams for testing.

prerequisites

To set up the environment, use the following commands:

conda env create -f environment.yml
conda activate kafka

The producer contains the following components:

A folder includes dummy data (images, latent, texts)
A folder for customized message templates (YAML)
A script scans the template, reads data from the data folder, composes a message, and sends the message to the corresponding queue.

To run the producer, enter the producer folder and use the following command:

python produce.py --tmp <name of the template file>

The consumer contains is a script reads the message of the designated topic and displays the content.

To run the consumer, enter the consumer folder and use the following command:

python consume.py --topic <name of the topic to listen to>