This is a Docker image based on the slim Debian 10 image with the addition of tcpreplay, Zeek 3.1.1, and librdkafka.
This image does a few things when it starts up:
- Brings up a dummy0 network interface
- Starts an instance of tcpreplay that
- Replays /pcaps/zeek_streamer.pcap
- Sends the replayed packets to dummy0
- Starts an instance of Zeek that
- monitors dummy0
- sends its output as JSON to an Apache Kafka broker
Zeek is configured to send its output to a host named broker.
There are a few other (better) Docker images out there that can use Zeek to read pcaps and send the output to Apache Kafka.
This one uses tcpreplay to read a pcap and stream it in real-time as its original pace instead of all at once. In other words, if it took six hours to generate your 100MB pcap, this will stream those packets at their original rate over the course of those six hours. In this way, streaming Zeek events stream into Apache Kafka at their original rate.
This Docker image is also (maybe) unique in that it uses a dummy0 network interface, so you can replay all kinds of nasty garbage without causing actual harm to network admins.
Specify your broker host, run from your current working directory with a file named zeek_streamer.pcap at ./pcaps/zeek_streamer.pcap
docker run -it \
-v `pwd`/pcaps/:/pcaps \
--cap-add=NET_ADMIN \
--add-host broker:192.168.1.108 \
bertisondocker/zeek-tcpreplay-kafka:latest
Or, run with your own local.zeek and send-to-kafka.zeek files:
docker run -it \
-v `pwd`/pcaps/:/pcaps \
-v `pwd`/local.zeek:/usr/local/zeek/share/zeek/site/local.zeek \
-v `pwd`/send-to-kafka.zeek:/usr/local/zeek/share/zeek/site/send-to-kafka.zeek \
--cap-add=NET_ADMIN \
--add-host broker:192.168.1.108 \
bertisondocker/zeek-tcpreplay-kafka:latest
This image is designed to start automatically replaying the file /pcaps/zeek_streamer.pcap
$ ls pcaps/
heavy_dns.pcap zeek_streamer.pcap
So if you already have a file named zeek_streamer.pcap as above, then you're all set. If your pcap isn't (or wasn't) named zeek_streamer.pcap when you started, you can still feed it to Zeek and have the results stream to Kafka.
Since ./pcaps/ is a bind mount to /pcaps in the docker image :
docker exec -d <CONTAINER ID> /usr/bin/tcpreplay -i dummy0 /pcaps/heavy_dns.pcap