Messaging unification: investigate kafka

Question

Closed this issue 5 years ago · 3 comments

Look into drastically reducing the messaging complexity for this system by leveraging kafka for all service communication:

Drop MQTT and use a websocket gateway to connect to microcontrollers. See https://tttapa.github.io/ESP8266/Chap14%20-%20WebSocket.html
Drop Redis pub/sub
Drop google cloud messaging and leverage the existing push_redis endpoint to move data up to the cloud. Retain the authorization layer

Answer 1 · 2019-07-25T18:12:40.000Z

Kafka system will use a lot of memory when running on raspberry pi

Answer 2 · 2019-08-01T11:45:27.000Z

Also see #69 - we could leave all messaging alone and add Prometheus to help track historical sensor reports

Answer 3 · 2020-05-23T15:15:37.000Z

It's too scalable, and a fully developed set of Kafka apps uses a lot of RAM. Stick with mqtt/Redis Streams