ml4ai/tomcat-baseline-tasks

Baseline task data collection running on ToM, storing data on CAT

Closed this issue · 2 comments

@adarshp suggested using ELK to store baseline task data on CAT while running the server on ToM.

There are three components that can live on separate machines or the same machine:

  • ELK stack (3 containers)
  • MQTT container
  • Minecraft container

Messages from the Minecraft container travel to the MQTT container, which then pushes them to the ELK stack. Depending on the desired latency and timestamp generation machine, we can put these three components on one machine (like we currently have with TOM), or split them among TOM and CAT.

After discussing with Chinmai and Kobus, it seems that if the time difference between ToM and CAT is less than a millisecond (they should both be syncing to the same time server), then there is no need to have CAT receiving the data, putting the timestamp, and storing locally on CAT.

We will run the baseline task server on ToM and writing to CAT NFS folder using ToM timestamp.