MarshFlattsFarm/NIBE-Uplink-API

InfluxDB and Grafana integration

mloskot opened this issue · 2 comments

mloskot commented

Thanks for the great write up at the https://www.marshflattsfarm.org.uk/wordpress/?page_id=4988 and for sharing the code.

Would you have any notes on integrating the data storage in the InfluxDB for visualisation with Grafana?

MarshFlattsFarm commented

Hi Mateusz; thanks for the positive feedback.

In very general terms, my approach to storing and graphing the data is based on MQTT which enables me to take data feeds from multiple different sources and 'rationalize' them into a set of MQTT Topics, masking the details of how the data points were acquired. For example:

  • I have some temperature data that comes in from OneWire sensors
  • I have some other temperature and air quality data that uses 433MHz Oregon Scientific sensors
  • I have real-time electricity consumption & generation data coming from CT clamp sensors
  • I have some API-derived data - like that from the NIBE Uplink API

For me, it's convenient to be able to consume all of those as MQTT Messages - no matter how they were derived.

I then use the MQTT Plugin for Telegraf to grab all of the MQTT messages and record those in InfluxDB, tagged with the MQTT Topic hierarchy label - and from InfluxDB I select various combinations of Topics to include in Grafana dashboards.

I then have various Scripts which run every few minutes and grab the data from the relevant source (in this case the NIBE Uplink API) and publish each reading as an MQTT Message - which then follows the Telegraf --> InfluxDB route ready for (possibly) visualizing with Grafana.

It's not essential to use MQTT and Telegraf - there's no reason why a Script couldn't write directly to InfluxDB - but there are some Messages it's useful to respond to in real time, so (for me) there are benefits in having an "Event" to react to, rather than just a reading going directly to InfluxDB.

Right now, my "production" data logging for the NIBE Uplink API is based on Perl scripts that I probably wouldn't recommend copying. For another project I need to log MQTT Messages from Python so happy to share further info on that when I get to it.

David

mloskot commented

David, thank you for the further explanation. I've got the more complete picture now.
I am going to set up the environment and experiment with your scripts.
For now, my aim is to store and visualise data from the NIBE Uplink API.

I will share whatever I come up with.