firmata-mqtt

Description

This project provides a Docker container to read signals from a microcontroller and publish them over MQTT.

Its main use case is to integrate impulse devices like S0 counters into a home automation system.

Prerequisites

A microcontroller running a Firmata-compatible firmware (e.g. an Arduino running Firmata or ConfigurableFirmata)
A running MQTT broker (e.g. mosquitto)
Docker
Optional and recommended: Docker Compose

The Docker Compose [documentation](https://docs.docker.com/compose/install/) contains a comprehensive guide explaining several install options. On recent Debian based systems, dependencies may be installed using the command

$ sudo apt-get install docker-ce docker-compose-plugin

TL;DR

Install Docker and MQTT broker.
Download Arduino IDE and flash Firmata.
Connect a hardware device (e.g. a S0 counter) to the Arduino:
- Device signal (+) → Arduino digital pin (e.g. pin 2)
- Device GND (-) → Arduino GND
Create compose.yml configuration file.
Run docker compose up -d.

Usage

Create a run configuration, e.g.

compose.yml

  ---
  services:
    firmata-mqtt:
      image: "ckware/firmata-mqtt"
      container_name: "firmata-mqtt"
      init: "true"
      restart: "unless-stopped"
      devices:
        - /dev/ttyUSB0
      environment:
        MQTT_OPTIONS: "-h broker-host"
        FIRMATA_PINS: "9:boiler,12"
        FIRMATA_TEMPLATE: "pullup:debounce:falling:1:W"

Start a container:
```
$ docker compose up -d
```

Receive value changes:

$ mosquitto_sub -h broker-host -v -t firmata/#
firmata/9/json {"pin":9,"name":"boiler","rate":155.193070189773,"unit":"W","value":0,"timestamp":1592731404.896913,"duration":23.1969120502472,"count":1,"total":2}
firmata/12/json {"pin":12,"rate":85.9448365143275,"unit":"W","value":0,"timestamp":1592731423.586332,"duration":41.8873331546783,"count":1,"total":2}
^C
$ docker compose down
$ docker compose logs
Connecting to Firmata device at /dev/ttyUSB0
Observed pin(s): 9, 12
Stopping due to signal TERM
Disconnected from Firmata device.

Configuration

Table 1. Firmata Options

Environment variable	Description	Allowed values	Default	Example
`FIRMATA_PINS`	Comma-separated list of pin number with optional name and pin configuration	pin[:name[:configuration]][,pin[:name[:configuration]] …]	no default	`2` `2,3,4` `2,3::input,4:boiler:pullup:debounce:falling:1:W`
`FIRMATA_DEVICE`	Serial firmata device to read messages from.	Any special character file	`/dev/ttyUSB0`	`/dev/ttyUSB1`
`FIRMATA_TEMPLATE`	Template containing the default pin configuration for all pins. See Pin Configuration for details.	`input` / `pullup` `:` `instant` / `debounce` : `none` / `rising` / `falling` / `any` : number : string	`pullup:instant:none`	`pullup:instant:falling:1:W` `pullup:debounce:falling:2000:m³/h` `pullup:debounce` `:debounce:::W`
`FIRMATA_INTERVAL`	Poll interval in milliseconds. When strategy is `instant`, value changes are reported immediately, not only between poll intervals (see Detection strategy).	Positive integer	`50`	`100`
`FIRMATA_COMMAND`	Command that is run for each message. See Command Line Interface for details.	Any executable file	`/opt/firmata-mqtt/mqtt-publish`	`/bin/echo`
`FIRMATA_VERBOSITY`	Log verbosity.	`0` / `1` (verbose) / `2` (debug)	`0`	`1`

Table 2. MQTT Options

Environment variable	Description	Allowed values	Default	Example
`MQTT_OPTIONS`	MQTT options	All options supported by `mosquitto_pub`	none	`-v -h broker`
`MQTT_TOPIC`	MQTT topic for publishing sensor data	Topic names	`firmata`	`devices/sensors`
`MQTT_TOPIC_APPEND_ID`	Append sensor ID to topic?	`true` / `false`	`true`	`true`
`MQTT_TOPIC_APPEND_FORMAT`	Append format (one of: `json`, `raw`) to topic?	`true` / `false`	`true`	`true`
`FORMAT_JSON`	Publish sensor data in JSON format?	`true` / `false`	`true`	`true`
`FORMAT_RAW`	Publish sensor data in raw format?	`true` / `false`	`false`	`false`
`FORMAT_RAW_SEPARATOR`	Field separator for raw format	String	Whitespace (`\u0020`)	`,`
`CLIENT_MOSQUITTO`	Use Mosquitto as MQTT client?	`true` / `false`	`true`	`false`

Table 3. Log Options

Environment variable	Description	Allowed values	Default	Example
`LOG_FILE`	If set, topic and message are appended to a log file	File path	not set	`/var/log/tfrec-mqtt.log`
`LOG_FORMAT`	Format to write topic and message to log file	`printf`-compatible syntax for 2 arguments	`%s %s\n`	`{"topic":"%s","message":%s}\n`

Pin Configuration

A pin configuration is a colon-separated list of the properties listed in this section. All properties are optional, trailing colons may be omitted.

Pin Mode

Supported pin modes: input, pullup. Default: input.

See Digital Pins in the Arduino Tutorial for details.

Detection strategy

Supported detection strategies: instant, debounce. Default: instant.

instant: Changes will be reported instantly.
debounce: Value changes will be buffered until the current poll interval has elapsed (see FIRMATA_INTERVAL). This may help when bouncing switches are connected to the Firmata device. Please note that this option effectively reduces the sample rate to the poll interval, thus the poll interval should be chosen carefully according to the connected hardware.

Rate trigger

Supported rate triggers: none, falling, rising, any. Default: none.

When rate trigger ist set, the value changes are supposed to come from a meter device, and a rate per hour is calculated as

3600 / (frequency * duration)

falling: The rate will be calculated for every value change from 1 to 0.
rising: The rate will be calculated for every value change from 0 to 1.
any: The rate will be calculated for every value change.
none: The rate will not be calculated.

Rate frequency

The frequency tells how many changes per hour the meter device emits.

Supported frequencies: positive numbers

Rate unit

The unit contains the unit of the meter device.

Examples

Example: S0 energy meters emitting 1000 impulses per kWh
```
  environment:
    FIRMATA_PINS: "2:boiler,3:washer"
    FIRMATA_TEMPLATE: "pullup:debounce:falling:1:W"
    MQTT_OPTIONS: "-h broker-host"
```
- Connect to firmata device at /dev/ttyUSB0 (default)
- Configure pin 2 with name boiler and pin 3 with name washer
- Enable pullup, software debouncing and rate calculation for all pins
- Publish to broker-host
Example: Different devices with lower sample rate
```
  environment:
    FIRMATA_DEVICE: "/dev/ttyUSB1"
    FIRMATA_PINS: "2::pullup,3::input"
    FIRMATA_INTERVAL: "20"
    FIRMATA_VERBOSITY: "1"
    MQTT_OPTIONS: "-h broker-host"
```
- Connect to firmata device at /dev/ttyUSB1
- Disable software debouncing (default)
- Configure pin 2 as digital input with pullup
- Configure pin 3 as digital input pin (without pullup)
- Poll every 20 ms for changes
- Log verbose messages
- Publish to broker-host
Example: Debugging
```
  environment:
    FIRMATA_PINS: "2"
    FIRMATA_VERBOSITY: "2"
    FIRMATA_COMMAND: "/bin/echo"
```
- Connect to firmata device at /dev/ttyUSB0
- Configure pin 2 as digital input with pullup
- Log debug messages
- Do not publish over MQTT but call /bin/echo instead.

Command Line Interface

The main part of this project is a command line program that connects to a microcontroller using the Firmata protocol and observes its input pins. For each value change of an observed pin, an external command is called with the following arguments:

pin name value timestamp duration count total rate unit

By default, the external command is a shell script that converts the arguments to JSON (or optionally keeps them raw) and publishes them over MQTT. The command option may be used to set a different command for custom processing.

Arguments

Illustration of a value change:

──┐               ┌──  1
  │←───── d ─────→│
  └───────────────┘    0
                  ↑    ↑
                  t    v
v: value
t: timestamp
d: duration

pin: An integer containing the pin number.
name: A string containing a name for the connected device.
value: The value as reported by Firmata, e.g. 0 or 1.
timestamp: A decimal containing the timestamp of the value change. The integer part contains a unix timestamp (seconds since epoch). The fractional part has a precision of 9 digits (nanoseconds).
duration: A decimal containing the duration since the previous value change with a precision of 9 digits (nanoseconds).
count An integer containing ths pin’s number of changes to the current value.
total An integer containing the pin’s total number of changes.
rate A string containing the value change rate per hour.
unit A string containing the unit (e.g. W or m³).

Example

'9' 'boiler' '1' '1591428675.880354881' '2.1215808391571' '3' '5' '1696.85' 'W'

Explanation: pin 9 with name boiler has changed to value 1 at 1591428675.880354881 (2020-06-06 07:31:15 and 880 ms, 385 µs, 881 ns). Before the change, the pin was stable for about 2.122 seconds (with value 0). This is the 3rd time that pin 9 changed to 1. The total number of value changes (either 0 → 1 or 1 → 0) of pin 9 is 5. If we suppose that an energy meter is connected and 1 Wh has been consumed within the duration, then the duration corresponds to the power of 1696.85 W.

FHEM integration

This section contains an example configuration to integrate a power meter with FHEM. The power meter is a S0 counter emitting 1000 impulses per kWh. It is connected to pin 9 of an Arduino. The pin value stays at 1 when idle and changes to 0 shortly (~70 ms) for every consumed Wh.