Pyaiot provides a set of services for interacting and transporting data coming from IoT nodes using regular web protocols (HTTP) and technologies. Pyaiot relies on Python asyncio core module and on other more specific asyncio based packages such as Tornado, aiocoap or HBMQTT. Pyaiot tries to only use standard protocols to connect the IoT nodes to the web: CoAP, MQTT, HTTP, etc
Pyaiot main goal is to provide high level services for communicating with constrained nodes. Those nodes are generally microcontrollers and thus not able to run Linux. Thus, we need a specific OS to run on those kind of nodes. For this, we initially chose RIOT because it provides an hardware independent layer along with the standard network stacks required to communicate with the nodes from a network.
The source code of RIOT firmwares running on the nodes is available in another repository on GitHub.
Other nodes with communication capabilities can also be used. In this repository, we also provide a Micropython script that can be used on Pycom nodes. This script only works with the mqtt gateway service.
Pyaiot is built around several microservices:
- A public central broker
- A public web application for the dashboard
- Private distributed gateways
The role of the broker is to put in relation gateways and web clients in order to be able to transfer in a bi-directionnal way messages coming from nodes, via the gateways, to clients and vice versa.
The broker is in charge of the management of the list of gateways. The role of the gateways is to convert protocols used by the nodes to the web protocols used internally by Pyaiot to transfer information between the different services. In order to guarantee reactivity and security, this internally used protocols rely on HTTP websockets.
The Dashboard is a web page with some embbeded javascript that displays the list of available nodes and their status. It also allows to interact with the nodes (LED control, Robot control, etc)
3 examples of gateways are provided by pyaiot:
- A CoAP gateway that manages a list of alive sensor nodes by running its own CoAP server
- A MQTT gateway that manages a list of alive sensor nodes by subscribing and publishing messages to a MQTT broker.
- A Websocket gateway dedicated to nodes: each node is connected via a websocket
Here we describe how the CoAP gateway interacts with nodes.
When a node starts, it notifies itself to its gateway by sending a CoAP post request. On reception, the gateway converts and forwards this message to the broker server. In the mean time, the gateway initiates a discovery of the resources provided by the node (using the CoAP .well-known/core resource). Once available resources on the node are known, the gateway sends to the broker update messages. The broker simply broadcasts those notification messages to all connected web clients.
To keep track of alive nodes, each node has to periodically send a notification message to its gateway. If a sensor node has not sent this notification within 120s (default, but this is configurable), the gateway automatically removes it from the list of alive nodes and notifies the broker.
MQTT do things differently from CoAP: the nodes and the gateway have to publish or subscribe to topics to exchange information.
A resource discovery mechanism is also required for the gateway to determine the list of available resources on a node. In Pyaiot, the gateway and the nodes are used as clients of the same MQTT broker.
Since the nodes are constrained, we decided to use the mosquitto.rsmb broker. Some documentation and a sample systemd service file is provided in the pyaiot/gateway/mqtt directory.
With the MQTT gateway, each node is identified by a <node_id> and this identifier is used to build the topics specific to a given node.
Topics the nodes publish to/the gateway subscribes to are:
node/check
with payload{'id': <node_id>}
. Once started, each node publishes periodically (every 30s) on this topic.node/<node_id>/resources
with payload[<resource 1>, <resource 1>]
. This comes as a reply to the gateway request for resources available on a given node.node/<node_id>/<resource>
with payload{'value': <resource_value>}
. Depending on the type of resource, a node can publish periodically or not or on demand on this topic.
Topics the nodes subscribe to/the gateway publishes to are:
gateway/<node_id>/discover
with 2 possible payloads:resources
: then the node publish onnode/<node_id>/resources
values
: then, for each resource, the node publish onnode/<node_id>/<resource>
gateway/<node_id>/<resource>/set
with a payload depending on the resource: update the value of the resource on the node (LED toggle, text, etc)
The behavior with a websocket gateway is similar to the CoAP gateway except that the node doesn't have to send notifications periodically: the node is lost when the connection is closed.
A basic authentication mecanism based on symmetric cryptography exists between the broker and gateways. This prevents unwanted gateways to connect to your broker.
Before any installation, a pair of keys needs to be generated using the
provided tool aiot-generate-keys
:
bin/aiot-generate-keys
The tool writes the keys in the user home directory in ~/pyaiot/keys
.
By default, the broker and gateway services look in this location but one can
specify custom key file with the --key-file
option when starting the
services.
Thanks to this, you can have gateways on different hosts connecting in a secured way to your central broker. The important thing is to have your broker reachable from the gateways and clients.
See Pyaiot in action within 2 demos:
-
RIOT: You can find a permanent demo instance configured as a showroom for RIOT. This showroom is available at http://riot-demo.inria.fr.
-
IoT-LAB open A8 demo This demo automatically submits an experiment on IoT-LAB with two open A8 nodes. The first node is configured as a border router and the second node runs a firmware that integrates automatically in the RIOT Demo Dashboard described above.
Here are the steps to install Pyaiot on a standalone system. The final setup will be as follows:
aiot-broker
andaiot-dashboard
running as systemd services- the
aiot-broker
websocket server listening on port 8082 - the
aiot-dashboard
web application listening on port 8080. All served pages open a websocket client on the port 8082 of the broker - the
aiot-coap-gateway
listening on UDP 5683 CoAP port - the
aiot-ws-gateway
websocket server listening on port 8083
The gateway services are optional.
For a custom setup, please edit the Environment
option of the
aiot-broker,
aiot-dashboard,
aiot-coap-gateway and
aiot-ws-gateway systemd service files
before deploying. The broker port should be the same in both service files
if you want them to work together.
Here are the installation steps:
- Clone this repository
$ git clone https://github.com/pyaiot/pyaiot.git
- Deploy the system (works for debian/raspbian/ubuntu):
$ make deploy
- Verify that the services are correctly running:
$ sudo systemctl status aiot-broker.service ● aiot-broker.service - Pyaiot Broker Application Loaded: loaded (/lib/systemd/system/aiot-broker.service; enabled) Active: active (running) since dim. 2016-12-18 14:59:56 CET; 35min ago Main PID: 32411 (python3) CGroup: /system.slice/aiot-broker.service └─32411 /usr/bin/python3 /usr/local/bin/aiot-broker --port=8082 --debug [...] $ sudo systemctl status aiot-dashboard.service ● aiot-dashboard.service - Pyaiot Dashboard Application Loaded: loaded (/lib/systemd/system/aiot-dashboard.service; enabled) Active: active (running) since dim. 2016-12-18 14:52:29 CET; 41min ago Main PID: 32321 (python3) CGroup: /system.slice/aiot-dashboard.service └─32321 /usr/bin/python3 /usr/local/bin/aiot-dashboard --port=8080 --broker-port=8082 --broker... [...]
- Deploy a coap gateway (optional):
$ make setup-coap-gateway-service
- Deploy a ws gateway (optional):
$ make setup-ws-gateway-service
You can also update the Environment
option in the services definition files
after deployment. The services files are located in `/lib/systemd/system.
Note that you'll have to reload the systemd daemon services and restart
services:
$ sudo systemctl daemon-reload
$ sudo systemctl restart aiot-broker
$ sudo systemctl restart aiot-dashboard
$ sudo systemctl restart aiot-coap-gateway-service
$ sudo systemctl restart aiot-ws-gateway-service
Example: Environments used in the online RIOT demo
- aiot-broker:
Environment='BROKER_PORT=8082'
- aiot-dashboard:
Environment='STATIC_PATH=/home/pi/demos/pyaiot/dashboard/static' \
'HTTP_PORT=8080' \
'BROKER_PORT=80' \ # This is because the broker is behind an apache proxy
'BROKER_HOST=riot-demo.inria.fr' \
'APP_TITLE=RIOT Demo Dashboard' \
'APP_LOGO=/static/assets/logo-riot.png' \
'APP_FAVICON=/static/assets/favicon192.png'
- aiot-coap-gateway:
Environment='BROKER_HOST=riot-demo.inria.fr' \
'BROKER_PORT=80'
- aiot-ws-gateway:
Environment='BROKER_HOST=riot-demo.inria.fr' \
'BROKER_PORT=80' \
'GATEWAY_PORT=8083'
Here we take as example the online demo available at http://riot-demo.inria.fr. The websocket server of the broker service is reachable on port 80. As the broker and the dashboard are decoupled in 2 distinct services, it's possible to run a local dashboard application serving dashboard web page that itself connects to the broker. This way your dashboard will display the available nodes on the online RIOT demo.
In this configuration, you don't need to install all the services but you still need to install the required development dependencies the first time:
$ make install-dev
Then you can start the dashboard application:
$ make run-dashboard
and open a web browser at http://localhost:8080. When the web page is loaded, thanks to its embedded javascript, it directly connects to the broker websocket server and starts to communicate with the nodes.
Of course the default environment variables can be changed in order to fit your
needs:
BROKER_PORT
, BROKER_HOST
, DASHBOARD_PORT
, DASHBOARD_TITLE
,
DASHBOARD_LOGO
, DASHBOARD_FAVICON
, CAMERA_URL
.
$ BROKER_PORT=8082 BROKER_HOST=localhost make run-dashboard