Securely stream data from ESP8266 MCUs to Azure IoT Hub over HTTPS/REST
Purpose
This solution securely streams sensor data directly to Azure IoT Hub or Azure Event Hub over HTTPS calling Azure REST APIs from ESP8266 MCUs.
Device Platform
The ESP8266 is a great commodity priced Arduino compatible MCU with integrated WiFi.
This project is implemented and tested on the following ESP8266 based development boards:-
Firmware
Arduino core for ESP8266 WiFi chip V2.0 firmware adds HTTPS (TLS 1.0 and 1.1) support, making this a viable platform for secure IoT data streaming. See Security Discussion for more information.
IoT Hub Certificate Verification
This sample verifies your IoT Hub Server Certificate to mitigate against Man in the Middle Attacks.
The Server Certificate Fingerprint was generated as follows:-
- From Bash on Ubuntu on Windows (10) or Linux
- echo -n | openssl s_client -connect IoTCampAU.azure-devices.net:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > cert.pem
- openssl x509 -noout -in cert.pem -fingerprint
- See Certificate Pinning article for more information
Azure IoT Hub
IoT Hub is designed to "Connect, monitor, and control millions of IoT assets" and is designed for internet scale data ingestion.
Stream Analytics, Power Bi and preconfigured IoT Hub solutions such as Remote monitoring provide ways to visualise and unlock the value of your data in Azure.
For more background information read this "Stream Analytics & Power BI: A real-time analytics dashboard for streaming data" article.
Acknowledgments
Thanks to Štěpán Bechynský "Proof of Concept – NodeMCU, Arduino and Azure Event Hub" project I've migrated my "Arduino NodeMCU ESP8266 MQTT" project and added IoT Hub support to stream data directly to Azure IoT Hub or Azure Event Hubs over HTTPS.
Setup and Deployment Summary
- Setup your Azure IoT Hub. There is a free 8000 message a day subscription to get started.
- Register your device with Azure IoT Hub.
- Update the main AzureClient.ino sketch
- Deploy the solution to your ESP8266 based device.
- View data with Device Explorer
- Optionally: Visualise your data in real time with Azure Stream Analytics and Power BI.
Setting Up Azure IoT Hub
Follow lab Setting Up Azure IoT to provision an Azure IoT Hub and an IoT Hub device.
Solution Configuration
const char* connectionString = "HostName=YourIoTHub.azure-devices.net;DeviceId=syd-board;SharedAccessKey=gsadE27VKloflZygS+Pvfye7cnm042uD4vPQdDC1yOE=";
const char* ssid = "Your WiFi SSID";
const char* pwd = "Your Wifi Password";
const char* geo = "Your geo location for the sensor";
Select Sensor
The sample includes support for these sensors: Fake Sensor, BMP180, BMP280, BME280, DHT11 and the DHT22.
Uncomment the sensor you wish to use.
Sensor sensor(&data); // Fake sample environmental data
//Bmp180 sensor(&data);
//Bmp280 sensor(&data);
//Bme280 sensor(&data);
//DhtSensor sensor(&data, &device, dht11);
//DhtSensor sensor(&data, &device, dht22);
Viewing Data
From Device Explorer, head to the Data tab, select your device, enable consumer group then click Monitor.
Visualising Data
Azure Stream Analytics
Azure Stream Analytics enables you to gain real-time insights in to your device, sensor, infrastructure, and application data.
See the Visualizing IoT Data lab. Replace the query in that lab with the following and be sure to change the time zone to your local time zone offset. Australia (AEDST) is currently +11 hours.
SELECT
iothub.connectiondeviceid deviceid,
Geo AS GeoLocation,
Max(DateAdd(Hour, 10, EventEnqueuedUtcTime)) AS TimeCreated, -- AU EST UTC + 10
Avg(Celsius) AS Temperature,
AVG(Humidity) AS Humidity,
AVG(Light) AS Light,
AVG(HPa) AS AirPressure
INTO
[PowerBI]
FROM
[TelemetryHUB] TIMESTAMP BY EventEnqueuedUtcTime
GROUP BY
iothub.connectiondeviceid, Geo,
TumblingWindow(minute, 30)
Power BI
Microsoft Power BI makes it easy to visualise, organize and better understand your data.
Follow the notes in the See the Visualizing IoT Data lab and modify the real time report as per this image.
Power BI Designer Setup
Power BI Report Viewer
View on the web or with the Power BI apps available on iOS, Android and Windows.
Data Schema
The AzureClient sketch streams data in the following JSON format, of course you can change this:)
{"Utc":"2017-01-12T08:00:38","Celsius":25.00,"Humidity":50.00,"hPa":1000,"Light":0,"Geo":"sydney","Schema":1,"Mem":21368,"Id":2}
ESP8266 Based Development Boards
There are a number of ESP8266 based development boards available so be sure to check out this great article "Comparison of ESP8266 NodeMCU development boards".
NodeMCU V2 with BMP180 Sensor
- NodeMCU v2
- BMP180 Barometric Pressure Sensor
- 1 x 400 Tie Point Interlocking Solderless Breadboard
- Wires
WeMos D1 Mini MCU
WeMos D1 Mini with BMP180 Sensor
- WeMos D1 Mini
- BMP180 Barometric Pressure Sensor
- 1 x Mini Breadboard
- Wires
WeMos D1 Mini with DHT Shield Sensor
No wiring required, just solder the supplied header pins for the WeMos and the DHT Sensor shield.
- WeMos D1 Mini
- DHT Shield or the DHT Pro Shield.
Software Requirements
Drivers
- NodeMCU - On Windows, Mac and Linux you will need to install the latest CP210x USB to UART Bridge VCP Drivers.
- WeMos - On Windows and Mac install the latest Ch340G drivers. No drivers required for Linux.
- ESP8266 Thing Development Board Hookup Guide
Arduino IDE
As at Jan 2017 use:-
- Arduino IDE 1.8.1.
- ESP8266 Board Manager 2.3 or better required for HTTPS/TLS Secure Client support.
Visual Studio
There an fantastic plugin for Visual Studio that adds Arduino support from Visual Micro. IntelliSence, auto complete, debugging, it doesn't get much better:)
Arduino Boards Manager ESP8266 Support
Arduino version 1.6.4 and above allows installation of third-party platform packages using Boards Manager.
- Install Arduino 1.6.8 from the Arduino website.
- Start Arduino and open Preferences window.
- Enter http://arduino.esp8266.com/stable/package_esp8266com_index.json into Additional Board Manager URLs field. You can add multiple URLs, separating them with commas.
- Open Boards Manager from Tools > Board menu and install esp8266 platform (and don't forget to select your ESP8266 board from Tools > Board menu after installation).
- Select NodeMCU or WeMos D1 Mini Board: Tools -> Board -> NodeMCU 1.0 (ESP-12E module) or WeMos D1 Mini
- Set Port and Upload Speed: Tools. Note, you may need to try different port speeds to successfully flash the device. Faster is better as each time you upload the code to your device you are re-flashing the complete ROM not just your code.
ESP8266 Arduino Core Documentation
Be sure to read the ESP8266 Arduino Core Documentation - there are some minor gotchas.