/thinx-device-api

Remote IoT Device Management Platform

Primary LanguageCSSOtherNOASSERTION

☢ thinx-device-api

IoT Device Management Server running on node.js.

Vulnerabilities Build Status CodeFactor Security Rating codebeat badge Language grade: JavaScript Codacy Badge Total alerts Coverity Scan Build Status Quality Gate Status Coverage Status License FOSSA Status Greenkeeper badge Twitter: @thinxcloud

The CircleCI build is limited and therefore returns mostly bad results. Closer look may show better numbers.

The Purpose

  • Update IoT device by pushing a code to a Git repository. We'll build it.
  • Swap operating system for another over-the-air.
  • Migrate multiple devices at once between WiFi networks.
  • THiNX provides complete IoT infrastructure for your device (where the data storage and visualisation can be fully up to you).
  • automatic updates for headless devices, or semi-automatic (with user consent after build and tests succeed)

As a user I have already many IoT new and/or legacy devices at home and new platforms are coming every day.

Sometimes we need to change WiFi credentials on a wireless switch mounted on a ceiling. The other day I we want to swap whole firmware for new one, but not always to rewrite working legacy Lua or Micropython code to PlatformIO.

That's why we have decided to create the über-platform: THiNX.

Supported hardware

Currently the platform supports building firmware for Arduino, PlatformIO (also with ESP-IDF), NodeMCU, Mongoose, Micropython and features JavaScript library that is intended to use on any hardware capable of running a Node.js server.

Features

  • Remote Things Management console for monitoring devices, attaching source code, pushing data, managing incoming payloads and firmware updates.

  • Continuous Integration practices to update device apps/configurations from a GitHub repository using commit hooks

  • Building secure MQTTS infrastructure as an optional side-chain transport layer.

  • Device registration endpoint while storing device data using CouchDB server and Redis session-store.

  • API is a back-end data provider (security agent) for RTM Admin Console Application.

  • Provides control to a dockerized build servers and pushes new firmware versions to client applications (FCM push) and devices (MQTT).

  • Provides HTTP-to-HTTPS proxy to secure legacy IoT devices that are not capable of TLS and/or AES-level encryption.

  • Allows transfer of device ownership (e.g. for pre-configured devices).

  • Custom firmware builder for MongooseOS, NodeMCU and Micropython (allow module selection, add THiNX as an OS-level library)

  • Transfer device to another owner along with sources/firmware.

  • Device status messages can be transformed using custom JavaScript lambda-style functions.

  • Supports OAuth login with Google and GitHub.

  • Supports InfluxDB/Grafana data storage and visualisation.

  • Supports LoRaWan server integration.

  • Supports Rollbar, Sqreen and Crisp.chat integrations.

  • Message-queue integration using docker-compose project on thinx_internal network

  • Supports Traefik for SSL offloading.

Supported IoT Platforms

  • PlatformIO and Arduino IDE (ESP8266P/ESP32)

  • Micropython

  • Lua

  • MongooseOS

  • NodeJS (Mac/Linux/Windows)

  • Tested on Wemos D1 Mini, Wemos D1 Mini Pro, RobotDyn D1, RobotDyn D1 Mini, RobotDyn MEGA WiFi and various NodeMCU (Lolin, AI-THINKER) boards with Mongoose, Arduino Core, ESP-IDF, Lua and Micropython-based core firmwares...

  • Expected: Arduino and BigClown with networking support

Base THiNXLib Platform Library in C++:

THiNXLib for ESP8266

THiNXLib for ESP32

THiNX Platform Library repositories for various IDEs and firmwares:

Platform.io

Arduino

NodeMCU/Lua

Micropython

MongooseOS

NodeJS

Custom Firmwares

With built-in THiNX Client Library:

NodeMCU/Lua

Micropython

Arduino, Plaform.io and MongooseOS are firmwares by nature.

Dockerized Firmware Builders

PlatformIO

Arduino

MongooseOS

NodeMCU/Lua

Micropython

Prerequisites for running own THiNX Server

  • Linux Server (min. 2 GB RAM, 32GB SSD, Ubuntu)
  • Docker

Port mapping

  • API runs on HTTP port 7442 (HTTPS 7443) and 7444 (web socket)
  • MQTTS runs on port 8883
  • Admin runs on HTTP/HTTPS port (80/443)
  • GitHub commit hooks are listened to on port 9002
  • Status Transformers (internal network only, 7445)

Logging

Use your favourite service and log-sender agent. Tested successfully with Logz.io, Instana and Sematext

Installation

Prerequisites

Suggested:

Optional:

Using docker-compose

Make sure you have valid directory structure available at /mnt/data (default) and edit the .env file to suit your needs.

You don't need Mailgun for developer installation, just copy-paste the activation URL from api log using docker-compose logs -f while creating your first admin account.

git clone http://github.com/suculent/thinx-device-api
cd thinx-device-api
cp .env.dist .env
cp .thinx_env.dist .thinx_env
nano .env
nano .thinx_env
./copy-envs.sh
docker-compose up -d --build

GitHub Webhook support

You can direct your GitHub web-hooks to https://thinx.cloud:9001/ after adding a valid deploy key from GitHub to THiNX RTM.

Endpoints

See 03-test.sh. There is no point of maintaining documentation for this at current stage of development and user base zero.

Platforms State of Union

Overall

Platform libraries are now stabilised on the basic level, approaching first release version 1.0 with default HTTPS with optional fallback to HTTP for development.

THiNX has now passed version 1.0 upgrading to docker-compose installation with separate container services (CouchDB, Redis, Transformers, THiNX, Traefik and optional monitoring services).

Data are being moved to configured location, which is by default /mnt/data:

deploy/ # build products ready for deployment to devices
mosquitto/ # auth, log, config, data, ...
repos/ # fetched/watched repositories
ssh_keys/ # will be moved to vault and provided exlusively to builder
ssl/ # shared SSL certificates, may be generated by Traefik/ACME/Letsencrypt

PlatformIO

  • Docker builder works.
  • Deployment update can be tested now.

Arduino

  • Docker builder has been recently updated.
  • Deployment update can be tested now.

NodeMCU

  • File-based update has been pre-tested. Docker builder works fine but needs tighter integration with sources ($workdir).
  • Deployment is not verified, therefore update cannot be tested.

Micropython

  • Docker builder works fine but needs tighter integration with sources.
  • Deployment is not verified, therefore update cannot be tested now.

License

FOSSA Status