Imagimob - AFarCloud project source code

Edge AI tractor tracker and cow activity monitoring collar

This repository contains the code for the sensors firmwares and backend services developped during the AFarCloud project as presented at the final demonstrator in Pisa, November 2021.

Principle

The platform consists of sensor modules placed on animals or legacy equipement (for the demonstration, we used sensors in collars on cows and on legacy tractors) that collect - among others - movement data, analyze it using a neural network on the sensor itself to detect high level behavioral patterns and send aggregated data about these over LoRa. The data is then processed and decoded by the backend before behind forwarded to the AFarCloud for further usage.

Data capture and labelling

Data capture and labelling is done using small bluetooth enabled sensors, the SmartBond SDK from dialog semiconductors and the free Imagimob Capture app.

The app simultaniously records video and accelerometer data, and allows the user to label data as the video is being recorded.

The labelling can be further improved after capture using Imagimob Studio

Training

Training of the neural network from labelled data is done using Imagimob AI devlopment platform.

After a satisfactory model has been trained, the platform exports it as pair of C files that contain all the routines and parameters for that particular model. (See the example models developped during AFarCloud in Model_CowCollars and Model_Tractors).

These models require no external dependencies and being nothing more C99-complaint C code, will run on practically any microcontroller platform.

Sensors

For the demonstrator, we used two types of boards from the TTGO family build around the widespread Expressif ESP32 CPU:

The T-Beam was used for the smart tractor trackers
The T-Display was used for the cow collars (due to smaller form factor)

The main difference between the boards is the lack of GPS and smaller form factor on the latter. The OLED display was not used in the project and was disabled on the cow collar.

The boards need at least an additional Bosch BMI160 accelerometer/gyroscope and optionally a Bosch BME280 environmental sensor soldered to the I2C bus.

Network

The final demonstrator used The Things Network, but integrations with the Finnish country-wide LoRa network Digita as well with self-hosted lorawan-server was tested and implementations are available in the backend.

Backend

The backend code consists of a few microservices written in python that receive input from the LoRa network provider and communicate with each-other over MQTT. The services process raw input from the sensors into high level json structures and forward the data to the AFarCloud platform in a suitable format.

The data is also captured locally in a Postgres database (or optionally InfluxDB) and can be visualized using Grafana, allowing some basic usage of the solution independently of the AFarCloud infrastructure if needed.

The backend stack runs on Docker and is currently configured to run entirely on one server. Running the stack in a fully distributed fashion has not been considered in this project.

All the configurations, states and data generated by the services are stored in the folder ./data, i.e. this and only this part needs regular backups.

Firmware

The firmware uses Arduino for ESP32. Please refer to Expressif's official documentation for setting up the toolchain.

Dependencies:

The following external libraries are required for compiling the firmware:

AXP202X : Library for controlling the programmable power supply on the V2 TTGO T-beam board. This is the standard library, forked for convenience.
BMI160 : Library for reading data from the BMI160 acceleremoter/Gyroscope. This is the standard library, forked for convenience.
BME280-I2C-ESP32 : only required if using the BME280. The version hosted on Imagimob's github repository is customized for this project and backwards compatible with the original library --- but the original version won't work in this project. Note that you will also need to uninstall the official Adafruit BME280 if you have it as it will cause conflicts with this one.

All these dependencies should be cloned into your ~/Documents/Arduino/libraries/ folder.

Flashing:

Have a look at the config.h file and make changes according to your needs and/or region. You also need to create a secrets.h with suitable LoRa app and dev UI keys (see sample file provided).

Then, compile and flash just like for any other Arduino-compatible board.

Backend :

Requirements

To run the stack, you will need Docker and docker-compose.

Configure the stack to your needs

You need to provide a suitable .env file with parameters for your specific use case. An example of .env file is provided in env.example.

Prepare the environment:

Download all necessary docker images:

docker-compose pull

Compile and build everything that needs to be compiled and built locally:

docker-compose build

Run the application

For development, you probably want to launch the stack as

docker-compose -f docker-compose.yml -f docker-compose.localdev.yaml up

On a production server, you should probably do docker-compose -f docker-compose.yml -f docker-compose.prod.yaml up -d

The latter will also call Let's encrypt to get the right certificates and configure an SSL proxy automagically.

To shut down the entire application: docker-compose -f ... -f ... down

Remember: Docker-compose will automaticall restore the services after a reboot untill a docker-compose down.

Troubleshooting

I get error messages about undefined environmental variables right after docker-compose up : your .env is mis-configured or you are running the command in a wrong folder.
It stopped working and I don't really understand why :

docker-compose down
docker-compose up -d --force-recreate

should get you out of most troubles...

Seems I f**** it up more than that:

docker-compose down
docker system prune -a
docker-compose up -d --force-recreate

That's maximum possible clean up without touching the application data. If it doesn't work after that, your problem is in your ./data folder and you shall be careful not to loose data...

No way - I can't connect to Postgres...: You might have attempted to change the user, password or database name in the .env file for an existing database! You should not do this...

Explanation: the info in .env file is used by Postgres only for initializing the database. Changing it in the .env will not "update" the information in the database automatically, but all clients will take the new user/password upon restart.

If you do not care about your data:

docker-compose down
rm -rf ./data/postgis/
docker-compose up -d --force-recreate

Otherwise, if you need your data:

docker-compose exec postgis /bin/bash

and go and fix your database manually using psql

Bottom line: if you want to change the Postgres password, change it .env first, then change it in Postgres with psql, and then restart the services!

nicoinn/AFarCloud