Knocking demo using the GIoTTO Stack. A Web version of this documentation is HERE
This is an open source application built on top of the GIoTTO stack to showcase how an IoT app can leverage the stack functionality to offer advanced features and complex interactions between multiple IoT entities. It also demonstrates how Edge Analytics reduces the end-to-end time required for IoT scenarios. The schematic diagram of the application is shown in Figure 1.
- [TI sensor tag] (http://www.ti.com/tool/cc2541dk-sensor)
- [Raspberry PI] (https://www.raspberrypi.org/blog/raspberry-pi-3-on-sale/)
- IP camera
- Internet connection
The [knockingSettings.json] (./knockingSettings.json) file holds the configurations required for the demo:
- camera_ip, the IoT camera IP in IPv4 format 0.0.0.0
- domain, url for GIoTTO
- port, DataService port @ GIoTTO
- client_id, client id for authentication
- client_key, client key for authentication
- res (reserved), checking resolution, how frequent to check on the sensor data
- knockingSensor_uuid, where knocking sensor posts time-series data
- postSensor_uuid, where google cards information are posted
- snapshot_fileName, file name of the knocker's face
The entire GIoTTO stack can be downloaded here. For full description, installation guide and API documentation, please refer to the BuildingDepot [webpage] (http://buildingdepot.andrew.cmu.edu/). Once installation is complete, you need to register with the platform and create a username and password. Use the registration credentials to login to the system and create 2 sensor IDs: knockingSensor_uuid to store the knocking sensor times series data and postSensor_uuid to post the Google Now card information. Copy the generated uuids of both sensors to respective lines in the [knockingSettings.json] (./knockingSettings.json).
The TI sensor tag is used to detect the knocking signal. The current version of the TI sensor tag can communicate only through BLE. Therefore, you need a gateway to communicate the data generated by the sensor to the platfrom. As such, we use a Raspberry PI as a gateway. The mac address of the TI sensor tag is a constant in the file virtualSensorMonitor.py as DEFAULT_MAC_ADDRESS = "replace-with-your-sensor-mac-address". Make sure that the raspberry pi is connected to the Internet. We also need to translate the raw sensor data generated by the TI sensor resulting from the knocking signal into a meaningful semantic information (e.g., knocking or silent). The machine leanring script in the virtual_sensor folder creates the required virtual sensor and post either "knocking" or "silent" to knockingSensor_uuid.
This demo uses OpenFace for face detection. OpenFace is a Python and Torch implementation of face recognition with deep neural networks. To install and setup OpenFace, please refer to the OpenFace guide in this link. However, OpenFace is pre-installed with the [GIoTTO] (http://cmu.buildingdepot.org/BuildingDepot.tar.gz) distribution as a docker container. You can access Openface in this folder /srv/openface. Nevertheless, Openface can be installed on any server, but make sure to update the proper changes to reach the server ip/domain in the code.
To train the classifier on your own photos, you need to place enough photos of yours in .png format (>3 is recommended) in /srv/openface/data/mydataset/raw/YOUR_NAME/ and run ./train.sh -- In a few minutes the classifier will be able to recognize your face. Note that this is done inside the docker container (i.e., run ./start.sh first)
Currently, the Google Now API is not available for public. You need to whitelist your email with Google to be ble to access their Now API. Then, you need to create an account with Google developer if you don't have one already. Once you register, you will receive a client id and a client secret tokens. Those go into client_id and client_secret parameters in the client_secret.json file. You will need to request and access token and replace any access_token instance in the files: makeCard.py. To get an access token, you just need to put your credentials in refreshtoken.py file. Google cards in this demo contains actions buttons (e.g., call and send SMS). Some of these actions require background processing, which we provide in an Android app in this folder AndroidDLApp. To successfully generate cards on a Google-whitelisted account, all card actions/conditions must be satisfied. Hence, you need to install this app on the mobile device participating in the demo. No need to run it though.
- Run install.sh (in privileged mode, i.e. sudo)
- Make sure the the TI sensor tag is running and the green light is flashing
- Connect to the raspberry pi serving to the TI sensor tag and run virtualSensorMonitor.py
- Make sure that the camera is connected to the Internet
- Connect to the GIoTTO platform (i.e., server running GIoTTO) and run start.sh
- start.sh executes the following instructions to ensure Docker is up and running and initialize a new container
service docker restart
docker run -p 9000:9000 -p 8000:8000 -v /srv/buildingdepot/Documentation/build/html/:/var/www/html -t -i bamos/openface /bin/bash- Go to cd ./knocking (i.e., inside the docker container)
- Run ./knocking.py
- Enjoy the demo
Unless otherwise stated, the source code and trained Torch and Python model files are copyright Carnegie Mellon University and licensed under the Apache 2.0 License. Portions from the following third party sources have been modified and are included in this repository. These portions are noted in the source files and are copyright their respective authors with the licenses listed.