SCALE Client
The Safe Community Awareness and Alerting Network (SCALE) project leverages the client code in this repo: a generic Python package designed for monitoring sensor devices, performing local processing on this data, and publishing results to a (possibly cloud-based) data exchange for further use. Use this package as a quick method to efficiently and flexibly set up a sensing platform (usually on a Raspberry Pi) that incorporates multiple sensors, actuators, networking technologies, and/or protocols.
Quickstart
Clone the Git repo, ensure the current directory (./) is in your PYTHONPATH, install dependencies, and run the client from inside the main repo top-level directory. This will run the default client configuration, which should print logging info to the console related to the dummy VirtualSensors enabled by the default configuration file (see Configuring for details).
For ease of reference, run all the following:
cd ~
git clone https://github.com/KyleBenson/scale_client.git
cd scale_client
pip install -r requirements.txt
python -m scale_client --config ~/scale_client/scale_client/config/test_config.yml \
--log-level infoNote that you may also install the requirements in the file scale_client/stats/requirements.txt if you plan to use the statistics package for parsing results and/or generating random sensor data from statistical distributions.
If you wish to run SCALE on a Raspberry Pi device, you can follow the quickstart instructions or more detailed directions.
You can view the data stream using the web-based dashboard found at the SCALE project's website. Alternatively, you can use the mqtt_subscriber.py script for a command-line alternative.
Installation
For easy installation, simply run sudo python setup.py install from inside the main directory. Assuming you have setuptools installed via pip, this should handle installing all of the dependencies. Note that the codebase should run fine on any machine supporting Python 2.7+, but that it is tested on a Raspberry Pi running Raspbian and on Linux/Mac OSX machines (note that the latter won't support most physical sensor devices!). You don't need to run with sudo; doing so will install the daemon file.
NOTE: the setup.py script is no longer supported and may not install everything properly. Make sure you look at the various requirements.txt files for additional needed dependencies that may be commented out since we have so many different configuration possibilities!
WARNING: you'll also need to install our custom fork of CoAPthon if you wish to use CoAP! You can do this by cloning the repository and simply putting the coapthon Python package folder inside of it on your PYTHONPATH.
If you don't wish to install the package and instead run it straight from the repository (as shown above in the Quickstart section), you can do so as long as the directory containing scale_client is in your PYTHONPATH or in the current directory.
The scripts directory contains some useful scripts for e.g. setting up the devices, running a daemon, etc.
Running
You can run the code with python -m scale_client or scale_client/core/client.py as long as you set it up properly (i.e. your PYTHONPATH is set correctly and/or the Python package has been installed).
Documentation
To aid others in leveraging the SCALE project for their own IoT deployments, we documented the SCALE codebase, devices, and deployment methodology.
Advanced Usage Documentation
SCALE is designed to be highly flexible, easily configurable, modular, and extensible. For a quick example configuration file that you can modify and use right away look at scale_client/core/example_config.yml. For more information about configuring the SCALE Client, including using the command line options and more details about the configuration files, see the Configuration documentation. For details about the SCALE client architecture and our general design methodology for it, see the Architecture documentation. If you wish to modify SCALE to incorporate new features in the core or simply extend it by adding a new module, see the Modifying SCALE documentation.
Source Code
This repository contains all of the SCALE client Python code that drives the individual IoT devices. The flexible nature of these devices allows them to include a lot of intelligence or as little as you want. We are currently working towards consolidating the SCALE client and server code so as to share as much event processing logic as possible between them.
Directory structure
.
+-- scale_client: contains main Python package
| +-- config: various example configuration files to run the client in different pre-set ways
| +-- core: core client logic, abstract base classes, etc.
| +-- client.py: driving main Python file for client
| +-- sensed_event.py: abstract representation of a sensor reading, higher-level event, action, etc.
| +-- event_reporter.py: handles the multiple EventSinks to upload (publish) SensedEvents that should be shared externally through the appropriate channel(s)
| +-- sensors: all physical sensor interface wrappers and VirtualSensors reside here, subdivided into different categories
| +-- applications: non-VirtualSensor applications that run various long-lived, analytics, or management-related operations live here
| +-- event_sinks: implementations for the various EventSinks
| +-- network: all modular networking-related functionality resides here
| +-- test: simplistic test files to verify the client is working
| +-- util: various utility scripts for setting up the client, patching libraries, etc.
| +-- mqtt_subscriber.py: simple debugging script for subscribing to an MQTT topic and printing out the incoming publications
| +-- defaults.py: various default values (especially for logging) used in the client
+-- docs: additional documentation about SCALE client architecture, configuring, extending, etc.
+-- scripts: various installation scripts for different platforms
| +-- scale_daemon: daemon script for running as a service
+-- csn_bin: contains binary files needed to integrate CSN sensor code for SCALE (not publicly supported or recommended)
+-- setup.py: installation script for default configuration/platform (NOTE: 2 other file versions for different configurations)
+-- requirements.txt: contains all Python pip requirements, some of which are commented out as they are only necessary for certain configurations
Branches
The master branch has all the code you will likely use. If you wish to experiment with our (currently unsupported) mesh networking functionality, the andy_mesh_network branch has extensions for using the BATMAN mobile ad-hoc protocol, but this branch is no longer up-to-date with our latest changes.
Other Repositories
A notable fork of scale_client is Qiuxi's repository for SCALECycle (look at the cycle branch).
The original SCALE server repo contains a Python Django web app-based analytics and alerting service. It requires some significant manual configuration and is no longer supported as we plan to migrate the logic in that repository to this one. This consolidation will allow us to demonstrate and experiment with running analytics logic at various locations of the SCALE pipeline (i.e. on devices, in-network, in local vs. edge cloud servers, etc.).
The ScaleSaltConfig repo contains configuration files and dependencies used to remotely manage and provision SCALE devices automatically. Note that it may be out of date from our latest iterations as we found it easier to simply clone Raspbian images with all the configuration already done when making several SCALE boxes of the same configuration.
Platform-specific Files
One may note that throughout the codebase there exist a few different versions of files (e.g. pi, sheeva, and mesh). These were used to configure different versions of the SCALE boxes on other platforms such as the Sheevaplug or for our BATMAN-based experimental mesh networking setup. We left these files intact in case anyone might find them useful, but they are not necessarilly supported (i.e. tested) platforms any longer.
Building a SCALE Box
For directions on how to build your own SCALE devices (including hardware purchase list, assembly, and running SCALE on a Raspberry Pi with Raspbian), see our evolving Google document.
Contributing
Please feel free to fork this repo for your own IoT research and development. If you've fixed a significant bug or wish for SCALE to incorporate a new feature you added, please submit a pull request. Your code should fit the same style and properly implement the SCALE Client architecture described in the documentation.
History
SCALE was originally envisioned as a demonstration of using the Internet of Things (IoT) for public and personal in-home safety applications as a response to the SmartAmerica Challenge in 2014. It has since evolved to include many different partnerships, organizations, and technologies. Thus, it demonstrates a generic IoT project's organic evolution over time.
You can view the SCALE project's website for additional information, publications, and a demonstration of the event-viewing dashboard.
Acknowledgements
Special thanks to the NSF; the University of California, Irvine; Montgomery County, MD; for sponsoring and supporting the SCALE project. Additional thanks to the many other partners involved in SCALE!
Special thanks to the main SCALE software contributors:
- Kyle Benson
- Guoxi Wang
- Qiuxi Zhu
Licensing
See LICENSE.txt file for official (permissive and business/open source friendly) licensing.