/SensorAnalysis

Examination of the Array of Things (AoT) Production Sensor Network

Primary LanguageHTML

Array Of Things (AoT)


Splash Page:https://tombresee.github.io/SensorAnalysis/
Heroku Page:https://michigan-milestone.herokuapp.com/
Authors:Tom Bresee, Michael Phillips
Version:2.0
University:Michigan
Course:SIADS 694/695: Milestone II
Focus:Anomaly Detection of Time-Series Sensor Cluster Data
Part:Part B
Observations:4,195,104,977

  • If you want to see the big picture of what I'm trying to do here, click the Splash Page, it gives you a thorough background. It is detailed and walks you through what precisely is going on, the depth of the data analysis, etc etc.
  • If you want to see the jupyter notebook code and results, just go to this section of the Splash Page, makes reviewing the code easy
  • If you want to see some cool results that were uploaded to the heroku page, click the Heroku Page link above
  • The part A of this project was Supervised Machine Learning for wearable sensors and HAR (human activity recognition), the initial plunge into sensor analysis, with code from Michael Phillips stored here

What is the Chicago AoT Program ?

  • The Array of Things (AoT) is an experimental urban measurement system comprising programmable, modular "nodes" with sensors and computing capability so that they can analyze data internally, for instance counting the number of vehicles at an intersection (and then deleting the image data rather than sending it to a data center). AoT nodes are installed in Chicago and a growing number of partner cities to collect real-time data on the city’s environment, infrastructure, and activity for research and public use. The concept of AoT is analogous to a “fitness tracker” for the city, measuring factors that impact livability in the urban environment, such as climate, air quality, and noise.
  • AoT is now an anchor partner in a new NSF-funded project called SAGE.
    • In late 2018 the AoT team proposed a new effort to the National Science Foundation's Mid-Scale Research Infrastuructre program, with an expanded vision, building on all of the lessons learned from the AoT project and creating a new hardware and software infrastructure. Successfully funded with a start of October 2019, the new NSF-funded project, called SAGE: A Software-Defined Sensor Network, will result in a migration of AoT functions to new devices in 2021. SAGE is led by Northwestern University in partnership with the Discovery Partners Institute (University of Illinois), University of Chicago, Argonne National Laboratory, the University of Colorado, the University of California-San Diego, Northern Ill


SAGE



Privacy

  • No active sensors have the capability to measure or identify individuals !
  • Microphones and cameras in public spaces do not collect sensitive personally identifiable information (PII). Microphone and camera images are processed in near-real-time within the installed equipment, not transmitted or stored, with the exception of less than 1% of images at random times, saved for the purposes of image processing software calibration. Although these images will not contain PII, they will be controlled and protected with the same measures typically afforded PII.
  • History - During the 2016-2017 pilot period, the cameras will be used for the purposes of detecting and publishing (a) count/flow of pedestrians, (b) count and flow of various vehicle types, and (c) extent to which road surface is covered with standing water (flooding). This list will be updated prior to publishing new types of data from images.


The Data

SubFiles:

data.csv.gz     # massive compressed file of all sensor data values and readings
nodes.csv       # list of nodes in the dataset and their individual metadata
README.md       # An explaination of the database fields
sensors.csv     # A list of active sensors and their pertinent metadata
offsets.csv     # data.csv.gz file byte offsets
  • What Data is Collected ?
    • The nodes will initially measure temperature, barometric pressure, light, vibration, carbon monoxide, nitrogen dioxide, sulfur dioxide, ozone, ambient sound pressure, and pedestrian and vehicle traffic. Continued research and development is using machine learning to create sensors to monitor other urban factors of interest such as solar light intensity (visible, UV, and IR) and cloud cover (important to building energy management), and flooding and standing water.


Reference Links



Citations

Journal of Open Source Software article.

L. McInnes, J. Healy, S. Astels, hdbscan: Hierarchical density based clustering In: Journal of Open Source Software, The Open Journal, volume 2, number 11. 2017

McInnes L, Healy J. Accelerated Hierarchical Density Based Clustering In: 2017 IEEE International Conference on Data Mining Workshops (ICDMW), IEEE, pp 33-42. 2017



map to buried treasure

Current Architecture