/uaenvironment.github.io

A showcase of our development and proof of concept research work at the Arizona Institutes for Resilience

Primary LanguageHTML



Collaborators | Proposals | Technology Transfer | Projects | Publications | Students


AIR Technical Research Applications Development

Overview

Who we are

We are the Technical Research Applications Development group found within the Arizona Institute for Resilience (AIR). We are a group of technology professionals and students who provide a variety of technical services and collaborations to the Centers and Programs found within AIR. Our project collaborations span the globe and make meaningful impacts from the desert southwestern United States to the Middle East and places in between. AIR explores and develops solutions with campus and community partners that serve human and natural communities on a global scale by engaging a full array of disciplines, professional schools, international capacity, and entrepreneurial opportunities.

The research applications group is led by Rey Granillo Director of Technology and Research Computing, Leland Boeman Research & Development Software Engineer, and Thomas Weiss Research & Development Systems Engineer.

Our proficiencies

  • Research focused software development
  • Cloud computing
  • Data assimilation, processing, and presentation
  • Database architecture and development
  • Application Programming Interface (API) development
  • Remote sensing
    • LoRaWAN network gateways
    • Designing and building microcontrollers
  • 3D printing
  • Machine learning

How we do it

Development Process

By developing and combining technologies, we incorporate novel technical solutions into new and existing research. Using a collaborative model, versus the typical service model found most in Information Technology, we can better understand project data, the research questions being asked, and provide higher quality research collaborations. Having that greater understanding has led us to find correlations in data that were not previously realized which have led to new research project ideas and proposals. We collaborate with researchers during the proposal process and can provide a proof-of-concept application or tool showing what is technically feasible if project funding is awarded. Upon project completion, we aid with the technical writing portions of publications which outline our findings and technical solutions that were implemented.

Student Support

We rely on student support and participation across all research projects. Student participation ranges from software development, technical hardware implementation, database architecture design and implementation, ML/AI research, and publication writing. To support these efforts, AIR has established a student research computing working group that primarily meets during the Fall and Spring semesters. This working group is a forum where we discuss new research technologies, ask questions, and where students can report on their current research activities. The intention is to create research focussed critical thinking processes that can generate new ideas and concepts across various projects and collaborations.

Our current student cohort includes:

Research Collaborators

In addition to our core research applications development team, we've had a number of collaborators across AIR programs and affiliated groups across the University of Arizona.

Ben McMahan
AIR, School of Anthropology
Mike Crimmins
CALS - Environmental Sciences
Zack Guido
AIR International Programs
Tom Evans
SBS - School of Geography and Development
Riley Duren
CEO Carbon Mapper
Wes Howden
Eller College of Management
Konan Hara
Eller Doctoral Student Economics
Rachel Rosenbaum
School of Anthropology
Nancy Petersen
AIR Haury Program
Dharma Hoy
Statistics Graduate Student Colorado State

Proposals

Integrating Data Science and ML/AI into Stakeholder Informed Assessments of Fire Weather Tools

Study/Project Aim(s) IDENTIFY fire weather forecast priorities and EXPAND the fire weather stakeholder network, DEVELOP experimental fire weather analysis and visualization prototypes, based on insights from data science, Ux testing, and ML/AI, CO-PRODUCE relevant, salient, and credible fire weather information collaboration with fire managers that ENHANCE fire weather impact decision support services (a NOAA/Npriority), and DEVELOP strategies to expand these products at scale for use by wider networks of fire managers, and EXPLORE opportunities for aggregation, management, visualization, and analysis of fire rand weather data

PI: Rey Granillo
Co-Applicants: Ben McMahan, Leland Boeman
$304,609

High-Resolution multitemporal urban heat mapping: innovations in data aggregation, community engaged citizen science, and student experiential learning, to develop a scalable heat mapping platform in the Southwest

Proof of concept for data aggregation work, development of heat island data and visualization, laid foundation for student training in interdisciplinary and stakeholder engaged research at intersection of climate/social/computer science

PI: Ben McMahan & Rey Granillo
$144,646

Can We Make a “RainLog for Heat Maps”? – Leveraging Data Aggregation and Citizen Science to Improve Temperature Maps and Enhance Outreach, Engagement and Education

Proof of concept for data aggregation work, would have helped lay foundation for future student collaborative work, and faculty led proposals

PI: Ben McMahan
Co-PI: Mike Crimmins
$40,554

Stakeholder Informed Assessments of Fire Weather Impact Decision Support Services

Heavy involvement on data aggregation and UX/visualization development. (why Rey was PI, as learning lab was heavily implicated in our proposed capacity)

PI: Ben McMahan
Co-PI: Rey Granillo, Mike Crimmins, Will Holmgren
$638,224

Anticipating Extreme Monsoon Rainfall: Integrating Machine Learning and Artificial Intelligence into Flood Risk Prediction Models

Use ML techniques to organize aggregated monsoon data to create improved flood decision support and data viz

PI: Ben McMahan & Rey Granillo
$24,385

Adapting and Enhancing the Monsoon Threat Index: Iterative Development of Stakeholder Informed Climate Services and Weather Information

Included development of stakeholder focused data viz and tools - would have used the learning lab to help with project work and give students experience

PI: Ben McMahan
Co-PI: Will Holmgren, Mike Crimmins
$365,117

Public Engagement in Southwest Monsoon Forecasting: A Prototype Game to Build Climate Awareness and to Evaluate Community Forecast Skill

PI: Z. Guido
Co-PI: McMahon, B., Crimmins, C., and Granillo, R.
$13,958

Innovative Visualization & Analysis Tools for the North American Monsoon - Connecting Citizen Science Data and Observations for Research, Policy, and Decision Support

Initial foray into data aggregation.

PI: Ben McMahan
Co-PI: Will Holmgren, Mike Crimmins
$25,000


Technology Transfer

National Science Foundation Innovation Corps (NSF I-Corps) Training Program

The U.S. National Science Foundation’s Innovation Corps (I-Corps™) program is an immersive, entrepreneurial training program that facilitates the transformation of invention to impact. This immersive, seven-week experiential training program prepares scientists and engineers to extend their focus beyond the university laboratory — accelerating the economic and societal benefits of NSF-funded and other basic research projects that are ready to move toward commercialization. Visit the NSF's I-Corps website for more information.

Desert and Pacific Regional Hub 2023 - Cohort 6 - Fire Data Dynamics

AIR's technology team is working with Tech Launch Arizona on licensing for our Fire Data Dynamics prototype to help make meaningful impacts in fire prediction, response, and analysis. As part of this licensing process, we participated in NSF's I-Corps where we learned about the Lean Startup Methodology with a strong focus on customer discovery and learning about the challenges being faced to ensure we are providing a solution with meaningful impact.


Projects

Fire Data Dynamics

With data from the National Interagency Fire Center's (NIFC) Southwest Coordination Center (SWCC), and with funding from the University of Arizona's Technology and Research Initiative Fund (TRIF), we developed a fire data dashboard prototype designed to be used by and inform decision makers during fire events. Using data from past fire events in Arizona and New Mexico, our dashboard displays fire occurrences and their locations, fire burn perimeters (when available), and relevant data from Remote Automated Weather Stations (RAWS) during those fire events. The current prototype also allows stakeholders to export this historical data for use with their own analysis. In addition, new features are being developed to incorporate a live fire viewer, perform predictive machine learning analysis for determining fire weather and occurrence likelihood, and incorporating Weather Research & Forecasting Model (WRF) to inform decision makers of the impact weather could have during fire events.

Fire Dashboard image

Remote Automated Weather Stations (RAWS) Machine learning

Leveraging historical RAWS data from the past 20 years alongside popular machine learning technologies, we were able to build models to predict wind speed, temperature, and relative humidity conditions for 6-, 12-, and 24-hour time periods.

RAWS Data

20 years of weather data was pulled for specific stations in AZ during this prototype phase. The data was then cleaned and scaled for use with the intended machine learning technologies.

Python Package

A custom RAWSTraining Python package was developed to easily clean and prepare data for use with Tensorflow and Keras to build multi-step forecasting models. With custom classes used for Training and making Predictions, training and testing took only a couple of hours for a single station to be deployed (with non-GPU enabled hardware).

Web Application

An application built with Vue3, Firebase, and Firestore was launched to show the predictions and their respective stations. This application featured flagging for potential fireweather conditions, and also assessed the models in realtime to showcase the validity of the predictions being made.

Future implementations of this project will exist within the Fire Data Dynamics prototype.

Fire ML Map image A screenshot showing the RINCON station and the application's flagging functionality.

Fire ML example graph A graph showing a trained model's predictions against real, measured data.

Carbon-Econ Plotter

Methane emissions have an outsized impact on climate, and are increasingly receiving attention from policymakers and the scientific community. We map methane plume emission rates measured by Carbon Mapper onto tract-level demographics from the U.S. Census Bureau to explore environmental justice issues associated with the methane emissions. Carbon Mapper collects methane emission data through their airborne pilot projects with advanced remote sensing technology. Census tract-level demographics are obtained from the 2009-2012 five-year moving average American Community Survey. This map and scatter plots can be viewed at Carbon Plotter website.

Carbon Plotter Map Image


LoRaWAN Remote Sensing Network

LoRaWAN is a Low Power Wide Area Networking (LPWAN) open communications protocol used in a variety of remote sensing and Internet of Things (IoT) applications. AIR supports the Desert Laboratory on Tumamoc Hill which is an 860-acre preserve located west of downtown Tucson. AIR has a number of LoRaWAN Gateways and is using Tumamoc as our current testbed for future remote sensing projects.

Tumamoc LoRaWAN image


Monsoon Game Repositories

The following projects all pertain to various aspects of the AIR Monsoon Fantasy Game. These repositories include the detailed scoring method and a post game analysis.

Monsoon Fantasy Game

In Monsoon Fantasy, players estimate the total monthly precipitation at each of the five major cities in the U.S. Southwest Monsoon region: Tucson, Phoenix, Flagstaff, Albuquerque, and El Paso. Points are awarded each month depending on the accuracy of the estimate compared to the actual observed rainfall. The goal is to accumulate the most points over the July, August, September period.

Monsoon Game Forecast Monsoon Game Leaderboard

Monsoon Game Scoring Method

Contributors:

DOI

This repository contains a description of the scoring system used in the Monsoon Fantasy game and a simulation that was created to compare different scoring methods before the final scoring method was decided upon. The final scoring method takes into account both the risk and accuracy of a players guess. First, a potential maximum points value is determined for a guess. This value is higher the further a guess is from the historical rainfall average. Then, the player gets a percentage of their potential maximum points value depending on how close their guess is to the actual rainfall. Visit our public GitHub repository for more detailed information.

Monsoon Game Scoring Graph


Monsoon Post Game Analysis

When signing up to play Monsoon Fantasy, players had the option to fill out profile questions. These questions asked things such as

  • How many monsoon seasons have you experienced while living in the southwest?
  • How would you rate your understanding of the monsoon system?
  • During monsoon season how often do you consult different types of weather forecasts?

All of the questions were optional and the responses were made anonymous before analysis. This data combined with the users' forecasts and points earned were used in this analysis which will be performed and the end of every monsoon season.

Monsoon Game Post Analysis




Monsoon Data Collection and Processing

The following projects are geared towards improving the availability of monsoon related meteorological data in Arizona by providing a centralized and persistent source for otherwise ephemeral observation data and demonstrating the value of that data through visualization and machine learning applications.


Monsoon Scraper

This project centralizes public data from several different Flood Control District (FCD) networks across the state of Arizona. This data is stored in a cloud based data warehouse and serves as the central data source for a number of monsoon related projects and research. To gather this data we have written a number of applications that run at different intervals dependent on the different FCD network implementations. These applications run on 15 minute to 1 hour intervals. These time intervals are required in order to obtain incremental precipitation data readings which are not available if gathering data on an hour or day interval. In addition to precipitation data, some FCD sensors also report temperature, pressure, humidity, and stream flow intensity in washes.

This dataset consists of the following remote sensing precipitation networks along with their API programmatic names. Additional networks will be added as they are implemented.

  • Pima County FCD - pima_fcd
  • Maricopa County FCD - maricopa_fcd
  • RainLog.org - rainlog
  • MesoWest - mesowest
  • Mohave County FCD - mohave_fcd (data beginning 2021)

Data from the networks above are updated at different frequencies and in some cases multiple times per day or hour. This is variable across networks based on their configuration and if precipitation sensors are experiencing rainfall.

The result of this work can be found in a Frontiers in Climate publication titled Curating and Visualizing Dense Networks of Monsoon Precipitation Data: Integrating Computer Science Into Forward Looking Climate Services Development.


Monsoon Data Application Programming Interface (API)

Once data from the Monsoon Scraper project was gathered, we developed a REST API to programmatically query the dataset. The API contains a number of custom routes designed to query specific sets of data. Some of these routes include our monsoon route which returns precipitation totals from specified networks or sensors between June 15th - September 30th of provided years, a flood route which returns data from flood gauge sensors typically found in washes, a sensors metadata route which returns metadata of specific sensors, and a readings route which queries specific sensors or networks using a provided date range.

Currently, API keys are only issued to researchers working with this dataset. There are plans to expand this audience in the future.


Monsoon Plotter

Monsoon Plotter is used to visually represent the data gathered via the Monsoon Scraper project which collects data from the state of Arizona flood control district (FCD) remote sensing networks. There are a handful of networks that can be plotted and more will be added as we expand our Monsoon Scraper project to gather more data. There is also a limited CSV export feature available of the specific data points chosen to be plotted. For full exports of data an API key is required to make programmatic calls to the Monsoon API.

Monsoon Plotter


Monsoon API Package/CLI Tool

This Python package serves as a wrapper to simplify REST API calls to the monsoon scraper data warehouse. This is the same dataset that is visually represented in our monsoon plotter found at monsoon.environment.arizona.edu. The plotter allows a limited export of the data dependent on the date range and sensor network being plotted. This package allows you to incorporate our monsoon dataset into a local codebase for processing.

This package also contains a Command Line Interface (CLI) tool for those who prefer to work within a CLI instead of the Python package.

This repository is in the process of being published to The Python Package Index (PyPi) which serves as a repository of Python software. Instructions will be added here to install the package once published.


Monsoon Machine Learning

This repository contains code in R and Python that demonstrates how to create basic machine learning algorithms. It then takes historical weather data from the Tucson International Airport, precipitation data from our Monsoon API, and storm data from NOAA and applies these machine learning algorithms in an attempt to accurately predict flooding using historic data and a database of notable flood and rainfall events.

Monsoon Machine Learning




Regenerate Lebanon

This project stemmed from a UN funding award through the Japanese Embassy to provide a mapping service of recycle centers across Lebanon. The project then branched out to include sustainable businesses and other environmentally friendly organizations.

This was a collaboration between a technology company in Lebanon called Imperium Code and other Lebanon non-profits. Our role in the overall project was to provide the mapping solution for the data set. This included mapping the points, displaying their categorizations, as well as implementing a tooltip/popup to display information about each data point. This work can be found on the Regenerate Hub site.

Regenerate Lebanon Map


Jamaica Coffee Leaf Rust

This was a proof-of-concept application that was built using Google's Vision AI to detect Coffee Leaf Rust which is a fungus that kills coffee plant crop fields. The idea was to detect this fungus via photos taken by farmers in Jamaica. We would then process those photos through the Vision AI algorithm we created to return the probability of positive coffee rust detection. In turn, this would help government officials to focus remediation efforts.

We developed this proof-of-concept model using plant life located in UArizona's ENR2 building. We trained this model by feeding it various images of different plant life to show how the image analysis would function and the different probability results.


Project Visualizations

This project was developed for the Agnese Nelms Haury Program in Environment and Social Justice proposal submission process for research funding. The idea was to use models and algorithms in Natural Language Processing (NLP) to find similarities or potential collaboration opportunities across abstract and proposal submissions. We incorporated a number of models/algorithms in this process to determine which is best suited for the particular set of research themes.

Project Visualization Graphic


Zambia Invasive Species Tracking

This invasive species tracking application was designed to plot the detection of the Fall Army Worm (FAW) across maize crops in Zambia. The result of this tracking helped governmental agriculture decision makers in focussing remediation efforts of FAW. To achieve this we worked with data aggregated via analog phone text messaging that was collected by an company in Africa called TextIt. We export that data in a format that allowed us to store responses in a relational way to more easily pull data into our front-end plotter. This data was gathered every 2 weeks during their grow season and plotted via the text campaign date. This application was architected in a way that would allow us to track other species in future research. More information about this project can be found on our FAW environment site.

Zambia FAW Distribution


Publications

Dharma H, Granillo R.L. III, Boeman L, McMahan B, Crimmins MA (2023) Data Aggregation, ML ready Datasets, and an API: Leveraging diverse data to create enhanced characterizations of monsoon flood risk. Frontiers in Climate. Front. Clim. doi: 10.3389/fclim.2023.1107363

Guido, Z., McMahan, B., Hoy, D., Larsen, C., Delgado, B., Granillo, R. L., III, & Crimmins, M. (2022). Public Engagement on Weather and Climate with a Monsoon Fantasy Forecasting Game, Bulletin of the American Meteorological Society. https://journals.ametsoc.org/view/journals/bams/aop/BAMS-D-22-0003.1/BAMS-D-22-0003.1.xml

Dharma Hoy, Calvin Larsen, Rey Granillo III, UA's Arizona Institute for Resilient Environments and Societies. (2022). uaenvironment/monsoon-game-scoring-method: v1.0.1 (v1.0.1). Zenodo. https://doi.org/10.5281/zenodo.6878318

McMahan B, Granillo R.L. III, Delgado B, Herrera M and Crimmins MA (2021) Curating and Visualizing Dense Networks of Monsoon Precipitation Data: Integrating Computer Science Into Forward Looking Climate Services Development. Front. Clim. 3:602573. doi: 10.3389/fclim.2021.602573


Works in Progress

Improving heat mapping using IoT weather station devices (Ben McMahan, Rey Granillo, Leland Boeman, Rose Prendergast, Dharma Hoy, Mike Crimmins)


Student Contributors

Current:

Roger Palmenberg
Lily McMullen

Former:

Dharma Hoy
Rose Prendergast
Shyambhavi Shyambhavi
Franny Slater
CJ Larsen
Benni Delgado
Mau Herrera
Lauren Tran