Combined Data Hub with Electricity Market and Coronavirus Case Data in U.S.

This data hub, COVID-EMDA (Coronavirus Disease - Electricity Market Data Aggregation), is orginally designed to track the potential impacts of COVID-19 on the electricity markets in the United States of America, and could also contribute to relevant interdisciplinary researches.\

Features

Overall, this data hub contains the coronavirus case, weather, generation mix, load and price data for all the existing U.S. electricity marketplaces (CAISO, MISO, ISO-NE, NYISO, PJM, SPP , ERCOT) and some typical cities in these markets (San Francisco, Boston, New York City, Philadelphia, Kansas City, Houston).
All the coronavirus case, weather and electricity market data are carefully verified and coordinated to match the geological scale.
Daily load profile, hourly generation mix and day-ahead price are recorded and tidied in a consistent, compact and ready-to-use data format that makes it easy for cross-market analysis.
Historical data dating back to 2017 are included as a time-series benchmark.

Suggested Reference

A paper containing comprehensive introduction of this data hub and preliminary analysis is available at: arXiv and EnerarXiv
Suggested reference for publication associated with this data hub:
G. Ruan, D. Wu, X. Zheng, S. Sivaranjani, H. Zhong, C. Kang, M. A. Dahleh and L. Xie, A Cross-Domain Approach to Analyzing the Short-Run Impact of COVID-19 on the U.S. Electricity Sector , arXiv preprint: https://arxiv.org/abs/2005.06631, 2020.

Navigation

This github repository is structured as follows.

  COVID-EMDA
  |
  ├── data_source/
  │    ├── caiso/
  │    ├── miso/
  │    ├── isone/
  │    ├── nyiso/
  │    ├── pjm/
  │    ├── spp/
  │    ├── ercot/
  │    ├── covid/
  │    ├── weather/
  │    ├── README.md
  ├── data_release/
  │    ├── caiso/
  │    ├── miso/
  │    ├── isone/
  │    ├── nyiso/
  │    ├── pjm/
  │    ├── spp/
  │    ├── ercot/
  │    ├── README.md
  ├── startup/
  ├── parser/
  ├── README.md

All the data source files are archived in folder date_source/, the cleaned and processed data are stored in folder data_release. The supporting team will provide daily updates to capture the latest sitation. All these files are properly collected by location. The file naming rule is: MARKET_AREA_CATEGORY.csv, e.g. nyiso_nyc_load.csv. This is a dataset of load profile in New York City, from 2017 to present.

Readers can turn to folder startup/ for information about basic applications. For the need of extension, we provide the parser codes, which can be found in folder parser/.

Data Source

This data hub contains three major sources: U.S. electricity market data, COVID-19 cases and weather data. For some categories, multiple data sources are carefully gathered to ensure accuracy.

Electricity Market Data:
CAISO, MISO, ISO-NE, NYISO, PJM, SPP, ERCOT, EIA, EnergyOnline.
COVID-19 Case Data:
John Hopkins CSSE.
Weather Data:
Iowa State Univ IEM.

Additional Dataset

Cellphone Dataset:
Description: This dataset aggregates and processes the real-time GPS location of cellphone users by Census Block Group. It includes social distancing data and patterns of visits to Point of Interests (POIs). To obtain the access, please click the link below and apply for SafeGraph's permission which is totally free.
Link: Cellphone Dataset from SafeGraph
Night Time Light Satellite Dataset:
Description: This dataset includes the raw satellite image taken at night time in each area.
Link: NTL Dataset from NASA