UNFCCC Flexible Queries Data Scraper

About

Dynamic Python data scraper and csv exporter for the UNFCCC annex and non-annex flexible queries website.

A collection of CSV exports was not able to be provided for download due to the data's size, but can be easily generated by running the scripts on your own system.

Generated CSV names	Generated CSV contents
`unit.csv`	`All unique unit types`
`all_measures_types.csv`	`All unique measurement types`
`measures.csv`	`All unique measures`
`classifications.csv`	`All unique classifications`
`years.csv`	`All unique years`
`categories.csv`	`All unique categories`
`gases.csv`	`All unique gases`
`annexes.csv`	`All unique annexes`
`parties.csv`	`All unique parties`
`queries.csv`	`All possible unique queries`

Setup

Ensure the latest version of Python is downloaded.

_{^{Tested and working on Python 3.10.4}}

Install the required Python libraries with PIP by running `pip install -r requirements.txt`.

_{^{Note: This project utilizes the UNFCCC DI API. Documentation for it if errors arise can be found here.}}

Verify that the following import statements are at the start of the Python script before running.

import unfccc_di_api import pandas as pd import numpy as np

Instructions

Once setup, run unfccc-query.py and .csv files will be created in the current directory. Note that the process may take up to 30 minutes or more.

Troubleshooting

If `pip install -r requirements.txt` fails, the necessary Python libraries can be manually downloaded utilizing pip with the commands below.

PIP install	Latest tested and working version
`pip install unfccc_di_api --upgrade --quiet`	`3.0.1`
`pip install pandas --upgrade --quiet`	`1.4.2`
`pip install treelib --upgrade --quiet`	`1.6.1`
`pip install numpy --upgrade --quiet`	`1.22.4`

Credits

This scraper was designed by a team organized by Greg Hay consisting of Parsa Khazaeepoul, Max Wang, Russell Eng, and Calvin Chan at the University of Washington for WattTime and The Green Software Foundation and integrates the UNFCCC DI API, credited below.

Mika Pflüger, Daniel Huppmann & Johannes Gütschow. (2021-12-03). pik-primap/unfccc_di_api: Version 3.0.1. Zenodo. https://doi.org/10.5281/zenodo.5751935

Want to utilize this data scraper and credit us?

Parsa Khazaeepoul, Max Wang, Russell Eng, and Calvin Chan. (06-15-2022). UNFCCC Flexible Queries Data Scraper: Version 0.0.2. Github. https://github.com/maxjj9710/unfccc-query.

parsakhaz/unfccc-query