miro-ka/countries

Simple gcp function for fetching & parsing countries

countries

A simple program to fetch and visualize restcountries.eu data.

Dashboard Link

About

Simple program for:

1. Fetching countries json from public https://restcountries.eu
2. Applied minor preprocessing
3. Extra/optional: Upload to GCP - BQ for visualization in Data-Studio

Requirements

Built and tested with python 3.8. If you want to upload data to GCP you need to provide big-query credentials gcp auth info

Python modules install (ideally in separate environment)

pip install -r requirements.txt

Usage

usage: countries.py [-h] [--bq_upload] --bq_dest BQ_DEST [--out_file OUT_FILE]

Simple job for fetching countries from restcountries.eu API and uploading it to GCP - BQ

optional arguments:
  -h, --help           show this help message and exit
  --bq_upload          Upload dataframe to GCP BigQuery (Default False)
  --bq_dest BQ_DEST    GCP destination table
  --out_file OUT_FILE  Output json file name (Default countries_dump.json)

Example run with uploading data to BigQuery

python countries.py --bq_dest genemoos-playground.sandbox.countries --bq_upload