Imitate an API for downloading data from Facebook Data for Good
Currently, Facebook Data for Good does not provide an API to automatically download or update datasets.
This program uses selenium webdriver to imitate the behaviour of an API for downloading the full timeseries of a data collection.
Disclaimer: This download routine will only work for those with access to Facebook Data for Good, and will only function for datasets to which the user has been granted access. This tool is not developed by or associated with Facebook, it is simply a utility to automate downloading data from the Facebook Data for Good Geoinsights Platform.
Clone and cd
into this repository.
git clone https://github.com/hamishgibbs/pull_facebook_data_for_good.git
cd pull_facebook_data_for_good
Currently functional for TileMobility
, TilePopulation
, and Colocation
datasets only.
The command line inteface follows the format:
$ python pull.py [Country_Name] [DatasetName]
For example, to pull the TileMobility
dataset for Britain:
$ python pull.py Britain TileMobility
The country name must exactly match the name stored in the .config
file. For multi-word names, each word will be separated by '_'
. ie. New_Zealand
If the .config
file is missing variables for a given dataset, please alter the .config
file and open a PR to share with others.
Run unit tests:
$ python -m unittest
Dataset attributes are stored in the remote .config
file of this repository. To add functionality for downloading another dataset, alter the .config
file with two pieces of information:
- The dataset id, embedded in the url of the Geoinsights download page. For example, the dataset ID for the collection stored at
https://www.facebook.com/geoinsights-portal/downloads/?id=243071640406689
is243071640406689
. - The date origin of the dataset, the earliest date of data publication, in the format:
year_month_day(_hour)
.
Please open a PR to share the config variables for a new dataset in the .config
file.
Credentials can be input manually on each download or stored in a .env
file.
If you are using a .env
file, set the environment variables: FB_USERNAME
, FB_PASSWORD
, and FB_OUTDIR
.
If you are not using a .env
file, input your Facebook login credentials when prompted.
Unless you use a .env
file, credentials are not stored on your computer but are passed directly to the Facebook login page by the web driver.
This is an early release with limited functionality, suggestions and contributions are welcome. To request a feature or report an issue with this tool, please open an issue.
More info on Facebook Data for Good: https://dataforgood.fb.com/