/WP1-data-transfer

Repository documenting our routine for transfering data from GCloud to our server

Primary LanguagePython

TABMON data transfer

TABMON uses BUGGs acoustic devices which by default send the data over to Google Cloud.

This repository documents our procedure to copy the data over Google Cloud to our NIRD server.

Google Cloud setup

Create service account with access to the Google Cloud bucket

First create a new service account:

IAM and Admin -> Service Account -> Create Service Account -> tabmon-data-upload

Then you need to change the permission for Storage Object User so that the service account can access the cloud bucket.

Then create a Key that will act as the GOOGLE_APPLICATION_CREDENTIALS, an environment variable that authentificate a user for accessing the Google Cloud Bucket (see main.py/fetch_audio_data).

Copy/Paste the .json file created from the key and copy it in a file called key-file.json that is to be stored both on your local directory.

Run the program locally

There is no need to use any login (gcloud auth login, gcloud config set project) as everything is handled by the script.

Installing the dependancies

pip install -r requirements.txt

Create a .env file

Document all the paths in an .env file.

Use the template

Download the data

python3 main.py

Acknowledgement

This program has been conceived by Benjamin Cretois as part of the TABMON project.