/avoviirsprocessor

I process VIIRS data at AVO

Primary LanguagePythonCreative Commons Zero v1.0 UniversalCC0-1.0

avoviirsprocessor

Build Status Code Climate

Docker container to process VIIRS data at AVO

Data directory

I expect to find a writable directory to work in to be mounted at /rsdata

Environment variables

I look to the environment for my bootstrap config. I require three enironrment variables.

  • RSPROCESSING_BASE local filesystem path of my working directory
  • CU_CONFIG_URL URL to a configupdater configuration file.
  • AVOVIIRS_CONFIG Local filesystem path of the configuration file.
  • COVERAGE_THRESHOLD Skip products without at least this much coverage. Somewhere between 0 and 1.

If authentication is required to retrieve the configupdater configuration it must be specified in the environment.

  • CU_USER Username, if required to retrieve configupdater config file.
  • CU_PASSWORD Password, if required to retrieve configupdater config file.

I will email logged errors if desired.

  • CU_CONTEXT_NAME Displayed in the subject of any email generated by configupdater.
  • MAILHOST Who can forward mail for me?
  • LOG_SENDER From: address
  • LOG_RECIPIENT To: address

Optionally, I'll cleanup downloaded files after some number of days.

  • DAYS_RETENTION Maximum file retention in $RSPROCESSING_BASE

docker-compose

Here is an example service stanza for use with docker-compose.

collectors:
  image: "tparkerusgs/rscollectors:release-2.0.2"
  user: "2001"
  environment:
    - RSPROCESSING_BASE=/rsdata
    - MIRROR_GINA_CONFIG=/tmp/mirrorGina.yaml
    - DAYS_RETENTION=7 
    - PYTHONUNBUFFERED=1
    - MAILHOST=smtp.usgs.gov
    - LOG_SENDER=avoauto@usgs.gov
    - LOG_RECIPIENT=tparker@usgs.gov
    - CU_CONFIG_URL=https://avomon01.wr.usgs.gov/svn/docker/rsprocessing/configupdater-collectors.yaml
    - CU_CONTEXT_NAME=collectors
    - CU_USER=user
    - CU_PASSWORD=password
  restart: always
  logging:
    driver: json-file
    options:
      max-size: 10m
  volumes:
    - type: volume
      source: rsdata
      target: /rsdata
      volume:
        nocopy: true

mirror_gina Configuration

I use a single YAML configuration file, an annotated example is provided.