bom_exporter - Prometheus exporter for the Australian Bureau of Meteorology

A Prometheus compatible exporter for forecast and observation data published by the Australian Bureau of Meteorology.

The exporter adapts the forecast and observation data feeds (from the anonymous FTP BOM server) into promscrape compatible format.

Build

go build cmd/bom_exporter.go

A binary called bom_exporter should be compiled.

Test

go test ./...

Scraping Data

bom_exporter will serve forecast scrape data on the following URL:

http://<server>:8080/metrics?id=<product_id>

Where the product identifier can be obtained from: http://www.bom.gov.au/catalogue/anon-ftp.shtml

The following product identifier types are currently supported:

forecast
observations

Scrape Configuration

Data is retrieved from the BoM on request, thus the configured scrape interval controls the frequency at which the BoM FTP server is accessed. It is recommended to match the scrape interval with the product being accessed:

observations - scrape every 5 minutes, updated every 10 minutes
forecast - scrape every 1-6 hours, updated every 12(?) hours

Example Scrape Single Product

Following is the configuration snippet to scrape the Sydney city forecast (IDN10064) hourly.

  - job_name: bom_forecast
    scrape_interval: 1h
    metrics_path: /metrics
    params:
      id: ['IDN10064']
    static_configs:
      - targets: ['localhost:8080']

Example Scrape Multiple Products

Following is the configuration snippet to scrape all the state observations every 5 minutes:

  - job_name: bom_observations
    scrape_interval: 5m
    metrics_path: /metrics
    static_configs:
      - targets:
        - 'IDS60920'
        - 'IDN60920'
        - 'IDD60920'
        - 'IDQ60920'
        - 'IDT60920'
        - 'IDV60920'
        - 'IDW60920'
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_id
      - source_labels: [__param_id]
        target_label: instance
      - target_label: __address__
        replacement: localhost:8080

Motivation

I've always wanted to have longer term climate data available with a user interface that I have familiarity (Grafana).

Known Issues/Future Ideas

Using a scrape interval >5 minutes results in Prometheus staleness
- Cache the data and timestamps internally
- Disconnect the external scrape interval from the data retrieval and use the 'next issue time' to intelligently schedule the next FTP retrieval.
Support other products, eg. tides
Release a Docker image
- Look into goreleaser
Improve test coverage
Add continuous integration

tardisx/bom_exporter