jdemaeyer/brightsky

DWD Alerts

Closed this issue · 11 comments

jwillmer commented

Is there a way to also get the DWD weather alerts? I found many services that offer an API to the weather or the alerts from DWD but none of them are offering both. Any specific reason this project does not offer it (or I don't find it)?

jdemaeyer commented

Heyho @jwillmer!

There is no specific reason other than that those warnings were out-of-scope for our initial release and no one has expressed interest in them since (plus now that the funding period is over I have limited time to work on Bright Sky).

Could you post an example request/response from one of those APIs that offer warnings? Judging from https://www.dwd.de/DWD/warnungen/warnapp/json/warnings.json it seems that they wouldn't fit too well into our existing timestamp-focused database/API structure and will require coming up with new tables and API endpoints (but maybe we can get away with just adding a new key to our weather response...), and with a way to figure out whether the requested location falls into a warning region (instead to just grabbing the nearest available measurement like we do for weather).

jwillmer commented

Request/response for this API is very basic. As you can see in the link they support different formats (CAP, SMS, TXT). They also offer filters for locations (areas) but I'm not sure how they work. Documentation is not very good 😞 Additionally they also offer German or English alert text. So this would be something to keep in mind when using the endpoint (filter, additional option).

I found the following repositories that work with alerts and might be the best location to get more information about the API:

jwillmer commented

@jdemaeyer I was thinking about it and came to the following conclusion:

I think you should not try to add the alerts in the current API. It is anyway a bad design to pull for alerts. I think the appropriate implementation would be an API to subscribe to alerts from an area. And then get notified (POST request) when an alert, matching the area.

API could look like this:

/// <summary>
/// Webhook endpoint for alerts
/// </summary>
/// <param name="point">Geo location to monitor</param>
/// <param name="webhook">Url to call when an alert was received</param>
/// <param name="km">Alert radius to include</param>
/// <returns>Webhook id</returns>
[HttpPost]
string SubscribeToAlerts(GeoPoint point, Uri webhook, int km = 10);

/// <summary>
/// Unsubscribe from alerts
/// </summary>
/// <param name="id">The subscription id</param>
[HttpDelete]
void UnsubscribeFromAlerts(string id);

Additionally I would add some rules to auto unsubscribe inactive subscribers. Determining inactive could be done by:

  • webhook timeout / not reachable
  • webhook endpoint not reachable for more then 6 months
  • webhook endpoint not reachable for more then 10 alerts

Months and alert count needs some additional logic in case we did not get any alerts in the last 6 months we should not remove a subscription. Or if we send 50 alerts in the last week we might like to remove the endpoint earlier.

jwillmer commented

Referencing repo with similar intentions:
earthobservations/wetterdienst#342

jdemaeyer commented

Thanks for all the input! I think the alerts would definitely be a great feature for Bright Sky.

IMHO a webhook-based approach would add too much new complexity (like the user/subscription management you outline above, post-update hooks inside Bright Sky, and making sure alerts are sent out reliably) and exclude a lot of users (anyone who can't supply a publicly available endpoint), so I'm very reluctant to go down that route. Such a big departure from our current architecture seems better suited for a separate project which internally polls the alerts from Bright Sky.

Even staying within our polling structure leaves us with the other big (and computation-heavy) problem of determining if a requested lat/lon is inside an alert area. However, we may get away with simply assigning any alert to the center of its area, and then using the distance-based approach that we also use for weather records. (I have no doubt people who know more about geocomputation than I do are throwing their hands up in horror right now. :))

Our roadmap could then be roughly as follows:

  • Investigate how bad using just the center of the alert areas is
  • Add a new table alerts containing columns for lat, lon, start, end, id, type, headline, description, area_name, and other event data
  • Add a parser for alert data (at first glance it seems that https://opendata.dwd.de/weather/alerts/cap/DISTRICT_DWD_STAT/Z_CAP_C_EDZW_LATEST_PVW_STATUS_PREMIUMDWD_DISTRICT_DE.zip is the file we're looking to parse) and adjust the poller URLs
  • Add a diff-based update mechanism for the alerts table (we will not be preserving historical alerts)
  • Add a new alerts endpoint returning all alerts for a given lat, lon, and max_dist
  • If retrieving alerts is computationally cheap, add a new key alerts to the weather endpoint response containing any alerts for the requested location/radius that have a timeframe overlapping the requested timeframe

That being said, I can't promise anything concerning when I'll find time to tackle this :/

jwillmer commented

As far as I know the alerts are already assigned to a region. They are not location based. And another good thing to know is that the system will also receive preplaned alerts. Like: "Glazed frost between 5 and 10 AM" that will be issued the evening before (with start/end timestamp).

MartinHennrich commented

It would be really convenient to send a ZIP code to the API and receive warnings for it.

jdemaeyer commented

Weather alerts landed in Bright Sky with v2.0.6! 🎉

This is currently in experimental feature and subject to change, below is a quick-n-dirty documentation while I work on the proper one.

Endpoint

https://api.brightsky.dev/alerts

Parameters

There are currently four parameters:

  • lat/lon: latitude, longitude that should be checked for active warnings
  • warn_cell_id: warn cell ID that should be checked for active warnings
  • tz: timezone for returned timestamps

You must supply either warn_cell_id or both lat and lon.

Example

Current alerts for Göttingen: https://api.brightsky.dev/alerts?lat=51.55&lon=9.9

Response:

{
  "alerts": [
    {
      "id": 75,
      "alert_id": "2.49.0.0.276.0.DWD.PVW.1687514160000.20999218-5d5e-4761-b271-6c243f695568",
      "effective": "2023-06-23T09:56:00+00:00",
      "onset": "2023-06-23T09:56:00+00:00",
      "expires": "2023-06-23T12:00:00+00:00",
      "category": "met",
      "response_type": "prepare",
      "urgency": "immediate",
      "severity": "moderate",
      "certainty": "likely",
      "event_code": 61,
      "event_en": "heavy rain",
      "event_de": "STARKREGEN",
      "headline_en": "Official WARNING of HEAVY RAIN",
      "headline_de": "Amtliche WARNUNG vor STARKREGEN",
      "description_en": "There is a risk of heavy rain (Level 2 of 4).\nPrecipitation amounts: 20-35 l/m²/6h",
      "description_de": "Es tritt Starkregen auf. Dabei werden Niederschlagsmengen zwischen 20 l/m² und 35 l/m² in 6 Stunden erwartet.",
      "instruction_en": "NOTE: Be aware of the following possible dangers: The downpours can cause temporary traffic disruption.",
      "instruction_de": "ACHTUNG! Hinweis auf mögliche Gefahren: Während des Platzregens sind kurzzeitig Verkehrsbehinderungen möglich."
    },
    {
      "id": 28,
      "alert_id": "2.49.0.0.276.0.DWD.PVW.1687470000000.fe90b61b-3755-4efb-8eda-b161251da9f7",
      "effective": "2023-06-22T21:40:00+00:00",
      "onset": "2023-06-23T06:00:00+00:00",
      "expires": "2023-06-23T14:00:00+00:00",
      "category": "met",
      "response_type": "prepare",
      "urgency": "immediate",
      "severity": "minor",
      "certainty": "likely",
      "event_code": 51,
      "event_en": "wind gusts",
      "event_de": "WINDBÖEN",
      "headline_en": "Official WARNING of WIND GUSTS",
      "headline_de": "Amtliche WARNUNG vor WINDBÖEN",
      "description_en": "There is a risk of wind gusts (level 1 of 4).\nMax. gusts: ~ 55 km/h; Wind direction: west then north-west; Increased gusts: in exposed locations ~ 65 km/h",
      "description_de": "Es treten Windböen mit Geschwindigkeiten um 55 km/h (15 m/s, 30 kn, Bft 7) anfangs aus westlicher, später aus nordwestlicher Richtung auf. In exponierten Lagen muss mit Sturmböen um 65 km/h (18 m/s, 35 kn, Bft 8) gerechnet werden.",
      "instruction_en": null,
      "instruction_de": null
    }
  ],
  "location": {
    "warn_cell_id": 803159016,
    "name": "Stadt Göttingen",
    "name_short": "Göttingen",
    "district": "Göttingen",
    "state": "Niedersachsen",
    "state_short": "NI"
  }
}
Notes
  • The DWD divides Germany's area into roughly 11,000 "cells" based on communes (Gemeinden), and issues warnings for each of these cells. Most warnings apply to many cells
  • Bright Sky will supply information on the cell that a given lat/lon belongs to in the location field
  • There is also a coarser set of roughly 400 cells based on districts (Landkreise) that is not supported by Bright Sky
  • The complete list of cells can be found on the DWD GeoServer (see below)
Additional resources
 

/cc @krystofbe

jdemaeyer commented

Closing as this is now officially part of the API :)

  • 🎉2
jdemaeyer commented

Demo of an interactive alerts map: live demo, source

  • ❤️1