openclimatefix/nowcasting_utils

Compare GSP-level PV forecasts against ESO's forecasts

JackKelly opened this issue · 3 comments

JackKelly commented

I've emailed Lyndon to ask for ESO's historical GSP-level PV forecasts :)

Ideally, compare in lots of different ways:

  • Different metrics
  • Different times-of-day
  • Different GSPs
  • Different "weather regimes" (cloudy day vs clear day, etc.)

Subtasks:

  • #36
  • Write a Jupyter Notebook which:
    • Separates out ESO's ASL and ML forecasts.
    • Aligns the ESO forecasts with the "ground truth" (the Sheffield Solar GSP-level PV Live data).
    • Selects just the first 4 hours (configurable) of the ESO forecasts.
    • Computes a bunch of metrics 🙂.
JackKelly commented

ESO are (hopefully) close to sending us their historical GSP-level PV forecasts.

  • ESO's GSP PV forecasts go back to June 2020
  • 5.42 GB compressed, 92 GB uncompressed
  • UPDATE: 5.8 GB compressed, 98.5 GB uncompressed CSVs
  • Both ASL and ML
JackKelly commented

The good news is that ESO have agreed to share their historical GSP-level PV forecasts and have sent me a 5.8 GB Zip file. The bad news is that Zip file appears corrupt (when opening on Linux using unzip or nautillus). I'll try a few more ways to open the Zip file; and I'll try re-downloading it. If that fails I'll ask ESO to re-send the zip file.

JackKelly commented

The zip file opened fine in Windows 10. So I've decompressed the CSV files. In total, there are 98.5 GB of (uncompressed) CSV files, one month per CSV file.

(Next time I try to open one of these zip flies in Linux, I should try a different unzipping programme. 7z might work).

  • 🎉1