/ElexonDataPortal

Python wrapper for the Elexon/BMRS API

Primary LanguageJupyter NotebookMIT LicenseMIT

Elexon Data Portal

DOI Binder PyPI version

The ElexonDataPortal library is a Python Client for retrieving data from the Elexon/BMRS API. The library significantly reduces the complexity of interfacing with the Elexon/BMRS API through the standardisation of parameter names and orchestration of multiple queries when making requests over a date range. To use the ElexonDataPortal you will have to register for an Elexon API key which can be done here.



Installation

The library can be easily installed from PyPi, this can be done using:

pip install ElexonDataPortal


Getting Started

We'll begin by initialising the API Client. The key parameter to pass here is the api_key, alternatively this can be set by specifying the environment variable BMRS_API_KEY which will then be loaded automatically.

from ElexonDataPortal import api

client = api.Client('your_api_key_here')

Now that the client has been initialised we can make a request!

One of the key abstractions within the ElexonDataPortal library is the handling of multiple requests over a date range specified through the start_date and end_date parameters. Each response will be automatically cleaned and parsed, then concatenated into a single Pandas DataFrame. If a settlement period and date column can be identified in the returned data then a new column will be added with the local datetime for each data-point. N.b. that if passed as a string the start and end datetimes will be assumed to be in the local timezone for the UK

start_date = '2020-01-01'
end_date = '2020-01-01 1:30'

df_B1610 = client.get_B1610(start_date, end_date)

df_B1610.head(3)
documentType businessType processType timeSeriesID curveType settlementDate powerSystemResourceType registeredResourceEICCode marketGenerationUnitEICCode marketGenerationBMUId marketGenerationNGCBMUId bMUnitID nGCBMUnitID activeFlag documentID documentRevNum resolution start end settlementPeriod quantity local_datetime
0 Actual generation Production Realised ELX-EMFIP-AGOG-TS-212 Sequential fixed size block 2020-01-01 Generation 48W000CAS-BEU01F 48W000CAS-BEU01F M_CAS-BEU01 CAS-BEU01 M_CAS-BEU01 CAS-BEU01 Y ELX-EMFIP-AGOG-22495386 1 PT30M 2020-01-01 2020-01-01 1 18.508 2020-01-01 00:00:00+00:00
1 Actual generation Production Realised ELX-EMFIP-AGOG-TS-355 Sequential fixed size block 2020-01-01 Generation 48W00000STLGW-3A 48W00000STLGW-3A T_STLGW-3 STLGW-3 T_STLGW-3 STLGW-3 Y ELX-EMFIP-AGOG-22495386 1 PT30M 2020-01-01 2020-01-01 1 28.218 2020-01-01 00:00:00+00:00
2 Actual generation Production Realised ELX-EMFIP-AGOG-TS-278 Sequential fixed size block 2020-01-01 Generation 48W00000GNFSW-1H 48W00000GNFSW-1H T_GNFSW-1 GNFSW-1 T_GNFSW-1 GNFSW-1 Y ELX-EMFIP-AGOG-22495386 1 PT30M 2020-01-01 2020-01-01 1 29.44 2020-01-01 00:00:00+00:00

If you've previously written your own code for extracting data from the Elexon/BMRS API then you may be wondering where some of the normal parameters you pass have gone. The reduction in the parameters passed are due to 4 core drivers:

  • Standardisation of date range parameter names
  • Removal of the need to specify ServiceType
  • Automatic passing of APIKey after client initialisation
  • Shipped with sensible defaults for all remaining parameters

The full list of data streams that are able to be requested can be found here. If you wish to make requests using the raw methods these are available through the ElexonDataportal.dev.raw module.

Further information can be found in the Quick Start guide.



What's Changed in v2

The latest release of the library includes a full rewrite of the code-base. We have endeavoured to make the new API as intuitive as possible but that has required breaking changes from v1, if you wish to continue using the historic library use pip install ElexonDataPortal==1.0.4. N.b v1 will not be maintained going forward, you are advised to change over to v2.0.0+.

The key feature changes are:

  • Coverage of more BMRS streams
  • Automated default values
  • Cleaner client API
  • A larger range of request types are compatible with the date range orchestrator


Programmatic Library Generation

One of the core features within the ElexonDataPortal library is that it is self-generating, by which we mean it can rebuild itself (including any new API request methods) from scratch using only the endpoints.csv spreadsheet. As well as generating the Python Client library a BMRS_API.yaml file is created, this provides an OpenAPI specification representation of the Elexon/BMRS API. In turn this allows us to automatically generate documentation, as well as run tests on the API itself to ensure that everything is working as expected - during this process we identified and corrected several small errors in the API documentation provided by Elexon.

To rebuild the library simply run the following in the root directory:

python -m ElexonDataPortal.rebuild

N.b. If you wish to develop the library further or use any of the programmatic library generation functionality then please install the development version of the library using:

pip install ElexonDataPortal[dev]

If you are not installing into a fresh environment it is recommended you install pyyaml and geopandas using conda to avoid any dependency conflicts. In future we are looking to release ElexonDataPortal as a conda package to avoid these issues.



Data Stream Descriptions

The following table describes the data streams that are currently retreivable through the API. The client method to retrieve data from a given stream follows the naming convention get_{stream-name}.

Stream Description
B0610 Actual Total Load per Bidding Zone
B0620 Day-Ahead Total Load Forecast per Bidding Zone
B0630 Week-Ahead Total Load Forecast per Bidding Zone
B0640 Month-Ahead Total Load Forecast Per Bidding Zone
B0650 Year Ahead Total Load Forecast per Bidding Zone
B0710 Planned Unavailability of Consumption Units
B0720 Changes In Actual Availability Of Consumption Units
B0810 Year Ahead Forecast Margin
B0910 Expansion and Dismantling Projects
B1010 Planned Unavailability In The Transmission Grid
B1020 Changes In Actual Availability In The Transmission Grid
B1030 Changes In Actual Availability of Offshore Grid Infrastructure
B1320 Congestion Management Measures Countertrading
B1330 Congestion Management Measures Costs of Congestion Management
B1410 Installed Generation Capacity Aggregated
B1420 Installed Generation Capacity per Unit
B1430 Day-Ahead Aggregated Generation
B1440 Generation forecasts for Wind and Solar
B1510 Planned Unavailability of Generation Units
B1520 Changes In Actual Availability of Generation Units
B1530 Planned Unavailability of Production Units
B1540 Changes In Actual Availability of Production Units
B1610 Actual Generation Output per Generation Unit
B1620 Actual Aggregated Generation per Type
B1630 Actual Or Estimated Wind and Solar Power Generation
B1720 Amount Of Balancing Reserves Under Contract Service
B1730 Prices Of Procured Balancing Reserves Service
B1740 Accepted Aggregated Offers
B1750 Activated Balancing Energy
B1760 Prices Of Activated Balancing Energy
B1770 Imbalance Prices
B1780 Aggregated Imbalance Volumes
B1790 Financial Expenses and Income For Balancing
B1810 Cross-Border Balancing Volumes of Exchanged Bids and Offers
B1820 Cross-Border Balancing Prices
B1830 Cross-border Balancing Energy Activated
BOD Bid Offer Level Data
CDN Credit Default Notice Data
DERSYSDATA Derived System Data
DETSYSPRICES Detailed System Prices
DEVINDOD Daily Energy Volume Data
DISBSAD Balancing Services Adjustment Action Data
FORDAYDEM Forecast Day and Day Ahead Demand Data
FREQ Rolling System Frequency
FUELHH Half Hourly Outturn Generation by Fuel Type
MELIMBALNGC Forecast Day and Day Ahead Margin and Imbalance Data
MID Market Index Data
MessageDetailRetrieval REMIT Flow - Message List Retrieval
MessageListRetrieval REMIT Flow - Message List Retrieval
NETBSAD Balancing Service Adjustment Data
NONBM Non BM STOR Instructed Volume Data
PHYBMDATA Physical Data
SYSDEM System Demand
SYSWARN System Warnings
TEMP Temperature Data
WINDFORFUELHH Wind Generation Forecast and Out-turn Data