/ODS_OpenExposureData

Open data standards curated by Oasis.

Primary LanguagePython

Latest version of ODS
ODS version PyPI version OpenData Build

                                                       

Mission Statement:

ODS should be the standard used for exposure data, result outputs and contract(s) terminology for the (Re)Insurance industry and other interested parties in catastrophe modelling and exposure management across all classes of business.

Steering Committee Members:

 

Open Data Standards (ODS)

ODS is curated by Oasis LMF and governed by the Open Data Standards Steering Committee (SC), comprised of industry experts representing (re)insurers, brokers, service providers and catastrophe model vendors. The SC will evolve over time and include Subject Matter Experts to assist with areas requiring specific domain experience, including data validation and support for enhancing and automating the interoperability of these standards.

The components of ODS are (but not limited to) the Open Exposure Data (OED) format and the Open Results Data (ORD) format. Both OED and ORD are designed to assist with solving interoperability problems current in the cat modelling community, where implementing a model-developer-independent exposure data and results format will assist in creating choice in the use of catastrophe models and analytical tools.

Further information and community views of ODS can be found on the ODS website: https://oasislmf.org/open-data-standards

It's important to know that ODS is NOT an Oasis standard, but an initiative developed by the market for the market.

 

Releases

Version Excel Spec JSON Spec Release Notes
2.0.0 OpenExposureData_Spec.xlsx OpenExposureData_Spec.json Release Page
2.1.0 OpenExposureData_Spec.xlsx OpenExposureData_Spec.json Release Page
2.2.0 OpenExposureData_Spec.xlsx OpenExposureData_Spec.json Release Page
2.3.0 OpenExposureData_Spec.xlsx OpenExposureData_Spec.json Release Page
3.0.0 OpenExposureData_Spec.xlsx OpenExposureData_Spec.json Release Page
3.1.0 OpenExposureData_Spec.xlsx OpenExposureData_Spec.json Release Page
3.2.0 OpenExposureData_Spec.xlsx OpenExposureData_Spec.json Release Page
3.3.0 OpenExposureData_Spec.xlsx OpenExposureData_Spec.json Release Page
3.4.0 OpenExposureData_Spec.xlsx OpenExposureData_Spec.json Release Page

Structure of ODS

The diagram below highlights the proposed, long-term structure of ODS and all the key components. Interoperability is vital to ensure efficient interaction across multiple databases, systems and external exposure management and data storage facilities.

 

 

ODS Implementation into Oasis: Oasis LMF are continuously expanding the ODS functionality they support on their platform, especially in their financial module (FM).

Detailed documentation on which financial fields are supported in the Oasis kernel can be found here: https://github.com/OasisLMF/OasisLMF/blob/master/docs/OED_financial_terms_supported.xlsx  

Open Exposure Data (OED)

The aim of OED is to provide the industry with a robust, open, and transparent data format. This will improve efficiency and transparency for the cat modelling community, facilitating data transfer and analytics across models and vendors. OED is a model agnostic data format and the detailed descriptions of each data field for property are covered in the 'Open Exposure Data Spec.xlsx' and reference and background OED information can be found in the 'docs’ folder (https://github.com/OasisLMF/ODS_OpenExposureData/tree/develop/OpenExposureData/Docs). Examples of how to code multiple financial structures in the input files are also covered within these documents.

The web (HTML) version of the OED documentation can be viewed here https://oasislmf.github.io/sections/OED.html

 

Open Data Transformation Framework (ODTF)

The ODTF is funded by the The Insurance Development Forum (IDF) and is an industry collaboration to develop a conversion tool that transforms exposure data to and from different formats including OED. More information can be found here:

https://github.com/OasisLMF/OpenDataTransform

 

Liability

The focus for OED has primarily been on property cat business since its inception but has now expanded to support other lines of business. The liability data schema was released in April 2022 - details and docs can be found here:

https://github.com/OasisLMF/ODS_OpenExposureData/tree/develop/OpenExposureData/Liability

 

Cyber

A cyber data standard was released in February 2023 and the data schema and associated docs can be found here:

https://github.com/OasisLMF/ODS_OpenExposureData/tree/develop/OpenExposureData/Cyber

 

Open Results Data (ORD)

ORD was initially developed during the Lloyd's Lab innovation project (Cohort 3) in 2019, by a working group led by Oasis, that focussed on constructing model agnostic results formats and appropriate data formats. These model outputs cover an extensive suite of results that can be isolated by aspects of the exposure data, financial and statistical perspectives.

ORD follows the same versioning format as OED (following the SemVer convention as described below) but is developed and versioned independently to OED in a separate repository that can be found here:

https://github.com/OasisLMF/ODS_OpenResultsData

 

Governance

ODS is governed by a steering committee that meets periodically and is chaired by Oasis LMF.

The governance and process around releases and updates of ODS can be found here but a schematic of the overview is shown below: https://github.com/OasisLMF/ODS_OpenExposureData/tree/develop/Docs

 

A Technical Working Group (TWG) will maintain and update the assets that define the data standards. Although the TWG has a few core members from Oasis LMF and Nasdaq, its dynamic and will involve the users from the community who have proposed the updates or changes.

 

Releases

All releases will follow the SemVer convention (https://semver.org/), so given a version number MAJOR.MINOR.PATCH.

Major Updates

Major version updates are when incompatible changes are made that can cause 'breakages' in processes or validations, examples are highlighted below. Major updates require signoff from the Steer Co and require a three month time frame for notice and any UAT by the community.

  • Removing fields from the schema.*
  • Adding ‘mandatory’ fields to the schema.
  • Moving fields between input files (i.e., from ‘ReinsScope’ to ‘ReinsInfo’ files)
  • Removing occupancy or construction codes.
  • Changing or removing any other codes currently used in the OED schema. I.e., any codes contained in the tabs of the OpenExposureData_Spec.xlsx (https://github.com/OasisLMF/ODS_OpenExposureData/blob/develop/OpenExposureData/Docs/OpenExposureData_Spec.xlsx)
  • Reducing the ‘valid value range’.
  • Changing the ‘allow blanks’ field.
  • Changing the ‘default’ values.

Minor Updates

Minor version updates are when functionality is added and is backwards compatible and WILL NOT cause breaking changes. These are highlighted below:

PATCH updates are for making backwards compatible bug fixes e.g. correcting a typo in a column label.

All new work will be done in feature branches, following the GitFlow model. The latest released version will be held in the master branch and the current development work will be in develop or specific feature branches.

 

Versioning and Back Compatibility

The 'versioning' tab in the OED spec (https://github.com/OasisLMF/ODS_OpenExposureData/tree/OED_v3.1_testing/OpenExposureData/Docs) contains the mapping and back compatibility of occupancy and construction codes from previous versions of OED. For example, if a user codes exposure data using an occupancy or construction code from OED v3, but the model implementation they are running is only using the OED v2 schema, then that code will be mapped back to an appropriate code in OED v2 so any validation processes will still work.

 

Tracking

Suggestions for updates will be raised in this repository and contain all correspondence and associated documents or data. Any GitHub user can raise an issue and the TWG will classify as a major or minor update as specified above and follow the governance and signoff process.

 

CSV to Parquet Data Conversion

Apache Parquet is an open-source columnar data format which has superior data compression and encoding schemes enabling more efficiency around the handling of large datasets. More info here (https://parquet.apache.org)

All source code and info on converting csv to Parquet data format can be found at the repo below: https://github.com/OasisLMF/ODS_Tools

 

Documentation Updates

These can be actioned directly by the TWG but are likely to be done by Oasis LMF.

 

Bugs

These can be actioned directly by the TWG, assuming they do not cause a breaking change. All communication around bugs should be submitted in an issue within this repo.

 

Attributions

OED was developed by a working group of industry practitioners in close collaboration with AIR Worldwide who generously made their CEDE schema available to the industry. The working group was chaired by Matthew Jones with technical work led by Aiste Kalinauskaite both of NASDAQ (formerly Simplitium). The initial OED assets were ported from https://github.com/simplitium/oed on 28 April 2020 with permission from NASDAQ.

 

License

The content of ODS including the documentation and the schema within both Open Exposure Data (OED) and Open Results Data (ORD) are licensed under the CC0 1.0 Universal license. Any code, MS SQL scripts or the development of tools are licensed under BSD 3-clause license.