loco-philippe/Environmental-Sensing

Environmental data interoperability (standard and connectors)

Environmental-Sensing

Make environmental data interoperable

Why a project for Environmental Data ?

There is a great diversity of solutions dealing with environmental data and paradoxically, the interoperability between these solutions remains very low, which requires the implementation of specific interfaces and leads to the original data being distorted through successive reprocessing.

This lack of interoperability results in a lack of transversal solution both at the level of use cases (integration of personal, family, professional, regional contexts) and at the level of the life cycle of environmental data: (1) the production of data (acquisition, observation, modeling), (2) their sharing (networks, storage), (3) the exploitation of information (restitution, analysis).

The establishment of strong interoperability is conditioned by:

• The existence of semantic data structure (standard)
• The availability of open tools for exploiting this structure (connectors),

Three additional conditions are necessary for this structure to be adopted and applied:

• A simple implementation adapted to each need,
• A construction of "convergence" rather than "questioning" of the existing,
• Compatibility with the "general interest" qualification of environmental data

The Environmental Sensing project (ES project)

The Environmental Sensing project aims to:

• Facilitate the use and sharing of environmental data
• Standardize both data acquisition equipment (sensors) and processing applications,
• Reduce coding/decoding operations (interfaces) by using standard connectors,
• Integrate a semantic level into the main existing standards (convergence)
• Optimize the volume of data exchanged

Work currently underway

• integrate a semantic dimension into JSON formats
• propose an alternative solution to the obsolete CSV format
• improve the quality of tabular data by better taking into account relationships between fields
• improve accessibility to data from sensors

Examples of achievements

sensors

• Bluetooth extension for Air Pollutants (available since sept-21)
  • Environmental Sensing Service (ESS) Bluetooth
  • Extension Air pollutants in ESS Bluetooth
  • ESS permitted Characteristics
• Tools for micro-controlers
  • C++ Implementation (experimental)

semantic JSON

• JSON-NTV : a semantic format for interoperability
  • Internet-Draft
  • NTV repository
• NTV-TAB : NTV tabular format
  • Internet-Draft
• JSON Schema extension to NTV data
  • Internet-Draft

tools for structured tabular data

• Tool for structuring Datasets(TAB-dataset repository)
• NTV-pandas repository : Semantic, compact and reversible JSON-pandas converter (pandas ecosystem)

tabular data analysis

• methodology
  • Concepts and principles for taking into account relationships between fields in tabular data
  • a simple implementation used to check the validity of relationship property
  • Add 'relationship' property in TableSchema issue #803, pattern #859
• tools
  • Tool for Dataset analysis(TAB-analysis repository)

If you are interested challenge us ! We will be very happy to show you the relevance of our approach

References

• Tabular data study

  • Tabular data management
  • What future for the CSV format ?
  • Etalab Talk

• Project presentation

  • overview of tabular structures (french)
  • interoperability of environmental data

The Environmental Sensing project is one of the six BlueHats Semester of Code projects selected among the 40 projects identified by in March 22.