/ngsi-timeseries-api

QuantumLeap: a FIWARE Generic Enabler to support the usage of NGSIv2 (and NGSI-LD experimentally) data in time-series databases

Primary LanguagePythonMIT LicenseMIT

QuantumLeap

FIWARE Core Context Management License: MIT Docker badge Support
Documentation badge CircleCI Coverage Status Status Swagger Validator CII Best Practices

QuantumLeap is the first implementation of an API that supports the storage of FIWARE NGSIv2 data into a time series database. It currently also experimentally supports the injection of NGSI-LD in a backward compatible way with NGSI-v2 API. I.e. you can retrieve NGSI-LD stored data via NGSI v2 API and retrieve data will be describe following NGSI v2 format.

QuantumLeap is not a real time API, its purpose is to process notifications received from the Context Broker and to create temporal records for them. In general, the whole FIWARE stack, being based on a micro-service architecture, cannot be regarded as real time in case you have requirements on guaranteed delivery in a given amount of time.

However, even though hard real time may not be FIWARE's forte, in our experience a properly tuned FIWARE stack can perform extremely well and handle very demanding IoT workloads without a glitch - you just need to configure your infrastructure to handle that :)

Want to know more? Refer to the docs or checkout the Extra Resources below.

This project is contributed to FIWARE by Martel Innovate as part of its Orchestra Cities platform. You can find more FIWARE components in the FIWARE catalogue.

📚 Documentation 🎓 Academy quay.io 🎯 Roadmap

Contents

Install

Refer to the Installation Guide.

QuantumLeap supports both Crate DB and Timescale as time-series DB backends. The following versions are currently covered in tests (see .circleci/config.yml):

  • Crate backend: Crate DB version 4.2.7 up to 4.6.7
  • Timescale backend: Postgres version 12 up to 13 + Timescale extension 1.7.5 up to 2.3.0 + Postgis extension 2.5.*.

As regards caching feature, QuantumLeap has been tested with Redis 4.* up to 6.2.3. Integration with Orion Context Broker is tested from version 2.4.2 up to 3.3.1.

Automated tests, to avoid too long testing time, are not combining all possible versions. This anyhow should not be an issue given that component integrations are not depending on each other. In general, only latest version listed is actively supported.

PR #373 introduced basic support for NGSI-LD. In short this means that using the current endpoint you are able to store NGSI-LD payloads with few caveats (see #398)

Docker Images

Docker images are available on docker hub:

  • latest refers to the last release. This behaviour will be introduced since 0.8.3 release.
  • edge refers to the version in master, this behaviour is introduced since 0.8.3-dev activities.
  • *.*.* refers to specific releases.
  • additionally (usage is not recommended), we release images for each branch.

Security note

CrateDB versions > 4.6 is affected by log4 security issue. We recommend to update your deployments to CrateDB 4.6.5. For CrateDB versions >= 3.2.0 it is possible to mitigate the issue setting -Dlog4j2.formatMsgNoLookups=true via CRATE_JAVA_OPTS or by setting the environment variable LOG4J_FORMAT_MSG_NO_LOOKUPS=true.

Usage

Refer to the User Manual.

Troubleshooting

Refer to the Troubleshooting section.

Contributing

Refer to the Contributing section and to the contribution guidelines.

Extra resources

The following is a collection of external guides and pages where you may find additional documentation about QuantumLeap. Note that these guides could be outdated (so could the official docs!), so we appreciate all efforts to keep consistency.

Adopters

Orchestra Cities QuantumLeap has been adopted by the FIWARE community and is one of the core components that make up the FIWARE core context management stack. The companies listed below use QuantumLeap as part of their IoT solutions. If you would like to add your company to this list, please mention it in this issue or create a PR to update the list.

Profirator

Linked projects

The following projects are linked to Orchestra Cities QuantumLeap, if you want to list it here, add your information on this issue or create a PR to update the list.


NGSI Go is a CLI supporting QL

Enterprise Support

Contact us if you need Enterprise Support for your QuantumLeap installation.


License

QuantumLeap is licensed under the MIT License

© 2017-2023 Martel Innovate