/VSDS-LDESWorkbench-NiFi

Linked Data Event Stream Workbench that prepares data for ingestion, built on Apache NiFi

Primary LanguageJavaMIT LicenseMIT

LDES Workbench NiFi

The workbench is a customized Apache NiFi docker image containing processors built by VSDS to allow processing of data that is intended to be ingested by an LDES server.

Available Processors

LDES Client

The LDES Client Processor is a Nifi wrapper around the LDES Client SDK which enables you to follow and synchronize with a Linked Data Event Stream.

Further details on the usage of the the Client can be found here.

NGSI V2 To LD

To support the ingestion of input data in NGSI V2 format, the NgsiV2ToLd processor will take the inputted NGSI V2 data and transform it to NGSI LD.

Further information on how to use this processor can be found in the in-NiFi documentation.

NGSI LD to LDES

To support the ingestion of input in NGSI LD format, the NgsiLdToLdes processor will take the inputted NGSI LD data and transform it into LDES members to be ingested into an LDES server.

Further information on how to use this processor can be found in the in-NiFi documentation.

Deliverables

NiFi nar packages

The above-mentioned processors are being packaged as individual nar files and can be manually added to a locally running NiFi instance by adding it to the /lib directory.

NiFi Docker

Additionally, a docker file is provided to avoid setting up a Nifi instance which contains a NiFi instance (based on the apache/nifi docker) and the above-mentioned processors.

The docker doesn't contain any workflows, these can be uploaded manually.

docker run --name ldes-workbench \
  -p 8443:8443 \
  -d \
  -e SINGLE_USER_CREDENTIALS_USERNAME=[username] \
  -e SINGLE_USER_CREDENTIALS_PASSWORD=[password consisting of 32 characters] \
  ghcr.io/informatievlaanderen/ldes-workbench-nifi:latest