As of 1 July 2020, Qlik Core is no longer available to new customers. No further maintenance will be done in this repository.
The Example GRPC S3 File Connector shows how to load data into Qlik Associative Engine from S3 using a dockerized connector built in Javascript. It streams the data using NodeJS streams from the S3 SDK via a simple transform to the grpc output stream.
The /example
directory defines a simple stack of services using docker-compose:
- Qlik Associative Engine
- S3 GRPC Connector
- Node Test Runner (only used for automated testing)
This example will show how to create a grpc file connector to retrieve the data (originally defined in example/airports.csv) using either corectl or node.
Run in a *nix environment (or Git Bash if on Windows), note that you must accept the
Qlik Core EULA by setting the ACCEPT_EULA
environment variable.
The connector reads S3 credentials from the following environment variables:
- CORE_S3_FILE_CONNECTOR_BUCKET_NAME_
- CORE_S3_FILE_CONNECTOR_BUCKET_ACCESS_KEY_ID
- CORE_S3_FILE_CONNECTOR_BUCKET_SECRET_ACCESS_KEY
- CORE_S3_FILE_CONNECTOR_BUCKET_REGION
as can be seen in example/docker-compose.yml.
To setup the docker containers simply
$ cd example
$ ACCEPT_EULA=yes docker-compose up -d --build
and then follow either the corectl or node instructions below to build an example app. Both examples use the loadscript example/script.qvs to load the data.
If you do not yet have corectl installed just follow the download instructions from corectl.
Once installed, try:
$ cd corectl
$ corectl build
$ corectl get tables
The build
command then builds the a qlik app using the information in example/corectl/corectl.yml
specifying for example the grpc connection. This command will build the app and load the data from your s3 bucket.
The get tables
command prints an overview of the tables present in the app.
To run the example using node.js do:
$ cd node
$ npm install
$ npm start
This will run the script example/node/index.js to create an app, populate it with data from the s3 bucket and print the resulting table contents.
We welcome and encourage contributions! Please read Open Source at Qlik R&D for more info on how to get involved.