Designed by Agile Lab, Witboost is a versatile platform that addresses a wide range of sophisticated data engineering challenges. It enables businesses to discover, enhance, and productize their data, fostering the creation of automated data platforms that adhere to the highest standards of data governance. Want to know more about Witboost? Check it out here or contact us!
This repository is part of our Starter Kit meant to showcase Witboost's integration capabilities and provide a "batteries-included" product.
This Python microservice implements an Authentication Webhook and Role Mapping service for GraphQL Output Ports based on Hasura. It is used by the corresponding Hasura Specific Provisioner.
Hasura is an open-source, real-time GraphQL engine that simplifies and accelerates API development for web and mobile applications. It connects to your data sources like databases or REST services and automatically generates a GraphQL API, making it easier to query and manipulate data. Hasura's real-time capabilities enable instant updates to clients when data changes, enhancing the responsiveness of applications. It's a popular tool for developers looking to streamline the process of building dynamic and interactive applications by providing a unified and efficient way to access, manage, and synchronize data.
This microservice is written in Python 3.11, using FastAPI for the HTTP layer. Project is built with Poetry and supports packaging as Wheel and Docker image, ideal for Kubernetes deployments (which is the preferred option).
The Python project for the microservice is in the hasura-authenticationwebhook-rolemapping-service subdirectory; this is probably what you're interested in. It contains the code, the tests, the docs, etc.
The rest of the contents of the root of the repository are mostly support scripts and configuration files for the GitLab CI, gitignore, etc.
Requirements:
- Python 3.11
- Poetry
To set up a Python environment we use Poetry:
curl -sSL https://install.python-poetry.org | python3 -
📝 If you are on Windows, you probably want to use pipx instead:
pipx install poetry
Once Poetry is installed and in your $PATH, you can execute the following:
poetry --version
If you see something like Poetry (version x.x.x), your install is ready to use!
Install the dependencies defined in hasura-authenticationwebhook-rolemapping-service/pyproject.toml:
cd hasura-authenticationwebhook-rolemapping-service
poetry install
Poetry automatically creates a Python virtual environment in which the packages are installed; make sure to read the next section to enable it.
📝 If you are on Windows, you may get an error about Visual C++ missing; follow the instructions provided by Poetry to fix it.
You just need to enable the Python virtual environment (venv) generated by Poetry:
source $(poetry env info --path)/bin/activate
As with any Python venv, your shell prompt will change to reflect the active venv.
You can also use:
poetry shell
Which spawns a subshell in the virtual environment; it is slighly different than the command above as this is not a login shell, hence your shell's profile file will likely be ignored.
Simply run:
pre-commit install
In case you need to commit and skip the pre-commit checks (eg, to push WIP code, or to test that the CI catches formatting issues), you can pass the --no-verify flag to git commit.
The recommended IDE is PyCharm, though other ones should work just fine.
In order to import the project, use the standard "Open..." dialog and point PyCharm to the hasura-authenticationwebhook-rolemapping-service subdirectory, not the repository root. This ensures that PyCharm correctly identifies this as a Poetry project and prompts you to set it up as such.
The Docker image can be built with:
docker build .
More details can be found here.
Application version: the version for the project is automatically computed using information gathered from Git, using branch name and tags. Unless you are on a release branch 1.2.x or a tag v1.2.3 it will end up being 0.0.0. You can follow this branch/tag convention or update the version computation to match your preferred strategy.
CI/CD: the pipeline is based on GitLab CI as that's what we use internally. It's configured by the .gitlab-ci.yaml file in the root of the repository. You can use that as a starting point for your customizations.
To run the server locally, use:
cd hasura-authenticationwebhook-rolemapping-service
source $(poetry env info --path)/bin/activate # only needed if venv is not already enabled
uvicorn src.main:app --host 127.0.0.1 --port 8092By default, the server binds to port 8092 on localhost. After it's up and running you can make provisioning requests to this address. You can also check the API documentation served here.
Application configurations are handled with environment variables:
| Environment Variable | Description |
|---|---|
| GRAPHQL_URL | URL of the Hasura instance |
| GRAPHQL_ROLE | Role to use when performing actions on Hasura |
| GRAPHQL_ADMIN_SECRET | Admin secret for the Hasura instance |
| JWKS_URL | JWKS URL, eg "https://login.microsoftonline.com/common/discovery/v2.0/keys" |
| JWT_AUDIENCE | JWT audience, eg "https://management.core.windows.net/" |
| JWT_ALGORITHMS | JWT algorithms, eg `["RS256", "RS512"] |
| JWT_OPTIONS | JWT options, eg "{\"verify_exp\": true,\"require\": [\"exp\", \"iat\"]}" |
| AZURE_SCOPES | For Azure AD JWTs; the JWT scopes, eg [\"https://graph.microsoft.com/.default\"] |
| AZURE_TENANT_ID | For Azure AD JWTs; the tenant id |
| AZURE_CLIENT_ID | For Azure AD JWTs; the client id |
| AZURE_CLIENT_SECRET | For Azure AD JWTs; the client secret |
| AUTHORIZATION_HEADER_FIELD_NAMES | List of headers fields to use, eg: "[\"authorization\", \"Authorization\"]" |
| ROLEMAPPING_TABLE_SCHEMA | Schema for the role mapping table on the database, eg "rolemapping" |
Those environment variables are already templated in the Helm chart (see below). Customize them according to your needs.
Logging is handled with the native Python logging module. The Helm chart provides a default logging.yaml that you can override. Check out the Helm docs for details.
To configure the OpenTelemetry framework refer to the OpenTelemetry Setup.
This microservice is meant to be deployed to a Kubernetes cluster with the included Helm chart and the scripts that can be found in the helm subdirectory. You can find more details here.
Refer to the HLD for the corresponding Specific Provisioner here.
This project is available under the Apache License, Version 2.0; see LICENSE for full details.
Agile Lab creates value for its Clients in data-intensive environments through customizable solutions to establish performance driven processes, sustainable architectures, and automated platforms driven by data governance best practices.
Since 2014 we have implemented 100+ successful Elite Data Engineering initiatives and used that experience to create Witboost: a technology-agnostic, modular platform, that empowers modern enterprises to discover, elevate and productize their data both in traditional environments and on fully compliant Data mesh architectures.
Contact us or follow us on: