This repository is a template for developers to create Dataiku DSS plugins from GitHub.
Use it and adapt it as you wish, and have fun with Dataiku!
We recommend supporting your development cycle with unit and integration tests.
To operate integration tests, you will need the help of the dataiku-plugin-tests-utils
package to automate their executions while targeting dedicated DSS instances.
dataiku-plugin-tests-utils
will be installed as a pytest plugin
. Install that package inside an environment dedicated to integration tests; otherwise, pytest
will complain about unused fixtures inside your unit tests.
To install the dataiku-plugin-tests-utils
package for your plugins, use the following line depending on your preferred way to managed packages.
git+https://github.com/dataiku/dataiku-plugin-tests-utils.git@<BRANCH>#egg=dataiku-plugin-tests-utils
Replace <BRANCH>
with the most accurate value
git+https://github.com/dataiku/dataiku-plugin-tests-utils.git@releases/tag/<RELEASE_VERSION>#egg=dataiku-plugin-tests-utils
Replace <RELEASE_VERSION>
with the most accurate value
Put the following line under [dev-packages]
section
dku-plugin-test-utils = {git = "https://github.com/dataiku/dataiku-plugin-tests-utils.git", ref = "<BRANCH>"}
TBD
First, ensure that you have personal API Keys for the DSS you want to target. Secondly, define a config file that will give the DSS you will target.
{
"DSSX":
{
"url": ".......",
"users": {
"usrA": "api_key",
"usrB": "api_key",
"default": "usrA"
},
"python_interpreter": ["PYTHON27", "PYTHON36"]
},
"DSSY":
{
"url": "......",
"users": {
"usrA": "api_key",
"usrB": "api_key",
"default": "usrB"
},
"python_interpreter": ["PYTHON36", "PYTHON39"]
}
}
BEWARE: User names must be identical in the configuration file between the different DSS instances.
Then, set the environment variable PLUGIN_INTEGRATION_TEST_INSTANCE
to point to the config file.
To use the package in your test files:
import dku_plugin_test_utils
import dku_plugin_test_utils.subpakcage.subsymbol
Look at the next section for more information about potential subpackage
and subsymbol
.
The python integration test files are indirections towards the "real" tests written as DSS scenarios on DSS instances. The python test function triggers the targeted DSS scenario and waits either for its successful or failed completion. Thence your test function should look like the following snippet :
# Mandatory imports
from dku_plugin_test_utils import dss_scenario
def test_run_some_dss_scenario(user_dss_clients):
dss_scenario.run(user_clients, 'PROJECT_KEY', 'scenario_id', user="user1")
# [... other tests ...]
With:
user_dss_clients
: representing the DSS client corresponding to the desired user.PROJECT_KEY
: The project that holds the test scenariosscenario_id
: The test scenario to runuser
: Specify the user to run the scenario with. It is an optional argument. By default, it is "default".
For each plugin, a folder named allure_report
should exist inside the test
folder; reports will be generated inside that folder.
To generate the graphical report, you must have Allure installed on your system as described on their installation guide. Once the installation is done, run the following :
allure serve path/to/the/allure_report/dir/inside/you/plugin/test/folder/
As it is a tooling package for integration tests, it will aggregate different packages with different goals.
The following hierarchy exposes the different sub-package contained in dku_plugin_test_utils
with their aim
and the list of public symbols:
run_config
:ScenarioConfiguration
: Class exposing the parsed run configuration as a python dictionary.PluginInfo
: Parse the plugin.json and the code-env desc.json files to extract plugin metadata as a python dictionary.
dss_scenario
:run
: Run the target DSS scenario and wait for its completion (either success or failure).