PaulFarault/tdp-lib

Python library to configure, manage and deploy TDP

TDP Lib

TDP lib is a Python library built on top of Ansible to manage clusters. It provides a set of tools to overcome the limitations of Ansible, such as:

• Defining a DAG of tasks based on relations between services and components
• Defining variables in a single place

tdp-lib can be used as a Python library, through an admin CLI, a REST API (see tdp-server) or a web interface (see tdp-ui).

Pre-requisites

TDP lib requires:

• Python 3.9+ with Poetry
• A RDBMS system (such as PostgreSQL or SQLite)

Optionally, you can install the following dependencies for DAG visualization:

• graphviz
• Python visualization dependency (poetry install -E visualization)

And to build the documentation:

• Python docs dependency (poetry install -E docs)

Installation

Install dependencies and the package in a virtual environment:

poetry install

Export the following environment variables:

• TDP_COLLECTION_PATH: path(s) to the collection(s). tdp-collection is mandatory. Other collections can be added, separated by a colon : (such as tdp-collection-extras, tdp-observability).
• TDP_RUN_DIRECTORY: path to the working directory of TDP deployment (where ansible.cfg, inventory.ini and topology.ini are located).
• TDP_DATABASE_DSN: DSN of the database to use.
• TDP_VARS: path to the folder containing the variables.

Note: Ansible must be configured to use the tosit.tdp.inventory plugin. For example, in ansible.cfg:

[defaults]
inventory=your_inventory,..,~/tdp_vars

[inventory]
enable_plugins = tosit.tdp.inventory,..,your_plugins

Finally, initialize the database and default variables:

poetry shell
tdp init

CLI usage

Full documentation can be found inside docs/cli.

Build the documentation

Documentation can be built with:

poetry run task docs-html

Built doc is available at docs/_build/html/index.html.

Contributing

Install pre-commit hooks:

poetry run pre-commit install --hook-type pre-commit
poetry run pre-commit install --hook-type commit-msg

Run the tests:

poetry run pytest tdp

Format the code:

poetry run black .

Developers documentation: docs/Developer