TDP Lib

tdp-lib is a Python library built on top of Ansible. It is designed for managing TDP clusters, offering advanced tools to extend Ansible's capabilities. Key features include:

Creating a Directed Acyclic Graph (DAG) of tasks to manage dependencies and relationships between services and components.
Centralized configuration management through a unified variable definition system.

tdp-lib can be utilized in various ways: as a Python library, through an admin CLI, via a REST API (see tdp-server), or through a web interface (see tdp-ui).

Pre-requisites

To use tdp-lib, ensure you have the following prerequisites:

Python 3.9 or higher. Higher version are not guarantee to work, only 3.9 is tested.
A relational database management system (RDBMS), such as PostgreSQL or SQLite.

Optional dependencies for DAG visualization:

Graphviz for graphical representation of DAGs.

Installation

Set the following environment variables:

TDP_COLLECTION_PATH: Specifies the file path(s) to the necessary collection(s). tdp-collection is mandatory. Multiple collections can be specified, separated by a colon : (e.g., tdp-collection-extras, tdp-observability).
TDP_RUN_DIRECTORY: Path to the working directory of the TDP deployment (where ansible.cfg, inventory.ini, and topology.ini are located).
TDP_DATABASE_DSN: Database DSN (Data Source Name) for the chosen RDBMS.
TDP_VARS: Path to the folder containing configuration variables.

Ensure Ansible is configured to use the tosit.tdp.inventory plugin. Example ansible.cfg:

[defaults]
inventory=your_inventory,..,~/tdp_vars

[inventory]
enable_plugins = tosit.tdp.inventory,..,your_plugins

Install the library:

# Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate
# Install the dependencies
pip install "tdp-lib[visualization]@https://github.com/TOSIT-IO/tdp-lib/tarball/master"
# Initialize the database and tdp_vars
tdp init

CLI Usage

Note

This section is a work in progress.

tdp --help

Contributing

Contributions are welcome! Here are some guidelines specific to this project:

Use Poetry for development:

# Install Poetry
curl -sSL https://install.python-poetry.org | python3 -
# Install the dependencies
poetry install -E postgresql-binary -E mysql

Commit messages must adhere to the Conventional Commits standard.

Run tests on all supported databases before submitting a PR:

docker compose -f dev/docker-compose.yaml up -d
poetry run pytest tests --database-dsn 'postgresql+psycopg2://postgres:postgres@localhost:5432/tdp' --database-dsn 'mysql+pymysql://mysql:mysql@localhost:3306/tdp' --database-dsn 'mysql+pymysql://mariadb:mariadb@localhost:3307/tdp'
docker compose -f dev/docker-compose.yaml down -v

Format and lint code using (Black) and (Ruff):
```
poetry run task precommit-fix
```

Developer Documentation

Developer documentation is available here: docs/Developer

Docstrings are used to generate Sphinx documentation. Install the `docs`` extra dependency:

poetry install -E docs

Build the documentation:

poetry run task docs

The built documentation is available at docs/_build/html/index.html.