Simplify config management with `.toml` configuration file

[subhashb]

Abstract

This proposal recommends transitioning Protean's configuration management from its current versatile but complex system to a more streamlined approach using .toml files. This change aims to simplify both the codebase and the workflow for users, promoting clarity and maintainability.

Background

Currently, Protean supports multiple configuration formats, including Python files, JSON, YAML, environment variables, and direct object references. While this flexibility allows various ways to manage configurations, it often leads to inconsistent implementation and overuse, where configurations are cluttered with diverse types of constants.

Proposal

Implement a unified configuration approach with the domain configuration in a domain.toml file. TOML is popular because of its readability, ease of use, and growing adoption in the Python community (as seen with pyproject.toml).

Justification

• Simplification of Code and Workflows: Adopting a single configuration file type reduces the complexity of the codebase. This simplification extends to the development workflow, making it easier for new developers to adopt and use Protean effectively.
• Prevention of Misuse: The structured nature of .toml files helps maintain a clean and organized configuration schema, discouraging the misuse of the configuration file for storing unrelated constants.
• Support for Multi-Domain Architecture: Typical projects start with one domain, and subdomains emerge over time with increased clarity. Protean's design supports the development of applications with multiple domains, often structured as microservices within a single repository. The new bounded contexts will be carved out of existing directories. Developers should be able to earmark one of the subdirectories as a subdomain, which indicates the outer domain to ignore the subdomain directory during traversal. An elegant way to earmark the directory is to have a file like domain.toml (a la pyproject.toml).

Implementation Considerations

• Existing projects must migrate to the new .toml format. Given the low number of active projects, this transition will be done manually.
• Protean can pick up configurations from a variety of files, like pyproject.toml (tool.protean.* sections), domain.toml, or .domain.toml.
• Rules around Config file discovery have to be clearly defined. Ruff has a good implementation that we could adopt: https://docs.astral.sh/ruff/configuration/#config-file-discovery