/composeit

Process composing utility

Primary LanguagePythonMIT LicenseMIT

composeit

Process composing utility

Notice

This package is still during prototyping / exploratory development.

Currently it is as a toy/experimental project, not a production ready solution. Checkout a rough plan of development in TODO.md and COMPOSE.md

It might be just good enough for your use case, but no guarantees. See the similar tools section at the end if you are looking for a mature solution.

Purpose

Have docker-compose without dockers.

Sure, just running processes is not the same as running dockers, but the interface of docker-compose seems quite ok for organizing processes.

The target is to have an intuitive interface based on the tool from the dockers' world. Surely not all concepts will translate well to just processes but we still get the advantage of readability and familiarity.

Some of the features also will not work because we do not have the docker daemon. This is ok. Let's see what we can get.

Installing

Direct installation via git is the recommended way of installing the package.

You can use:

Note that these point to the main branch of the project. This project does not have a released version/tag yet.

pipx

pipx is a tool for installing Python command line tools in dedicated environments. Seems like a perfect match for composeit

Get pipx and then:

pipx install https://github.com/Bajron/composeit

(alternatively you can use git+ssh://git@github.com/Bajron/composeit)

pip

Separate environment is recommended because of the custom python-dotenv

pip install --upgrade https://github.com/Bajron/composeit

A package with a direct dependency (custom python-dotenv in our case) cannot be uploaded to PyPI, so simpler pip installation is rather unlikely.

Quick start hacking this repository

git clone https://github.com/Bajron/composeit
cd composeit
python -m venv venv

source ./venv/bin/activate

or

.\venv\Scripts\activate.bat

pip install -e .
cd examples/unios
composeit -up # ctrl+c to stop

Also see CONTRIBUTING.md

Documentation

See COMPOSE.md for features comparison to docker-compose. The document is created for tracking and explaining ideas.

Run the tool itself for more info about certain options.

composeit --help

or

composeit <command> --help

Ad hoc process server

To achieve certain features with the docker daemon missing, an ad hoc HTTP server is created that listens on a named pipe.

The pipe is created in the project directory on Linux. On Windows the pipe is created in .\\pipe\\. Default access rights are applied.

Known issues

Windows

  • Process handling on Windows is very different from Linux (e.g. signal support). There is no real graceful shutdown for processes at the moment.

  • Note that on Windows environment variable names in Python are always uppercase! This may lead to some unexpected expansions if your variables are not all uppercase.

python-dotenv

python-dotenv was a great example of setting up a Python project for me.

Loading environment and variable expansion uses its implementation.

I did not have patience to wait for my proposals there. This package uses my fork of the package. https://github.com/Bajron/python-dotenv/tree/v1.1.0

For this reason separate virtual environment or pipx is recommended to avoid a version clash.

If for some reason you want to use composeit with official python-dotenv, note there will be only a limited support for variable expansions in such combination.

To get the custom python-dotenv later you can run

pip install --upgrade python-dotenv@git+https://github.com/Bajron/python-dotenv.git@v1.1.0

Why another tool?

I wanted to organize a couple of processes on Windows, but I could not find a package that does that.

I wanted docker-compose without Docker.

There is supervisor that could do the trick, but it is not supported on Windows. It also is quite a serious package...

There are similar things like pypyr or prefect, but they do a different thing. I wanted to organize a set of long running processes, not a pipeline.

Later I also found honcho, seems ok for the task.

There is also a package called just-compose (wanted that name! ;( ). I found it late. It seems to be kind of what I want. The example looks quite promising. The PyPy page suggests some specific use case, and I did not try it really. It does seem similar to docker-compose yaml. Could be what I wanted, but...

I figured I can write it myself, have some fun and learn a little bit of Python and asyncio by the way.

Similar tools

It seems creating own compose-like tool is not that uncommon :)

Checkout these projects too for process organization:

You might be interested in these as well: