harrystech/arthur-redshift-etl

Tie requirements.txt file to the Arthur version

thomas-vogels opened this issue · 0 comments

Describe the bug

The upload script that deploys Arthur copies the latest version of the code into S3 while keeping a version number, so for example: dist/redshift_etl-1.56.0.tar.gz. But the file with the requirements for that version is always copied as requirements.txt. That means that an accidental upload of the requirements file or a downgrade of the version leads to the "wrong" requirements file being active.

Expected behavior
The package requirements should be part of the tar ball or a wheel so that we can run something like:

python3 -m pip install arthur-etl

without relying on an un-versioned requirements.txt file.

Additional context

See also: #574
to bring in a process around requirements.