botify-labs/simpleflow

Installation from git is broken

Opened this issue · 10 comments

jbbarth commented

Problem:

pip uninstall -y simpleflow; pip install git+https://github.com/botify-labs/simpleflow.git@62c4fd24e87c754d253438d78fd85b2e733133f2

results in:

% ls .venv/lib/python3.11/site-packages/simpleflow
__init__.py  __pycache__     activity.py  canvas.py   constants.py  exceptions.py  executor.py  futures.py  log.py              marker.py     registry.py  storage.py  workflow.py
__main__.py  _decorators.py  base.py      command.py  download.py   execute.py     format.py    history.py  logging_context.py  metrology.py  signal.py    task.py

We're missing all the subdirectories.

Versions in case it matters:

% python --version
Python 3.11.5
% pip --version
pip 23.1.2 from /Users/jean-baptiste.barth/Library/Caches/pypoetry/virtualenvs/xxxx-4VfeYLr8-py3.11/lib/python3.11/site-packages/pip (python 3.11)

I'm digging

jbbarth commented

Thanks @ewjoachim I just saw exactly that!

For posterity:

  • ❌ I confirmed it was the same with an old commit (pip uninstall -y simpleflow; pip install git+https://github.com/botify-labs/simpleflow.git@1e755cea0693f55dcc1013a3235507c88df79e39 => same result)
  • ❌ the behaviour is the same locally: pip uninstall -y simpleflow; pip install --verbose ../../botify-labs/simpleflow => makes it easy to test
  • ✅ changing "tool.setuptools.packages" from a static value to find = {} works

I'll check everything twice and submit a PR this morning

ybastide commented

Oopsie—I've been bitten by this a few times but didn't investigate it; thank you both!

jbbarth commented

The issue is that as soon as I include all sub directories correctly, I get "build", "examples", "venv" and "tests" installed into my site-packages. The "exclude" directive doesn't seem to work, at least not as documented. I'm digging more. Send help if you don't hear about me in 2 days.

jbbarth commented

@ybastide @ewjoachim I randomly stumbled upon Hatch via Stackoverflow, and it works nicely out of the box. I'm tempted to dig into that direction, and in particular check if it has any impact on "script/release". Any opinion on the topic?

ybastide commented

Looks like it would simplify script/release bigly 🙂
Joachim has a lot of experience in such subjects, I'll defer to both of you.

  • 👍2
ewjoachim commented

Not a huge fan of hatch compared to poetry (both are equivalent tools) but it's true that hatch, being born later, uses the standard PEP-defined format for pyproject.toml while poetry uses its own format.

Honnestly, I'm ok with both.
Let's go with hatch, this way I'll learn.

  • 🚀2
jbbarth commented

Alternatively we can keep setuptools if we find a config that works (== that doesn't leak "examples" in my site-packages 😆)

About poetry: I'm using it daily and I absolutely hate it. It failed us so many times (not in the context of a lib though). I'd be very happy to discuss this with you over a beer one day!

  • 👍1
  • 😄1
ybastide commented

👎 on keeping setuptools, we all want to learn 🙂

jbbarth commented

@ybastide @ewjoachim I tested hatch a bit more and submitted a PR here => #435. Lmk what you think, no hurry!