OpenFn/apollo

Find a better solution to manage python version

Opened this issue · 0 comments

We have a little problem in our node-python set up

Right now, the node version is hard-coded in pyproject.toml and platform/src/bridge.ts. If the local python version drifts from this, the python bridge will break.

The problem is that we need to tell node-calls-python where the poetry environment is. And the poetry environment uses the minor python version in the path:

.venv/lib/python${PYTHON_VERSION}/site-packages

I have tried to write a function to scrap the python version from the shell. Bun has lovely apis for this. But the solution feels so flaky.

Hard-coding the python version, weirdly, actually feels like the best solution.

Possible solutions:

  • I've love venv to just sit in a static dir like .venv/lib. How can we configure poetry to generate that path with -in-project true? If the user installs a new python version then modules might break, but you're only a rimraf .venv && poetry install away from a fix.
  • Alternatively, there must be some poetry command which will give us the full path to the local lib? Surely? But I can't find it.