golemfactory/ray-on-golem

enable `pip install` from within the provider

mateuszsrebrny opened this issue · 0 comments

Why

We want ray worker running on provider to be able to get any needed libs from pip before starting the actual computations

How

  • On worker side we should be able to run something like this:

     pip install --index-url https://pypi.dev.golem.network/simple redis
    
  • We should be able to put pip command into yaml file to execute at the setup phase (for now it is ok to make user write --index-url, it might require hiding later though)

  • there is example of how this can be done in https://github.com/golemfactory/golem-kernel-python/tree/ms-misc

Tasks