AlexsLemonade/refinebio

Create mechanism to run pipelines locally

kurtwheeler opened this issue · 1 comments

Context

#2584

We can't run Batch locally like we could with Nomad. This means that there's not currently a way to run a sample all the way through locally without using the tester.sh script which hasn't gotten a lot of use in the last couple years.

Problem or idea

I think we need a script that will monitor the database and will start containers for any jobs that are waiting to be run. This is necessary because no one docker image has all the dependencies for all jobs.

Solution or next step

Create that script or a similar mechanism to be able to run all jobs locally, along with running downstream jobs.

Closed by #2799