Make the Buildbot independent

[sampsyo]

I don't know why this realization came to me only now, but I have come to a startling epiphany about the soul of the Buildbot.

I originally made it to address three problems:

• tools that are super slow and therefore annoying to run on your dev machine
• tools that are inconvenient to run at all because of licensing and platform dependencies
• the need for access to special hardware

None of these problems apply to the Seashell compiler itself. It's all about dealing with HLS and other FPGA tools. It's not inconvenient, slow, or expensive to run our compiler locally (despite Scala ). At this point, we're even compiling everything locally before submitting jobs anyway.

So I think we should separate the Buildbot from the compiler. In its soul, the Buildbot doesn't want to be Seashell-specific—it just wants to be a convenient wrapper for inconvenient FPGA tools and hardware. We can give it a nice name, like Cyclops or Polyphemus, and send it on its way. Freed from the need to compile Seashell programs, it can be simpler and better fulfill its true purpose—wrapping the FPGA toolchains.

Concretely, this means we should:

• move it to a separate git repository
• drop the Seashell stage (we can keep the HLS stage)
• remove the silly "live interface" and the "non-Seashell" config option
• simplify the Docker setup on our server

This will be satisfying for me to do, so I'll plan on doing it. It is, however, not a terribly high priority and we have bigger fish to fry at the moment.

[rachitnigam]

Woot! No more python code! Finally, I can justify rewriting everything in monadic (and excruciatingly slow) Scala!!

/s

[sampsyo]

Alright, here are some concrete steps I had in mind:

• Resolve or decide to move any open PRs—perhaps that's just #242 and #240 at the moment. (And #240 is 100% ready to go, I think?)
• Move the buildbot directory to a new repository.
• Delete the Dockerfile from this repository. We'll probably just start from scratch in the new repo.
• Delete the Seashell stage and the "no-Seashell" job config option from the Buildbot.
• Bring our local config stuff up to speed with the new repo.
• Move any relevant issues to the new repo from this one.

And mostly as a note to myself, here are a few issues I was intending to create in the new repo once it exists:

• Make a workflow for packaging the Buildbot as an AMI for easy deployment, including on non-F1 instances if you don't actually want to run stuff (it's cheaper!).
• @ViviYe has talked about adding a "delete" button to jobs to free up space.
• Not the highest priority, but: lots of operations will slow down linearly with the number of jobs because they scan the entire database; cutting down on this would be awesome.
• Feature request: "priority" jobs that jump the queue and run immediately for interactive execution. A client CLI tool could help submit jobs and wait for their results, streaming the log interactively, to sort of simulate the feeling of running commands on your own machine.