bodywork-ml/bodywork-core

Create a `configure cluster` command

AlexIoannides opened this issue · 0 comments

As a Machine Learning Engineer, I would like all k8s cluster setup to be done for me, so that I don't need to understand k8s before deploy my pipelines.

Tasks

  • Create a bodywork.cli.configure_cluster module.
  • Implement a setup_cluster_for_bodywork function that will perform all the necessary steps to configure k8s for use with bodywork - e.g.,
    • create a bodywork-deployment-jobs namespace, into which all deployment (workflow) jobs will be run.
    • create service accounts and roles for bodywork-deployment-jobs, so that workflow-jobs will be able to create resources. This could be done with bodywork.k8s.auth.setup_workflow_service_account, but it may have to be modified with the option to grant permission for namespaces to be created, which will be required by workflow-jobs in bodywork-deployment-jobs (as we want to automate namespace creation).