/cloud-compute

Set up and use pandas for off-core compute

MIT LicenseMIT

cloud-compute

Set up and use pandas for off-core compute

Installation

Cluster

In order to do meaningful compute, we will need a remote cluster. There are a number of ways to do this, but following current trends we will aim to deploy using Kubernetes and for the sake of my sanity, we will throw that at Azure. It is important to note that everything we do should run locally (but then why are we getting off of Pandas?), and more importantly should work on any modern cloud compute platform (wherever Kubernetes is welcome).

Azure

Let's get AKS working ...

  1. Get a Docker Hub account!
  2. Install the Azure CLI
  3. JDK 8, get it ready
  4. Scala Build Tool (SBT)