NixOS/equinix-metal-builders

Manage spot market bids with Terraform

Closed this issue · 2 comments

Right now the Packet spot market bids are just managed by hand. I think it'd be better and more transparent to manage them via terraform, right here.

Here is a starter config:

variable "project_id" {
  default = "86d5d066-b891-4608-af55-a481aa2c0094"
}

resource "packet_spot_market_request" "req" {
  project_id    = var.project_id
  max_bid_price = 1.80
  facilities    = ["ams1", "sjc1", "dfw2", "nrt1", "ewr1"]
  devices_min   = 1
  devices_max   = 1

  instance_parameters {
    hostname         = "c2.large.arm"
    billing_cycle    = "hourly"
    operating_system = "custom_ipxe"
    always_pxe       = true
    plan             = "c2.large.arm"
    ipxe_script_url  = "https://netboot.gsc.io/hydra-aarch64-linux/netboot.ipxe"
    project_ssh_keys = []
    user_ssh_keys    = []
    tags             = [ "hydra" ]
  }
}

resource "packet_spot_market_request" "req2" {
  project_id    = var.project_id
  max_bid_price = 1.70
  facilities    = ["ams1", "sjc1", "dfw2", "nrt1", "ewr1"]
  devices_min   = 1
  devices_max   = 1

  instance_parameters {
    hostname         = "c2.large.arm"
    billing_cycle    = "hourly"
    operating_system = "custom_ipxe"
    always_pxe       = true
    plan             = "c2.large.arm"
    ipxe_script_url  = "https://netboot.gsc.io/hydra-aarch64-linux/netboot.ipxe"
    project_ssh_keys = []
    user_ssh_keys    = []
    tags             = [ "hydra" ]
  }
}

Setting up the shell.nix, adding it to the build pipeline, etc. is still to-do. We should assume the packet API key will come from the environment, though not sure exactly what the environment variable should be named, it is currently named PACKET_TOKEN.

You'll need to use a nixpkgs with this commit in it: NixOS/nixpkgs#105790

Ideally this would use terraform's for_each support, and accept config in the format of:

aarch64-bids = [
  { bid = 1; count = 1; }
]

like this:

variable "docker_ports" {
  type = list(object({
    internal = number
    external = number
    protocol = string
  }))
  default = [
    {
      internal = 8300
      external = 8300
      protocol = "tcp"
    }
  ]
}

from https://www.terraform.io/docs/configuration/variables.html