twosigma/waiter

CLI should support temporarily ejecting an instance

shamsimam opened this issue · 0 comments

The Waiter CLI should expose a subcommand to temporarily (timeout flag) ejecting a specific instance.
Alternatively, we can expose ejecting and unejecting an instance.

There should be a mechanism to select instances.

In addition to marking the instance as ejected, it also needs to be tracked as an ejected instance in the responder.
Sample code that updates an annotation in a Kubernetes pod:

(defn mark-pod-for-ejection
  "Marks the pod for ejection  that marks the resulting instance with the flag :ejected."
  [{:keys [api-server-url] :as scheduler} {:keys [id] :as instance}]
  (let [pod-url (instance->pod-url api-server-url instance)]
    (log/info "marking instance for scale down" {:instance-id id})
    (patch-object-json pod-url
                       [;; The backslash becomes "~1" so "waiter/prepared-to-scale-down-at" becomes "waiter~1prepared-to-scale-down-at"
                        ;; Source https://stackoverflow.com/questions/55573724/create-a-patch-to-add-a-kubernetes-annotation
                        ;; Here is the RFC-6901 reference https://www.rfc-editor.org/rfc/rfc6901#section-3
                        {:op :add :path "/metadata/annotations/waiter~1pod-ejected" :value "true"}]
                       scheduler)))