/gcpfire

Run Short jobs on GCP Spot Instances

Primary LanguagePython

GCP Fire

Next Steps

  • Reduce Docker Image download time by having some version of the docker images that is semi-recent baked into the image.
  • Use optional git pull & default image instead of Docker to reduce runtime.
  • PIPE SSH Key
  • Add Profiling
  • Run Multiple Jobs in Succession
  • Parallelize & Proper parallelized logging
  • Get rid of "Any" type hints from Google API

Changelog

0.2.1

  • Better error handling and propagation of stdout/stderr.

0.2.0

  • Add Type hints
  • Minor improvements

0.1.1

  • Make this a package (so we can import it on other packages)
  • Add Exporter Task
  • Docker uses GPU now
  • Analysis Task is now working.
  • Set proper service account & make SSH-Login inherit permissions.
  • Enabble Docker Client on VM access to docker registry.

0.1.0

  • Disable Serial Port
  • Custom Image incl. driver and all code to have better startup times
  • Split process (analyze + export into 2 parts)
  • Add Remote Code Execution to run the script
  • Propagate error code up