vultr/docker-machine-driver-vultr

[BUG] - Vultr instance never arrives at "ok" state in docker-machine

Static-Flow opened this issue · 7 comments

Static-Flow commented

Describe the bug
I am trying to use docker-machine to spin up a vultr instance but it never fully finishes. It gets stuck at the "(vultr) waiting for server state to become 'ok'..." stage. On the vultr panel I can see the instance did actually spin up fully but docker-machine doesn't seem to think so. I run into this issue even using the very basic example from the docs.

To Reproduce
Steps to reproduce the behavior:

  1. docker-machine.exe --debug create -d vultr --vultr-api-key= vultr
  2. Hangs

Expected behavior
For docker-machine to finish and the instance show up as a docker container

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information where applicable:

  • OS: Win10
  • Language Version: go version go1.14.3 windows/amd64
  • Docker-Machine Version: docker-machine.exe version 0.12.2, build 9371605

Additional context

Docker-machine output:

PS C:\Users\Tanner\go\src\github.com\vultr\docker-machine-driver-vultr> docker-machine.exe --debug create -d vultr --vultr-api-key=****** vultr
Docker Machine Version: 0.12.2, build 9371605
Found binary path at C:\Users\Tanner\go\bin\docker-machine-driver-vultr.exe
Launching plugin server for driver vultr
Plugin server listening at address 127.0.0.1:61773
() Calling .GetVersion
Using API Version 1
() Calling .SetConfigRaw
() Calling .GetMachineName
(flag-lookup) Calling .GetMachineName
(flag-lookup) Calling .DriverName
(flag-lookup) Calling .GetCreateFlags
Found binary path at C:\Users\Tanner\go\bin\docker-machine-driver-vultr.exe
Launching plugin server for driver vultr
Plugin server listening at address 127.0.0.1:61775
() Calling .GetVersion
Using API Version 1
() Calling .SetConfigRaw
() Calling .GetMachineName
(vultr) Calling .GetMachineName
(vultr) Calling .DriverName
(vultr) Calling .GetCreateFlags
(vultr) Calling .SetConfigFromFlags
Running pre-create checks...
(vultr) Calling .PreCreateCheck
(vultr) Calling .GetConfigRaw
Creating machine...
(vultr) Calling .Create
(vultr) DBG | Generating SSH key...
(vultr) waiting for ip address to become available...
(vultr) Created Vultr VPS with ID: 42476196, Public IP: 207.246.87.6
(vultr) waiting for server state to become 'ok'...

docker-machine ls output:

PS C:\Users\Tanner> docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
job - vultr Error Unknown Invalid server.

Static-Flow commented

I also updated docker-machine to 0.16.2 and I still get the same issue.

ddymko commented

@Static-Flow I'll look into this for you

  • ❤️1
Static-Flow commented

quick update: I tried it on OSX and a linux subsystem for windows both with fresh from source builds and they both still hung on "waiting for server state to become ok"

ddymko commented

@Static-Flow

This error is a bit of a false-positive.

When you initially deploy a instance it will get three states

  "42484495": {
        "SUBID": "42484495",
        "status": "active",
        "power_status": "running",
        "server_state": "installingbooting",

So while the instance is up - there still maybe install/boot processes running. That is why the server_state is in installingbooting

It will eventually change the server_state to ok however...this isn't all that great because it would require you to wait.

I'll look into changing these status checks to not wait until ok and run on active and running

Static-Flow commented

hmm alright, I might change my local copy to do that and see how my mileage is. I'm trying to run a highly distributed workload with a lot of instances so the lag time waiting for docker-machine to actually register them as "ok" will dampen runtime.

ddymko commented

I made a few changes and got it to work with a more reasonable time

➜  ~ time docker-machine create -d vultr --vultr-api-key= 123489
Running pre-create checks...
Creating machine...
(123489) waiting for ip address to become available...
(123489) Created Vultr VPS with ID: 42485369, Public IP: 45.77.201.223
(123489) waiting for server state to become 'ok'...
(123489) stopped
(123489) stopped
(123489) stopped
(123489) stopped
(123489) stopped
(123489) stopped
(123489) running
(123489) installing and booting processes on the server are complete, server is ready to use
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with ubuntu(systemd)...
Installing Docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env 123489
docker-machine create -d vultr  123489  0.34s user 0.15s system 0% cpu 2:52.98 total

➜  ~ docker-machine ls
NAME     ACTIVE   DRIVER   STATE     URL                         SWARM   DOCKER      ERRORS
123489   -        vultr    Running   tcp://45.77.201.223:2376            v19.03.14

I will submit a pr tomorrow that changes the status checks to check for running as it seems to be running fine without any issue.

Static-Flow commented

Awesome! Yea my monkey patch worked as well to get it to a quicker response time. Thanks for the attentiveness to the issue!

  • 👍1