buildkite-plugins/docker-buildkite-plugin

Cancelling step does not does not `docker stop` gracefully

evandam opened this issue · 1 comments

In particular, I'm looking to have a pipeline that runs Packer builds using the hashicorp/packer image.

I noticed after cancelling a step running Packer, it doesn't give Packer a chance to clean up and exit gracefully like normally hitting ctrl + C in Packer and seeing logs like `Cancelling build after receiving interrupt...Cleanly cancelled builds after being interrupted.

In my opinion, cancelling a step in Buildkite should run docker stop or similarly pass a SIGTERM to the container, but it doesn't look like that's happening.

PS: Sorry if this is an issue outside of the Docker plugin. I can reopen wherever it makes sense 😇

toote commented

@evandam Hi and sorry for the delay in getting back to you. I've been thinking and trying to understand exactly your scenario; but as far as I understand the plugin's behaviour is quite straight-forward: lots of preparation and finally docker run ... in a single bash script. When a job is cancelled, the agent send a SIGTERM to the bash script, which should be sending it through to the subprocesses (docker run in this case... which will send it to packer in turn). So things should work... but you are saying it is not happening that way.

I can only think of two explanations:
1- (most likely) the job is taking more than the agent's cancel grace period, so it eventually gets killed
2- there is another - more serious - issue that we will need to further debug. For that we will need you to provide more information (like public execution when this happened or the minimal information possible and some screenshots/logs to see what you are seeing)