Performance issues during `bundle exec kitchen destroy`

Question

Performance issues during `bundle exec kitchen destroy`

lmayorga1980 opened this issue 4 years ago · 10 comments

lmayorga1980 commented 4 years ago

Terraform Version: 0.14.4
Kitchen-Terraform: Latest
Ruby: 2.6.6

It takes a long time to destroy resources using AWS as the provider(Longer than 30 minutes).

Answer 1 · 2021-01-19T15:08:22.000Z

Hello again @lmayorga1980!

What you described looks like an issue with AWS or the AWS provider as Kitchen-Terraform just shells out to run terraform destroy during the destroy action. Have you been able to run the same Terraform configuration without Kitchen-Terraform and verified that it does not take a long time to destroy?

Answer 2 · 2021-01-19T15:14:29.000Z

I thought of that before but terraform apply & terraform destroy are very fast but its the moment when I run bundle exec kitchen create or bundle exec kitchen destroy that takes long periods of time.

Answer 3 · 2021-01-19T17:41:29.000Z

Is it the actual Terraform run that's taking a long time, like resources are "still destroying..." for 30+ minutes, or is it a delay in running the Terraform commands? Do you have a repository which reproduces the issue that you can share?

Answer 4 · 2021-01-19T17:50:53.000Z

I am not sure if this is related to ruby or not but ruby 2.7.2 fixed my problem ...

Answer 5 · 2021-01-19T18:12:18.000Z

I see the problem now.

Steps to Reproduce

bundle exec kitchen converge - Creates infrastructure but fails in lets say the provisioning powershell script
re-run bundle exec kitchen converge - This will destroy the failing resource and rebuild again.
.kitchen/windowsapp.yml - It keeps the old IP Address from #1.
Try to run verify and fails because the old IP does not exist. and takes a long time
bundle exec kitchen destroy experiments timeouts.

Fixed by manually editing the .kitchen/windowsapp.yml file and injecting the newer ip.

Answer 6 · 2021-01-20T15:50:37.000Z

Can you please clarify what IP address you're referring to in windowsapp.yml? Is it under the kitchen_terraform_outputs key?

Answer 7 · 2021-01-20T16:22:53.000Z

The windowsapp.yml contains the output results and is created under .kitchen/ directory.

Answer 8 · 2021-01-20T17:40:53.000Z

Sorry; what I meant was is this IP address a Terraform output? Those values should be updated each time kitchen converge is executed, so if that is not the case then this sounds like a bug in the output management.

Answer 9 · 2021-01-21T00:11:10.000Z

It seems that during the bundle kitchen destroy it was trying to connect to an old instance. When I cancel the long process with Ctrl+C then for some reason, several attempts to run bundle exec kitchen converge keep the old ip address.

Answer 10 · 2021-01-21T14:46:12.000Z

Are you able to isolate the point where the Terraform outputs are updated with a new IP address but windowsapp.yml still has the old IP address? I would like to understand how to reproduce this issue because the outputs within the Kitchen instance state file should always be updated with the latest Terraform outputs.