equinix/terraform-provider-metal

Determine if TF acceptance tests are racing and sweeping each other

Closed this issue · 2 comments

More interesting errors that seem unrelated. I wonder if the two versions of TF acceptance tests are running side-by-side and the sweeper from one test is knocking out the test resources of the other test. Most of these errors could be explained by a resource being deleted by an external actor.

If sweeping is the problem:

  • and tests across PRs are racing, can we prevent that?
  • and tests in the same PR are racing, can we prevent that?

Tests failures that may be evidence of sweeping:

--- FAIL: TestAccMetalDataSourceProject_Basic (2.42s)
    testing.go:569: Step 0 error: errors during apply:
        
        Error: Not found
        
          on /tmp/tf-test475229150/main.tf line 6:
          (source code not available)
--- FAIL: TestAccMetalProject_BGPBasic (0.77s)
    testing.go:569: Step 0 error: errors during apply:
        
        Error: Not found
        
          on /tmp/tf-test218057823/main.tf line 2:
          (source code not available)
--- FAIL: TestAccMetalProject_BGPUpdate (1.39s)
    testing.go:569: Step 1 error: errors during apply:
        
        Error: Not found
        
          on /tmp/tf-test871845553/main.tf line 2:
          (source code not available)
--- FAIL: TestAccMetalPortVlanAttachment_L2Individual (1017.40s)
    testing.go:569: Step 1 error: errors during apply:
        
        Error: POST https://api.equinix.com/metal/v1/ports/4779f6fa-e488-47fb-9ca1-b5c683224e1b/convert/layer-2: 403 You are not authorized to view this resource 
        
          on /tmp/tf-test315818338/main.tf line 29:
          (source code not available)

Originally posted by @displague in #21 (comment)

I believe the reason for this test is one of the following:

  • The testing account is not privileged to create BGP configs on projects
  • The project is not ready when the POST /project/{id}/bgp-config request is sent

I tried to insert a 4-second delay between creating the project and posting to the bgp-config but that did not resolve the issue.

https://github.com/equinix/terraform-provider-metal/blob/e399bbb/metal/resource_metal_project.go#L123-L133

2021/03/16 23:48:37 [DEBUG] POST https://api.equinix.com/metal/v1/projects
2021/03/16 23:48:43 [DEBUG] POST https://api.equinix.com/metal/v1/projects/5ffb94c3-4933-448a-9b7b-0dd5b4b5af3e/bgp-config
2021/03/16 23:48:43 [ERROR] <root>: eval: *terraform.EvalApplyPost, err: Not found
2021/03/16 23:48:43 [ERROR] <root>: eval: *terraform.EvalSequence, err: Not found
    testing.go:569: Step 0 error: errors during apply:
        
        Error: Not found
        
          on /var/folders/rg/np923dpx40s17lqfpm27v3980000gn/T/tf-test710565247/main.tf line 6:
          (source code not available)  

The 404s here will be resolved by $45.
The 403 will be resolved by #44 (although it is not clear to me at a glance why we attempted to convert a resource to layer-2 and then received a 403, perhaps because of sweeper racing).