409 conflict when creating resources is delayed

[cliffchapmanrbx]

Describe the bug

• Artifactory Enterprise Plus 7.68.7
• Terraform 1.6.2
• Provider 9.8.0

Our Artifactory server is under fairly significant load these days, and while we work on mitigation that we get to encounter fun issues with delayed requests.

Our permission model is heavily autogenerated through terraform, which causes a large number of artifactory_permission_target objects to be managed by Terraform. When we create a new service account it's not unusually to have 5+ of these generated and created immediately after the user.

Terraform has started behaving inconsistently when creating these objects, taking minutes+ to actually complete the creation operation. Often these fail with a 409 CONFLICT HTTP error code:

Error: 
409 POST https://artifactory.example.com/artifactory/api/v2/security/permissions/users-service-teamname-publish-a3ec3672a
{
  "errors" : [ {
    "status" : 409,
    "message" : "Can't create permission target 'users-service-teamname-publish-a3ec3672a' for type repo. It already exists."
  } ]
}

  with module.accounts["./../common-primary-cluster/users.json"].artifactory_permission_target.user_repos["service-teamname-publish-a3ec3672a"],
  on ../../modules/users/main.tf line 18, in resource "artifactory_permission_target" "user_repos":
  18: resource "artifactory_permission_target" "user_repos" {

And sure enough, if I go look at Artifactory's UI I can see the permission target was created successfully. It seems like Terraform ends up retrying the request somewhere along the lines, gets the 409 conflict and errors out.

This is particularly frustrating because I can't just re-run another plan and apply. The 409 error gets logged as "failure to create", the actually created item doesn't get into the statefile, and I'm doomed to get more 409s until I write a manual import block to pull the resource into the statefile. This is, of course, frustrating :)

We're considering looking into a way to autogenerate an import statement for these situations as a workaround.

Requirements for and issue

• A description of the bug
• A fully functioning terraform snippet that can be copy&pasted (no outside files or ENV vars unless that's part of the issue). If this is not supplied, this issue will likely be closed without any effort expended.
• Your version of artifactory (you can curl it at $host/artifactory/api/system/version
• Your version of terraform
• Your version of terraform provider

Expected behavior

When terraform encounters network troubles creating a new resource and retries an API call, it should gracefully handle a 409 error response as "created".

Even better would be some form of check on whatever is conflicting to determine if it was created as expected (and thus successful) or if it differs from the expected config (and generates an error).

[alexhung]

@cliffchapmanrbx Thanks for the report. I've added this issue to our plan.

[NiklasRosenstein]

We're in the exact same boat. It happens quite frequently and means much of the resources we Terraform manage in conjunction with the permission targets are only half-initialized and mostly unusable until we've manually remediated by either importing or manually deleting the permission target and rerun (usually importing is more work).