Use of create_selector

[raihle]

I'm trying to use create_selector in restful_resource to work with an awkward API. Most of the resources work "normally", but this specific one requires create via PUT (rather than POST) with a "data" property containing an array containing the object I want to create, and creation returns an array with the object created. The API also embeds the "scopeConfig" in its response (identified by the "scopeConfigId" in the POST data).

An example of using the API via curl

DEVLAKE="***"
AUTH="***"

echo "CREATE"
curl -X PUT "${DEVLAKE}/api/plugins/github/connections/3/scopes" \
  -H "Authorization: Bearer ${AUTH}" \
  -d '{
    "data": [{
      "githubId": 123456789,
      "scopeConfigId": 22,
      "name": "telia-company/fft-devlake-projects",
      "fullName": "telia-company/fft-devlake-projects"
    }]
  }'

echo "GET"
curl -X GET "${DEVLAKE}/api/plugins/github/connections/3/scopes/123456789" \
  -H "Authorization: Bearer ${AUTH}" \

echo "DELETE"
curl -X DELETE "${DEVLAKE}/api/plugins/github/connections/3/scopes/123456789" \
  -H "Authorization: Bearer ${AUTH}" \

And the output (formatted for readability)

CREATE

[
  {
    "connectionId": 3,
    "fullName": "xxx/yyy",
    "githubId": 123456789,
    "name": "xxx/yyy",
    "scopeConfig": {
      // Omitted for readability
    },
    "scopeConfigId": 22
  }
]

GET

{
  "connectionId": 3,
  "fullName": "xxx/yyy",
  "githubId": 123456789,
  "name": "xxx/yyy",
  "scopeConfig": {
    // Omitted for readability (same as above)
  },
  "scopeConfigId": 22
}

DELETE

{
  "success": true,
  "message": "success",
  "causes": null,
  "data": null
}

I have tried to accommodate this by using create_selector (and various shots in the dark) as follow:

resource "restful_resource" "this" {
  path      = "/api/plugins/github/connections/3/scopes"
  read_path = "$(path)/123456789"

  create_method   = "PUT"
  create_selector = "0"

  body = jsonencode({
    data = [{
      githubId      = 123456789
      scopeConfigId = 22
      name          = "xxx/yyy"
      fullName      = "xxx/yyy"
    }]
  })
}

When I run terraform apply, the module consistently fails (although all resources were successfully created in the system):

restful_resource.this: Creating...
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to restful_resource.this, provider "provider[\"registry.terraform.io/magodo/restful\"]" produced an unexpected new value:
│ .body: was
│ cty.StringVal("{\"data\":[{\"fullName\":\"xxx/yyy\",\"githubId\":123456789,\"name\":\"xxx/yyy\",\"scopeConfigId\":22}]}"),
│ but now cty.StringVal("{}").
│ 
│ This is a bug in the provider, which should be reported in
the provider's own issue tracker.

It seems to me that the response body was not successfully extracted. It's quite possible that I'm just misusing gjson query syntax - I have never used it before, and every example seems to assume that the "root" data is an object, not an array.

[magodo]

@raihle I believe create_selector = "0" is the right usage for your case. Whilst I'm not sure why it failed with an inconsistent result. Would you mind sniffer the api sequences to see what's actually happening (e.g. via mitmproxy)?

[raihle]

These are the relevant flows:

• GET the object ("scope") while planning
• DELETE it because Terraform considers it tainted (due to the error, I guess)
• PUT (create) it
• GET it again (not sure why)

If I terraform destroy before apply, the initial GET and DELETE are omitted, but the PUT and final GET look the same.

The bodies look like I expect:

Initial GET

{
  "connectionId": 3,
  "fullName": "xxx/yyy",
  "githubId": 123456789,
  "name": "xxx/yyy",
  "scopeConfig": {
    // Omitted for readability
  },
  "scopeConfigId": 23
}

DELETE

{
  "success": true,
  "message": "success",
  "causes": null,
  "data": null
}

PUT request

{
  "data": [
    {
      "fullName": "xxx/yyy",
      "githubId": 123456789,
      "name": "xxx/yyy",
      "scopeConfigId": 23
    }
  ]
}

PUT response

[
  {
    "connectionId": 3,
    "fullName": "xxx/yyy",
    "githubId": 123456789,
    "name": "xxx/yyy",
    "scopeConfig": {
      // Omitted for readability
    },
    "scopeConfigId": 23
  }
]

Final GET

{
  "connectionId": 3,
  "fullName": "xxx/yyy",
  "githubId": 123456789,
  "name": "xxx/yyy",
  "scopeConfig": {
    // Omitted for readability
  },
  "scopeConfigId": 23
}

(123456789 in my examples is 659690394 in my actual config - that's just too much work to fix in a screenshot...)

[magodo]

The create_selector works as expected. The issue resides in the mismatch between the PUT request body and GET response body.

Using create_selector doesn't violate the expectation that the body match both the request/response body.

[raihle]

Is there a way to bypass that expectation/check? Every other API resource seems to work "normally" so this provider has been a big help so far.

[magodo]

You can try write_only_attrs, but in that case, terraform has no way to keep track of this resource at all..

[raihle]

Oh, yeah, if I say all of data is write-only...
This particular resource is effectively immutable so that should be good enough, thanks!