pact-foundation/pact_broker

Can I deploy json output should return provider version in the matrix when verification did not happen yet

uittorio opened this issue · 8 comments

uittorio commented

Pre issue-raising checklist

I have already

  • Upgraded to the latest Pact Broker OR
  • Checked the CHANGELOG to see if the issue I am about to raise has been fixed
  • Created an executable example that demonstrates the issue using either a:
    • Dockerfile
    • Git repository with a Travis or Appveyor (or similar) build

Software versions

  • OS: e.g. Any
  • pact broker client details:** pact-ruby-standalone CLI v 1.91.0
    • pact gem 1.62.0
    • pact-mock_service gem 3.9.0
    • pact-support gem 1.18.1
    • pact-provider-verifier gem 1.36.1
    • pact_broker-client gem 1.65.0
    • pact-message gem 0.11.1

Expected behaviour

When checking if a Consumer with a specified version can be deployed in an environment for a specific Provider
And the pact is not verified yet
Then the JSON output includes the version of the Provider

Actual behaviour

When checking if a Consumer with a specified version can be deployed in an environment for a specific Provider
And the pact is not verified yet
Then the JSON output does not include the version of the Provider

Step to reproduce

  • Publish a pact that changes the content (a new pact)
  • Do not verify the pact
  • run the pact broker can-i-deploy command

I was expecting this command to return the version of the provider in prod but instead it returns null!

pact-broker can-i-deploy --pacticipant {Consumer} --version {consumerVersion}  --pacticipant {Provider} -l --in_environment {environment} --broker-base-url {broker.base.url} -k {token} -o json

Actual Response.

I've only included the relevant part in the json output example!

{
  "summary": {
    "deployable": null,
    "reason": "There is no verified pact between version s712-consumerVersion-g1723 of AConsumer and the latest version of AProvider (b123-providerVersion-901)",
    "success": 0,
    "failed": 0,
    "unknown": 1
  },
  "notices": [
    {
      "type": "error",
      "text": "There is no verified pact between version s712-consumerVersion-g1723 of AConsumer and the latest version of AProvider (b123-providerVersion-901)"
    }
  ],
  "matrix": [
    {
      "consumer": {
        "name": "AConsumer",
        "version": {
          "number": "s712-consumerVersion-g1723"
        }
      },
      "provider": {
        "name": "AProvider",
        "version": null
      },
      "verificationResult": null
    }
  ]
}

Expected Response.
I was hoping to receive a json that contains the provider version. Example

{
  "summary": {
    "deployable": null,
    "reason": "There is no verified pact between version s712-consumerVersion-g1723 of AConsumer and the latest version of AProvider (b123-providerVersion-901)",
    "success": 0,
    "failed": 0,
    "unknown": 1
  },
  "notices": [
    {
      "type": "error",
      "text": "There is no verified pact between version s712-consumerVersion-g1723 of AConsumer and the latest version of AProvider (b123-providerVersion-901)"
    }
  ],
  "matrix": [
    {
      "consumer": {
        "name": "AConsumer",
        "version": {
          "number": "s712-consumerVersion-g1723"
        }
      },
      "provider": {
        "name": "AProvider",
        "version": "b123-providerVersion-901"
      },
      "verificationResult": null
    }
  ]
}

Why this would be useful

We would like to use the ouput of this command to determine the version of the Provider in a specific environment. It's probably not the best place to look at but it feels correct to return the version of the provider deployed in that environment considering that is part of the error message.

Let me know if you need anything else, I am happy to open a Pull request if this makes sense!

tplass-ias commented

I am in the same situation trying to set up a webhook-less flow for cross-verification of pacts. Similarly, the matrix endpoint does not return the environment when a provider verification doesn't exist. E.g. in the example above, the actual response is like:

{
  "environments":[]
}

but I expect full environment details like:

{
  "environments":[
    {
      "uuid":"uuid",
      "name":"prod",
      "displayName":"Prod",
      "production":true,
      "createdAt":"timestamp",
      "_links":{
        "self":{
          "title":"Environment",
          "name":"prod",
          "href":"https://pact-broker/environments/uuid"
        }
      }
    }
  ]
}

For now, I am working on a workaround to use list-environments from the pact-broker cli and hitting the /environments/#{environment.uuid}/deployed-versions/currently-deployed endpoint to get my list of versions to verify.

bethesque commented

Rather than describing a change to the existing tool, can you describe what you're trying to do, and what information you need @uittorio . We may be able to build a better tool specifically for your usecase.

anto-ac commented

@bethesque We want to get the Prod version of a specific provider in order to be able to check out that version and run the provider tests in a webhook-less flow.

We're aware of the endpoint that you can call to get a list of application versions within an environment, which you pointed out to me (thanks!), and we've implemented some logic using that (as @tplass-ias is doing). However it would be useful to extend describe-version or have something similar.

What @uittorio described above comes off the back of a conversation and he and I have had. We noticed that the provider version is null when can-i-deploy returns no, and we're not sure if it's intended or not. After all, it might make sense to have the version in there when asking for --latest --in_environment

bethesque commented

Thanks Anto. So, it sounds like the "what version(s) is in this environment" command is actually a better fit for your problem. I think I created a card for it the other day. I'll double check it tomorrow and make sure it's on the pactflow roadmap. It's not a big piece of work.

bethesque commented

For background, the reason the provider version is null is because left outer join. The query starts with the consumer version, and tries to join it to an appropriate provider version via the verifications table. If there is no matching provider version, the consumer version is still returned, but the provider version columns are empty.

I can see why adding it would be potentially useful, but if I'm understanding things correctly, the other command would be more useful for you.

  • 👍2
anto-ac commented

For background, the reason the provider version is null is because left outer join. The query starts with the consumer version, and tries to join it to an appropriate provider version via the verifications table. If there is no matching provider version, the consumer version is still returned, but the provider version columns are empty.

Interesting! Where is the provider version coming from in the the summary.reason and in notices[].text then?

  • 👍1
uittorio commented

Thank you Anto for claryfing! Thank you Beth for all the details

I agree, the other command would be more useful and appropriate to the use case!

Is there a chance that I/we could help with that piece of work by opening a PR or it needs to go through the internal team?

tplass-ias commented

For us it is not just "what version(s) are in this environment" but "what provider version(s) have unverified pacts in this env". I am using results from can-i-deploy to filter the results of /environments/#{environment.uuid}/deployed-versions/currently-deployed for this for now.