can-i-deploy fails for provider in a transitive relation edge case
praveen-em opened this issue · 10 comments
Pre issue-raising checklist
I have already (please mark the applicable with an x):
- Upgraded to the latest Pact Broker OR
- Checked the CHANGELOG to see if the issue I am about to raise has been fixed
- Created an executable example that demonstrates the issue using either a:
- Dockerfile
- Git repository with a Travis or Appveyor (or similar) build
Software versions
pact-broker docker 2.95.0.0
Setup
Integration 1(consumer: "foo-consumer-1", provider: "bar-provider-1");
Integration 2(consumer: "foo-consumer-2", provider: "bar-provider-1");
Integration 3(consumer: "foo-consumer-2", provider: "bar-provider-2");
All three integrations verified;
foo-consumer-1 deployed in prod;
bar-provider-1 deployed in prod;
foo-consumer-2 is still in development (not deployed in prod);
Expected behaviour
can-i-deploy bar-provider-2 to prod returns "yes"
Actual behaviour
can-i-deploy bar-provider-2 to prod returns "no".
can-i-deploy bar-provider-2 version 1 to prod: no
---
deployable:
reason: There is no verified pact between the latest version of foo-consumer-1 with
tag prod (1) and version 1 of bar-provider-2
bar-provider-2 has relationship only with foo-consumer-2. It has nothing to do with foo-consumer-1. right?
Steps to reproduce
require "bundler/inline"
gemfile do
source "https://rubygems.org"
gem "faraday"
gem "faraday_middleware"
end
begin
$LOAD_PATH << "#{Dir.pwd}/lib"
require "pact_broker/test/http_test_data_builder"
base_url = ENV["PACT_BROKER_BASE_URL"] || "http://localhost:9292"
td = PactBroker::Test::HttpTestDataBuilder.new(base_url)
td.delete_integration(consumer: "Foo", provider: "Bar")
.delete_integration(consumer: "foo-consumer-1", provider: "bar-provider-1")
.delete_integration(consumer: "foo-consumer-2", provider: "bar-provider-1")
.delete_integration(consumer: "foo-consumer-2", provider: "bar-provider-2")
.publish_pact(consumer: "foo-consumer-1", consumer_version: "1", provider: "bar-provider-1", content_id: "111", branch: "feat/x")
.get_pacts_for_verification(
enable_pending: true,
provider_version_branch: "main",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ branch: "main" }]
)
.verify_pact(
index: 0,
provider_version_tag: "main",
provider_version: "1",
success: true
)
.deploy_to_prod(pacticipant: "bar-provider-1", version: "1")
.publish_pact(consumer: "foo-consumer-2", consumer_version: "2", provider: "bar-provider-1", content_id: "111", branch: "feat/y")
.get_pacts_for_verification(
enable_pending: true,
provider_version_branch: "main",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ branch: "main" }]
)
.verify_pact(
index: 0,
provider_version_tag: "main",
provider_version: "1",
success: true
)
.deploy_to_prod(pacticipant: "foo-consumer-1", version: "1")
.publish_pact(consumer: "foo-consumer-2", consumer_version: "1", provider: "bar-provider-2", content_id: "112", branch: "feat/z")
.get_pacts_for_verification(
enable_pending: true,
provider_version_branch: "main",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ branch: "main" }]
)
.verify_pact(
index: 0,
provider_version_tag: "main",
provider_version: "1",
success: true
)
.can_i_deploy(pacticipant: "bar-provider-2", version: "1", to: "prod")
rescue StandardError => e
puts "#{e.class} #{e.message}"
puts e.backtrace
exit 1
end
Relevant log files
You're right, that looks wrong. Thanks for the repo. I'll look into it.
Hi @praveen-em. Can you double check your script? When I run it, I get "yes" for can-i-deploy. See https://github.com/pact-foundation/pact_broker/runs/5595285401?check_suite_focus=true#step:3:1625
ah, sorry @bethesque , there was a problem with my script. I was trying different scenarios and ended up sending you the wrong script. The scenario I described wasn't quite right either. Here is the right script and the scenario.
Scenario
Integration 1(consumer: "foo-consumer-1", consumer_version: "1", provider: "bar-provider-1");
Integration 2(consumer: "foo-consumer-2", consumer_version: "1", provider: "bar-provider-1");
Integration 3(consumer: "foo-consumer-2", consumer_version: "2", provider: "bar-provider-2");
All three integrations verified;
foo-consumer-2(version 1) deployed in prod;
Expected Result
can-i-deploy bar-provider-1 to prod returns "yes"
Actual Result
can-i-deploy bar-provider-1 to prod returns "no"
can-i-deploy bar-provider-1 version 1 to prod: no
---
deployable:
reason: There is no verified pact between the latest version of foo-consumer-2 with
tag prod (1) and version 1 of bar-provider-1
The prod deployed version of foo-consumer-2 has been verified successfully against version 1 of bar-provider-1. But still it thinks it hasn't been verified? I could be missing something here.
=============================================================
Publishing pact for consumer foo-consumer-2 version 1 and provider bar-provider-1
=============================================================
Fetching pacts for verification for bar-provider-1
---
:providerVersionTags: []
:providerVersionBranch: main
:consumerVersionSelectors:
- :branch: main
:includePendingStatus: true
:includeWipPactsSince: '2020-01-01'
Pacts for verification (2):
---
url: http://pact-broker:9292/pacts/provider/bar-provider-1/consumer/foo-consumer-1/pact-version/a2456ade40d0e148e23fb3310ec56831fef6ce8e/metadata/dz10cnVl
wip: true
pending: true
why:
- The pact at http://pact-broker:9292/pacts/provider/bar-provider-1/consumer/foo-consumer-1/pact-version/a2456ade40d0e148e23fb3310ec56831fef6ce8e/metadata/d2lwPXRydWU
is being verified because it is a 'work in progress' pact (ie. it is the pact for
the latest version of foo-consumer-1 from branch 'feat/x' and is still in pending
state). Read more at https://docs.pact.io/go/wip
- This pact is in pending state for this version of bar-provider-1 because a successful
verification result for a version of bar-provider-1 from branch 'main' has not yet
been published. If this verification fails, it will not cause the overall build
to fail. Read more at https://docs.pact.io/go/pending
---
url: http://pact-broker:9292/pacts/provider/bar-provider-1/consumer/foo-consumer-2/pact-version/c5428f526532e46edcb066e8923aac2fb4b50cf5/metadata/dz10cnVl
wip: true
pending: true
why:
- The pact at http://pact-broker:9292/pacts/provider/bar-provider-1/consumer/foo-consumer-2/pact-version/c5428f526532e46edcb066e8923aac2fb4b50cf5/metadata/d2lwPXRydWU
is being verified because it is a 'work in progress' pact (ie. it is the pact for
the latest version of foo-consumer-2 from branch 'feat/y' and is still in pending
state). Read more at https://docs.pact.io/go/wip
- This pact is in pending state for this version of bar-provider-1 because a successful
verification result for a version of bar-provider-1 from branch 'main' has not yet
been published. If this verification fails, it will not cause the overall build
to fail. Read more at https://docs.pact.io/go/pending
=============================================================
Deploying foo-consumer-2 version 1 to prod
Creating tag 'prod' for foo-consumer-2 version 1
=============================================================
Steps to reproduce
require "bundler/inline"
gemfile do
source "https://rubygems.org"
gem "faraday"
gem "faraday_middleware"
end
begin
$LOAD_PATH << "#{Dir.pwd}/lib"
require "pact_broker/test/http_test_data_builder"
base_url = ENV["PACT_BROKER_BASE_URL"] || "http://localhost:9292"
td = PactBroker::Test::HttpTestDataBuilder.new(base_url)
td.delete_pacticipant("foo-consumer-1")
.delete_pacticipant("foo-consumer-2")
.delete_pacticipant("bar-provider-1")
.delete_pacticipant("bar-provider-2")
.publish_pact(consumer: "foo-consumer-1", consumer_version: "1", provider: "bar-provider-1", content_id: "111", branch: "feat/x")
.get_pacts_for_verification(
enable_pending: true,
provider_version_branch: "main",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ branch: "main" }]
)
.verify_pact(
index: 0,
provider_version_tag: "main",
provider_version: "1",
success: true
)
.publish_pact(consumer: "foo-consumer-2", consumer_version: "1", provider: "bar-provider-1", content_id: "112", branch: "feat/y")
.get_pacts_for_verification(
enable_pending: true,
provider_version_branch: "main",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ branch: "main" }]
)
.verify_pact(
index: 0,
provider_version_tag: "main",
provider_version: "1",
success: true
)
.publish_pact(consumer: "foo-consumer-2", consumer_version: "2", provider: "bar-provider-2", content_id: "113", branch: "feat/z")
.get_pacts_for_verification(
enable_pending: true,
provider_version_branch: "main",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ branch: "main" }]
)
.verify_pact(
index: 0,
provider_version_tag: "main",
provider_version: "1",
success: true
)
.deploy_to_prod(pacticipant: "foo-consumer-2", version: "1")
.can_i_deploy(pacticipant: "bar-provider-1", version: "1", to: "prod")
rescue StandardError => e
puts "#{e.class} #{e.message}"
puts e.backtrace
exit 1
end
I tried another scenario with .deploy_to_prod(pacticipant: "foo-consumer-2", version: "2") ; can-i-deploy's answer is again "No". It should be "yes' in this scenario as well?
Logs
I ran the script above, and this is correct:
can-i-deploy bar-provider-1 version 1 to prod: no
---
deployable:
reason: There is no verified pact between the latest version of foo-consumer-2 with
tag prod (1) and version 1 of bar-provider-1
success: 0
failed: 0
unknown: 1
verification_result_urls: []
As you can see, there is no verification between foo-consumer-2 version 1 and bar-provider-1.
This code is publishing a verification from bar-provider-2
.publish_pact(consumer: "foo-consumer-2", consumer_version: "2", provider: "bar-provider-2", content_id: "113", branch: "feat/z")
.get_pacts_for_verification(
enable_pending: true,
provider_version_branch: "main",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ branch: "main" }]
)
.verify_pact(
index: 0,
provider_version_tag: "main",
provider_version: "1",
success: true
)
I tried another scenario with .deploy_to_prod(pacticipant: "foo-consumer-2", version: "2") ; can-i-deploy's answer is again "No". It should be "yes' in this scenario as well?
foo-consumer-2 version 2 has no pacts, so that's a tricky question to know what it should do in that scenario. I'll give it some thought.
I think you're right - in the case that the consumer version in an environment has no pacts with the provider, then the provider should be allowed to deploy. I've added a failing test for that scenario, as you can see in the notification above.
With the latest Pact Broker release (2.95.1), using the script from above, but changing the "1" to a "2" in the deploy_to_prod:
=============================================================
Deploying foo-consumer-2 version 2 to prod
Creating tag 'prod' for foo-consumer-2 version 2
=============================================================
can-i-deploy bar-provider-1 version 1 to prod: yes
---
deployable: true
reason: There are no missing dependencies
success: 0
failed: 0
unknown: 0
verification_result_urls: []
=============================================================
Excellent! Thanks @bethesque for doing a new release so quickly. Much appreciated. The latest release works fine now in our internal setup where this problem originally came up. Going back to the other scenario - I was scratching my head as to why
.publish_pact(consumer: "foo-consumer-2", consumer_version: "1", provider: "bar-provider-1", content_id: "112", branch: "feat/y")
.get_pacts_for_verification(
enable_pending: true,
provider_version_branch: "main",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ branch: "main" }]
)
.verify_pact(
index: 0,
provider_version_tag: "main",
provider_version: "1",
success: true
)
wasn't publishing verify results for the pact between foo-consumer-2 version 1 and bar-provider-1. It looks like i need to add another .verify_pact block with index: 1 in the script. Is that correct?
.verify_pact(
index: 1,
provider_version_tag: "main",
provider_version: "1",
success: true
)
Specify the provider name explicitly, provider: "..." in the get_pacts and verify_pact methods. If not specified, it will use the name from the most recently published pact.
It was able to fetch both the pacts for the provider bar-provider-1 but not able to verify both - it was verifying just one. I have added provider: "bar-provider-1" as you suggested but didnt make any difference. I have also removed the second provider to keep things simple. Here is the script.
td = PactBroker::Test::HttpTestDataBuilder.new(base_url)
td.delete_pacticipant("foo-consumer-1")
.delete_pacticipant("foo-consumer-2")
.delete_pacticipant("bar-provider-1")
.delete_pacticipant("bar-provider-2")
.publish_pact(consumer: "foo-consumer-1", consumer_version: "1", provider: "bar-provider-1", content_id: "111", branch: "feat/x")
.get_pacts_for_verification(
enable_pending: true,
provider_version_branch: "main",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ branch: "main" }]
)
.verify_pact(
index: 0,
provider_version_tag: "main",
provider_version: "1",
success: true
)
.publish_pact(consumer: "foo-consumer-2", consumer_version: "1", provider: "bar-provider-1", content_id: "112", branch: "feat/y")
.get_pacts_for_verification(
enable_pending: true,
provider: "bar-provider-1",
provider_version_branch: "main",
include_wip_pacts_since: "2020-01-01",
consumer_version_selectors: [{ branch: "main" }]
)
.verify_pact(
index: 0,
provider: "bar-provider-1",
provider_version_tag: "main",
provider_version: "1",
success: true
)
.deploy_to_prod(pacticipant: "foo-consumer-2", version: "1")
.can_i_deploy(pacticipant: "bar-provider-1", version: "1", to: "prod")
the second pact is not being verified unless I change the index value.
This is just an observation while using the script. The original problem I raised in this issue is fixed in the latest release, so we can close this issue.