pact-foundation/pact_broker

Provider verification results not reflected in all consumers sharing same pact - deduplication side effect?

praveen-em opened this issue · 4 comments

praveen-em commented

Pre issue-raising checklist

I have already (please mark the applicable with an x):

  • Upgraded to the latest Pact Broker OR
  • Checked the CHANGELOG to see if the issue I am about to raise has been fixed
  • Created an executable example that demonstrates the issue using either a:
    • Dockerfile
    • Git repository with a Travis or Appveyor (or similar) build

Software versions

Docker image: pactfoundation/pact-broker:2.91.0.0

Setup:

  • Two consumers (consumer1 and consumer2) integrating with same provider.
  • Both consumers have exactly same pact.
  • Provider is configured to verify pacts from both of its consumers. Pending and wip pacts are enabled as well.
@ExtendWith(SpringExtension.class)
@Provider("${pact.provider}")
@PactBroker(url = "${pactbroker.url}",
        authentication = @PactBrokerAuth(username = "${pactbroker.auth.username}",password = "${pactbroker.auth.password}"),
        consumerVersionSelectors = {
                @VersionSelector(tag = "master"),
                @VersionSelector(tag = "dev"),
                @VersionSelector(tag = "test"),
                @VersionSelector(tag = "production")
        }
)

Expected behaviour

When the provider run their contract tests, it verifies the pact just once as both of its consumers share the same pact. This works fine - I guess pact broker does deduplication (which is actually quite useful) behind the scenes to make this happen. All good so far. However, when the verification results are published to the pact broker, I would expect the results to be reflected for both of its consumers in the broker.

Actual behaviour

Provider verification results are updated only for one of its consumers in the broker. If you look at the attached snapshot, the second row (consumer2) doesn't have any last verified entry but the first row says it was updated 1 hour ago and that is when the provider tests were run through it's main pipeline (not webhook triggered).

image

Steps to reproduce

  • Setup consumer1 that generates pactA between consumer1 and provider1.
  • Setup consumer2 that generates pactA between consumer2 and provider1. Yes, exactly same pact (pactA).
  • Setup provider1 to verify against all of its consumers.
  • Run consumer1 tests which generates/publish pactA to the pactbroker.
  • Run consumer2 tests which generates/publish pactA to the pactbroker.
  • Run provider1 tests which should publish verification results to the pactbroker.
  • you will see only one of the consumers(consumer 1 or 2) getting updated with the verification results. This is where the problem is. It should update both consumers?

Relevant log files

Due to strict corporate security policy, it is bit difficult to upload the logs here. It is possible but I need to navigate through some process which is very time consuming. Hopefully the steps to reproduce makes sense. if not, please let me know.

bethesque commented

The broker de-duplicates pacts within a consumer, but not across consumers. If you are only seeing one consumer being verified, there is some other bug or misconfiguration.

I tried to replicate your issue using a little client I've written that simulates a pact consumer and provider tests, but I can't see any issues from the Pact Broker side.

Here is the script to reproduce the issue:

https://github.com/pact-foundation/pact_broker/blob/issue/pact-broker-548/script/reproduce-issue.rb

The build output is here:

https://github.com/pact-foundation/pact_broker/runs/5548180005?check_suite_focus=true#step:3:1116

The script publishes 2 identical pacts for one consumer, then a third identical pact for another consumer.

The response in the 'pacts for verification' API from the Pact Broker shows two pacts are returned, one of which is de-duplicated for the first consumer, and the second which belongs to the other consumer.

 Pacts for verification (2):
 ---
 url: http://pact-broker:9292/pacts/provider/bar-provider/consumer/foo-consumer/pact-version/a2456ade40d0e148e23fb3310ec56831fef6ce8e/metadata/c1tdW2JdPWZlYXQlMkZ4JnNbXVtsXT10cnVlJnNbXVtjdl09MTAmc1tdW2JdPW1haW4mc1tdW2xdPXRydWUmc1tdW2N2XT05JnA9dHJ1ZQ
 wip:
 pending: true
 why:
 - |-
   The pact at http://pact-broker:9292/pacts/provider/bar-provider/consumer/foo-consumer/pact-version/a2456ade40d0e148e23fb3310ec56831fef6ce8e is being verified because the pact content belongs to the consumer versions matching the following criteria:
       * latest version from branch 'feat/x' (2)
       * latest version from branch 'main' (1)
 - This pact is in pending state for this version of bar-provider because a successful
   verification result for bar-provider has not yet been published. If this verification
   fails, it will not cause the overall build to fail. Read more at https://docs.pact.io/go/pending
 ---
 url: http://pact-broker:9292/pacts/provider/bar-provider/consumer/foo-consumer-2/pact-version/a2456ade40d0e148e23fb3310ec56831fef6ce8e/metadata/c1tdW2JdPW1haW4mc1tdW2xdPXRydWUmc1tdW2N2XT0xMSZwPXRydWU
 wip:
 pending: true
 why:
 - |-
   The pact at http://pact-broker:9292/pacts/provider/bar-provider/consumer/foo-consumer-2/pact-version/a2456ade40d0e148e23fb3310ec56831fef6ce8e is being verified because the pact content belongs to the consumer version matching the following criterion:
       * latest version from branch 'main' (1)
 - This pact is in pending state for this version of bar-provider because a successful
   verification result for bar-provider has not yet been published. If this verification
   fails, it will not cause the overall build to fail. Read more at https://docs.pact.io/go/pending
 
 =============================================================
bethesque commented

See if you can replicate the issue using these instructions https://github.com/pact-foundation/pact_broker/blob/issue/pact-broker-548/ISSUES.md

You'll need docker.

praveen-em commented

Hi @bethesque , I managed to recreate the problem by modifying your script slightly. I have set all pacts to be on feature branches and didn't specify those branches in consumer version selectors (they should be picked up automatically as wip is enabled?). Submitted the modified script/example in this pull request.

$ docker-compose -f docker-compose-issue-repro-with-pact-broker-docker-image.yml run repro-issue
Creating pact_broker_repro-issue_run ...
Creating pact_broker_repro-issue_run ... done
2022/03/15 17:43:40 Waiting for: http://pact-broker:9292
2022/03/15 17:43:40 Received 200 from http://pact-broker:9292
Deleting all data for the integration between Foo and Bar
404
{"error"=>"No consumer with name 'Foo' found"}

=============================================================

Deleting all data for the integration between foo-consumer and bar-provider

=============================================================

Deleting all data for the integration between foo-consumer-2 and bar-provider

=============================================================

Publishing pact for consumer foo-consumer version 1 and provider bar-provider

=============================================================

Publishing pact for consumer foo-consumer version 2 and provider bar-provider

=============================================================

Publishing pact for consumer foo-consumer-2 version 1 and provider bar-provider

=============================================================

Fetching pacts for verification for bar-provider
---
:providerVersionTags: []
:providerVersionBranch: main
:consumerVersionSelectors:
- :branch: main
:includePendingStatus: true
:includeWipPactsSince: '2020-01-01'

Pacts for verification (1):
---
url: http://pact-broker:9292/pacts/provider/bar-provider/consumer/foo-consumer-2/pact-version/a2456ade40d0e148e23fb3310ec56831fef6ce8e/metadata/dz10cnVl
wip: true
pending: true
why:
- The pact at http://pact-broker:9292/pacts/provider/bar-provider/consumer/foo-consumer-2/pact-version/a2456ade40d0e148e23fb3310ec56831fef6ce8e/metadata/d2lwPXRydWU
  is being verified because it is a 'work in progress' pact (ie. it is the pact for
  the latest versions of foo-consumer-2 from branches 'feat/x' and 'feat/y' (all have
  the same content) and is still in pending state). Read more at https://docs.pact.io/go/wip
- This pact is in pending state for this version of bar-provider because a successful
  verification result for a version of bar-provider from branch 'main' has not yet
  been published. If this verification fails, it will not cause the overall build
  to fail. Read more at https://docs.pact.io/go/pending

=============================================================

2022/03/15 17:43:43 Command finished successfully.
bethesque commented

Thanks, that repro made it super easy to identify the problem. I've released the fix in 2.95.0.

  • 👍1