Matching seems to fail when JToken evaluates types other than what's in the spec

[daghb]

Thanks for a great library, @bartschotten!

If I try to evaluate a provider message with a contract that has a matching rule on a field, and my test data evaluates into a type that is not supported by Pact v3 (like boolean or DateTime for example), matching always fail on type difference.

Given a consumer contract of

ComPact.Builders.V3.Pact.JsonContent.With(
  Some.Element.Named("MyField".WithTheExactValue(true.ToString())

matching rules always fail with

Property 'MyField' has a different type in the actual response. Expected value: True, actual value: True

probably because the JTokenType is evaluated into Boolean, but there is no boolean type in Pact v3. For JToken types that do not have their Pact equivalence, type matching should not be performed.

The same is true with DateTime types (JTokenType: Date).

Or have I misunderstood something somewhere?

[bartschotten]

Hi Dag,

JToken is just an implementation detail. What matters here is the JSON standard. JSON does support boolean values, so it should work if you use WithTheExactValue(true).

Date or DateTime is not supported by JSON though, so if you're looking for a specific value you'd have to do RegEx matching.

Maybe the ComPact DSL isn't strict enough here, because it allows any type as a parameter of WithTheExactValue(), while it should only accept types that have a JSON representation.

[bartschotten]

Fixed in 7f1b2dc

[daghb]

Sorry for reopening this, but earlier I tried and discarded the WithTheExactValue method, because it fails even on boolean values. Also, I tried out using LikeRegex of the same reason.

Regardless of what I try, the same evaluation errors occur:

    "tests": [
      {
        "testDescription": "a VehicleArchived event.",
        "success": "failed",
        "issues": [
          "Property 'MyBooleanField' has a different type in the actual response. Expected value: True, actual value: True",
          "Property 'MyDateField' has a different type in the actual response. Expected value: 10.02.2020 15:00:00, actual value: 23.04.2020 10:15:47",
        ]
      }
    ]

The output is the same for both WithTheExactValue. I even tried Regex on both the boolean and the DateTime field, also with the same output.

Pact:

          "$.MyBooleanField": {
            "matchers": [
              {
                "match": "regex",
                "regex": "^$|[tT]rue|[fF]alse"
              }
            ]
          },
          "$.MyDateField": {
            "matchers": [
              {
                "match": "regex",
                "regex": "(\d{2})\.(\d{2})\.(\d{4}) (\d{2}):(\d{2}):(\d{2})"
              }
            ]
          },

[bartschotten]

Maybe I'm misunderstanding the problem. What does the actual JSON message that you're verifying look like?

[daghb]

I made a gist with all the details for you - available here: https://gist.github.com/daghb/8c59b62fcc5e4cdc3471388c621a6e28

[bartschotten]

Ah I see. You are expecting { "MyBooleanField": "True" } but you're producing { "MyBooleanField": True }. The message you're getting is correct, the types don't match.

[daghb]

Ah, OK. So I should really serialize all fields produced that do not match any known Pact type?

[daghb]

Ah, thanks! This is so cool!

[bartschotten]

Yes, since Pact is language and implementation independent, everything must ultimately be reduced to just JSON types, so if it's not a boolean or a number it will have to be serialized as a string.

[sfiodorov]

@daghb, How did you solved it?
I have same issue with dates? Did you changed your cosnumer expectations to object or some serialization workaround at producer side?

[daghb]

@sfiodorov
Serialization at producer side. I don't know if it will work - I have not implemented this into production code yet, only played around a bit.

[sfiodorov]

@daghb Could you share the code, please?
Serialization on producer creates json string with "mydate":"2020-04-01T00:00:00"
But when string deserialized back to object it will be DateTime mydate, so it will fail on type compare.
We can not change the deserialization , since it is hidden.
So, its not so clear for me, how to manage it from produce side.
The only way i see it by changing the contract from:
"$.ToDateUtc": { "matchers": [ { "match": "regex", "regex": "(\\d+)(?:-(\\d+))?" } ] }

to:

$.ToDateUtc": {
            "matchers": [
              {
                "match": "type"
              }
            ]
          }

In such case i will lose the regex validation but this is ok becouse it will try to serialize the string to DateTime and if string is not valid then it will fail.

[bartschotten]

@sfiodorov I've created a new issue for this.