Feature - Snapshot JSON schema assertion

[thim81]

Hi Freek,

I'm fan of the idea of snapshot testing where the result of a test is used as an assertion the next time the test runs.

The package currently matches on the following options:

• assertMatchesSnapshot($actual)
• assertMatchesJsonSnapshot($actual)
• assertMatchesXmlSnapshot($actual)
• assertMatchesFileSnapshot($filePath)
• assertMatchesFileHashSnapshot($filePath)

We use a lot of JSON based API's, so the snapshot assertMatchesJsonSnapshot is suitable for testing the results of the JSON API.

But it requires the JSON response to match every time, which is great for unit tests but less for "integration "test.

I had the idea to use the same manner of taking snapshots for "integration" tests, where the data in the JSON response will change depending on the input you give to the API but the structure / schema of the JSON should be the same.

Example: JSON response

{
  "result": {
    "code": 200,
    "message": "success"
  },
  "parcel": {
    "id": "57d6b6ad88cdf78b461777",
    "alias": "null",
    "name": "Laptop"
  }
}

The name and ID in the response will be different every time we create a parcel.

But the JSON schema should be the same otherwise my test should fail.

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "result": {
      "type": "object",
      "properties": {
        "code": {
          "type": "number",
          "description": "The result code"
        },
        "message": {
          "type": "string",
          "description": "The result message"
        }
      }
    },
    "parcel": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string",
          "description": "The unique ID of the Parcel."
        },
        "alias": {
          "type": "string",
          "description": "not used currently"
        },
        "name": {
          "type": "string",
          "description": "The name/description of the parcel,"
        }
      }
    }
  }
}

The test would fail if for example "name" would be an integer

{
  "result": {
    "code": 200,
    "message": "success"
  },
  "parcel": {
    "id": "57d6b6ad88cdf78b461777",
    "alias": "null",
    "name": 1234567890
  }
}

The test would fail if for example the JSON response would not match the snapshot schema, when a new property is added or an expected one is missing

Missing property example

{
  "result": {
    "code": 200,
    "message": "success"
  },
  "parcel": {
    "id": "57d6b6ad88cdf78b461777",
    "alias": "null",
  }
}

New unexpected property example

{
  "result": {
    "code": 200,
    "message": "success"
  },
  "parcel": {
    "id": "57d6b6ad88cdf78b461777",
    "alias": "null",
    "name": "Laptop",
    "code": "4567890KHJL"
  }
}

How it could work

1. Do API Call
2. Convert JSON response to JSON schema
3. Snapshot JSON schema result
4. Match JSON response with the snapshot JSON schema

Step 1, 3 is something that the current package already supports, more or less
Step 2, 4 would be the new part.

For Step 4: I have found this: https://github.com/estahn/phpunit-json-assertions
Which could be used to validate if the JSON response matches the schema.

For step 2:
This is the big unknow, how would you convert a JSON response and into a JSON schema.

I have found this:
https://github.com/solvire/php-json-schema-generator

Questions

I want to develop this feature to extend the package, but I'm looking for your advice to do it properly, so I'm looking for your advice / guidance.

• Would this be a valuable add-on for this package?
• Would the steps be the correct way to implement it?
• I have looked for something like this not found anything suitable. Perhaps you ran into something that does phpunit JSON schema assertions?
• Would the available classes by up-to-standard to use?

Looking forward for your input.

[thim81]

Some example reference build in Javascript:
https://github.com/bahmutov/schema-shot

Which seems to do exactly what I want to achieve, except it is in Javascript which would be the strange duck in the PHPunit test pond.

[sebastiandedeyne]

Hi @thim81,

First of all, thanks for the detailed proposal! I wouldn't mind having this included in the package if we're able to flesh out the details, but I don't want to maintain too much code for it, since it's not part of the core functionality here.

I'm not sure if we'll be able to do a good conversion though... For example, we won't be able to automatically determine nullable types.

Regarding in implementing something like this, check out custom drivers.

https://github.com/spatie/phpunit-snapshot-assertions#writing-custom-drivers

Lastly, if you find a solution but we don't feel good supporting it here, we'll gladly link to your package so no work is lost :)

[spatie-bot]

Dear contributor,

because this issue seems to be inactive for quite some time now, I've automatically closed it. If you feel this issue deserves some attention from my human colleagues feel free to reopen it.