build-canaries/nevergreen

Improve configuration validation

GentlemanHal opened this issue · 1 comments

We currently have a json schema that is used to validate configuration. This performs simple checks on the structure, making sure certain keys exist for example. However it doesn't perform any semantic validation.

Some semantic validations we could perform:

  • Check trays key match the trayId in the object
  • Check projects keys match a known trayId
  • Check projects key match the trayId in the object
  • Check selected keys match a known trayId
  • Check selected array matches known projectId
  • Check backupRemoteLocations key match the internalId in the object (once #271 is done)

Note: if #301 is done before this improvement some of the keys about trays may have been renamed.

Turns out we can't always check projects and selected match known tray IDs as it's possible to partially import any subset of the state (this is on purpose so we don't have to write migrations for every change to state).

In theory we could check these things IF the trays key was also present, but that seems like overkill for now.