gbif/crawler

Add new FinishReason: NOT_VALIDATED

Opened this issue · 0 comments

This new FinishReason would signify that the DwC-A was downloaded, but it didn't pass validation.

Currently, FinishReason.ABORT is given, no matter if the DwC-A failed to be downloaded or if the DwC-A successfully downloaded but failed validation. It is extremely important to differentiate between these two cases, otherwise datasets that are online but don't validate will get mistaken for being offline.

As an example, refer to https://registry.gbif.org/web/#/dataset/1c714c57-da3e-457e-9752-edd1cd5e0463/crawl which is online, but the FinishReason is ABORT