Add check after every Dump.ps1 step to review .csv structure
Fist0urs opened this issue · 1 comments
Fist0urs commented
Sometimes .csv produced by Dump.ps1 can be corrupted (Internet connection disabled, user forced to stop process, etc.). This corrupted files are only noticed when trying to import results in neo4j database as it raises an error.
Adding a check (or an extra option to Dump.ps1 to be able to verify if target .csv files are well-formed) would prevent user from restarting dump from the beginning (which can be very long).
gdedrouas commented
This is undesirable, as we can easily end with a valid csv but an incomplete set of objects, leading to false results.
Users have to take errors into account and will be redirected to a future checkpoint feature in the dump script, to restart previously failed steps.