Create a data validation from downloaded data
samuelhei opened this issue · 1 comments
The data downloaded from Binance will be used to test a trading strategy, it is important that:
- There is no duplicated data.
- There is no missing data.
@samuelhei with the new parallel implementation, duplicated data is already scrubbed before saving. This is in part because Binance sometimes has some missing slots (i.e., when they were down for a maintenance period), and so normal intervals didn't always grab all the data (interval starts didn't line up to the second post-maintenance). So I added some intentional overlap, and then scrubbed out dupes.
For the missing data, there will still be some even in the best case due to above-mentioned maintenance period. It would be possible to generate a quick report showing what intervals are not present in the data set if you think it'd be useful