bullsignals/binance-downloader

Create a data validation from downloaded data

samuelhei opened this issue · 1 comments

The data downloaded from Binance will be used to test a trading strategy, it is important that:

  1. There is no duplicated data.
  2. There is no missing data.

@samuelhei with the new parallel implementation, duplicated data is already scrubbed before saving. This is in part because Binance sometimes has some missing slots (i.e., when they were down for a maintenance period), and so normal intervals didn't always grab all the data (interval starts didn't line up to the second post-maintenance). So I added some intentional overlap, and then scrubbed out dupes.

For the missing data, there will still be some even in the best case due to above-mentioned maintenance period. It would be possible to generate a quick report showing what intervals are not present in the data set if you think it'd be useful