Can you put the example data somewhere for download, so one can run the example more easily and quickly?
cutebomb opened this issue · 16 comments
the data in the examples like
'data/ethusdt_20221003.npz',
'data/ethusdt_20221004.npz',
'data/ethusdt_20221005.npz',
'data/ethusdt_20221006.npz',
'data/ethusdt_20221007.npz'
Do you know a good platform for uploading large amounts of data for free? Initially, I uploaded the sample data on GitHub LFS, but it quickly reached the bandwidth limit.
kaggle datasets may be one option.
or other cloud filesystem like google file
i'll check it out. since i need to collect data newly, it will take time.
@nkaz001 can you please please upload this data
'data/ethusdt_20221003.npz',
'data/ethusdt_20221004.npz',
'data/ethusdt_20221005.npz',
'data/ethusdt_20221006.npz',
'data/ethusdt_20221007.npz' like maye be on kaggle
i'll set up a new data collection to avoid any licensing issues and provide the sample data publicly. But, it also costs me to run a server in the cloud, so please wait for a moment. or you can directly collect the data yourself using https://github.com/nkaz001/collect-binancefutures.
I began collecting the feed and may provide sample data which will be about for a week. Once the collection and preparation of the sample data are complete, I will attempt to upload it and update the demo accordingly.
Great, are you done?
Still collecting it. need to select pairs for upload. already now it's ~hundreds of gb. maybe I'll try to upload it next week?
wow
i couldn't create a public dataset.
Meanwhile, I've added a Rust version of the data collector for Binance Futures USDM. It's a quick way to gather the data and test it. https://github.com/nkaz001/collect-binancefutures/tree/master/rust
can you please please upload this data
'data/ethusdt_20221003.npz',
'data/ethusdt_20221004.npz',
'data/ethusdt_20221005.npz',
'data/ethusdt_20221006.npz',
'data/ethusdt_20221007.npz'
ue to the size of the data files, it is difficult to find a place to upload them. Instead, I have added an example showing the complete process of backtesting, including downloading the data from Tardis and converting it. This would be the quickest way unless you collect the data yourself using collect-data.
I'm impressed with the work you're doing @nkaz001 . I'd be happy to host some data for you if it would help the community. I guess I'll be able to find a few TB for it. Also, would possibly be able to thin it without impacting the backtests much. Please, feel free to reach out with the details if you're interested, on Telegram @kronael .
Good to hear that. Thanks for the suggestion. By the way, do you have Discord, or can you let me know your email? Or, please get in touch with me through Discord nkaz001 or email nkaz001@protonmail.com