Trouble with acquiring the dataset (DMC)
luccachiang opened this issue · 4 comments
Hello Ale, thanks for your awosome work and I am really into it. However, when I follow your README I encounter some problems in the section 'Downloading Data'. I tried to run your python command on my Linux server but found the following exceptions:
Processing dataset download for dmc
Saving dataset to:
/root/misc/viper_rl/viper_rl_data/datasets
dmc_dataset_aa: failed to retrieve item metadata - errors
mv: cannot stat 'dmc_dataset_aa/dmc.tar.partaa': No such file or directory
rmdir: failed to remove 'dmc_dataset_aa': No such file or directory
dmc_dataset_ab: failed to retrieve item metadata - errors
mv: cannot stat 'dmc_dataset_ab/dmc.tar.partab': No such file or directory
rmdir: failed to remove 'dmc_dataset_ab': No such file or directory
dmc_dataset_ac: failed to retrieve item metadata - errors
mv: cannot stat 'dmc_dataset_ac/dmc.tar.partac': No such file or directory
rmdir: failed to remove 'dmc_dataset_ac': No such file or directory
dmc_dataset_ad: failed to retrieve item metadata - errors
mv: cannot stat 'dmc_dataset_ad/dmc.tar.partad': No such file or directory
rmdir: failed to remove 'dmc_dataset_ad': No such file or directory
dmc_dataset_ae: failed to retrieve item metadata - errors
mv: cannot stat 'dmc_dataset_ae/dmc.tar.partae': No such file or directory
rmdir: failed to remove 'dmc_dataset_ae': No such file or directory
cat: 'dmc.tar.part*': No such file or directory
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
rm: cannot remove 'dmc.tar.part*': No such file or directory
I went through the code and the problem seems to lie in this line:
ia download dmc_dataset_$i dmc.tar.part$i
I guess the data is not correctly downloaded. Since I have no idea about what the dataset looks like or how the project is structured, I cannot figure out a way to download the dataset and upload to my server. I guess similar problems will show up if I continue to download Atari dataset as well as both the checkpoints.
Would you plz help me solve the problem or provide an alternative to get the data?
Interesting, what OS are you running this command on? it still work for me on linux. I recommend properly downloading internetarchive as outline here:
https://archive.org/developers/internetarchive/installation.html
Yes, I am running on Ununtu 20.04 on a remote server. The ia has been installed but somehow I cannot download the dataset. Maybe it is because that the server has blocked some websites, which I can do nothing about. Sorry if this question is a little bit stupid, are there any alternative ways to acquire the dataset?
how to acquire the RLbench datasets and checkpoint? and how to use train videogpt and policy in RLbench?
@luccachiang I dont know if this problem is still present or relevant for you but I only ran into this when i didnt execute one of the steps before. As long as you follow all the steps including the conda environment stuff you should be good so the question is: are you sure you applied all the steps before in the same terminal?