Soft target download failed
Closed this issue · 10 comments
When downloading the official Google Cloud Disk soft label, it will fail when the download reaches a certain progress. Is there any other way to download it?
@VsionQing It seems this is a mechanism of protection from Google Drive and it may take up to 24 hours to be able to download the file again. I made a copy of it, can you try this link: https://drive.google.com/file/d/1LaWOlkTfCVGQvUR_0cA976epxAZ-XlIu/view?usp=sharing.
I also uploaded an fp16 version on OneDrive which is much smaller: https://gohkust-my.sharepoint.com/:u:/g/personal/zhiqiangshen_ust_hk/EfS0IqKwJc5FlSVLmfP09HQBHaFvjPVxhobxofzUXFGVNA?e=u1JkAZ
@VsionQing It seems this is a mechanism of protection from Google Drive and it may take up to 24 hours to be able to download the file again. I made a copy of it, can you try this link: https://drive.google.com/file/d/1LaWOlkTfCVGQvUR_0cA976epxAZ-XlIu/view?usp=sharing.
I also uploaded an fp16 version on OneDrive which is much smaller: https://gohkust-my.sharepoint.com/:u:/g/personal/zhiqiangshen_ust_hk/EfS0IqKwJc5FlSVLmfP09HQBHaFvjPVxhobxofzUXFGVNA?e=u1JkAZ
Thank you for your reply
The backup link has also failed to download. I hope to launch the domestic version as soon as possible
Hi @VsionQing, Tencent weiyun is uploading but is a little bit slow. Can you try this link of OneDrive: https://gohkust-my.sharepoint.com/:u:/g/personal/zhiqiangshen_ust_hk/EUeQbrwWcxpGrGwb63_7jpwBXkXlQKEqnZx2mPl-0zpsNw?e=6n722H.
I will provide weiyun's link here once it is ready.
Hi @VsionQing, Tencent weiyun is uploading but is a little bit slow. Can you try this link of OneDrive: https://gohkust-my.sharepoint.com/:u:/g/personal/zhiqiangshen_ust_hk/EUeQbrwWcxpGrGwb63_7jpwBXkXlQKEqnZx2mPl-0zpsNw?e=6n722H.
I will provide weiyun's link here once it is ready.
Hi @VsionQing, the link of Tencent weiyun is: https://share.weiyun.com/BdwXNxsh.
Thank you for your reply, the problem has been solved.
But，the following errors occurred during training,
PS E:\PythonFile\FKD-main> python train_FKD.py -a resnet50 --lr 0.1 --num_crops 4 -b 1024 --cos --softlabel_path "G:\OPEN\FKD_soft
_label_500_crops_marginal_smoothing_k_5.tar_7\FKD_soft_label_500_crops_marginal_smoothing_k_5" "G:\OPEN\ImageNet\data\ImageNet201
7\object_detection_from_video\ILSVRC2017_VID_new\ILSVRC\Data\VID"
=> creating model 'resnet50'
D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py:478: UserWarning: This DataLoader will create 24 worker p
rocesses in total. Our suggested max number of worker in current system is 16 (cpuset is not taken into account), which is small
er than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow
or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
warnings.warn(_create_warning_msg(
Traceback (most recent call last):
File "E:\PythonFile\FKD-main\train_FKD.py", line 528, in
main()
File "E:\PythonFile\FKD-main\train_FKD.py", line 138, in main
main_worker(args.gpu, ngpus_per_node, args)
File "E:\PythonFile\FKD-main\train_FKD.py", line 328, in main_worker
train(train_loader, model, criterion_sce, optimizer, epoch, args)
File "E:\PythonFile\FKD-main\train_FKD.py", line 363, in train
for i, (images, target, soft_label) in enumerate(train_loader):
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 521, in next
data = self._next_data()
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 1229, in _process_data
data.reraise()
File "D:\Anconda\envs\pytorch\lib\site-packages\torch_utils.py", line 434, in reraise
raise exception
FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data_utils\worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data_utils\fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data_utils\fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "E:\PythonFile\FKD-main\utils_FKD.py", line 98, in getitem
label = torch.load(label_path, map_location=torch.device('cpu'))
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\serialization.py", line 594, in load
with _open_file_like(f, 'rb') as opened_file:
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\serialization.py", line 211, in init
super(_open_file, self).init(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'G:\OPEN\ImageNet\data\ImageNet2017\object_detection_from_video\ILSV
RC2017_VID_new\ILSVRC\Data\VID\train\ILSVRC2017_VID_train_0000\ILSVRC2017_train_00054000\000188.tar'
My imagenet dataset is downloaded in third party resource sharing.Why do I look for compressed files in the dataset？
Hi @VsionQing, in our default code, the name of soft label files is aligned with the standard ImageNet-1K dataset. In your case, I think you can simply modify the code at https://github.com/szq0214/FKD/blob/main/utils_FKD.py#L95 to match your ImageNet path and soft label files.
Hi @VsionQing, in our default code, the name of soft label files is aligned with the standard ImageNet-1K dataset. In your case, I think you can simply modify the code at https://github.com/szq0214/FKD/blob/main/utils_FKD.py#L95 to match your ImageNet path and soft label files.
sorry, i can't understand this code.The code means that the corresponding hard label information is extracted from the dataset directory? If possible, can you provide the following standard data sets? I can't find the data set on the official website
Hi @VsionQing, in our default code, the name of soft label files is aligned with the standard ImageNet-1K dataset. In your case, I think you can simply modify the code at https://github.com/szq0214/FKD/blob/main/utils_FKD.py#L95 to match your ImageNet path and soft label files.
I hava downloaded another version in official website which structure meets code requirements.
Hi @VsionQing, in our default code, the name of soft label files is aligned with the standard ImageNet-1K dataset. In your case, I think you can simply modify the code at https://github.com/szq0214/FKD/blob/main/utils_FKD.py#L95 to match your ImageNet path and soft label files.
I hava downloaded another version in official website which structure meets code requirements.
Good, I will close this issue.
To those who have the same issue: this appears to be the Google Drive forbidden download error. It can be fixed by enabling Chrome's incognito mode or switching to another browser.