Speed up checking for already downloaded files
Aariq opened this issue · 0 comments
Aariq commented
Currently get_prism_*()
functions check for already downloaded files one at a time. This might be faster if all the files were checked at once. It looks like prism_webservice()
acts on a single record and both retrieves the filename and checks if it is already downloaded. Maybe a solution would be to do all the HEAD requests at once to generate the filenames, then check which ones are missing, then download those one at a time.