Add file (size) validation when files already exist
Opened this issue · 0 comments
BSchilperoort commented
We could try to enhance the check for file existance by validating the file size. This way we can differentiate fully downloaded files from files where the download was interrupted.
import cdsapi
c = cdsapi.Client()
r = c.retrieve(...) # make a request and start the downloading process
r.content_length # bytes
>>> 65337064
r.content_length
will return the size of the file in bytes. This can be compared to Path(...).stat().st_size
.
This way we can skip existing files if they match the right content_length
, and otherwise overwrite them.
Originally posted by @BSchilperoort in #155 (comment)