oPromessa/flickr-uploader

order of uploads

Closed this issue · 4 comments

coltongrainger commented

Describe the bug./What's your Question?

Any idea why this function call is loading photos in arbitrary order?

Expected behavior

This

[325][2019.09.29 00:01:46]:[1936       ][PRINT   ]:[uploadr] Successful file:[/home/colton/flickr/upload/20190929T0001_p001.jpg]
[325][2019.09.29 00:01:48]:[1936       ][PRINT   ]:[uploadr] Successful file:[/home/colton/flickr/upload/20190929T0001_p002.jpg]
[325][2019.09.29 00:01:48]:[1936       ][PRINT   ]:[uploadr] Successful file:[/home/colton/flickr/upload/20190929T0001_p003.jpg]
[325][2019.09.29 00:01:48]:[1936       ][PRINT   ]:[uploadr] Successful file:[/home/colton/flickr/upload/20190929T0001_p003.jpg]

Not this

[325][2019.09.29 00:01:46]:[1936       ][PRINT   ]:[uploadr] Successful file:[/home/colton/flickr/upload/20190929T0001_p001.jpg]
[325][2019.09.29 00:01:48]:[1936       ][PRINT   ]:[uploadr] Successful file:[/home/colton/flickr/upload/20190929T0001_p004.jpg]
[325][2019.09.29 00:01:49]:[1936       ][PRINT   ]:[uploadr] Successful file:[/home/colton/flickr/upload/20190929T0001_p003.jpg]
[325][2019.09.29 00:01:51]:[1936       ][PRINT   ]:[uploadr] Successful file:[/home/colton/flickr/upload/20190929T0001_p002.jpg]

To Reproduce

Agh! I wish I had time to go and get them. But I've commented out most of the functionality in lib.FlickrUploadr that it probably won't be any more specific than the comment below.

My Platform

Any details about your specific platform:
* What version of the flickr-uploader tool?

Lastest version on git.
* What Operating system (OS): Synology DSm, Linux, Windows, Mac, other?
Linux Debian Buster
* Use -m/--mask-sensitive option to mask sensitive data on log files
like your pics filenames and set/albums names.
-->

Additional context

Thanks for putting up with a poorly worded issue. It's more an opportunity for me to learn wtf why the order of images found and actually uploaded to flickr is not based on file name.

coltongrainger commented 
[325][2019.09.29 00:01:46]:[1936       ][PRINT   ]:[uploadr] Successful file:[/home/colton/flickr/upload/20190929T0001_p001.jpg]
[325][2019.09.29 00:01:48]:[1936       ][PRINT   ]:[uploadr] Successful file:[/home/colton/flickr/upload/20190929T0001_p004.jpg]
[325][2019.09.29 00:01:49]:[1936       ][PRINT   ]:[uploadr] Successful file:[/home/colton/flickr/upload/20190929T0001_p003.jpg]
[325][2019.09.29 00:01:51]:[1936       ][PRINT   ]:[uploadr] Successful file:[/home/colton/flickr/upload/20190929T0001_p002.jpg]
coltongrainger commented

1, 4, 3, 2

coltongrainger commented

I've tried to dissect the source code as much as possible. Maybe this is not the tool I needed. I really only want to push, and the only ordering I care about is based on filename (which is based on a timestamp from the scan of each image). I tried flickr-rsync, but am not sure how to rig the urllib to work with flickr's new oauth.

Thanks, sorry for the sparse request.

I just don't have time to keep fighting with it.

oPromessa commented

Hello @coltongrainger I'll take a look in more detail...

  1. Current behaviour
  • As far as I recall the uploading order comes from an os.walk file scan... which may be arbitrarly.
  • But hent they are sorted.
  • But once uploaded into Flickr, with the correct EXIF timestamps, I guess you can sort any way you like, or am I missing something?

  1. If you enable DEBUG (in uploadr file) and verbosity over 3 (argument -vvvv) the tool will show you at one point the list of files (and their order) that will be processed. for checking. Look for output Pretty Print Output for [files]-------

  2. On this line of code the list is actually being sorted.

  3. Please note that if you use the multiprocessing option to allow multiple uploads at the same time (and faster), the files to be uploaded are split into groups and loaded up in paralell.