jjjake/internetarchive

download --stdout option should also assume --no-directories

Opened this issue · 1 comments

hornc commented

This is a suggestion / enhancement.

I don't think there is a valid usecase for the --stdout download option sending the downloaded files to STDOUT while creating empty directories? That is the behavior I am observing.

I am trying to download a lot of text files from many items and concatenate them to a single output file:

 while read identifier; do ia download $identifier ${identifier}_suffix.txt --stdout  ; done >> combined.txt < list-of-identifiers.tsv

Which does what I want, but creates a lot of empty directories.

I get the behavior I want by adding --no-directories to the command, but I think it's always safe to assume no directories with --stdout.

If this is deemed a good suggestion, I'd be prepared to implement it. I haven't looked at the code in a while, but I'm assuming it's pretty straightforward to pass another option.

@hornc That sounds like a good suggestion to me. I think this is where we should make the change.

For example, this:

            if parent_dir != '' and return_responses is not True:
                os.makedirs(parent_dir, exist_ok=True)

Should be changed to something like:

            if parent_dir != '' and return_responses is not True and stdout is not True:
                os.makedirs(parent_dir, exist_ok=True)

Let me know if you have any questions, or if you'd like me to make the change. Thank!