Better support for creating new directories

[tv42]

I wanted to experiment with a separate build result directory, something that can be rm -rf'ed to restore a pristine source tree. Along the lines of https://github.com/apenwarr/redo#can-i-put-my-o-files-in-a-different-directory-from-my-c-files

This fails because redux tries to put the temp file in the directory before it exists.

[0 tv@brute ~]$ mkdir zz
[0 tv@brute ~]$ cd zz
[0 tv@brute ~/zz]$ echo echo foo >bar.do
[0 tv@brute ~/zz]$ redo-init
[0 tv@brute ~/zz]$ redo build/bar
Error: /home/tv/bin/redo: open /home/tv/zz/build/bar-redo-tmp-120491795: no such file or directory
[1 tv@brute ~/zz]$

[gyepisam]

This is an interesting problem.

The fundamental reason for the state of affairs is that if the tmp file were placed elsewhere
(probably /tmp), it could end up on a different device from the target dir and
redux could not guarantee that the output file is only seen in a complete state.

redux requires the directory to exist because it depends on the atomicity guarantees
of rename (2) to ensure that the output file is always seen in a complete state and is
never seen in an incomplete state. However, rename (2) cannot work across devices so
redux limits your choices.

But upon further thought, I think we can relax the requirement and still maintain
the guarantee using this pseudo python:

when creating temp file
    if target dir does not exist, use a temp dir and note the fact.

run .do script

if using temp dir
    if temp dir is on same device as target dir
       rename # avoids uneccesary file copy
    else
       copy temp file to target dir
       rename
else
    rename

[tv42]

Yeah, I was thinking the same, potential for cross-dev complicates. You could just always make temp file under .redo/tmp (more likely to be same device than $TMPDIR or /tmp), then if rename from there to final destination fails with EXDEV, copy it to a temp file in the target dir, and then rename that. The extra EXDEV step would in practice be rare.

[gyepisam]

This problem has now been corrected. redux, by default, creates a temp directory inside the .redo directory so do scripts can create target directories as necessary. I realized upon finishing this, that this also makes it possible to use fast(er) drives for creating tmp files, so there's now an environment variable REDO_TMP_DIR for the purpose. Also in the docs.