Parallel doesn't play well with filenames containing a quote
WyohKnott opened this issue · 7 comments
Hello, it's me again with my filenames problems
I'm gonna take an example to explain the situation.
Say we have two folders with files to move.
mkdir folder1 folder2
touch "folder1/Let'sgo.txt" "folder1/Let'sgo-q1.txt" "folder1/Let'sgo-q2.txt" "folder1/Let'sgo-q3.txt"
export file="Let'sgo"
I can move a file containing a quote with mv without any problem:
mv "folder1/${file}.txt" "folder2/."
However if I try to use parallel to do the same thing, I get an error:
seq 1 3 | parallel mv "folder1/${file}-q{}.txt" "folder2/."
parallel: reading inputs from standard input
mv: missing destination file operand after 'folder1/Letsgo-q1.txt folder2/.'
Try 'mv --help' for more information.
mv: missing destination file operand after 'folder1/Letsgo-q2.txt folder2/.'
Try 'mv --help' for more information.
mv: missing destination file operand after 'folder1/Letsgo-q3.txt folder2/.'
Try 'mv --help' for more information.
It seems parallel remove the quote from my filename or misinterpret it, despite it being inside an enclosed variable.
If I escape the quote, it works:
seq 1 3 | parallel mv "folder1/Let\'sgo-q{}.txt" "folder2/."
but it's not really a workable solution because I feed filenames as a variable inside a bigger script. I think there shouldn't be any difference of behaviour between a non-parallelized function and a parallelized one.
Basically, your shell expands the variable before it passes it to the program, so there's no way to tell what is or isn't inside a variable. Although this is just a general parsing problem that should be simple to fix.
I'll have the fix released sometime tomorrow. Basically, I'll be escaping input parameters by default, and implementing the -q
/ --quote
parameter which will escape the command argument. With the input you're trying, this will generate a parsing error without --quote
due to having a non-terminated quote, but will work as you want with --quote
enabled.
This commit should resolve the issue. If the problems are fixed, you can close this issue and I'll make a new release.
Try it out like so:
seq 1 3 | parallel -q mv -v "folder1/${file}-q{}.txt" "folder2/."
There is also a custom error message if you don't use quoting that will tip you to use the --quote parameter to escape special characters.
It doesn't seem to work, the {} is not replaced with the input anymore when using -q.
seq 1 3 | parallel -q mv "folder1/${file}-q{}.txt" "folder2/."
parallel: reading inputs from standard input
mv: target '1' is not a directory
mv: target '2' is not a directory
mv: target '3' is not a directory
If I use echo to see what happens, parallel thinks there is no {} in my command so it appends the variables at the end:
seq 1 3 | parallel -q echo "folder1/${file}-q{}.txt" "folder2/."
parallel: reading inputs from standard input
folder1/Let'sgo-q{}.txt folder2/. 1
folder1/Let'sgo-q{}.txt folder2/. 2
folder1/Let'sgo-q{}.txt folder2/. 3
It should be fixed now. The braces are no longer escaped.
It is working fine now. Thank you.