git-for-windows/git

Custom Commands: single quoted arguments not passed unchanged (especially \\)

episource opened this issue · 6 comments

episource commented

Git for windows strips some double backslashes from the arguments passed to a custom git command. Whether this happens, depends on the surrounding string. I was able to reproduce this with arguments containing any of []{}?~". Characters ()+-%:=$(backtick) seem not to be troublesome. I'am always using single quotes (see 'details'):

  • (not as expected) '[\\' is converted to [\
  • (not as expected) '\\]' is converted to \]
  • (not as expected) '[a\\b' is converted to '[a\b'
  • (as expected) '\\' is passed unchanged
  • (as expected) '\\a' is passed unchanged
  • (as expected) \a is passed unchanged

More information:

  • This does not happen when invoking the custom command script file directly.
  • This cannot be reproduced using git for linux (git version 2.11.0 @ arch linux).
  • This might be related to msys/mingw path conversion as '/a\\b' is changed to A:/b, but it cannot be solved by setting MSYS_NO_PATHCONV=1.



  • I was not able to find an open or closed issue matching what I'm seeing

#239 - Closest one providing some context about how bash handles backslash escapes.

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
$ git --version --build-options
git version 2.11.0.windows.1
sizeof-long: 4
machine: x86_64
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

Windoes 10 64 Bit

$ cmd.exe /c ver
Microsoft Windows [Version 10.0.14393]
  • What options did you set as part of the installation? Or did you choose the
    defaults?

It's a portable installation.

  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

Don't think so.

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Git-Bash started via git-cmd.exe --no-cd --command=usr/bin/bash.exe --login -i.

echo -e '#!/bin/bash\necho "$@"' > git-test
chmod +x git-test
PATH="$PWD:$PATH"

# 2. Issue triggered by invoking a custom command
git test '[\\'
# 3. Not triggered when invoking custom command script directly
git-test '[\\'

echo "--- path name conversion? ---"
# 4. Might be related to mingw/msys path conversion
git test '/a\\b'
# 5. ... but setting MSYS_NO_PATHCONV=1 only solves (4)
MSYS_NO_PATHCONV=1 git test '/a\\b'
MSYS_NO_PATHCONV=1 git test '[\\'
  • What did you expect to occur after running these commands?

I'd expect to see the following lines written to stdout:

[\\
[\\
--- path name conversion? ---
/a\\b
/a\\b
[\\
  • What actually happened instead?

The following is written to stdout:

[\
[\\
--- path name conversion? ---
A:/b
/a\\b
[\
  • If the problem was occurring with a specific repository, can you provide the
    URL to that repository to help us with testing?

No repository required at all.

dscho commented

I can confirm that behavior. It seems to be caused by MSYS2's command-line parsing, indeed, as this even smaller MCVE shows:

C:\Program Files\Git> usr\bin\bash.exe -c 'echo "$0"' '[\\'
[\

So the bug is actually with MSYS2's runtime...

episource commented

Git and cmd.exe are native applications invoking the msys application bash. When invoked by a native application, msys applies wildcard expansion and path conversion to the argument string received.

Wildcard expansion can be disabled by setting the environment variable MSYS="noglob". Path conversion can be disabled by setting MSYS_NO_PATHCONV=1.

Probably git should make sure that MSYS="noglob" is set when invoking bash (or any external application)?

Explanation

Linux (Posix?) applications pass an argument array (argv) from caller to callee. It's also the caller (shell) doing wildcard expansion. The same happens when an msys-application invokes another msys-application. Regarding native windows applications, only a single argument string (argc) is passed by the caller and received by the callee via GetCommandline(W). Usually, this string is then parsed into an argv array using CommandLineToArgvW.

When invoked by a native windows application, msys-applications don't use CommandLineToArgvW to parse the argc string, but use their own rules. These rules include wildcard expansion and path conversion.

  • Wildcard expansion is triggered by characters ~?*["'(){} and can be disabled by setting the environment variable MSYS="noglob". It causes patterns like [\\ to fail.
  • Path conversion rules have been documented by the mingw team and can be disabled by setting the environment variable MSYS_NO_PATHCONV=1. They cause patterns like /a\\b to fail.
dscho commented

Probably git should make sure that MSYS="noglob" is set when invoking bash (or any external application)?

As Git itself spawns plenty of things (e.g. hooks) via shell, this may very well cause more harm than good.

It looks, though, as you found a good workaround?

episource commented

Indeed, I found a workaround.

Nevertheless I consider this a bug, not a high priority one, though: I'd expect git for windows (the whole bundle, not just the main executable) to behave like git for linux as long as windows does not enforce differences.

episource commented

As Git itself spawns plenty of things (e.g. hooks) via shell, this may very well cause more harm than good.

Did you consider applying MSYS="noglob" to custom commands only?

dscho commented

Did you consider applying MSYS="noglob" to custom commands only?

Yes. And I came to the conclusion that this approach is fraught with danger, as

  1. future updates to said commands may allow for passing paths, in which case we want to allow globs, and
  2. setting noglob for a custom set of commands is very likely to not include the command for which you need the handling.

So I think that the workaround you found is the best possible solution.