Node.js doesn't run as tty on windows / cygwin
martinheidegger opened this issue · 47 comments
Just had a report @ nodeschool that node -p -e "Boolean(process.stdout.isTTY)" on Windows 8.1 Pro with Node.js 4.1.0 on Cygwin 2.2.1 returns false. It does seem to work in the regular command prompt.
IIRC, that's because the cygwin shell isn't really a tty, it's a pipe.
I am confused (general state of mind these days): Ain't I supposed to be able to do things like process.stdin.resume() on cygwin? Is process.stdin.resume() not tied to it being a tty? Am I not understanding something?
process.stdin can be a tty, pipe, file, or even a socket. node.js detects what type of device it is and adjusts accordingly.
Let me rephrase the question: In my understanding tty is indicating that there is some user input loop that can be .resume()ed. Is that correct? A pipe would not indicate that there is ui attached to it?!
Note: clarification on Cygwin: I meant the Cygwin bash.
You can .resume() either way, it's not specific to ttys. stdin being a pipe doesn't say anything about the other end having a UI.
I am sorry to ask differently yet again: Does this mean that there is no reliable way to figure out whether node (or any other app) is running inside a terminal?
(I am asking because http://stackoverflow.com/questions/4426280/what-do-pty-and-tty-mean tells me that tty means having a terminal...)
If by 'terminal' you mean 'user-visible window', then no, not in general and not reliably.
I need to ask again to form a good question here (so sorry for taking your time - and anyone who reads this). I am still not clear what tty means in the node context. I am still assuming it has to do with "user input". As far as I read in the api documentation, depending on process.stdout.isTTY other parts of the stdio of node (tty.ReadStream) change. If I can't rely on isTTY to tell me if the terminal supports asks for user input or not, can I force node to use the tty.Read/WriteStream ?
Short version: Cygwin sucks. Don't use it. 😄
Medium version: Cygwin (and others) that attempt to provide "terminal windows" have a lot of problems with standard Windows programs (e.g. node).
Long version...
POSIX has pty's, which allows programs to work the same way on the console, serial terminals, ssh sessions, and GUI terminal windows (e.g. XTerm). The program sees a standard "tty" interface in all cases, and can easily differentiate this interface from a pipe or file redirection.
Windows has no equivalent feature. Windows consoles are a special facility built in to the OS, and generally aren't extensible. The Windows console can do things that no other application can do.
cygwin (and others) attempt to emulate POSIX pty's and provide "terminal windows" with varying degrees of success. These terminal windows are generally NOT Windows consoles, and can confuse standard Windows programs (e.g. node). Some of them use pipes, but pipes have problems. A program with a pipe for stdin or stdout has no way to know if it's being run in a traditional shell pipeline (e.g. "ls -l | head"), or if the other end of the pipe is a process attempting to emulate an interactive tty.
@mcnameej Thank you for your detailed explanation. The problem with this situation is that the bash of cygwin and the bash of github have been promoted widely in the nodeschool community. I also meet many windows developers that use it in their corporate computers.
I seem to have the following problems after processing all your great input:
- In node.js 4 (probably before that in one of the iojs versions) something seemingly changed in the tty detection. The result is that now it is
falsewhere before it probably wastrue(have not tracked non-issues). Those problems have only been reported since the 4.0 release. Does anyone know what has changed? Have people similar issues? - Some logic inside of node.js seem to change when
isTTYis detected but since we can't guarantee that it is detected properly I would like to be able to enforce that behaviour - making the code assume that it is a tty. Would that be possible? Is that even necessary? (Keep in mind that I don't know what the internals are about) - I have added an
isTTYcheck to my code in response to some strange error that occurs when you try to usesetRawModeandresume(). It seems I have usedisTTYwrong. When and how is it appropriate to do aisTTYcheck?
bash itself is not the problem -- the problem is running node inside a "fake" console. If you run bash in a regular Windows console, it gets along fine with node. See screen shots below. .
In node.js 4 (probably before that in one of the iojs versions) something seemingly changed in the tty detection.
I don't know what changed. A review of the code (probably in libuv) could probably determine the cause, but IMHO, that's asking the wrong question. You're trying to do something that is inherently unreliable on Windows, and any "fix" made today may break again tomorrow.
Would it be possible to make the code assume that it is a tty?
I'm not the right person to answer that; perhaps @bnoordhuis will chime in. I have a lot of experience with Win32, and can talk about the OS-level issues, but I'm not an expert on node's internals.
@mcnameej Don't have time for a lengthy reply but there is one thing to point out: those screenshots are made with win 7, the problem has been reported with windows 8.1
I have the same result with Windows 8.1...
Hi! I'm the one who reported the issue, yesterday the problem was presented to me in my work's desktop, running win8. Later at home I tried it in my pc running Windows 10, same issue on cygwin, but on cmd worked perfectly. Now I'm at work again(win8) trying to make it through the tutorial, it opens OK in cmd but the key arrows doesn't work. Here's a screenshot I took yesterday in my work's desktop.
Please tell me if you need more info and I will see what can I do for you!
(My lengthy reply)
bash itself is not the problem -- the problem is running node inside a "fake" console. If you run bash in a regular Windows console, it gets along fine with node. See screen shots below. .
It should but somehow it seems it doesn't as hinted by the screenshots of @LuigiR0jas above. Do you have an idea why his bash returns something different than yours? Is there a way to find out the difference between these two setups?
You're trying to do something that is inherently unreliable on Windows, and any "fix" made today may break again tomorrow.
It is not about trying to "fix" it windows for me. It is however important to me to write command line tools that work predictably and reliably on windows. I haven't had the chance to test it but it seems like the cli of other tools seems to work fine?! If the bash is up and running what is the practical difference between isTTY being true and false? (why does node need to adapt to it?)
@LuigiR0jas: Please try this...
Use Windows Explorer to navigate to wherever you have Cygwin installed. There should be a Cygwin.bat file in the base directory. Double click on that file to run it from Explorer. This will open Cygwin bash inside a standard Windows console.
Change to the node directory (e.g. cd "/cygdrive/c/Program Files/nodejs").
Run ./node -p -e "Boolean(process.stdout.isTTY)" from there.
It should print true.
I just did a fresh install of Cygwin on Windows 8.1 X64 and verified these results.
The "Cygwin Terminal" shortcut runs bash inside Cygwin's fake console. The fake console doesn't get along with node.
Do you have an idea why his bash returns something different than yours? Is there a way to find out the difference between these two setups?
See my message above to LuigiR0jas. I suspect he was using Cygwin's fake console. The problem isn't bash -- it's the console.
It is however important to me to write command line tools that work predictably and reliably on windows
Unfortunately, the Cygwin people have different priorities. They want their environment to be as "POSIX-ish" as possible, even if that causes problems with some native Windows programs.
I haven't had the chance to test it but it seems like the cli of other tools seems to work fine?
Tools that work with pure ASCII input and output streams generally work fine inside fake consoles. They don't care if their stdin/stdout are pipes. Tools that do special input processing (e.g. read single characters rather than lines), or that colorize their output, will fail.
why does node need to adapt to it
The problem is that Window has console-specific API's, which node uses when run inside a real Windows console. Node falls back to generic I/O functions when stdin/stdout are anything other than a real console (e.g. pipe, redirected to file, etc.). Having console-specific I/O functions is a key difference between Windows and POSIX.
Tried it, it worked.
I hope this is useful, i took it in my home PC with Windows 10 (it had the same problem), on the left the regular cygwin terminal (showing false), on the right the cygwin bash on cmd.
Short question: in the bash without false we can see colors, does vim or emacs work (keyword: stdin)
on the left the regular cygwin terminal (showing false), on the right the cygwin bash on cmd.
Proves what I'm saying. The problem isn't bash -- it's the console.
does vim or emacs work
I'm not sure what that test will prove. cygwin's own programs (bash, vim, emacs, etc.) know how to deal with both real Windows consoles and cygwin's terminal. They'll work either way. The problem is with standard Windows console applications (e.g. node) running inside cygwin's terminal.
@mcanameej I never doubted your word. Just need to go 100% clear in understanding to phrase it correctly when passing on. There is still some question open:
The problem is with standard Windows console applications (e.g. node) running inside cygwin's terminal.
Just for the sake of argument: could you compile nodejs with cygwin and it would work?
Could it be determined if you run bash in a "not-regular-console"?
Tools that do special input processing (e.g. read single characters rather than lines),
So it is possible to read lines and output text in those terminals?!
Does it allow other characters (like ignore the color on windows)?
Does this assumption exclude control characters like [;H or are only the colors a problem?
I went over and looked at workshopper/workshopper#120 and nodeschool/discussions#1448 that were mentioned by @SomeoneWeird.
I strongly suspect there is a bug in node v4 (probably in libuv) related to setting Windows 8/8.1/10 consoles into raw mode. I can reproduce the failure in a standard console. This is a completely separate issue from running node inside cygwin's fake console.
Just for the sake of argument: could you compile nodejs with cygwin and it would work?
AFAIK, node doesn't support building under cygwin. Somebody might be able to hack it, but I'm afraid it won't be me.
Could it be determined if you run bash in a "not-regular-console"?
I don't understand the question.
So it is possible to read lines and output text in those terminals?!
Yes.
When running node in a fake console, it's useful to imagine that you're running it with input and output redirected to files (e.g. node foobar.js <infile.txt >outfile.txt). That's basically what node thinks is going on (a pipe and a file look more-or-less the same to node). Things that would work with file redirections will work in the fake console.
Does this assumption exclude control characters like [;H or are only the colors a problem?
Windows consoles don't implement ANY escape sequences. node (via libuv) emulates them when it detects itself running in a console. Behind the scenes, libuv parses the escape sequences and makes the appropriate Windows console API calls to implement them.
It seems all started from stream_base rewrite.
cc @indutny ^
Was this also fixed in 5.1.0?
Not sure if this is directly related to this issue, but sending output through a pipe is broken with Git Bash.
Sending output through a pipe works fine in Cygwin (unfortunatelly it also adds error about writing EOF) and cmd.exe.
The winpty tool I wrote might be relevant to this discussion. It creates a hidden console and marshals I/O between it and Cygwin's emulated pty:
rprichard@vbwin7 ~
$ node -p -e "Boolean(process.stdout.isTTY)"
false
rprichard@vbwin7 ~
$ console node -p -e "Boolean(process.stdout.isTTY)"
true
Someone packaged it for MSYS2, and IIRC, it's distributed with Git for Windows. The EXE was renamed from console.exe to winpty.exe.
I don't know if this helps, but when installing Git for Windows 2.7.0 there is a prompt now (I don't recall seeing it in some fairly earlier version) that defaults to using MinTTY, but if you choose Use Windows' default console window, $ node -p -e "Boolean(process.stdout.isTTY)" prints true and I don't experience any of the problems that I do with MinTTY (some gulp hangs, inquirer prompts not working correctly, etc.)
That fixed it for me! Great solution, thanks. I also enabled QuickEdit Mode in the terminal properties (enables features such as wheel scrolling).
I was having the same issue while trying to redirect stdout to a file from node.js in git bash. I wrapped my node call in a bash script and redirected it in the script. It worked. It's not ideal, but it is fine for my use case.
Yeah. Create git bash script and wrapped node into this script solves the problem.
Is this fixed in Node v6?
Looks like the solution was:
I don't know if this helps, but when installing Git for Windows 2.7.0 there is a prompt now (I don't recall seeing it in some fairly earlier version) that defaults to using MinTTY, but if you choose
Use Windows' default console window,$ node -p -e "Boolean(process.stdout.isTTY)"prints true and I don't experience any of the problems that I do with MinTTY (some gulp hangs, inquirer prompts not working correctly, etc.)
I'm not sure we can actually fix this.
Is this fixed in Node v6?
or 7 ?
No. I tried to fix it, but gave up, see #2908 (comment)
I just rebased my branch at https://github.com/sam-github/node/tree/spawn-detached-window-hide-option, feel free to adopt it if you want.
I completely failed to make the pop-up windows happen at all, so could not verify that my change was capable of hiding the pop-up... if you can build and verify that branch, I could re-PR it and I'm sure we could get it merged in short order.
I had this problem and I tried running a 'sub-shell' (mean I ran bash and the command line and created another shell). Then I ran node program < infile and I don't have the problem anymore
It is not a node problem, I investigated this in context of yarnpkg. The way git-bash "fixed" the problem is by wrapping the node process in a program called winpty, which emulates some/enough of the unix teletype to make it work: https://github.com/rprichard/winpty
If you have a "newer" version of git bash installed, you will find a hack in /etc/profile.d/aliases.sh wrapping common programs like node, php and python to run through winpty.
WinPTY works with cygwin as well, so if you introduce a similar "hack/fix" your bash profile for, it should work "just as well" in cygwin.
Checkout this pull-request for further details: yarnpkg/yarn#2230
@thetrompf I am sorry but that seems like it is a Node.js problem after all: If you have to wrap it with some other method that "does the compatibility" it means - by what I understand - that the Node.js compatibility is not given.
@martinheidegger Node.js works fine with the standard windows shells (cmd.exe and power shell), using cygwin/msys shells is the users who has decided to use shells that looks like teletype terminals but are not, and the common way to test if you are command line programs is running in an interactive shell is by checking for process.stdout.isTTY or in a unix shell script http://stackoverflow.com/questions/911168/how-to-detect-if-my-shell-script-is-running-through-a-pipe CygWin ans Msys terminals has implemented some of the teletype layer, but unfortunately not enough to work in all cases. If you ask me cygwin and msys/mingw should include whatever winpty is doing in their terminals so they actually work.
@thetrompf I do know that cygwin does not implement certain parts correctly. But: if cygwin would break interactive mode then it wouldn't be possible for winpty to "fix" it. In other words: if an application running in cygwin can fix it node - an application running in cygwin can fix that too. That being said Cygwin is afaik not a officially supported platform. It would be good if it became one though (since several tools on windows promote cygwin as "best shell for windows".
note : if it can help someone there is a workaround for this issue using msys2 I guess it would be the same for cygwin
the issue I face is how to write npm's output to a file using msys2.
it's not the exact same issue but it has the same root as this issue.
I am using msys2w64 under windows 10
the workarround is:
use npm.cmd instead of npm
my case is to run npm and gulp only
this fails
$npm > /tmp/test
stdout is not a tty
$this works
$npm.cmd > tmp/test
$@pmalhaire I use msys2 (from msys2-x86_64-20160205.exe) and do not seem to need the workaround, calling the npm shell script directly works fine.
I do need winpty for running node itself, though.
@geonanorch did you try to redirect it's output to a file as done in my example ?
$npm > /tmp/test
stdout is not a tty
$
@pmalhaire yes I did:
$/C/Program\ Files/nodejs/npm > test
$cat test
Usage: npm <command>
. . . .
(remark: I installed node/npm independently of MSYS2)
Note that things aren't perfect for me either though: when invoking npm init for instance I have to type CTRL-C to exit the program, it just hangs after generating package.json (and going through winpty first does not help in this case).
I wish there existed a 'standard' library which allowed to build windows console programs with dual support native-console/mintty...
@geonanorch it's a workaround
fixing the issue implies fixing mysys2 it self (not nodejs)