microsoft/WSL

Unable to pipe Bash output to other programs

RaeesBhatti opened this issue · 41 comments

RaeesBhatti commented

I'm unable to spawn Bash from NodeJS. If you try to, it will return this error: Error: 0x80070057

How to reproduce:

  • Install NodeJS
  • Run node in CMD
  • Type:
cp = require('child_process');
proc = cp.spawnSync('cmd', ['/c', 'bash -c "ls"'], {encoding: 'utf8'}).output

It outputs:

[ null, 'Error: 0x80070057\r\r\n', '' ]

ref: MicrosoftDocs/WSL#8

patryk9200 commented

Try by changing encoding. I don't have problem to run bash -c "ls" from CMD.
Try UTF16 or Latin II (852). I have in CMD Latin II encodig by default.

RaeesBhatti commented

NodesJS says unknown encoding for both UTF-16 and ISO-8859-2. And it works just find in the CMD. Just not in NodeJS

RaeesBhatti commented

This is a problem with Bash having problems with output piping. If I specify { stdio: 'inhert' }, it works but that way I then have no way of getting that data. { stdio: 'pipe' } produces the error above

tkelman commented

Same from mintty (the terminal emulator used by cygwin and msys2), which uses named pipes for stdin/stdout:

$ /cygdrive/c/windows/system32/bash
Error: 0x80070057
russalex commented

Sorry for the late comment on this one. Main reason for the delay is that I did not want to comment until I have tried it myself and I did not have node.exe on my system.

I still get an error here on the latest build, but it at least is a different error (no more helpful).

We will look into this one, but I can not promise anything.

RaeesBhatti commented

This is actually very crucial to many applications of WSL. For example in my case I have to make Atom-Hack (HackLang plugin for Atom Editor) work by executing a Hack Linter inside the WSL and pipe its output to Atom's linter. There are a lot of use cases for this.

You can actually try this on Python too as suggested here

russalex commented

We have a known limitation where we only support the console for stdin and stdout. If either of those are not a console then we will fail. That said, I believe I'm telling you things you already know with your last comment in #20.

We do understand that this is a limitation and does block some scenarios. It is on the backlog. My suggestion at this point would be to add it to our User Voice page and go for the up votes to help raise the priority.

wywzxxz commented

node.js spawn have the same problem.
and popen in C++ have the problem(tested with Dev-c++).

however,If there is no output,then It works fine.
here is an example:
a.cpp

#include"stdio.h"
int main()
{
    FILE *fp=fopen("out.txt","w");
    fprintf(fp,"asdadasd\n");
    fclose(fp); 
}

main.cpp

#include"stdio.h"
#include"windows.h"
int main(int argc,char** argv)
{
    //if (argc<=1) return 0;
    system("bash -c \"./a\"");
    return 0;
}

g++ a.cpp -o a then compile and run main.cpp in windows. the out.txt file appear with the right content.

so here is a temporary solution:
1.use a wraper to write everything into a file
2.read that file in windows

2016/04/15 update:
It turn out system works just fine in C++(at least in Dev-C++/TDM-GCC),both bash -c "ls >out.txt" and bash -c "ls" works fine. However, when node.js called it, above problem appeared.

  • 👍1
RaeesBhatti commented

I've just purposed a UserVoice Idea for this problem. Anyone who wants to see this fixed, can vote for it.

  • 👍8
benhillis commented

Thanks for creating the UserVoice page. I wanted to be clear that this is definitely very high on our internal list of things we want to implement for future updates.

mjoppich commented

Is there any quick fix around the corner ... i'd desperately love to grab the output from bash -c :S

As a first "quick fix" is it somehow possible to save the output from bash to file?

aseering commented

I haven't tried this, but if you want to write the output to a file, could you just do that within bash? bash -c 'ls > /mnt/c/Users/me/tmp.txt'

wywzxxz commented

"ls > file" doesn't work for node-webkit,and as I remember,doesn't work for node.js either.
But as I mentioned above,system("bash -c 'ls > file' ") in C++ works.

russalex commented

Update on our oldest open issue.

The new Insider build, 14901, has some updates here. The following commands now work:

> C:\tmp>bash -c "ls -la" | findstr foo
> drwxrwxrwx 2 root root 0 Aug 12 12:01 foo

C:\tmp>dir | bash -c "grep foo"
08/12/2016  12:01 PM    <DIR>          foo

In Node on Windows:

cp = require('child_process');
proc = cp.spawnSync('cmd', ['/c', 'bash -c "ls"'], {encoding: 'utf8'}).output
outputs:
[ null,
'node_etw_provider.man\nnode.exe\nnode_modules\nnode_perfctr_provider.man\nnodevars.bat\nnpm\nnpm.cmd\n',
  '' ]

The following does not work.

C:\tmp>bash -c "ls -la" > out.txt

Out.txt only has the text: Error: 0x80070057. We are aware of this one and working on it.

  • 👍8
DanielRamosAcosta commented

Hi guys, previor to this build, I couldn't spawn bash without getting the error. Now works almost fine, but I can't send delayed commands, try this:

const spawn = require('child_process').spawn;

const bash = spawn('/Windows/System32/bash.exe');

bash.stdout.on('data', (data) => {
  console.log(data.toString());
})

bash.stderr.on('data', (data) => {
  console.log(`stderr: ${data}`);
})

bash.on('close', (code) => {
  console.log(`child process exited with code ${code}`);
})

bash.stdin.setEncoding('utf-8');

bash.stdin.write('pwd\n');
bash.stdin.write('uptime\n');
bash.stdin.write('apt-get moo\n');
// All this executes fine

setTimeout(() => {
  bash.stdin.write('apt-get --version\n');
  bash.stdin.end();
}, 5000)
// But these don't

I'm at 14905 build.

FallenAvatar commented

I have been fighting this bug myself for a few hours and found this issue. I have managed to create a TEMPORARY workaround for devs fighting this bug. You can find the .Net project here and you can either use it as is, or adapt the code for other languages. This work-around/project/code is MIT licensed, so have at it.

Basically, it forwards any args passed to it to Bash and tells Bash to pipe its output (using /mnt/c/...) to a temp file and then after Bash exits the file is read and output. This output CAN be redirected using any of the standard methods.

PLEASE note, this is VERY quick and dirty. Feel free to submit pull requests as you see fit.

  • 🎉2
RaeesBhatti commented

I'm having some problems with Bash output in GoLang. I've tried three things:

cmd := exec.Command("bash", "-c", `"ls /"`)
stdout, err := cmd.StdoutPipe();
if err != nil {
    panic(err)
}
stderr, err := cmd.StderrPipe();
if err != nil {
    panic(err)
}
if err := cmd.Start(); err != nil {
    panic(err)
}
defer cmd.Wait()
go io.Copy(os.Stdout, stdout)
go io.Copy(os.Stderr, stderr)
cmd := exec.Command("bash", "-c", `"ls /"`)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
err := cmd.Run()
if err != nil {
    panic(err)
}
cmd := exec.Command("bash", "-c", `"ls /"`)
var out bytes.Buffer
cmd.Stdout = &out
err := cmd.Run()
if err != nil {
    panic(err)
}
fmt.Println(out.String())

All of these result in Error: 0x80070006. @russalex Can you please take a look!

benhillis commented

@RaeesBhatti We have some changes coming to Insider builds which will allow pipe and file redirection to and from bash.exe. Stay tuned to the release notes.

  • 👍2
rbohac commented

Same problem when calling bash from a batch file if I set it up under windows scheduler.

If I call bash from a batch file and run the batch file from the command prompt directly it works

If I call that same batch file (that calls bash from within it) from Windows Scheduler I receive E r r o r : 0 x 8 0 0 7 0 0 5 7

mheden commented

Is there any ETA when this fix will be rolled out to non-insiders?

  • 👍5
russalex commented

New features and bug fixes will roll out to non-insiders when the next version of Windows ships to the public. We do know this is not ideal but these changes have a strong dependency on the Windows kernel version which ships with the Insider program.

Some required fixes will be backported. Examples would be any security issues found and the bug which was blocking Java from running, but these are the exception and not the rule.

If at all possible I highly recommend jumping onto the insider program. We are rolling out new features and bug fixes all the time there, including our new interoperability functionality.

pradyuman commented

Has this been fixed in an insider build? Bash still cannot run as a shell in emacs (I'm on build 14965): syl20bnr/spacemacs#6751

  • 👍1
felixfbecker commented

This unfortunately prevents me from integrating gcc under WSL as a task in Visual Studio Code to compile Unix programs right from VS Code under Windows, and get problems reported directly in VS Code as red squiggles. VS Code is Node-based and spawns bash as a child process as described in the issue and then displays the output in the output panel. I can of course just compile from a terminal, but then the errors won't get matched with a regexp to get red squiggles right in the editor. Please support this use case

benhillis commented

@felixfbecker - Are you running Anniversary Update or an insider build? This functionality is currently available via the Windows Insider program.

felixfbecker commented

@benhillis I will update instantly. Slow or Fast ring?

mjoppich commented

@felixfbecker they should be currently the same on 14965 or so...

rogalmic commented

I am trying to implement bash debugger extension based on bashdb to VS Code.

Will there ever be a possibility to do following in nodejs/Windows?

var process = spawn("bash", ["-c", `echo whatever; echo whatever2 >&3`
		], {stdio: ["pipe", "pipe", "pipe", "pipe"]});
felixfbecker commented

@rogalmic It's already possible in Insiders release

rogalmic commented

@felixfbecker

I just tried with fresh W10 14965.1001 from ISO, but I am having some problems still (though the line in my previous post does not throw any error).

The problem occurs in write method plus the traces from output pipes do not appear:

		this.sendEvent(new OutputEvent(`STARTING\n`, 'stderr'));

		try
		{
			var process = spawn("bash", ["-c", `echo WHATEVER >&3`]
			, {stdio: ["pipe", "pipe", "pipe", "pipe"]}
			);

			process.stdin.write(`DATA DATA DATA \n`);

			process.stdio[3].on("data", (data) => {
					this.sendEvent(new OutputEvent(`${data}`, 'stderr'));
			});
		}
		catch(ex)
		{
			this.sendEvent(new OutputEvent(`EXCEPTION ${ex}\n ${ex.stack}\n`, 'stderr'));
		}

		this.sendEvent(new OutputEvent(`STARTED\n`, 'stderr'));

Output shows Error: write EPIPE in Socket._writeGeneric():

STARTING
EXCEPTION Error: write EPIPE
 Error: write EPIPE
    at exports._errnoException (util.js:1026:11)
    at Socket._writeGeneric (net.js:706:26)
    at Socket._write (net.js:725:8)
    at doWrite (_stream_writable.js:307:12)
    at writeOrBuffer (_stream_writable.js:293:5)
    at Socket.Writable.write (_stream_writable.js:220:11)
    at Socket.write (net.js:651:40)
    at BashDebugSession.launchRequest (bashDebug.js)
...
STARTED

Am I doing something wrong?

felixfbecker commented

What is stdio[3] supposed to be? Maybe that's the issue, that there is only stdin/stdout/stderr? Anyway, Imo this is getting offtopic for this thread and pretty node-specific, you may want to open a new issue to not spam everyone. You also don't handle your errors correctly, those functions will never throw synchronously, you need to attach an error event handler.

rogalmic commented

@felixfbecker Well, stdio[3] is just another (non-standard) pipe passed to child process. It works fine when using cmd.exe instead of bash.exe.

Anyway, this is off-topic, so I will end here (and check the error correctly).

...Thanks to proper error handling, the error turned out to be in spawn ("ENOENT"), which I was able to fix by providing full bash path ("C:\Windows\sysnative\bash.exe"). Thank you for help...

chidea commented

I've wrote a program to solve this by managing temporary pipe/argument files.
http://github.com/chidea/GttBoW

alanzanattadev commented

Any news for official windows 10 support (not insider) of bash running under node on windows 10 ?

aseering commented

Hi @alanzanattadev -- WSL changes (excluding serious security flaws and similar) are batched and merged to stable Windows 10 only as part of major Windows releases. The next major Windows release is the "Creators" update. Note that this will be the first update after WSL's initial public release, so basically everything active or pending in this ticket-tracker is targeted at it :-)

Microsoft has not stated a release date yet for Creators, but the latest Internet rumors have it coming out in a month or so.

  • 👍1
alanzanattadev commented

@aseering Ok :) thank you for your quick answer, do you recommand going into Insider ? Can I rollback to stable version somehow if I switch ? I need this feature :(

aseering commented

So, I have two Windows computers. (First-order approximation :-) ) My main one runs the latest Insider build. Most of the time I'm very happy with it. Every now and then (a few times per year?) there's a bad build, in which case I usually just use my other computer for a while. It's been quite stable recently; they have been keeping things atable in preparation for the upcoming release. (I expect it will be a smooth upgrade from Insider builds to the Creators release when it ships.)

You can generally revert to previous Insider builds (though I usually don't bother) as long as you have a system restore point. You can try to roll all the way back to the latest stable release but they're so different at this point that the revert might not go smoothly? I only tried that once, months ago, but it worked ok.

I guess I haven't answered your question :-) In short -- in exchange for features, how much risk are you comfortable dealing with?

alanzanattadev commented

I need to have a stable platform for development :( I guess I have to wait, thank you for all !

benhillis commented

@alanzanattadev - As @aseering mentioned you won't have to wait long. Creators update is coming soon.

  • 👍2
benhillis commented

Sorry I was just trying to clean up misleading information.