Read from Pty.getInputStream() hangs

[trofimander]

Sometimes read from Pty stream hangs. Have you experienced that? What exactly can cause it?
That is a corresponding dump:

"ConnectRunnable@1796" prio=5 tid=0x13 nid=NA runnable
  java.lang.Thread.State: RUNNABLE
      at jtermios.linux.JTermiosImpl$Linux_C_lib_DirectMapping.read(JTermiosImpl.java:-1)
      at jtermios.linux.JTermiosImpl.read(JTermiosImpl.java:557)
      at jtermios.JTermios.read(JTermios.java:411)
      at jpty.Pty$1.read(Pty.java:132)
      at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
      at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
      at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
      - locked <0x717> (a java.io.InputStreamReader)
      at java.io.InputStreamReader.read(InputStreamReader.java:167)

InputStreamReader here is created on Pty.getInputStream()

I experience the hang both on Ubuntu Linux and Mac OSX 10.7.5, but maybe I'm doing something wrong.
The most suspecious thing is that the problem began to appear more often - I have it in 99% on my Mac while debugging. Before(I don't know when it changed) it was not so often.

Also sometimes I get following exception

java.io.IOException: Underlying input stream returned zero bytes
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:268)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
    at java.io.InputStreamReader.read(InputStreamReader.java:167)
    at com.jediterm.terminal.ProcessTtyConnector.read(ProcessTtyConnector.java:47)
...

[jawi]

The only thing I can think of why the read() "hangs" is that the underlying process is already terminated (perhaps with due to another signal?). I've to think about how to fix this (if it is fixable at all).

As for the other issue, I've seen it before, but I cannot remember what might be the cause of this. Looking at how I used JPty in combination with InputStreamReader does not reveal much detail...

[trofimander]

Strange thing that the child process isn't killed by a signal - it's not started. I've debugged it:
I execute /bin/bash
After m_jpty.forkpty( master, name, termios, null ) the child process is created and it looks like

22974   ??  Z      0:00.00 (java)

even after execve( command, argv, environment ) it stays the same.

And it dissapears after waitpid( m_childPid, stat, 0 ).

Exit code is 5.

P.S.
Btw I'm also implementing a terminal emulator(https://github.com/traff/jediterm) and have found your jVT220 project quite recently(It's a pity I didn't find it before) - I must admit it is quite impressive and I've even used some implementation details from it (namely charsets handling, God thanks it is Apache 2 licensed!).

[trofimander]

I've just checked JPty tests and the test JPtyTest.testExecInPTY is failing on my machine:

junit.framework.AssertionFailedError: Unexpected process result! expected:<0> but was:<5>
    at jpty.JPtyTest.testExecInPTY(JPtyTest.java:244)

[trofimander]

Hi, I think I can try to fix it myself. Any ideas and hints where to start are greatly appreciated.

[jawi]

Could you try to run the following program through your terminal and post me back the results:

// compile with: g++ -o isatty isatty.c
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <termios.h>
#include <unistd.h>
#include <iostream>
using namespace std;
int main() {
    char tty[L_ctermid+1] = {0};
    ctermid(tty);
    cout << "ID: " << tty << '\n';
    int fd = ::open(tty, O_RDONLY);
    if (fd < 0) perror("Could not open terminal");
    else {
        cout << "Opened terminal\n";
        struct termios term;
        int r = tcgetattr(fd, &term);
        if (r < 0) perror("Could not get attributes");
        else cout << "Got attributes\n";
    }
    if (isatty(fileno(stdin))) cout << "Is a terminal\n";
    else cout << "Is not a terminal\n";
    struct stat stats;
    int r = fstat(fileno(stdin), &stats);
    if (r < 0) perror("fstat failed");
    else {
        if (S_ISCHR(stats.st_mode)) cout << "S_ISCHR\n";
        else if (S_ISFIFO(stats.st_mode)) cout << "S_ISFIFO\n";
        else if (S_ISREG(stats.st_mode)) cout << "S_ISREG\n";
        else cout << "unknown stat mode\n";
    }
    return 0;
}

[trofimander]

You mean to run it inside normal terminal on my Mac?

I will do that today in the evening as now I don't have access to my Mac. But note that the problem is reproduced in 100% of cases when is executed in java debug mode.
If it is not a debug mode it fails only sometimes.

[jawi]

No, I meant to run this program through JPty. I'll try to debug JPty myself as well, but time is rather limited at the moment, unfortunately...

[trofimander]

I've tried to rewrite JPty and was experimenting with different parts of it and in the end I have found out that the hang appears unpredictable between fork and execv and the probability of the hang is higher when there is code to execute there. I've search in the Internet and found an opinion that "calling fork in a java program is kind of unsafe and non-portable between JVM since the JVM does not expect this to happen. The new process will be sharing some OS resources with the original process like semaphores, shared memory, etc, so this might cause unexpected results". After reimplemented the code to plain C code(exact the same code but in C) the constant problem on my Mac under debug dissapeared. Also dissapeared rare individual hangs on Linux.
So the conculsion that I can personaly make is that the exec-part: fork_tty-execv or fork-login_tty-execv should be native(not JNA) to be stable.

[trofimander]

I've also made some kind of fork of JPty and elt(eclipse terminal) where implemented all this: https://github.com/traff/pty4j