Customising line endings used for `PrintWriter Terminal.writer()`

[brett-smith]

Is there any way to change the behavior of any writer.println() wrt the types of line separators that are used?

We have a JLine enabled application being accessed over SSH (use xterm.js as the emulator). This particular configuration is expecting a CRLF sequence, but any line printed via JLine is only getting a LF.

The reason for this appears to be the use of the default java.io.PrintWriter which has this for its newline implementation.

 private void implNewLine() {
        try {
            ensureOpen();
            out.write(System.lineSeparator());
            if (autoFlush)
                out.flush();
        } catch (InterruptedIOException x) {
            Thread.currentThread().interrupt();
        } catch (IOException x) {
            trouble = true;
        }
    }

Note that it is using a fixed System.lineSeparator(). I happen to be running on Linux, so that is going to return LF.

The TerminalBuilder offers no way to customise the PrintWriter, so the only work around I have got at the moment is to never use println(), using print() only, and then output my own delimiters.

if(lineEndingMode == LineEndings.CRLF) // windows
     writer.print("This will work\r\n");
else if(lineEndingMode == LineEndings.LF) // unix
     writer.print("This will work\n");
else if(lineEndingMode == LineEndings.CR) // ancient macs
     writer.print("This will work\r");

However, this falls down when used with other JLine components, such as picocli integration. This internally is also using the PrintWriter from the terminal as it should, and I have no way to customise what it is doing.

In short, I think JLine either needs to provide a way to configure line endings through the builder, or to provide a way to customise the PrintWriter so that we can do so ourselves.

Am I missing something?

[gnodet]

If you're using JLine to implement a shell over SSH and the remote terminal requires CRLF, it should send the OCRNL in the SSH PtyMode. It should end up as a flag on the terminal's attribute.
The Terminal created using the TerminalBuilder should obey this setting

[brett-smith]

Thanks for the fast response. It pointed me in the right direction. My Attributes handling was naive at best. I am now doing more or less exactly the same as the Apache Mina example above. It still was wasn't working at this point though.

It turned out the client in fact wasn't sending any modes relating to newline conversion at all. I found that if I set both OPOST and ONLCR, it now works as expected. These modes are getting passed into JLine, and I now understand how JLine does the conversion for you.

However, this was with an SSH client I am in control of the configuration of, so I was able add these modes to the pseudo terminal request. If I try the same thing from an entirely different SSH client (plain old OpenSSH ssh command), then I am back to missing the CR characters, because of course it is not sending OPOST and ONLCR (in fact it appears to send no modes at all).

So again I am wondering if this is to do with the default System.lineSeparator() of LF on Linux. If none of these output post processing attributes are being set, then I assume that will be the default? I intend to check the behavior on Windows tomorrow to see if its any different. Will report back.

Edit: When I said "in fact it appears to send no modes at all", it looks like that may be a bug in the SSH proxy server that is involved here as well.

[gnodet]

Thanks for the fast response. It pointed me in the right direction. My Attributes handling was naive at best. I am now doing more or less exactly the same as the Apache Mina example above. It still was wasn't working at this point though.

It turned out the client in fact wasn't sending any modes relating to newline conversion at all. I found that if I set both OPOST and ONLCR, it now works as expected. These modes are getting passed into JLine, and I now understand how JLine does the conversion for you.

However, this was with an SSH client I am in control of the configuration of, so I was able add these modes to the pseudo terminal request. If I try the same thing from an entirely different SSH client (plain old OpenSSH ssh command), then I am back to missing the CR characters, because of course it is not sending OPOST and ONLCR (in fact it appears to send no modes at all).

So again I am wondering if this is to do with the default System.lineSeparator() of LF on Linux. If none of these output post processing attributes are being set, then I assume that will be the default? I intend to check the behavior on Windows tomorrow to see if its any different. Will report back.

I doubt there's a real issue in JLine. The SSH layer has been used for years in Apache Karaf in various environments. But it could be a regression or an edge case.

When you run the SSH client, are you in a real linux terminal ? OpenSSH will grab the terminal configuration and pass it through the SSH protocol to the other side if an interactive SSH session is created. When the session is not interactive (i.e. the ssh client does not request a PTY), there's no real way the SSH server can know about the line endings of the client, as there's no PTY config sent afaik.

If your SSH proxy is not propagating the Pty request / modes, this would certainly fail. But I suggest you ensure that those modes are conveyed correctly, in addition to the terminal type/size and all attributes, else the output may not be correct.

In case a single SSH command is sent (i.e. the session is not interactive), I don't think the SSH server is responsible for doing any translation. You may find more information in those cases on the web by just looking at the SSH part. You may replicate the problem by simply issuing a remote cat command with files with various line endings. The SSH protocol should simply convey the data, and it's up to the local client/terminal to handle those. If the session is not interactive and the line endings don't match, you'll most probably see output errors.... No JLine involved here imho !

[brett-smith]

You are absolutely right, there is no problem with JLine here.

Along with my own bad handling of Attributes, the rest was all down to a now confirmed bug in the SSH proxy not passing on any terminal modes at all, and me not realising earlier that the results of this would would be undefined.

Thanks for your patience, i've learned a fair bit on this little journey, and am always impressed by this project.

Closing as solved.