jupyter/terminado

Tests are failing with readline 8.1: ValueError: invalid literal for int() with base 10: '\x1b[?2004l\r150520\r'

mgorny opened this issue · 4 comments

It seems that readline now spews some ANSI codes:

============================= test session starts ==============================
platform linux -- Python 3.9.1, pytest-6.1.2, py-1.10.0, pluggy-0.13.1
rootdir: /tmp/terminado-0.9.2
plugins: forked-1.3.0, timeout-1.4.2, xdist-2.2.0, freezegun-0.4.2, shutil-1.7.0, virtualenv-1.7.0, asyncio-0.14.0, localserver-0.5.0, flaky-3.7.0, requests-mock-1.8.0, mock-3.5.0, hypothesis-5.47.0
collected 8 items

terminado/tests/basic_test.py ..FF.FFF                                   [100%]

=================================== FAILURES ===================================
______________________ NamedTermTests.test_max_terminals _______________________

self = <terminado.tests.basic_test.NamedTermTests testMethod=test_max_terminals>

    @tornado.testing.gen_test
    def test_max_terminals(self):
        urls = ["/named/%d" % i for i in range(MAX_TERMS+1)]
        tms = yield self.get_term_clients(urls[:MAX_TERMS])
>       pids = yield self.get_pids(tms)

terminado/tests/basic_test.py:216: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.9/site-packages/tornado/gen.py:735: in run
    value = future.result()
/usr/lib/python3.9/site-packages/tornado/gen.py:742: in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
terminado/tests/basic_test.py:127: in get_pids
    pid = yield tm.get_pid()
/usr/lib/python3.9/site-packages/tornado/gen.py:735: in run
    value = future.result()
/usr/lib/python3.9/site-packages/tornado/gen.py:748: in run
    yielded = self.gen.send(value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <terminado.tests.basic_test.TestTermClient object at 0x7f0fd6557f40>

    @tornado.gen.coroutine
    def get_pid(self):
        """Get process ID of terminal shell process"""
        yield self.read_stdout()                          # Clear out any pending
        self.write_stdin("echo $$\r")
        (stdout, extra) = yield self.read_stdout()
        if os.name == 'nt':
            match = re.search(r'echo \$\$\x1b\[0K\r\n(\d+)', stdout)
            pid = int(match.groups()[0])
        else:
>           pid = int(stdout.split('\n')[1])
E           ValueError: invalid literal for int() with base 10: '\x1b[?2004h\x1b]0;mgorny@pomiot:/tmp/terminado-0.9.2\x07\x1b[01;32mmgorny@pomiot\x1b[01;34m /tmp/terminado-0.9.2 $\x1b[00m echo $$\r'

terminado/tests/basic_test.py:98: ValueError
------------------------------ Captured log call -------------------------------
WARNING  tornado.general:autoreload.py:129 tornado.autoreload started more than once in the same process
________________________ NamedTermTests.test_namespace _________________________

self = <terminado.tests.basic_test.NamedTermTests testMethod=test_namespace>

    @tornado.testing.gen_test
    def test_namespace(self):
        names = ["/named/1"]*2 + ["/named/2"]*2
        tms = yield self.get_term_clients(names)
>       pids = yield self.get_pids(tms)

terminado/tests/basic_test.py:206: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.9/site-packages/tornado/gen.py:735: in run
    value = future.result()
/usr/lib/python3.9/site-packages/tornado/gen.py:742: in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
terminado/tests/basic_test.py:127: in get_pids
    pid = yield tm.get_pid()
/usr/lib/python3.9/site-packages/tornado/gen.py:735: in run
    value = future.result()
/usr/lib/python3.9/site-packages/tornado/gen.py:748: in run
    yielded = self.gen.send(value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <terminado.tests.basic_test.TestTermClient object at 0x7f0fd6557760>

    @tornado.gen.coroutine
    def get_pid(self):
        """Get process ID of terminal shell process"""
        yield self.read_stdout()                          # Clear out any pending
        self.write_stdin("echo $$\r")
        (stdout, extra) = yield self.read_stdout()
        if os.name == 'nt':
            match = re.search(r'echo \$\$\x1b\[0K\r\n(\d+)', stdout)
            pid = int(match.groups()[0])
        else:
>           pid = int(stdout.split('\n')[1])
E           ValueError: invalid literal for int() with base 10: '\x1b[?2004h\x1b]0;mgorny@pomiot:/tmp/terminado-0.9.2\x07\x1b[01;32mmgorny@pomiot\x1b[01;34m /tmp/terminado-0.9.2 $\x1b[00m echo $$\r'

terminado/tests/basic_test.py:98: ValueError
------------------------------ Captured log call -------------------------------
WARNING  tornado.general:autoreload.py:129 tornado.autoreload started more than once in the same process
_____________________ SingleTermTests.test_single_process ______________________

self = <terminado.tests.basic_test.SingleTermTests testMethod=test_single_process>

    @tornado.testing.gen_test
    def test_single_process(self):
        tms = yield self.get_term_clients(["/single", "/single"])
>       pids = yield self.get_pids(tms)

terminado/tests/basic_test.py:227: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.9/site-packages/tornado/gen.py:735: in run
    value = future.result()
/usr/lib/python3.9/site-packages/tornado/gen.py:742: in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
terminado/tests/basic_test.py:127: in get_pids
    pid = yield tm.get_pid()
/usr/lib/python3.9/site-packages/tornado/gen.py:735: in run
    value = future.result()
/usr/lib/python3.9/site-packages/tornado/gen.py:748: in run
    yielded = self.gen.send(value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <terminado.tests.basic_test.TestTermClient object at 0x7f0fd55172b0>

    @tornado.gen.coroutine
    def get_pid(self):
        """Get process ID of terminal shell process"""
        yield self.read_stdout()                          # Clear out any pending
        self.write_stdin("echo $$\r")
        (stdout, extra) = yield self.read_stdout()
        if os.name == 'nt':
            match = re.search(r'echo \$\$\x1b\[0K\r\n(\d+)', stdout)
            pid = int(match.groups()[0])
        else:
>           pid = int(stdout.split('\n')[1])
E           ValueError: invalid literal for int() with base 10: '\x1b[?2004h\x1b]0;mgorny@pomiot:/tmp/terminado-0.9.2\x07\x1b[01;32mmgorny@pomiot\x1b[01;34m /tmp/terminado-0.9.2 $\x1b[00m echo $$\r'

terminado/tests/basic_test.py:98: ValueError
------------------------------ Captured log call -------------------------------
WARNING  tornado.general:autoreload.py:129 tornado.autoreload started more than once in the same process
______________________ UniqueTermTests.test_max_terminals ______________________

self = <terminado.tests.basic_test.UniqueTermTests testMethod=test_max_terminals>

    @tornado.testing.gen_test
    def test_max_terminals(self):
        tms = yield self.get_term_clients(['/unique'] * MAX_TERMS)
>       pids = yield self.get_pids(tms)

terminado/tests/basic_test.py:240: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.9/site-packages/tornado/gen.py:735: in run
    value = future.result()
/usr/lib/python3.9/site-packages/tornado/gen.py:742: in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
terminado/tests/basic_test.py:127: in get_pids
    pid = yield tm.get_pid()
/usr/lib/python3.9/site-packages/tornado/gen.py:735: in run
    value = future.result()
/usr/lib/python3.9/site-packages/tornado/gen.py:748: in run
    yielded = self.gen.send(value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <terminado.tests.basic_test.TestTermClient object at 0x7f0fd6557ee0>

    @tornado.gen.coroutine
    def get_pid(self):
        """Get process ID of terminal shell process"""
        yield self.read_stdout()                          # Clear out any pending
        self.write_stdin("echo $$\r")
        (stdout, extra) = yield self.read_stdout()
        if os.name == 'nt':
            match = re.search(r'echo \$\$\x1b\[0K\r\n(\d+)', stdout)
            pid = int(match.groups()[0])
        else:
>           pid = int(stdout.split('\n')[1])
E           ValueError: invalid literal for int() with base 10: '\x1b[?2004h\x1b]0;mgorny@pomiot:/tmp/terminado-0.9.2\x07\x1b[01;32mmgorny@pomiot\x1b[01;34m /tmp/terminado-0.9.2 $\x1b[00m echo $$\r'

terminado/tests/basic_test.py:98: ValueError
------------------------------ Captured log call -------------------------------
WARNING  tornado.general:autoreload.py:129 tornado.autoreload started more than once in the same process
ERROR    asyncio:base_events.py:1738 Exception in callback None()
handle: <Handle cancelled>
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 522, in read
    s = self.fileobj.read1(size)
OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/terminado-0.9.2/terminado/management.py", line 204, in pty_read
    s = ptywclients.ptyproc.read(65536)
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 824, in read
    b = super(PtyProcessUnicode, self).read(size)
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 527, in read
    raise EOFError('End Of File (EOF). Exception style platform.')
EOFError: End Of File (EOF). Exception style platform.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 139, in _handle_events
    handler_func(fileobj, events)
  File "/tmp/terminado-0.9.2/terminado/management.py", line 216, in pty_read
    client.on_pty_died()
  File "/tmp/terminado-0.9.2/terminado/websocket.py", line 110, in on_pty_died
    self.send_json_message(['disconnect', 1])
  File "/tmp/terminado-0.9.2/terminado/websocket.py", line 77, in send_json_message
    self.write_message(json_msg)
  File "/usr/lib/python3.9/site-packages/tornado/websocket.py", line 339, in write_message
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
ERROR    asyncio:base_events.py:1738 Exception in callback None()
handle: <Handle cancelled>
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 522, in read
    s = self.fileobj.read1(size)
OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/terminado-0.9.2/terminado/management.py", line 204, in pty_read
    s = ptywclients.ptyproc.read(65536)
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 824, in read
    b = super(PtyProcessUnicode, self).read(size)
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 527, in read
    raise EOFError('End Of File (EOF). Exception style platform.')
EOFError: End Of File (EOF). Exception style platform.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 139, in _handle_events
    handler_func(fileobj, events)
  File "/tmp/terminado-0.9.2/terminado/management.py", line 216, in pty_read
    client.on_pty_died()
  File "/tmp/terminado-0.9.2/terminado/websocket.py", line 110, in on_pty_died
    self.send_json_message(['disconnect', 1])
  File "/tmp/terminado-0.9.2/terminado/websocket.py", line 77, in send_json_message
    self.write_message(json_msg)
  File "/usr/lib/python3.9/site-packages/tornado/websocket.py", line 339, in write_message
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
ERROR    asyncio:base_events.py:1738 Exception in callback None()
handle: <Handle cancelled>
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 522, in read
    s = self.fileobj.read1(size)
OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/terminado-0.9.2/terminado/management.py", line 204, in pty_read
    s = ptywclients.ptyproc.read(65536)
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 824, in read
    b = super(PtyProcessUnicode, self).read(size)
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 527, in read
    raise EOFError('End Of File (EOF). Exception style platform.')
EOFError: End Of File (EOF). Exception style platform.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 139, in _handle_events
    handler_func(fileobj, events)
  File "/tmp/terminado-0.9.2/terminado/management.py", line 216, in pty_read
    client.on_pty_died()
  File "/tmp/terminado-0.9.2/terminado/websocket.py", line 110, in on_pty_died
    self.send_json_message(['disconnect', 1])
  File "/tmp/terminado-0.9.2/terminado/websocket.py", line 77, in send_json_message
    self.write_message(json_msg)
  File "/usr/lib/python3.9/site-packages/tornado/websocket.py", line 339, in write_message
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
____________________ UniqueTermTests.test_unique_processes _____________________

self = <terminado.tests.basic_test.UniqueTermTests testMethod=test_unique_processes>

    @tornado.testing.gen_test
    def test_unique_processes(self):
        tms = yield self.get_term_clients(["/unique", "/unique"])
>       pids = yield self.get_pids(tms)

terminado/tests/basic_test.py:234: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.9/site-packages/tornado/gen.py:735: in run
    value = future.result()
/usr/lib/python3.9/site-packages/tornado/gen.py:742: in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
terminado/tests/basic_test.py:127: in get_pids
    pid = yield tm.get_pid()
/usr/lib/python3.9/site-packages/tornado/gen.py:735: in run
    value = future.result()
/usr/lib/python3.9/site-packages/tornado/gen.py:748: in run
    yielded = self.gen.send(value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <terminado.tests.basic_test.TestTermClient object at 0x7f0fd65735e0>

    @tornado.gen.coroutine
    def get_pid(self):
        """Get process ID of terminal shell process"""
        yield self.read_stdout()                          # Clear out any pending
        self.write_stdin("echo $$\r")
        (stdout, extra) = yield self.read_stdout()
        if os.name == 'nt':
            match = re.search(r'echo \$\$\x1b\[0K\r\n(\d+)', stdout)
            pid = int(match.groups()[0])
        else:
>           pid = int(stdout.split('\n')[1])
E           ValueError: invalid literal for int() with base 10: '\x1b[?2004h\x1b]0;mgorny@pomiot:/tmp/terminado-0.9.2\x07\x1b[01;32mmgorny@pomiot\x1b[01;34m /tmp/terminado-0.9.2 $\x1b[00m echo $$\r'

terminado/tests/basic_test.py:98: ValueError
------------------------------ Captured log call -------------------------------
WARNING  tornado.general:autoreload.py:129 tornado.autoreload started more than once in the same process
ERROR    asyncio:base_events.py:1738 Exception in callback None()
handle: <Handle cancelled>
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 522, in read
    s = self.fileobj.read1(size)
OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/terminado-0.9.2/terminado/management.py", line 204, in pty_read
    s = ptywclients.ptyproc.read(65536)
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 824, in read
    b = super(PtyProcessUnicode, self).read(size)
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 527, in read
    raise EOFError('End Of File (EOF). Exception style platform.')
EOFError: End Of File (EOF). Exception style platform.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 139, in _handle_events
    handler_func(fileobj, events)
  File "/tmp/terminado-0.9.2/terminado/management.py", line 216, in pty_read
    client.on_pty_died()
  File "/tmp/terminado-0.9.2/terminado/websocket.py", line 110, in on_pty_died
    self.send_json_message(['disconnect', 1])
  File "/tmp/terminado-0.9.2/terminado/websocket.py", line 77, in send_json_message
    self.write_message(json_msg)
  File "/usr/lib/python3.9/site-packages/tornado/websocket.py", line 339, in write_message
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
ERROR    asyncio:base_events.py:1738 Exception in callback None()
handle: <Handle cancelled>
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 522, in read
    s = self.fileobj.read1(size)
OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/terminado-0.9.2/terminado/management.py", line 204, in pty_read
    s = ptywclients.ptyproc.read(65536)
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 824, in read
    b = super(PtyProcessUnicode, self).read(size)
  File "/usr/lib/python3.9/site-packages/ptyprocess/ptyprocess.py", line 527, in read
    raise EOFError('End Of File (EOF). Exception style platform.')
EOFError: End Of File (EOF). Exception style platform.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 139, in _handle_events
    handler_func(fileobj, events)
  File "/tmp/terminado-0.9.2/terminado/management.py", line 216, in pty_read
    client.on_pty_died()
  File "/tmp/terminado-0.9.2/terminado/websocket.py", line 110, in on_pty_died
    self.send_json_message(['disconnect', 1])
  File "/tmp/terminado-0.9.2/terminado/websocket.py", line 77, in send_json_message
    self.write_message(json_msg)
  File "/usr/lib/python3.9/site-packages/tornado/websocket.py", line 339, in write_message
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
=============================== warnings summary ===============================
terminado/tests/basic_test.py:38
  /tmp/terminado-0.9.2/terminado/tests/basic_test.py:38: PytestCollectionWarning: cannot collect test class 'TestTermClient' because it has a __init__ constructor (from: terminado/tests/basic_test.py)
    class TestTermClient(object):

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED terminado/tests/basic_test.py::NamedTermTests::test_max_terminals - Va...
FAILED terminado/tests/basic_test.py::NamedTermTests::test_namespace - ValueE...
FAILED terminado/tests/basic_test.py::SingleTermTests::test_single_process - ...
FAILED terminado/tests/basic_test.py::UniqueTermTests::test_max_terminals - V...
FAILED terminado/tests/basic_test.py::UniqueTermTests::test_unique_processes
=================== 5 failed, 3 passed, 1 warning in 22.99s ====================

This still seems to be an issue with terminado 0.9.4. Any chance of progress here?

It is still an issue with 0.10.1.

Workaround is

echo "set enable-bracketed-paste off" > .inputrc
export INPUTRC=$PWD/.inputrc