Test suite requires real terminal

[wavexx]

There's the possibility to run package tests directly using the debian infrastructure (see http://packaging.ubuntu.com/html/auto-pkg-test.html) and it's actually encouraged, which is great. Tests would be actually run on all archs supported by the infrastructure.

However, tests are run in a headless environment. There's no terminal attached, thus our tests fail.
Simply put: you cannot run python test/test_tabview.py > log.

There's also to note that initializing curses actually clears some amount of scrollback in a real terminal, thus in case of errors 1) you cannot log it 2) you cannot scroll back.

This could be fixed by running each individual test under a "screen" instance. screen -D -m [cmd] does almost what we need, but it doesn't return the exit code of the underlying process (why is a pity). tmux is identical in this behavior.

You could potentially run the entire process (not each test) in a screen instance, but you either lose the ability to capture the actual output, or the output will be interspersed with terminal sequences (enable screen logging to see this).

Annoying.

[firecat53]

Well, I spent some time playing with the tests and trying to create a Debian package so I can even attempt messing with autopkgtest.

I was able (finally) to create a Debian package using stdeb (worked fine in an Ubuntu 14.04 Docker container, but in a Jessie container I had to use pip to install the latest stdeb version).

I wasn't able to get the normal tabview tests to pass inside the containers...various Unicode errors either with the tests or setup.py.

I tried running adt-run --output-dir /tmp/output-dir ./ --- null from the tabview Debian source package directory, but didn't seem to accomplish anything. I think I'm missing something...

Haven't even made it where I can even try the screen/tmux bit yet.

Sorry for being a bit dense here. Out of energy tonight...if you have any suggestions or a quick description of your packaging and autopkgtest setup it would be appreciated!

Thanks,
Scott

[wavexx]

Strictly speaking you do not need to run adt to reproduce the issue. Just unset TERM really, or redirect the output to a log file.

In your case, the tests fail because stdeb probably tries to run the tests using nose or setup.py tests. Unfortunately, if we need a terminal in an isolated context for each test, I'm unsure we can actually use normal python-based test discovery, because we'd need to re/attach the I/O of the python process to a tty at each step.

Maybe we can customize the logging, but we'd still run into the issue that I'd like stdout to go to the log, and each TEST to write to a separate tty.