Consume input from sys.stdin

[pdawyndt]

I would have a assumed that batch input could also be consumed this way

import sys

for name in sys.stdin:
    print(f'Hello, {name}')

but apparently the batch input isn't bound deep enough for the sys.stdin iterator to be able to consume it.

[winniederidder]

We override the builtin function input and the method readline of sys.stdin. Apparently, its iterator does not use readline. I must say that I have also never tried to use sys.stdin as an iterable.

[pdawyndt]

Isn’t it the other way round? Readline using the iterator protocol to fetch the next line from stdin?

[winniederidder]

Isn’t it the other way round? Readline using the iterator protocol to fetch the next line from stdin?

I don't know what you mean by this. What I meant is that, since we override readline, sys.stdin.next apparently does not use readline, as then it would work as expected.

[pdawyndt]

Readline using sys.stdin.next?

[winniederidder]

Readline using sys.stdin.next?

Well, readline has some extra options. It might be the case that it internally calls next, so then we could override that instead of readline. I'm currently doing some experimenting, but sadly the internals of sys.stdin are implemented in https://github.com/python/cpython/blob/main/Python/sysmodule.c, making it not quite as straightforward to figure out what happens precisely.

[winniederidder]

Overriding sys.stdin.__next__ is easy and works. sys.stdin.__iter__ / iter(sys.stdin) simply return sys.stdin, so that is also no issue. However, next(sys.stdin) doesn't seem to call sys.stdin.__next__, it seems to call something else, resulting in an immediate StopIteration. Perhaps @alexmojaki can shine some light on this?

[alexmojaki]

Thanks @pdawyndt for pointing this out. I had no idea about this bug and it'd be nice to fix.

@winniederidder I think you're on the right track. I tried investigating and came to similar conclusions and was pretty confused about why next wasn't calling __next__. But now I think I know why, it looks like a case of https://docs.python.org/3/reference/datamodel.html#special-lookup:

For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary. That behaviour is the reason why the following code raises an exception:

>>> class C:
...     pass
...
>>> c = C()
>>> c.__len__ = lambda: 5
>>> len(c)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: object of type 'C' has no len()

So setting sys.stdin.__next__ doesn't seem to work. And setting the method on the class doesn't seem to be an option:

>>> sys.stdin.__class__
<class '_io.TextIOWrapper'>
>>> sys.stdin.__class__.__mro__
(<class '_io.TextIOWrapper'>, <class '_io._TextIOBase'>, <class '_io._IOBase'>, <class 'object'>)
>>> sys.stdin.__class__
<class '_io.TextIOWrapper'>
>>> sys.stdin.__next__
<method-wrapper '__next__' of _io.TextIOWrapper object at 0x7f64741bead0>
>>> sys.stdin.__class__.__next__ = lambda *_: 'foo'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't set attributes of built-in/extension type '_io.TextIOWrapper'

So I can't even confirm if it's the special method lookup problem or something else like next skipping straight to some C method. But either way I think the solution is to entirely replace sys.stdin with an object of a custom class in basically the same way that we replace stdout/stderr.

[alexmojaki]

Released a fix in python_runner 0.6.1

[winniederidder]

Integrated by re-deploying, so this issue has been resolved. Thanks @alexmojaki

[pdawyndt]

I would suggest to raise a StopIteration when reaching the end of batch input.

[winniederidder]

We chose to keep the batch input editable at runtime. Therefore, it would make no sense to raise StopIteration when reaching the "end", as there actually is no end. The actual behaviour of sys.stdin as an iterable is also to keep going. Users using this method always need some sort of while-loop to break out of this themselves

[pdawyndt]

I would think of it as redirecting batch input as a file to stdin, as an alternative to the interactive input. If we keep it this way, at least there should be an explicit EOF option when providing interactive input (achting as ctrl+D on command line), and an option to get an EOF after the last line of batch input.

[winniederidder]

When trying this locally in a terminal, Ctrl-D does not work while iterating over sys.stdin, so this would seemingly deviate from the standard. As mentioned earlier, exercises in Dodona will usually ask the user to use a while-loop that tests for something like line == "stop". Furthermore, the batch input is editable at runtime, meaning the StopIteration by default would prevent the user from giving the expected input in the same run if the missing line is noticed. Finally, the runtime environment does not know of the existence of any input modes (as we opted to not provide it at all at the start, because it is editable).
In conclusion, I'm not convinced that the benefits outweigh having to deviate from the standard and how input is handled in Dodona.