Log/print execution status to stdout
wenting18 opened this issue · 2 comments
Feature request
Currently, the execution status is being written to stderr. This does not show up when running bonobo etl jobs using certain programs (such as docker-compose).
In console.py, there are print statements within the write function that writes to stderr:
print(prefix + _line + CLEAR_EOL, file=self._stderr)
It would be useful to have a configurable flag that can redirect the prints to stdout, or better yet allow these statements to be written as log messages.
A way that was found to work is creating a flag in settings.py and set the prints within the write function in console.py as an instance of a logger:
def write(self, context, prefix='', rewind=True, append=None):
...
for i in context.graph.topologically_sorted_indexes:
...
if settings.LOG_EXECUTION_STATUS:
logger = logging.getLogger()
print = logger.info
print(prefix + _line + CLEAR_EOL)
...
...
Of course, this approach alters the print calls and changes the behaviour of all the prints which may have further consequences.
Versions
-
Bonobo version:
bonobo v.0.6.3 -
Python version:
python 3.7 -
Platform:
Darwin Kernel Version 17.4.0
The fact bonobo writes to stderr is indeed a choice, so that you can redirect stdout to something and only get your own things there, while still getting the status output.
Probably, the absence of output is related to the fact that your execution process is not in a tty/pseudo-tty. Bonobo does not output status by default for non-tty executions.
How would you get the output when running in a docker container?