Make progress bar separate from log level
rabernat opened this issue · 2 comments
I am running map jobs with Lithops with O(100000) tasks. I greatly appreciate the ability to have a progress bar to monitor completion.
However, currently, the progress bar only shows up if the log level is exactly equal to INFO (not less or not more):
Lines 116 to 117 in 033e725
The problem here is that INFO also prints out a message for every single task I start:
2024-05-02 16:37:34,528 [INFO] invokers.py:174 -- ExecutorID 55eecc-0 | JobID A006 - Starting function invocation: process_chunk() - Total: 1 activations
That means if I want to have a progress bar for my 100000 tasks, I also need to accept 100000 lines being printed to my console. That is definitely NOT what I want.
I propose to simply remove the log-level check from the code cited above and make progress bar completely separate from log level.
@JosepSampe will you be able to look into this?
The idea behind this was not to show the progress bar during DEBUG logs, as DEBUG logs show a lot of lines, and print the progress bar is a bit messy. I agree in your case is also a bit messy, so we can remove the logger.getEffectiveLevel() == logging.INFO
check as you did in the PR. In addition, IMO, I would automatically disable the progress bar if logs are DEBUG
to keep the same behavior we had up until now.