Use Default Loggers/Handlers with Abseil `app.run`

[felixy12]

Hi, I am currently using abseil flags in my codebase, which involves using app.run. As mentioned in #148, this causes all existing root stderr handlers to be replaced with the absl handler instead.
Below is a minimum working example displaying this phenomenon:

from absl import app
import logging

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
logging.basicConfig()

def main(argv):
    logger.info("Run after app.run")
    print(logging.root.handlers)


if __name__ == "__main__":
    logger.info("Run before app.run")
    print(logging.root.handlers)
    app.run(main=main)

Output:

INFO:__main__:Run before app.run
[<StreamHandler <stderr> (NOTSET)>]

I1027 13:27:51.273060 139686705379136 test_case.py:10] Run after app.run
[<ABSLHandler (NOTSET)>]

Since the rest of the codebase uses standard python logging, I'd ideally like to not deal with absl's version of logging/handlers. Is there a way to avoid getting all root handlers replaced?

On a related note, is there a reason this replacement is currently implemented the way it is?

[yilei]

Currently using app.run() assumes you also want absl's log handlers (and formatter), unfortunately. It also uses some heuristics to remove existing handlers, which are commonly automatically added by calling standard logging.warning functions. This must be done to avoid log duplications when you don't configure your own handler (much more common), but it also means it makes harder for your case (much less common). Unfortunately there is no way to tell whether those handlers were added this way, or done by an explicit call of logging.basicConfig().

To use your own handler together with app.run, you have to configure the handlers inside your main() function (and replace absl's).

Is there a specific reason you want to use app.run?

[felixy12]

For why app.run is used, we like abseil's flag handling for its modular properties, so we've been using it over the generic argparse. It's possible to switch off of it, but in an ideal world we'd have a solution where we could continue to use the abseil flag module without being explicitly tied to its logging module.

A solution we've found for logging that is in our control is to specify our own handler and to have our loggers not propagate to root.

formatter = logging.Formatter(fmt=logging.BASIC_FORMAT)
console_handler = logging.StreamHandler()
console_handler.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.addHandler(console_handler)
logger.propagate = False

The main concern that I have now comes from 3rd party packages which may use the default logging module, and cannot do the above workaround. What are the potential issues that come from having root handler replaced in these cases?

[yilei]

You could do something like this at the beginning of your main function to replace absl's handler with yours:

absl_handler = absl.logging.get_absl_handler()
if absl_handler in logging.root.handlers:
  logging.root.removeHandler(absl_handler)
  logging.root.addHandler(console_handler)

This also works for logs coming from 3rd party packages. That's also the reason why absl attaches the absl handler to root, to work with 3rd party libraries.

Longer term, we want to support Abseil Python integration with Abseil C++ Libraries, and part of that integration is to synchronize the logs from both languages. Internally we already have that, but we haven't been able to release them. In designing that, we might have to add an option (an enum) to tweak how logs are handled, e.g. in the absl.app.run(), and one of the option can be, don't use absl's logging handler at all.)

[felixy12]

I appreciate the clarification and addressing the concern! From what you're saying, the replacing of the root handler will most likely not negatively impact 3rd party packages (and if it does, I'll open a separate issue). I will close the issue since a workaround was found (specified in my above comment). I would personally love it if there was an option in app.run() to be configurable to not replace handlers in the future. Thanks!