OSError: [Errno 48] Address already in use

[ChristianMichelsen]

Hi and thanks for a great library!

I have a problem where I want to run separate jobs, each using logger_tt. When I do that, I run into the following error:

OSError: [Errno 48] Address already in use

How do I specify to use another address or port?

My setup is to run:

from logger_tt import setup_logging, logger
from datetime import datetime

now = datetime.now()
now_str = now.strftime("%Y-%m-%d__%H-%M")
log_path = f"logs/log__{now_str}.log"

setup_logging(
    config_path=here_my_config_path,
    log_path=log_path,
)

with the following config file:

version: 1
disable_existing_loggers: False
formatters:
  simple:
    format: "[%(asctime)s] | %(name)s:%(lineno)d | %(levelname)s | %(message)s"
    datefmt: "%Y-%m-%d %H:%M:%S"
  brief:
    format: "[%(asctime)s] | %(levelname)s | %(message)s"
    datefmt: "%Y-%m-%d %H:%M:%S"

handlers:
  console:
    class: logging.StreamHandler
    level: INFO
    formatter: brief
    stream: ext://sys.stdout

  error_file_handler:
    class: logging.handlers.TimedRotatingFileHandler
    level: DEBUG
    formatter: simple
    filename: logs/log.txt
    backupCount: 15
    encoding: utf8
    when: midnight

loggers:
  urllib3:
    level: WARNING
    handlers: [console, error_file_handler]
    propagate: no

root:
  level: DEBUG
  handlers: [console, error_file_handler]

logger_tt:
  suppress: ["exchangelib", "numba", "matplotlib", "absl"]
  suppress_level_below: "WARNING"
  capture_print: False
  strict: False
  guess_level: False
  full_context: 0
  use_multiprocessing: True
  limit_line_length: 1000
  analyze_raise_statement: False
  default_logger_formats:
    normal: ["%(name)s", "%(filename)s"]
    thread: ["%(message)s", "%(threadName)s %(message)s"]
    multiprocess: ["%(message)s", "%(processName)s | %(message)s"]
    both: ["%(message)s", "%(processName)s %(threadName)s %(message)s"]

[AlphonsG]

Hello, I haven't tried using a different port, but if on Ubuntu if you run lsof-i:9020 on the terminal and then kill the processes with the IDs that show up, you'll be able to run your application with logger_tt again. Maybe a (filthy?) way of avoiding this manual process would be to have your python code run this command before initializing the logger? You can also read the source code to figure out if you can avoid binding the logger to addresses/ports, but not sure if it's possible.

By the way, the OSError only seems to happen on Linux and not Windows for me, and only if my python app using logger_tt is killed without properly exiting.

[Dragon2fly]

@ChristianMichelsen
Can you tell me which python version you are using?
Also, did your program exit normally from the previous run before you re-run it and encountered this problem?

As @AlphonsGwatimba said if it does not exit completely and gratefully, the port may still be in use.

If you run 2 separate programs at the same time, the second one may have this problem.
A workaround is to set the port differently before calling setup_logging.

from logger_tt import setup_logging, internal_config

internal_config.port = 9021         # change this number to suit your need
setup_logging()

You may even be tempted to set the port to 0 for the system to pick any free port for you.
That may not work since the spawn method of creating the child process will call that code again.
And thus, the system may pick up a different port for the child process.

I'll look into the case of running 2 separate programs both with multiprocessing enabled.

[ChristianMichelsen]

Right now I am running Python version 3.8.7.

I think I have managed to get a fix working by using the internal_config trick as you mentioned, so thanks a lot!

Maybe this should be mentioned in the docs?

[Dragon2fly]

The name of the object, internal_config, implies that the user doesn't need to care about it.

I'm thinking of different ways to solve this problem.
To expose this port setting to users or should they care about it at all?

In the end, people will only notice the port problem when they did not shut down the previous run properly.

[ChristianMichelsen]

I think it would be great to expose the port as a user-configurable setting, although I also recognise that most users should just use the default port.

I disagree about your last statement; it also affects users that run two programs at the same time.

[Dragon2fly]

Hi @ChristianMichelsen,

With v1.7.0, you can officially set host and port for each program separately through setup_logging or log config file.

Ideally, you should be able to pick a random port and pass it to setup_logging each time your program runs. But right now there is no way to communicate the dynamic picked port to the child processes. So the port is still a fixed number.

If you find a way to communicate the port number to child processes, a pull request is welcomed.

Have a nice weekend

[ChristianMichelsen]

This seems to be working. Thanks a lot!