TCP log listener server minimally adapted from Vinay Sajip's Python logging cookbook.
This is a standalone process, as described by Vinay:
"When deploying Web applications using Gunicorn or uWSGI (or similar), multiple worker processes are created to handle client requests. In such environments, avoid creating file-based handlers directly in your web application. Instead, use a SocketHandler to log from the web application to a listener in a separate process. This can be set up using a process management tool such as Supervisor - see Running a logging socket listener in production for more details."
There are several SO posts (see for example here and here incidentally answered by Vinay) around this issue, but remarkably, Django does not mention this issue in its documentation.
log_listener.py is a straight copy of Vinay's code for a server listening for log records on a TCP socket.
Log files will be kept in the sub-folder logs/.
ATTENTION: the transmission is not encrypted and this is therefore not suitable for use over a public network A log server requires a process to run it, taking four arguments:
python3 log_listener.py --help
-p PORT, --port PORT Which TCP socket port (default: 9020)
-fn FILENAME, --filename FILENAME Path to log file (default: log_file)
-l LEVEL, --level LEVEL Logging level (default: DEBUG)
-f FORMAT, --format FORMAT Logging string format (default: %(asctime)s [%(name)-15s] [%(levelname)-8s] %(message)s) `
This long-running process is best served by having it overseen by supervisor and log_listener.conf is a
configuration file for supervisor to handle three instances; one for default logging, one for Django and one for
audit logging.
Finally, as log file management has now been decoupled from Django, log_listener_logrotate.conf is a configuration
file for logrotate to rotate the log files daily. This is an intermittent operation, best set up as a cronjob. For
example:
5 12 * * * /usr/sbin/logrotate log_listener_logrotate.conf --state /home/cloo/log_listener/logrotate-state
Specifying a custom state file for the log rotation resolves an otherwise possible permission issue with the default
location /var/lib/logrotate/status.tmp.