soda480/mplogp

A log parser for parsing logs generated from multi-processing based tools.

mplogp

A Python script to parse a logfile generated from multi-processing based tools. The script will parse the logfile and create logs for each process under a specified timestamped folder. Supports log files generated from a log Formatter whose first two fields are: %(asctime)s %(processName)s ....

mplogp

usage: mplogp [-h] --log LOG --folder FOLDER [--regex REGEX]

A Python script to parse a logfile generated from multi-processing based tools

optional arguments:
  -h, --help       show this help message and exit
  --log LOG        The location of the logfile to parse
  --folder FOLDER  The folder where to write the parsed logs
  --regex REGEX    Regular expression to alias process log filename - must contain a matched group

Examples

Parse the example3.log file and write the parsed logs under the logs folder, add alias to each process log with its matched processor id:

mplogp --log example3.log --folder logs --regex ".*processor id (.*)$"

The log file contained logs from 10 processes and produced the following logs:

logs
└── 2021-05-13_18-48-46
    ├── MainProcess.log
    ├── Process-1-69b41899.log
    ├── Process-10-c40f3916.log
    ├── Process-2-a6ffc5b5.log
    ├── Process-3-14175157.log
    ├── Process-4-90e875c2.log
    ├── Process-5-ef35f44f.log
    ├── Process-6-c7268544.log
    ├── Process-7-7557d6d6.log
    ├── Process-8-e248e861.log
    └── Process-9-0bb23bab.log

Development

Build the Docker image:

docker image build \
-t \
mplogp:latest .

Run the Docker container:

docker container run \
--rm \
-it \
-v $PWD:/code \
mplogp:latest \
bash

Build the project:

pyb -X