logpai/logparser

Spell optimization in python using multiprocessing

sandyflute opened this issue · 1 comments

Thanks for sharing this repo.

Has anyone tired optimizing the parsers ? Spell/drain ?

It is taking lot of time to create templates for large logs.

Seconding this,
just adapted spell to my workflow and compared to my ibm/drain3 derived drain implementation it is insanely slow.
Drain parsing: 5 minutes, Spell parsing 2h.

Not only is the spell implementation not optimized for multi-processors, there are also many expensive and inefficient python function calls that add up.

The whole LCS implementation is questionable and I wonder how they did their experiments for their tools and benchmarks paper.