Spell optimization in python using multiprocessing
sandyflute opened this issue · 1 comments
sandyflute commented
Thanks for sharing this repo.
Has anyone tired optimizing the parsers ? Spell/drain ?
It is taking lot of time to create templates for large logs.
DerDoni commented
Seconding this,
just adapted spell to my workflow and compared to my ibm/drain3 derived drain implementation it is insanely slow.
Drain parsing: 5 minutes, Spell parsing 2h.
Not only is the spell implementation not optimized for multi-processors, there are also many expensive and inefficient python function calls that add up.
The whole LCS implementation is questionable and I wonder how they did their experiments for their tools and benchmarks paper.