process multiple files in parallel

[avh4]

Processing of each file is independent. Would it be faster overall if they were allowed to run in parallel? I'd expect that elm-format is mostly disk-bound, but maybe that's not the case with SSD drives or with caching in modern systems?

If it can be faster, we'd want to be careful to use a haskell library that would appropriately manage resources and not start additional threads if CPU usage or memory usage is already high.

[lydell]

This would be great, I think!

At work we have 726 Elm files. Letting one elm-format --validate check them all takes 15 seconds, while spreading the files out evenly on multiple elm-format --validate calls in parallel takes down to 4 seconds.

I tried a bunch of different numbers of parallel elm-format --validate, up to 12 which is the number of cores on my laptop.

• 1: 14.37 s
• 2: 8.26 s
• 4: 4.88 s
• 6: 4.31 s
• 8: 3.96 s
• 10: 3.93 s
• 12: 3.85 s

Conclusions:

• There looks to be a speedup from parallelization here!
• Even using just 2 cores gives a speedup, so even GitHub Actions would benefit.