Client is not nicely parallelizable
Opened this issue · 0 comments
davisjam commented
runspec.py
uses eventlet for concurrency, which uses green threads under a single Python process (under the GIL). Thus only one core will be used by a single runspec.py instance. To saturate the server I need to use multiple cores.
There appears to be a multi-instance mode for the client (runspec.py) but it is undocumented. It does seem to be used in run_multiple_instance.sh
though.
I prefer a "one-stop shop" client like ab
that handles multiple threads itself.
I created a patch that uses the multiprocess
module for the requestBased mode here. Each process runs the existing client logic (eventlet pool, etc.), and runspec.py then merges the resulting log files for a comprehensive summary.
Thoughts? Interested?