Concurrency Options

[WVAviator]

The level of concurrency with a test suite or a collection of suites should be customizable in some manner.

Sequential testing may be desired in cases where requests may follow a logical pattern - for example:

1. Authenticate
2. POST a new resource
3. GET the new resource that was created
4. PATCH the resource
5. GET the resource again and verify the updates
6. DELETE the resource
7. GET the resource again to verify it no longer exists

This workflow would not work with concurrency. Inversely, making a series of unrelated GET requests to different endpoints can be concurrent, and doing so would speed up testing.

Perhaps a suite-level option, something like an optional "parallel: true". I do think that sequential should be the default for all tests in a suite, since it is probably the most intuitive and unlikely to cause issues if the option is not specified. However, at the suite-level, I think that suites should by default run in parallel, and the user should be encouraged to write their test suites in isolation. Optionally a command-line argument could be supplied to run them one-at-a-time.