Testing the behavior of CircleCI parallelism option
In order to run CircleCI steps in parallel, you need to set a value greater than 1 to the parallelism option first.
version: 2
jobs:
test:
docker:
- image: circleci/node:latest
parallelism: 3
...Then you can use the CircleCI CLI for spliting tests
circleci tests glob "__test__/*.spec.ts" | circleci tests split --split-by=timingCircleCI advertises that by passing --split-by=timing to the circleci tests split command, your tests will be split evenly.
To split by test timings, use the --split-by flag with the timings split type. The available timings data will then be analyzed and your tests will be split across your parallel-running containers as evenly as possible leading to the fastest possible test run time
There are many strategies that can be potentially used for splitting tests.
One strong candidate is the Greedy Algorithm but it's not always the most optimized one.
So I decided to run the following tests in order to investigate the CircleCI's behavior.
I prepared the following tests.
I investigated how these tests are split when the parallelism option is set as 3.
If CircleCI chooses the most optimized strategy, then the tests should complete within 20 seconds.
- container 1: delay 20 seconds =
20 secnds - container 2: delay 12 seconds + delay 8 seconds =
20 seconds - container 3: delay 10 seconds + delay 6 seconds + delay 4 seconds =
20 seconds
CircleCI seems to choose the Greedy Algorithm and it ended up with the following result
- container 1: delay 20 seconds =
20 secnds - container 2: delay 12 seconds + delay 6 seconds + delay 4 seconds =
22 seconds - container 3: delay 10 seconds + delay 8 seconds =
18 seconds
