[rush] Feature Request: Add support for sharding

[dmichon-msft]

Summary

Jest (and other tools) support the concept of shards, wherein by passing an argument on the CLI like --shard K/N, the tool performs a deterministic partition of the work to be performed into N parts and executes the Kth such partition. This allows parallelization and scheduling to be controlled by an external orchestrator instead of relying on Jest's internal workers.

Rush should support a configuration option in rush-project.json to tell Rush that a particular phase supports sharding (with either some heuristic for selecting shard count, or a value written in the config). When the option is specified, during graph construction, Rush should split the operation into N shards that can be scheduled independently to more correctly reflect the active CPU usage.

Details

To facilitate interaction with the build cache, cache restore will need to happen as a dependency of all the shards, and cache write as a consumer of all shards, otherwise there will be issues with overlapping outputs.