trivago/cucable-plugin

Grouping scenarios to prevent that scenarios from a group runs at the same time

boris779 opened this issue · 10 comments

boris779 commented

name: Grouping by tags (and run this group serial)
about: It would be great if it would be possible to specify that one group of scenarios will run on the same Runner. (via cucumber tags)

Is your feature request related to a problem? Please describe.
We have some scenarios which should not run in parallel at the same time. Switching to feature-base parallelization does not allow to use cucumbertags.

Describe the solution you'd like
Define a group of scenarios (via cucumbertags) which runs either on the same runner or maybe cucumable will prevent that these scenarios runs in parallel.

Additional context
We have 4 scenarios which reset data in a microservice. While one of these is running, no others should run to prevent issuses with resetting the data.

bischoffdev commented

I will look into this request next week - I am currently on holidays 🙂

  • 👍1
boris779 commented

A solution could be parallelization mode to scenarios and allow excluding with tags --> remove all @exluded scenarios from the runners.
Another approach could be to add a parameter for include/exclude tagged features (not scenarios)
Last idea from my side: add a parameter for a group-prefix (e.g. group) and all scenarios with the same group-tag (e.g. @group_a) are running on the same runner or are added to the same runner-file.

bischoffdev commented

Interesting ideas, @boris779 - I will consider those as soon as I have a little more time on my hands again.

boris779 commented

If I can contribute let me know.

  • 👍1
bischoffdev commented

I will add that to the next release.

  • 👍1
boris779 commented

What does it mean, how long will it take? I'm just asking because in our sprint which will start today (and takes two weeks) we have the story in our backlog to contribute that. Should we wait for your implementation or should we start and make a pull request. We should not do it twice :-)

bischoffdev commented

If you have this planned, you can contribute any time. I am struggling to understand your proposed solution #1 though:

A solution could be parallelization mode to scenarios and allow excluding with tags --> remove all @exluded scenarios from the runners.

boris779 commented

As I said we did a spike in our sprint to see how cucable works and how we can implement the grouping with tags to prevent parallel runs with „grouped tagged“ Features. There are some possibilities we’ve discovered so far:

  1. First place
    a. We add an additional Maven Parameter which defines the „grouptags“
    b. When creating the Features a Kind of a Pair will be returned instead of only the feature names (in generateParallelizableFeatures).
    c. Another possibility, we’ve considered was wrapping the generatedFeaturePaths and the new groupedByGeneratedFeaturePaths into a wrapper object which contains both Lists. With this approach, the Impact to the Code would’ve been as big as our first approach.
    d. These three solutions have a massive impact on the Code so we stopped researching on that.
  2. Split and Merge
    a. After the creation of all Features, we get all the tags for every feature and instead of generating a runner for each we „summarize“ some runners
  3. Split and Merge 2nd
    a. As solution 2, before the runners are generated after the Scenarios are split we merge scenarios which have the same „grouptags“
  4. Resource Lock
    a. While execution we take care of what is actually running, but this part of surefire so we didn’t look further into it

For us very important is that we create a solution which fits also to you to get the changed into the master, we want that you accept our PR 😊

Best regards
Boris

bischoffdev commented

Hello @boris779 ,

after a lot of thinking about this project, I decided to make one final release (which should be out today) and then gradually phase out this project.

The reasons are:

  1. Cucable is basically obsolete with Cucumber 5 being able to parallelize natively (even on Scenario level)
  2. I need more time to concentrate on Cluecumber (apart from my regular work) as it is the more important and more widely used project.

Also, after reading about your problem and the proposed solutions again, I have the strong feeling that this is a very specific use case for you that should not be added to Cucable's feature set. In general, scenarios should always be independent of each other and never depend on others.

A quick solution to your grouping problem could be this:

  • Have different features containing tagged scenarios
  • Use the feature based parallelization but handle the tag filtering in the runner and not via Cucable

This wouldn't need any code change in Cucable.

If you however want to add the feature you have been discussing, feel free to fork and continue it. Personally, I will only add bugfixes in the future if necessary.

Thanks for your help,
Benjamin

bischoffdev commented

Closed due to no reaction.