Wrong execution order when using SpecificationWithJUnit with BeforeAfterAll on specs2 version 4.8.2 and up

[daniellasha]

I’m running tests using SpecificationWithJUnit with BeforeAfterAll trait on Scala 2.12.7, sbt 1.5.8, specs2 4.16.1.

When I have SpecStructure which contains several Specifications, these Specifications always run before the beforeAll() method.
See in the code example below, the order of the execution should be:

1. beforeAll()
2. Tests1
3. Tests2
4. afterAll()

However Tests1 & Tests2 are running first and then beforeAll() and afterAll().
I see this issue reproduce from specs2 version 4.8.2 (works ok in 4.8.1).

class AzureTest extends SpecificationWithJUnit with LoggerBase with BeforeAfterAll {
    override def beforeAll(): Unit = {
        logger.info("Test started ")
    }

    def is: SpecStructure = sequential ^ s2"""
          ${"Tests Specification #1" ~ Tests1}
          ${"Tests Specification #2" ~ Tests2}
         """

   override def afterAll(): Unit = {
         logger.info("Test finished ")
    }
}

  object Tests1 extends Specification {
    def is: SpecStructure = sequential ^ s2"""
      test case 1     $e1
      test case 2     $e2
      test case 3     $e3
      """

    def e1: MatchResult[Any] = {
      ok
    }
    def e2: MatchResult[Any] = {
      ok
    }
    def e3: MatchResult[Any] = {
      ok
    }
}

  object Tests2 extends Specification {
    def is: SpecStructure = sequential ^ s2"""
      test case 4     $e4
      test case 5     $e5
      test case 6     $e6
       """
       
    def e4: MatchResult[Any] = {
      ok
    }
    def e5: MatchResult[Any] = {
      ok
    }
    def e6: MatchResult[Any] = {
      ok
    }
  }

[etorreborre]

Hi @daniellasha. I think this scenario is best supported by embedding the examples of each spec into the main one:

class AzureSpec extends Specification with BeforeAfterAll {
    def beforeAll() =
      println("Test started ")

    def is = sequential ^ s2"""
          Tests Specification #1
          ${Tests1.is}

          Tests Specification #2
          ${Tests2.is}
         """

   def afterAll() =
     println("Test finished ")
}

object Tests1 extends Specification {
  def is = sequential ^ s2"""
    test case 1 ${e(1)}
    test case 2 ${e(2)}
    test case 3 ${e(3)}
  """

  def e(n: Int) = { 
    println(n)
    n === n.pp 
  }
}

object Tests2 extends Specification {
  def is = sequential ^ s2"""
    test case 4 ${e(4)}
    test case 5 ${e(5)}
    test case 6 ${e(6)}
  """

  def e(n: Int) = { 
    println(n)
    n === n.pp 
  }
}

Two drawbacks:

• it will make the AzureSpec pretty large
• you will have to exclude Test1 and Test2 from execution depending on your build tool to avoid those specs to be executed twice

But the execution order will be what you expect. I think it might have seemed to work in previous version, the way you wrote it, but it was purely by accident. I need to think more about how to support your example. Part of that was the introduction of a notion of "global resources" in specs2-5 to allow several concurrent specifications to share a global resource, initialized once and finalized once at the end of the run.

[etorreborre]

I am going to close this issue because I don't think I can really do something sensible in the 4.x series. I would recommend that you try out specs2-5.x with global contexts if you can.