Using Step inside [OneTimeSetUp] fails with `Value cannot be null`

[Lewka]

Hello there, I've been playing with new versions (not released ones) and
I've faced an issue, when I'm trying to use AllureStep inside a OneTimeSetUp method, it fails with following error:

OneTimeSetUp: System.ArgumentNullException : Value cannot be null. (Parameter 'key')

I'm submitting a ...

• bug report

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

    [AllureNUnit]
    [AllureParentSuite("Root Suite")]
    public class BaseTest
    {
        [OneTimeSetUp]
        public void CleanupResultDirectory()
        {
            AllureExtensions.WrapSetUpTearDownParams(() => { AllureLifecycle.Instance.CleanupResultDirectory(); },
                "Clear Allure Results Directory");
            Method(); // FAILS
        }

        [SetUp]
        public void Setup()
        {
            Method(); // doesnt fail
        }

        [AllureStep()]
        private void Method()
        {
        }
    }

What is the expected behavior?

I can use Step inside OneTimeSetUp

Please tell us about your environment:

• Allure version: 2.9.1-preview.6-nunit-fixtures

Other info

Looks like the problem in AllureNUnitAttribute class, the BeforeTest method is called only after OneTimeSetUp, so Allure thinks there is no tests started yet when we call a method with Step attribute inside OneTimeSetUp method, that's why it fails.

[neparij]

Hello there, @Lewka !
In order to use steps within OneTime* methods we need to have test context (and steps context).
NUnit provides it only after OneTimeSetUp method being called (using ITestAction interface, either using event listener).

[Lewka]

@neparij thanks for the reply. Yeah, I've got that (and I described it in Other info section), so my guess it the final implementation? If so, could you throw then an exception that would tell user what's going on? As of now it's not clear that you cannot use Step inside OneTime methods (as I remember you can do so in Java)

[neparij]

Throwing exception is not available due to same reasons.
I will describe it in README, so this behaviour will be documented as well :)

[tkeerthivel]

@neparij I'm facing the same issue with XUnit. In XUnit we use constructors for one time setup. In my scenario, I am not able to use Steps inside the constructor. Version: 2.9.2-preview.1. (With the latest release (2.1.3) it is working fine ) Any suggestions?

[neparij]

@tkeerthivel hi!
I'll check it out using constructors.
As a workaround you can implement IAsyncLifetime interface and use InitializeAsync method (returning Task)

[vladdex]

@neparij
This problem happens even if you have a base class decorated with [AllureNunit] containing only [OneTimeSetup] or a class that's defined as a Setup Fixture. This is problematic because without decorating these classes the report lacks the setup and teardown logs.
In version 2.9.1-preview 5 the exception is not thrown, only a log entry is being made.
Any way we can get something similar in place until a more permanent fix is found ?

[delatrie]

Hi, everyone!
We've just released a version that fixes lots of context-related issues including this one. Please, try it: 2.10.0.

Please, note, that currently it's mandatory to apply [AllureBefore] or [AllureAfter] to [OneTimeSetUp]/[OneTimeTearDown] methods if you want to use steps inside them:

[TestFixture]
[AllureNUnit]
class MyTestClass
{
    [OneTimeSetUp]
    [AllureBefore]
    public void OneTimeSetUp()
    {
        this.MyStep();
    }

    [OneTimeTearDown]
    [AllureAfter]
    public void OneTimeTearDown()
    {
        this.MyStep();
    }

    [AllureStep("MyStep")]
    public void MyStep() {}

    [Test]
    public void MyTest() {}
}

If the bug still exists, please, let me know here.

[DenisKorbovskyTds]

Hello @delatrie, is it expected that allure-results files are not generated (only empty folder) if OneTimeSetUp marked with AllureBefore in case of failure in this method? When OneTimeSetUo method passes steps from OneTimeSetUp are in Set Up section of report but if it fails report files are not generated at all

[delatrie]

Hi, @DenisKorbovskyTds
It happens because no test is run by NUnit once OneTimeSetUp fails (regardless of AllureBefore) Hence, there are no test results. It might be possible to create skipped results or at least create a surrogate result that holds the information about the error. But that needs to be implemented.
I've created a feature request on that matter: #479.

PS: it's better to ask questions like that in discussions. You will receive an answer quicker that way.