arquillian/arquillian-governor

Tests with timeout are not skipped by Governor

Opened this issue · 10 comments

jpechane commented

Modify ITJiraGovernorTest to have method

    @Test(timeout = 60000)
    @Jira("ARQ-831")
    public void willBeSkippedBecauseUnresolved()
    {
        Assert.assertTrue(false);
    }

The test then fails

smiklosovic commented

does it fail just when that timeout is there or if there is any other value in it? e.g. expected exception or a message

jpechane commented

The problem is that the test is executed although it should not be.
The ExecutionDecision is correctly evaluated but it seems that when the
timeout is present it is added to the list of executions again.

On Fri, May 13, 2016 at 9:45 AM, Štefan Miklošovič <notifications@github.com

wrote:

does it fail just when that timeout is there or if there is any other
value in it? e.g. expected exception or a message

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#24 (comment)

Jiri Pechanec
QA Engineer
Red Hat
Brno, Czech Republic

jpechane commented

It seems it is timeout specific. The other paramter supperted by @test is
expected exception and using this one it work correctly.

On Fri, May 13, 2016 at 9:59 AM, Štefan Miklošovič <notifications@github.com

wrote:

yeah i got it, the question is whether it fails only in case there is
timeout if there is any other value in test annotation set

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#24 (comment)

Jiri Pechanec
QA Engineer
Red Hat
Brno, Czech Republic

smiklosovic commented

could you post the stacktrace please?

jpechane commented 
diff --git a/jira/src/test/java/org/arquillian/extension/governor/jira/ITJiraGovernorTest.java b/jira/src/test/java/org/arquillian/extension/governor/jira/ITJiraGovernorTest.java
index ee2bd3e..13c2cf5 100644
--- a/jira/src/test/java/org/arquillian/extension/governor/jira/ITJiraGovernorTest.java
+++ b/jira/src/test/java/org/arquillian/extension/governor/jira/ITJiraGovernorTest.java
@@ -34,7 +34,7 @@ public class ITJiraGovernorTest
         Assert.assertTrue(true);
     }

-    @Test
+    @Test(timeout = 30000)^M
     @Jira("ARQ-831")
     public void willBeSkippedBecauseUnresolved()
     {

maven log is attached
mvn.log.zip

smiklosovic commented

@basovnik could you look at this issue, please?

basovnik commented

The problem is on this line: https://github.com/arquillian/arquillian-core/blob/master/test/impl-base/src/main/java/org/jboss/arquillian/test/impl/EventTestRunnerAdaptor.java#L170
Variable serviceLoader is null. From some reason no context is active and therefore no TestExecutionDecider is resolved.

If you try to execute this code before mentioned line then it will work. But I have not investigated yet why the ApplicationContextImpl is inactive.

if (!manager.getContext(ApplicationContextImpl.class).isActive()) {
    manager.getContext(ApplicationContextImpl.class).activate();
}
basovnik commented

I found one possible fix:
https://github.com/arquillian/arquillian-core/blob/master/core/spi/src/main/java/org/jboss/arquillian/core/spi/context/AbstractContext.java#L38
We can change ThreadLocal to InheritableThreadLocal and it will work. When we use timeout attribute then new thread is started...
https://github.com/junit-team/junit4/blob/r4.12/src/main/java/org/junit/internal/runners/statements/FailOnTimeout.java#L125

@smiklosovic Shall I create pull request to arquillian-core with my fix or do you have some other idea?

basovnik commented

The solution will be not so easy... https://issues.jboss.org/browse/ARQ-1071