Stop skipping certain tests under Windows
Opened this issue · 13 comments
Guava tests build is failing on windows. PFB the failure snippet from surefire-reports.
guava-tests/target/surefire-reports/com.google.common.base.ThrowablesTest.txt:Tests run: 30, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.026 sec <<< FAILURE!
Binary file guava-tests/target/surefire-reports/com.google.common.io.ResourcesTest.txt matches
guava-tests/target/surefire-reports/com.google.common.reflect.ClassPathTest.txt:Tests run: 32, Failures: 5, Errors: 0, Skipped: 0, Time elapsed: 0.463 sec <<< FAILURE!
And here is the text for those two files
Test set: com.google.common.base.ThrowablesTest
Tests run: 30, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.026 sec <<< FAILURE!
testGetStackTraceAsString(com.google.common.base.ThrowablesTest) Time elapsed: 0.002 sec <<< FAILURE!
junit.framework.AssertionFailedError: null
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at com.google.common.base.ThrowablesTest.testGetStackTraceAsString(ThrowablesTest.java:498)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:35)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:115)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.maven.surefire.booter.ProviderFactory$ClassLoaderProxy.invoke(ProviderFactory.java:103)
at com.sun.proxy.$Proxy0.invoke(Unknown Source)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:150)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:91)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)
Test set: com.google.common.reflect.ClassPathTest
Tests run: 32, Failures: 5, Errors: 0, Skipped: 0, Time elapsed: 0.463 sec <<< FAILURE!
testGetClassPathEntry(com.google.common.reflect.ClassPathTest) Time elapsed: 0 sec <<< FAILURE!
junit.framework.AssertionFailedError: expected:file:/C:/usr/test/dep.jar but was:file:/usr/test/dep.jar
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.failNotEquals(Assert.java:283)
at junit.framework.Assert.assertEquals(Assert.java:64)
at junit.framework.Assert.assertEquals(Assert.java:71)
at com.google.common.reflect.ClassPathTest.testGetClassPathEntry(ClassPathTest.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:35)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:115)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.maven.surefire.booter.ProviderFactory$ClassLoaderProxy.invoke(ProviderFactory.java:103)
at com.sun.proxy.$Proxy0.invoke(Unknown Source)
at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:150)
at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:91)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)
testResourceScanner(com.google.common.reflect.ClassPathTest) Time elapsed: 0.279 sec <<< FAILURE!
-Ankush
Thanks for filing the bug!
ClassPathTest is a pretty clear case of our assuming that we're running on a Linux-style filesystem. ThrowablesTest is more interesting -- probably a bad assumption that newlines are always represented by \n.
@cgdecker @cgruber Do we have any way of running our tests regularly on Windows? These shouldn't be hard to fix, but it would be good to have an easy way to test them and a way to make sure errors won't creep back in.
No problem..!:)
Let me know if you want me to fix this.
Sure, we are really not Windows-enabled here, so we would greatly appreciate any patches.
@kevinb9n, I understand that you cannot run any tests on Windows internally, but I've noticed that for the Github-hosted version of Guava you use Travis CI a lot (I don't know if you also use Travis internally).
This makes me wonder if Google policy would allow you to use additional external CI services, and if so whether you'd consider using AppVeyor.
It seems to be a rather good complement to Travis, as it allows one to run CI tasks on Windows VMs, and it seems to be free for open source projects.
What are your thoughts on this?
On a related note, is the bug in ThrowablesTest::testGetStackTraceAsStringcurrently being worked on?
If not, then I have a relatively trivial fix for the bug which I'd be happy to submit a Pull Request for.
(Although I've already made the 'fix' on my fork, I understand if it simply does not meet Guava's standards and gets rejected. 😄)
jbduncan got the Error Prone team to successfully adopt Appveyor: google/google-java-format@1961e6a
@cpovirk, did you mean to say "jbduncan got...", or did you mean "jbduncan, (I) got..."?
I think "jbduncan got," as in "You influenced the Error Prone team to do it." In either case, the idea is that we now have an example of how to do this by another team at Google that we know well, so Guava might be able to adopt it more easily, too.
OK, we now run CI under Windows. So what's left to do here is to address the remaining failures. The places we need to fix are identifiable from their isWindows() checks. Of the isWindows() checks, some are specifically related to my Files.createTempDir and FileBackedOutputStream problems, which I'm taking care of. So what should be left after that is all the locations that cite "b/136041958" (the mirrored copy of this bug in our internal bug tracker). I think I have theories for the reasons for all of the failures, so I'm hoping the remaining work will be straightforward. The main exception would be the AbstractFuture tests, which are very slow for some reason. But we could at least figure out which ones are slow so that we can run the rest. And we could consider reducing the number of iterations used for the rest under Windows.
This should be one of those occasional issues in which we could rubber stamp a PR that addresses one or more of the failures.