AndroidJUnitRunner now marking failures where before it had not
Opened this issue · 1 comments
GoogleCodeExporter commented
What steps will reproduce the problem?
1. Update from previous runner and espresso core libraries to newer versions
including rules support
2. Leave test cases unchanged or update them to use the ActivityTestRule, the
results are the same.
3. Run shell am instrument to execute the tests from the command line and
observe the difference in output via stdout
What is the expected output? What do you see instead?
The difference in output and where the failures are marked is interesting. In
the stdout for the tests themselves, everything is identical. Afterward, in the
summary, the stream shows a handful of "could not find class" errors. When I
update to the newer build of the runner, those errors which had been benign
before now show up as failures but only at the summary phase, the cases are
untouched. In my reporting mechanism, I wouldn't have even noticed since I trim
the files from the Time: line onward per:
http://www.everybodytests.com/2015/04/jenkins-devices-androidjunitrunner.html
### Previous result:
Time: 28.751
OK (7 tests)
INSTRUMENTATION_CODE: -1
### New result:
Time: 28.515
There were 10 failures:
(lots of spew)
FAILURES!!!
Tests run: 7, Failures: 10
INSTRUMENTATION_CODE: -1
What version of the product are you using? On what operating system?
### Previous build.gradle dependencies:
androidTestCompile 'com.android.support.test.espresso:espresso-core:2.0'
androidTestCompile 'com.android.support.test:testing-support-lib:0.1'
### New build.gradle dependencies:
androidTestCompile 'com.android.support.test:runner:0.2'
androidTestCompile 'com.android.support.test:rules:0.2'
androidTestCompile 'com.android.support.test.espresso:espresso-core:2.1'
See included attachment for detailed output.
Original issue reported on code.google.com by russellc...@gmail.com on 24 Apr 2015 at 7:17
Attachments:
GoogleCodeExporter commented
Original comment by nkors...@google.com on 5 May 2015 at 5:29