gojuno/composer

If the application process crashes, the JUnit xml file is not generated

nacuteodor opened this issue · 6 comments

Ran the tool like this:

java -jar composer-0.3.3.jar --apk ${APP_APK} --test-apk ${TEST_APK} --output-directory test-outputs --instrumentation-arguments class ${SUITE_CLASS_NAME} --shard false --install-timeout 300 --verbose-output true
15:17:41 [Fri Jul 06 15:17:41 UTC 2018]: Args(appApkPath=... testApkPath=..., testRunner=, shard=false, outputDirectory=test-outputs, instrumentationArguments=[class, ...], verboseOutput=true, keepOutputOnExit=false, devices=[], devicePattern=, installTimeoutSeconds=300)
15:17:41 [Fri Jul 06 15:17:41 UTC 2018]: 1 connected adb device(s): [AdbDevice(id=emulator-5556, model=Android SDK built for x86, online=true)]
15:17:41 [Fri Jul 06 15:17:41 UTC 2018]: [emulator-5556] Installing apk... pathToApk = ...
15:18:08 [Fri Jul 06 15:18:08 UTC 2018]: [emulator-5556] Successfully installed apk in 27 seconds, pathToApk = ...
15:18:08 [Fri Jul 06 15:18:08 UTC 2018]: [emulator-5556] Installing apk... pathToApk = ...
15:18:14 [Fri Jul 06 15:18:14 UTC 2018]: [emulator-5556] Successfully installed apk in 6 seconds, pathToApk = ...
15:18:14 [Fri Jul 06 15:18:14 UTC 2018]: [emulator-5556] Starting tests...
15:23:10 [Fri Jul 06 15:23:10 UTC 2018]: [emulator-5556] Test 1/7 passed in 4 minutes 51 seconds: ...
15:26:52 [Fri Jul 06 15:26:52 UTC 2018]: [emulator-5556] Test 2/7 passed in 3 minutes 41 seconds: ...
15:30:39 [Fri Jul 06 15:30:39 UTC 2018]: [emulator-5556] Test 3/7 passed in 3 minutes 46 seconds: ...
15:34:32 [Fri Jul 06 15:34:32 UTC 2018]: [emulator-5556] Test 4/7 passed in 3 minutes 52 seconds: ...
15:37:16 [Fri Jul 06 15:37:16 UTC 2018]: [emulator-5556] Test 5/7 passed in 2 minutes 42 seconds: ...
15:39:36 [Fri Jul 06 15:39:36 UTC 2018]: [emulator-5556] Test 6/7 failed in 2 minutes 19 seconds: ...
15:39:36 [Fri Jul 06 15:39:36 UTC 2018]: [emulator-5556] Error during tests run: java.lang.Exception: Application process crashed. Check Logcat output for more details.
15:39:36 Exception in thread "main" java.lang.RuntimeException: java.lang.Exception: Application process crashed. Check Logcat output for more details.
15:39:36 	at rx.exceptions.Exceptions.propagate(Exceptions.java:57)
15:39:36 	at rx.observables.BlockingObservable.blockForSingle(BlockingObservable.java:463)
15:39:36 	at rx.observables.BlockingObservable.first(BlockingObservable.java:166)
15:39:36 	at com.gojuno.composer.MainKt.runAllTests(Main.kt:187)
15:39:36 	at com.gojuno.composer.MainKt.main(Main.kt:65)
15:39:36 Caused by: java.lang.Exception: Application process crashed. Check Logcat output for more details.
15:39:36 	at com.gojuno.composer.InstrumentationKt.throwIfError(Instrumentation.kt:67)
15:39:36 	at com.gojuno.composer.InstrumentationKt.access$throwIfError(Instrumentation.kt:1)
15:39:36 	at com.gojuno.composer.InstrumentationKt$readInstrumentationOutput$2.call(Instrumentation.kt:113)
15:39:36 	at com.gojuno.composer.InstrumentationKt$readInstrumentationOutput$2.call(Instrumentation.kt)
15:39:36 	at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:69)
15:39:36 	at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:77)
15:39:36 	at rx.internal.operators.OnSubscribeCreate$BufferEmitter.drain(OnSubscribeCreate.java:366)
15:39:36 	at rx.internal.operators.OnSubscribeCreate$BufferEmitter.onNext(OnSubscribeCreate.java:299)
15:39:36 	at com.gojuno.composer.FilesKt$tail$1$1.handle(Files.kt:15)
15:39:36 	at org.apache.commons.io.input.Tailer.readLines(Tailer.java:528)
15:39:36 	at org.apache.commons.io.input.Tailer.run(Tailer.java:459)
15:39:36 	at java.lang.Thread.run(Thread.java:748)
15:39:36 Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: INSTRUMENTATION_RESULT: shortMsg=Process crashed.
15:39:36 	at rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:118)
15:39:36 	at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:73)
15:39:36 	... 7 m

Also, the HTML is not generated and further tests are not run (#131).

Seeing the same problem.

This may or may not be related, but i'm seeing failed tests causing the whole process to crash (even through ATO), which then doesn't generate the xml.

I gave up using composer. I use Android Test Orchestrator for robustness and it will crash only the affected test, but not the entire tests class and the xml is always generated.

@nacuteodor android test orchestrator slows my test builds down by ~300-400%. What are you doing to handle the slow speeds?

@abjerstedt I am not using gradle to build with the Android Test Orchestrator. I just deploy the orchestrator apks to the emulator and then run the tests with the orchestrator. So, that doesn't seem to noticeble slow down the tests runtime. The tests builds are built independently of the Android Test Orchestrator.