Weird Failure when executing multiple tests with shutdown simulator
yongjincho92 opened this issue · 1 comments
yongjincho92 commented
Hi,
So I have set up a KIF on a small simple app which only has a table view with 50 cells.
And I have written two tests like this.
- (void)testSwipeAction
{
// Go find one cell out of the screen.
[tester waitForCellAtIndexPath:[NSIndexPath indexPathForRow:40 inSection:0]
inTableViewWithAccessibilityIdentifier:@"table"];
// Move back to top of the table.
[tester waitForCellAtIndexPath:[NSIndexPath indexPathForRow:1 inSection:0]
inTableViewWithAccessibilityIdentifier:@"table"];
}
- (void)testTapVisibleCell
{
[tester tapViewWithAccessibilityIdentifier:@"Cell 1"];
}
So there are two scenarios when I execute the tests: 1) when the destination simulator is shutdown and 2) when the destination simulator is already booted.
- When I run tests via Xcode, in both scenarios, tests worked fine.
- However, when I run the tests via BUCK, it would work fine on scenario 2. But on scenario 1, it would hang on the second test method
testTapVisibleCell. And the failure I see here is thatKIFUITestActor(IdentifierTests) tapViewWithAccessibilityIdentifier:would lead toNSBundle accessibilityBundleWithLastPathComponent:which only exists until iOS 9.0.
Below is the trace exception
Test Suite 'All tests' started at 2020-10-14 13:53:59.502
Test Suite 'SCSampleScreenTests.xctest' started at 2020-10-14 13:53:59.503
Test Suite 'SCSampleScreenTests' started at 2020-10-14 13:53:59.503
Test Case '-[SCSampleScreenTests testSwipeAction]' started.
Test Case '-[SCSampleScreenTests testSwipeAction]' passed (2.482 seconds).
Test Case '-[SCSampleScreenTests testTapVisibleCell]' started.
2020-10-14 13:54:02.191 TestHostApplication[37224:37071515] +[NSBundle accessibilityBundleWithLastPathComponent:]: unrecognized selector sent to class 0x7fff87d0b248
2020-10-14 13:54:02.192 TestHostApplication[37224:37071515] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '+[NSBundle accessibilityBundleWithLastPathComponent:]: unrecognized selector sent to class 0x7fff87d0b248'
*** First throw call stack:
(
0 CoreFoundation 0x00007fff23c7127e __exceptionPreprocess + 350
1 libobjc.A.dylib 0x00007fff513fbb20 objc_exception_throw + 48
2 CoreFoundation 0x00007fff23c91ed4 +[NSObject(NSObject) doesNotRecognizeSelector:] + 132
3 CoreFoundation 0x00007fff23c75c4c ___forwarding___ + 1436
4 CoreFoundation 0x00007fff23c77f78 _CF_forwarding_prep_0 + 120
5 UIKit 0x0000000112007084 __46+[AXUIKitGlue _handleWebKitLegacyInstallation]_block_invoke + 112
6 libdispatch.dylib 0x00007fff5223f7b9 _dispatch_client_callout + 8
7 libdispatch.dylib 0x00007fff522409e5 _dispatch_once_callout + 20
8 UIKit 0x0000000112007012 +[AXUIKitGlue _handleWebKitLegacyInstallation] + 34
9 libdispatch.dylib 0x00007fff5223f7b9 _dispatch_client_callout + 8
10 libdispatch.dylib 0x00007fff52241b75 _dispatch_continuation_pop + 440
11 libdispatch.dylib 0x00007fff522529cc _dispatch_source_invoke + 1997
12 libdispatch.dylib 0x00007fff5224bb38 _dispatch_main_queue_callback_4CF + 857
13 CoreFoundation 0x00007fff23bd4049 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 9
14 CoreFoundation 0x00007fff23bceca9 __CFRunLoopRun + 2329
15 CoreFoundation 0x00007fff23bce066 CFRunLoopRunSpecific + 438
16 SCSampleScreenTests 0x0000000108505f29 -[KIFTestActor tryRunningBlock:complete:timeout:error:] + 537
17 SCSampleScreenTests 0x000000010850614a -[KIFTestActor runBlock:complete:timeout:] + 138
18 SCSampleScreenTests 0x0000000108506256 -[KIFTestActor runBlock:complete:] + 150
19 SCSampleScreenTests 0x0000000108506331 -[KIFTestActor runBlock:] + 65
20 SCSampleScreenTests 0x000000010850e600 -[KIFUITestActor waitForAccessibilityElement:view:withElementMatchingPredicate:tappable:] + 208
21 SCSampleScreenTests 0x000000010850e28e -[KIFUITestActor waitForAccessibilityElement:view:withIdentifier:tappable:] + 318
22 SCSampleScreenTests 0x000000010850b856 -[KIFUITestActor(IdentifierTests) tapViewWithAccessibilityIdentifier:] + 118
23 SCSampleScreenTests 0x00000001084f5fca -[SCSampleScreenTests testTapVisibleCell] + 154
24 CoreFoundation 0x00007fff23c7820c __invoking___ + 140
25 CoreFoundation 0x00007fff23c753af -[NSInvocation invoke] + 319
26 XCTest 0x00000001081f2037 __24-[XCTestCase invokeTest]_block_invoke_2 + 52
27 XCTest 0x00000001081f1fe3 __24-[XCTestCase invokeTest]_block_invoke.206 + 320
28 XCTest 0x000000010824cdc2 +[XCTestCase(Failures) performFailableBlock:testCase:testCaseRun:shouldInterruptTest:] + 69
29 XCTest 0x000000010824ccd4 -[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 115
30 XCTest 0x00000001081f19f6 -[XCTestCase invokeTest] + 1183
31 XCTest 0x00000001081f3329 __26-[XCTestCase performTest:]_block_invoke_2 + 43
32 XCTest 0x000000010824cdc2 +[XCTestCase(Failures) performFailableBlock:testCase:testCaseRun:shouldInterruptTest:] + 69
33 XCTest 0x000000010824ccd4 -[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 115
34 XCTest 0x00000001081f3260 __26-[XCTestCase performTest:]_block_invoke.359 + 86
35 XCTest 0x000000010825fa0d +[XCTContext runInContextForTestCase:block:] + 211
36 XCTest 0x00000001081f2b14 -[XCTestCase performTest:] + 566
37 XCTest 0x000000010823938e -[XCTest runTest] + 57
38 XCTest 0x00000001081ecd50 __27-[XCTestSuite performTest:]_block_invoke + 354
39 XCTest 0x00000001081ec4a2 __59-[XCTestSuite _performProtectedSectionForTest:testSection:]_block_invoke + 24
40 XCTest 0x000000010825fa0d +[XCTContext runInContextForTestCase:block:] + 211
41 XCTest 0x00000001081ec459 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 148
42 XCTest 0x00000001081ec7be -[XCTestSuite performTest:] + 348
43 XCTest 0x000000010823938e -[XCTest runTest] + 57
44 XCTest 0x00000001081ecd50 __27-[XCTestSuite performTest:]_block_invoke + 354
45 XCTest 0x00000001081ec4a2 __59-[XCTestSuite _performProtectedSectionForTest:testSection:]_block_invoke + 24
46 XCTest 0x000000010825fa0d +[XCTContext runInContextForTestCase:block:] + 211
47 XCTest 0x00000001081ec459 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 148
48 XCTest 0x00000001081ec7be -[XCTestSuite performTest:] + 348
49 XCTest 0x000000010823938e -[XCTest runTest] + 57
50 XCTest 0x00000001081ecd50 __27-[XCTestSuite performTest:]_block_invoke + 354
51 XCTest 0x00000001081ec4a2 __59-[XCTestSuite _performProtectedSectionForTest:testSection:]_block_invoke + 24
52 XCTest 0x000000010825fa0d +[XCTContext runInContextForTestCase:block:] + 211
53 XCTest 0x00000001081ec459 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 148
54 XCTest 0x00000001081ec7be -[XCTestSuite performTest:] + 348
55 XCTest 0x000000010823938e -[XCTest runTest] + 57
56 XCTest 0x000000010826ef14 __44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 171
57 XCTest 0x000000010826f001 __44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke.100 + 96
58 XCTest 0x0000000108207746 -[XCTestObservationCenter _observeTestExecutionForBlock:] + 682
59 XCTest 0x000000010826ec9f -[XCTTestRunSession runTestsAndReturnError:] + 615
60 XCTest 0x00000001081d0744 -[XCTestDriver runTestsAndReturnError:] + 456
61 XCTest 0x000000010825b64c _XCTestMain + 2496
62 libXCTestBundleInject.dylib 0x0000000108056bfa __copy_helper_block_e8_32s + 0
63 CoreFoundation 0x00007fff23bd429c __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
64 CoreFoundation 0x00007fff23bd3a08 __CFRunLoopDoBlocks + 312
65 CoreFoundation 0x00007fff23bce894 __CFRunLoopRun + 1284
66 CoreFoundation 0x00007fff23bce066 CFRunLoopRunSpecific + 438
67 GraphicsServices 0x00007fff384c0bb0 GSEventRunModal + 65
68 UIKitCore 0x00007fff48092d4d UIApplicationMain + 1621
69 TestHostApplication 0x0000000107fa9900 main + 112
70 libdyld.dylib 0x00007fff5227ec25 start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
2020-10-14 13:54:17.073 xcodebuild[37193:37070122] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
What's really weird here is that it only happens when I am running the KIF tests with shutdown simulator (i.e. running test will start the simulator and then run the tests).
My Environment
- Xcode version 11.6
- Simulator I tried : iPhone 8 (iOS 12.4, iOS 13.3, iOS 13.5)
dostrander commented
Very interesting I haven't run into this before adding it to the bug list, thanks for letting us know!