wix/Detox

App uninstalls and reinstalls for every test file when using Jest

META-DREAMER opened this issue · 11 comments

META-DREAMER commented

Describe the bug
After refactoring our tests to use Jest instead of Mocha, Detox started uninstalling / reinstalling the app for every test file. This greatly slows down our tests and is not ideal.

To Reproduce

  • I have tested this issue on the latest Detox release and it still reproduces

Any Detox setup with Jest will exhibit this behaviour while Mocha setups will not.

Expected behavior
The app should not be reinstalled for every test file unless the user tells detox to do so.

Environment (please complete the following information):

  • Detox: 12.2.1
  • Jest: 24.4.0
  • React Native: 0.58.6
  • Node: 8.11.2
  • Device: iPhone X
  • Xcode: 10
  • iOS: 12.1
  • macOS: Mojave

Device and Verbose Detox Logs
Example of it reinstalling between tests

detox[14058] DEBUG: [exec.js/EXEC_CMD, #11] /usr/bin/xcrun simctl shutdown FCC3C45C-8F27-44F6-8E0B-AB5A271D1532
detox[14058] DEBUG: [exec.js/EXEC_TRY, #11] Shutting down FCC3C45C-8F27-44F6-8E0B-AB5A271D1532...
detox[14058] DEBUG: [DetoxServer.js/DISCONNECT] role=tester, sessionId=386a6d9f-0821-fab3-433c-e7e7d1428729
detox[14058] DEBUG: [DetoxServer.js/DISCONNECT] role=testee, sessionId=386a6d9f-0821-fab3-433c-e7e7d1428729
PASS e2e/tests/Contacts.spec.ts (380.165s)
  Checkins
    ✓ User adds a new contact to the space via checkin screen|user can add contact note on new checkin (53889ms)
    ✓ User adds a new contact via the select contact screen (49995ms)
    ✓ user performs checkin with existing contact|user adds inspection note on checkin|user edits contact while checking it in (84214ms)
  Contacts
    ✓ user can search for existing contact by name and mobile (26458ms)
    ✓ Create new contact and check contact is created (65661ms)
detox[14058] DEBUG: [exec.js/EXEC_SUCCESS, #11] FCC3C45C-8F27-44F6-8E0B-AB5A271D1532 shut down
detox[14058] INFO:  [DetoxServer.js] server listening on localhost:50929...
detox[14058] DEBUG: [AsyncWebSocket.js/WEBSOCKET_OPEN] opened web socket to: ws://localhost:50929
detox[14058] DEBUG: [DetoxServer.js/LOGIN] role=tester, sessionId=cbfb8f18-2734-9e0d-1ff8-5d6c61727b7d
detox[14058] DEBUG: [DetoxServer.js/LOGIN_SUCCESS] role=tester, sessionId=cbfb8f18-2734-9e0d-1ff8-5d6c61727b7d
detox[14058] DEBUG: [exec.js/EXEC_CMD, #0] /usr/bin/xcrun simctl list -j
detox[14058] DEBUG: [exec.js/EXEC_CMD, #1] applesimutils --list --byType "iPhone X" --byOS "12.2"
detox[14058] DEBUG: [exec.js/EXEC_TRY, #1] Searching for device matching iPhone X...
detox[14058] DEBUG: [exec.js/EXEC_CMD, #2] applesimutils --list --byId "FCC3C45C-8F27-44F6-8E0B-AB5A271D1532"
detox[14058] DEBUG: [exec.js/EXEC_CMD, #3] xcodebuild -version
detox[14058] DEBUG: [exec.js/EXEC_CMD, #4] /usr/bin/xcrun simctl boot FCC3C45C-8F27-44F6-8E0B-AB5A271D1532
detox[14058] DEBUG: [exec.js/EXEC_TRY, #4] Booting device FCC3C45C-8F27-44F6-8E0B-AB5A271D1532
detox[14058] DEBUG: [exec.js/EXEC_CMD, #5] /usr/bin/xcrun simctl bootstatus FCC3C45C-8F27-44F6-8E0B-AB5A271D1532
detox[14058] DEBUG: [exec.js/EXEC_CMD, #6] /usr/bin/xcrun simctl uninstall FCC3C45C-8F27-44F6-8E0B-AB5A271D1532 com.homepass.consumer
detox[14058] DEBUG: [exec.js/EXEC_TRY, #6] Uninstalling com.homepass.consumer...
detox[14058] DEBUG: [exec.js/EXEC_SUCCESS, #6] com.homepass.consumer uninstalled
detox[14058] DEBUG: [exec.js/EXEC_CMD, #7] /usr/bin/xcrun simctl install FCC3C45C-8F27-44F6-8E0B-AB5A271D1532 "/Users/vagrant/git/ios/build/Build/Products/Release-iphonesimulator/Homepass.app"
detox[14058] DEBUG: [exec.js/EXEC_TRY, #7] Installing /Users/vagrant/git/ios/build/Build/Products/Release-iphonesimulator/Homepass.app...
detox[14058] DEBUG: [exec.js/EXEC_SUCCESS, #7] /Users/vagrant/git/ios/build/Build/Products/Release-iphonesimulator/Homepass.app installed
detox[14058] DEBUG: [exec.js/EXEC_CMD, #8] /usr/bin/xcrun simctl terminate FCC3C45C-8F27-44F6-8E0B-AB5A271D1532 com.homepass.consumer
detox[14058] DEBUG: [exec.js/EXEC_TRY, #8] Terminating com.homepass.consumer...
detox[14058] DEBUG: [exec.js/EXEC_SUCCESS, #8] com.homepass.consumer terminated
detox[14058] DEBUG: [exec.js/EXEC_CMD, #9] applesimutils --simulator FCC3C45C-8F27-44F6-8E0B-AB5A271D1532 --bundle com.homepass.consumer --setPermissions notifications=YES,location=always
detox[14058] DEBUG: [exec.js/EXEC_TRY, #9] Trying to set permissions...
detox[14058] DEBUG: [exec.js/EXEC_SUCCESS, #9] Permissions are set
detox[14058] DEBUG: [exec.js/EXEC_CMD, #10] /bin/cat /dev/null >/Users/vagrant/Library/Developer/CoreSimulator/Devices/FCC3C45C-8F27-44F6-8E0B-AB5A271D1532/data/tmp/detox.last_launch_app_log.out 2>/Users/vagrant/Library/Developer/CoreSimulator/Devices/FCC3C45C-8F27-44F6-8E0B-AB5A271D1532/data/tmp/detox.last_launch_app_log.err && SIMCTL_CHILD_DYLD_INSERT_LIBRARIES="/Users/vagrant/Library/Detox/ios/7f772a17b6b8438f781c4abce46983bf49cbca74/Detox.framework/Detox" /usr/bin/xcrun simctl launch --stdout=/tmp/detox.last_launch_app_log.out --stderr=/tmp/detox.last_launch_app_log.err FCC3C45C-8F27-44F6-8E0B-AB5A271D1532 com.homepass.consumer --args -detoxServer ws://localhost:50929 -detoxSessionId cbfb8f18-2734-9e0d-1ff8-5d6c61727b7d
detox[14058] DEBUG: [exec.js/EXEC_TRY, #10] Launching com.homepass.consumer...
detox[14058] INFO:  [AppleSimUtils.js] com.homepass.consumer launched. The stdout and stderr logs were recreated, you can watch them with:
        tail -F /Users/vagrant/Library/Developer/CoreSimulator/Devices/FCC3C45C-8F27-44F6-8E0B-AB5A271D1532/data/tmp/detox.last_launch_app_log.{out,err}
detox[14058] DEBUG: [DetoxServer.js/CANNOT_FORWARD] role=testee not connected, cannot fw action (sessionId=cbfb8f18-2734-9e0d-1ff8-5d6c61727b7d)
detox[14058] DEBUG: [DetoxServer.js/LOGIN] role=testee, sessionId=cbfb8f18-2734-9e0d-1ff8-5d6c61727b7d
detox[14058] DEBUG: [DetoxServer.js/LOGIN_SUCCESS] role=testee, sessionId=cbfb8f18-2734-9e0d-1ff8-5d6c61727b7d
noomorph commented

Just a wild suggestion - do you run with --runInBand?

At the moment I'm not sure how to do that exactly, but in your shoes, I'd try my luck with https://jestjs.io/docs/en/configuration#globalsetup-string, do you agree?

Is there a blocker from Detox side preventing using globalSetup?

META-DREAMER commented

@noomorph Problem with --runInBand is that detox automatically adds the --maxWorkers 1 option when calling jest. Jest doesn't allow specifying both runInBand and maxWorkers.

How exactly would globalSetup help with the issue? What would I be doing in the globalSetup?

noomorph commented

@HammadJ , I suggest to reinstall the app on the emulator/simulator in a global setup. To stop reinstalling on each suite run, make sure to follow this:

https://github.com/wix/Detox/blob/master/docs/APIRef.Configuration.md#faster-test-runs-with-app-reuse

META-DREAMER commented

@noomorph

I tried this by adding the following to my globalSetup.js:

const detox = require('detox');
const config = require('../package.json').detox;

module.exports = async () => {
    await detox.init(config, { launchApp: false });
    await detox.device.launchApp({
        newInstance: true,
        permissions: { notifications: 'YES', location: 'always' },
    });
}

And removing the init and launchApp from the beforeAll hook in the init.js file. However, this caused every single test to fail with errors like this:
image

I believe this is because the globalSetup is executed in a completely different context than the actual tests so the actual tests don't have a properly initialized detox.

noomorph commented

@HammadJ , I am still not sure if the guys from Detox team will approve my pull request to an example project, but that's basically it: #1328

WARNING: the approach you want is supported only in -w 1 (1 worker aka --runInBand) mode.

  • 👍1
support commented

We use the issue tracker exclusively for bug reports and feature requests. This issue appears to be a general usage or support question. Instead, please ask a question on Stack Overflow with the detox tag.

rotemmiz commented

I am reopening the issue just to explain what I know about this behaviour.

The nature of Jest as a multiworker test runner is causing this behaviour.

The test runner is spinning up workers, and dispatching jobs (in that case, test files) to each of them. AFAIK, there is no way to configure an env setup file per worker, only per test file.

--runInBand is exactly to --maxWorkers=1 (https://github.com/facebook/jest/blob/8e246a4f60107cd9c6738b5f48d158ee634fa82c/packages/jest-config/src/getMaxWorkers.ts#L15), so don't expect any different behaviour.

@noomorph, your suggested idea is interesting, but again, there's no lifecycle hook for per-worker startup AFAIK, therefore any setup solution with Jest will be limited.

  • 👍1
noomorph commented

@HammadJ, could you please create a feature request for support of single reinstall in multi worker mode? We'll continue the discussion there.

META-DREAMER commented

@noomorph Created a new issue here: #1331

Thanks for your time and effort on this!

stale commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.
Thank you for your contributions.

stale commented

The issue has been closed for inactivity.