xctool fails to launch simulator and install application when running tests against XCode 6 Beta 5

Question

xctool fails to launch simulator and install application when running tests against XCode 6 Beta 5

Closed this issue 9 years ago · 50 comments

Running application tests for iOS using xctool with XCode 6 Beta 5 fails. This worked with XCode 6 Beta 4.

export DEVELOPER_DIR="/Applications/Xcode6-Beta4.app/Contents/Developer"
<path to xctool>/xctool.sh -workspace <workspace>.xcworkspace -scheme <schema> test -sdk iphonesimulator8.0

Result:
[Info] Collecting info for testables... (1542 ms)
  run-test appTests.xctest (iphonesimulator8.0, application-test)
    [Warning] Tried to install the test host app 'com.domain.app' but failed. (51 ms)
    [Info] Preparing test environment failed; will retry 2 more times (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (120 ms)
    [Info] Reset iOS simulator content and settings at path "/Users/jenkins/Library/Developer/CoreSimulator/Devices/BD4FCA64-F2E1-4A49-9DBF-A3F1A5E6E897/data" (103 ms)
    [Warning] Tried to install the test host app 'com.domain.app' but failed. (15 ms)
    [Info] Preparing test environment failed; will retry 1 more time (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (120 ms)
    [Info] Reset iOS simulator content and settings at path "/Users/jenkins/Library/Developer/CoreSimulator/Devices/BD4FCA64-F2E1-4A49-9DBF-A3F1A5E6E897/data" (120 ms)
    [Warning] Tried to install the test host app 'com.domain.app' but failed. (12 ms)
    [Warning] Preparing test environment failed. (0 ms)
      -[TEST_BUNDLE FAILED_TO_START]
--------------------------------------------------------------------------------
There was a problem starting the test bundle: Simulator 'iPhone 4s' was not prepared: Failed for unknown reason.
----|---------------------------------------------------------------------------
    X -[TEST_BUNDLE FAILED_TO_START] (0 ms) (0)
      -[APICompletionHandlerTests testBlockCallsLogoutAppropriately]

...cut out all other tests fail...

Answer 1 · 2014-08-12T14:58:01.000Z

We'll take a look

Answer 2 · 2014-08-14T18:44:02.000Z

I'm experiencing this error specifically when running from an ssh session. It doesn't happen when I run from Terminal locally. Maybe Apple changed something about how xcodebuild gathers information about the current environment?

Answer 3 · 2014-09-03T02:25:16.000Z

Can you try with the latest HEAD of master?

Answer 4 · 2014-09-03T08:33:24.000Z

I have a similar problem with xcodebuild from Xcode 6 beta 7. It fails to run the tests when running from an ssh session.

iPhoneSimulator: Timed out waiting 120 seconds for simulator to boot, current state is 1.
Project Tests 2 encountered an error (The Simulator can't be launched because it is already in use. If ...

This happens with Xcode 6 beta 7 also. I did report this to apple a month ago. But no fixes so far.

Answer 5 · 2014-09-03T17:46:58.000Z

There are several hacks in xctool that are intended to workaround this issue. @xfreebird could you confirm that this reproduces in the latest xctool as well? If yes, steps to reproduce would be helpful.
Thanks.

Answer 6 · 2014-09-03T19:47:59.000Z

To run the simulator you need to have an active GUI login session (ugh, yay Apple). SSH sessions don't get all the launchd stuff. See:

https://developer.apple.com/library/mac/technotes/tn2083/_index.html#//apple_ref/doc/uid/DTS10003794-CH1-SECTION9

and

travis-ci/travis-ci#1630

So I don't think this is a real bug (are you sure you weren't logged into the GUI when previously testing?), or if it is it is a bug in Xcode.

Answer 7 · 2014-09-03T19:51:51.000Z

In my case I already have an active GUI login session for the same user logged in using ssh. This does work with Xcode 5, but not with Xcode 6 beta 7.

I'm using xcodebuild directly.

Answer 8 · 2014-09-03T19:54:52.000Z

Ah. We are still on Beta 6, were looking to update tomorrow to 7. Sounds like an Apple bug we probably can't work around :-/

Answer 9 · 2014-09-03T20:14:54.000Z

@ExtremeMan I tested with the xctool from master HEAD, you can see the compele log here http://pastebin.com/WW8fQuxz

It fails to start the simulator.

How to reproduce:

Start an active GUI login session on the mac (with Xcode 6 beta 5 or beta 6 or beta 7)
From a different station connect through ssh
Run tests from command line

What happens:
Tests fail to run due to simulator.

What is expected:
Tests run in the simulator successfully

Answer 10 · 2014-09-03T20:19:11.000Z

@xfreebird does xcodebuild work in the same situation when xctool fails?

Answer 11 · 2014-09-03T20:20:03.000Z

No, it fails too.

Something like:

iPhoneSimulator: Timed out waiting 120 seconds for simulator to boot, current state is 1.
Project Tests 2 encountered an error (The Simulator can't be launched because it is already in use. If ...

Answer 12 · 2014-09-03T20:20:59.000Z

Got it. Most likely xctool will not be able to launch simulator as well then.

Answer 13 · 2014-09-04T12:43:58.000Z

I got the exact same problem with xcodebuild as @xfreebird

Answer 14 · 2014-09-04T19:38:59.000Z

Similar issue has been reported on the developer forum:
https://devforums.apple.com/message/1033457#1033457

Answer 15 · 2014-09-10T09:52:54.000Z

Still happens and with the latest Xcode 6 (A313).

Answer 16 · 2014-09-10T17:46:55.000Z

Same issue here, have tried the freshinstall option as well as launching the simulator before tests, no luck

Answer 17 · 2014-09-12T05:31:08.000Z

Does this issue have a workaround of any kind? Apple seems not going to change this behavior.

Answer 18 · 2014-09-12T07:18:29.000Z

This has been under discussion at https://devforums.apple.com/message/1040016#1040016. I opened a bug today (18311014) at the request of an Apple engineering manager who finally chimed in to that thread. Hopefully it will get addressed soon.

Answer 19 · 2014-09-19T01:24:11.000Z

The new release with Xcode 6 support (v0.2) is now in Homebrew. Please confirm it fixes this issue for you.

Answer 20 · 2014-09-19T21:48:46.000Z

I'm still seeing this issue using v0.2.1 (i.e. HEAD) via brew.

Answer 21 · 2014-09-19T21:50:28.000Z

I think we can do nothing from xctool perspective.

Answer 22 · 2014-09-19T21:55:22.000Z

Ya I agree; it's all in Apple's hands at this point.

Answer 23 · 2014-09-29T10:36:31.000Z

So, one workaround (not very like-able though) is reportedly to use the Java Web Star agent in Jenkins instead of an SSH agent. Any other known workaround? I'd certainly expect more Jenkins users complaining here... or is everyone moving to Xcode Bots?

Answer 24 · 2014-09-29T15:47:30.000Z

My temporary solution was to start another ssh daemon on a different port from UI session by adding a script to Login Items.

Answer 25 · 2014-09-30T09:58:33.000Z

@xfreebird that would work out for me too as a temp solution. Can you share the script?

Answer 26 · 2014-09-30T10:16:11.000Z

https://gist.github.com/xfreebird/93901ce603cbe087329e

Answer 27 · 2014-09-30T12:28:53.000Z

👍 that worked, thank you! I've added my ssh daemon as a user LaunchAgent instead though, cause I had some issues getting the login item executed, plus then there is no need to programmatically hide the terminal window. Anyhow, this workaround should do fine until Apple fixes their tools (if ever...).

Answer 28 · 2014-09-30T12:30:37.000Z

I did try adding it to be started by LaunchAgent, but I had the same issue. Could you please share your LaunchAgent config ?

Answer 29 · 2014-09-30T12:37:34.000Z

Sure, it's extremely simple though:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>Label</key>
   <string>com.redacted.customsshd</string>
   <key>Program</key>
   <string>/Users/jenkins/custom_sshd.sh</string>
   <key>RunAtLoad</key>
   <true/>
</dict>
</plist>

Added it as /Users/jenkins/Library/LaunchAgents/com.redacted.customsshd.plist. After rebooting the machine it still worked, but I'm crossing my fingers...

Answer 30 · 2014-09-30T21:16:40.000Z

Thanks ! Indeed it is working. Not sure why it was not working before. I did update the script, now it has an option to install itself as a launchctl agent

Answer 31 · 2014-10-21T15:07:03.000Z

I am still seeing this issue. I use TeamCity as my build server and the builds are kicked off from a LaunchAgent process. Is there anything else that I should be doing to work around this issue?

Answer 32 · 2014-10-25T22:36:50.000Z

I'm still seeing this issue, using version 0.2.1 from brew. Running xctool works fine when logged in as the user through terminal, but when commands are sent over ssh with the user still logged in, it fails to launch the simulator correctly.

You can visually see the simulator being launched, then relaunched twice without being given enough time to fully launch and install the application for testing.

Have Xcode 6.1 and OSX 10.10 installed, with both iOS 7.1 and 8.1 simulators downloaded.

Answer 33 · 2014-10-31T20:09:55.000Z

Seeing the same issue. Running 10.10 / 6.1 / 0.2.1 as well. Pasted the output from a failed xctool run-tests -showTasks below. Works if I'm not using ssh, fails if I am.

My .xctool-args

[
  "-jobs", "4",
  "-reporter", "pretty",
  "-workspace", "AnApp.xcworkspace",
  "-scheme", "AnApp",
  "-sdk", "iphonesimulator8.1",
  "-destination", "name=iPhone 6"
]

This is stopping us from running our continuous integration so I am very willing to help looking for the issue and try things out, though it should be fairly easy to test. Just ssh to localhost and run the test. I also see the simulator starting and then something fails. Worked with Xcode 5.x/iOS7.x.

================================================================================
LAUNCHING TASK (finding Xcode path via xcode-select --print-path):

  /usr/bin/xcode-select \
    --print-path
================================================================================

================================================================================
LAUNCHING TASK (finding Xcode path via xcode-select --print-path):

  /usr/bin/xcode-select \
    --print-path
================================================================================

================================================================================
LAUNCHING TASK (spawning reporter task):

  /usr/local/Cellar/xctool/0.2.1/libexec/reporters/pretty
================================================================================

================================================================================
LAUNCHING TASK (querying available SDKs):

  /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild \
    -sdk \
    -version
================================================================================

================================================================================
LAUNCHING TASK (gathering build settings for a target):

  SHOW_ONLY_BUILD_SETTINGS_FOR_FIRST_BUILDABLE=YES \
  DYLD_INSERT_LIBRARIES=/usr/local/Cellar/xctool/0.2.1/libexec/lib/xcodebuild-fastsettings-shim.dylib \
  /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild \
    -workspace \
    AnApp.xcworkspace \
    -scheme \
    AnApp \
    -sdk \
    iphonesimulator8.1 \
    -destination \
    "name=iPhone 6" \
    -destination-timeout \
    10 \
    -jobs \
    4 \
    PLATFORM_NAME=iphonesimulator \
    build \
    -showBuildSettings
================================================================================
[Info] Loading settings for scheme 'AnApp' ... (1759 ms)

=== RUN-TESTS ===

  [Info] Collecting info for testables...
================================================================================
LAUNCHING TASK (running xcodebuild -showBuildSettings for 'AnAppTests' target):

  SHOW_ONLY_BUILD_SETTINGS_FOR_TARGET=AnAppTests \
  DYLD_INSERT_LIBRARIES=/usr/local/Cellar/xctool/0.2.1/libexec/lib/xcodebuild-fastsettings-shim.dylib \
  /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild \
    -configuration \
    Debug \
    -sdk \
    iphonesimulator8.1 \
    -destination \
    "name=iPhone 6" \
    -destination-timeout \
    10 \
    -jobs \
    4 \
    PLATFORM_NAME=iphonesimulator \
    -project \
    AnApp.xcodeproj \
    -target \
    AnAppTests \
    OBJROOT=/Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Intermediates \
    SYMROOT=/Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Products \
    SHARED_PRECOMPS_DIR=/Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Intermediates/PrecompiledHeaders \
    TARGETED_DEVICE_FAMILY=1 \
    test \
    -showBuildSettings
================================================================================

================================================================================
LAUNCHING TASK (running otest-query):

  SIMSHIM_DYLD_FALLBACK_FRAMEWORK_PATH=/Applications/Xcode.app/Contents/Developer/Library/Frameworks:/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Library/Frameworks \
  SIMSHIM_DYLD_INSERT_LIBRARIES=/usr/local/Cellar/xctool/0.2.1/libexec/lib/otest-query-lib-ios.dylib \
  DYLD_INSERT_LIBRARIES=/usr/local/Cellar/xctool/0.2.1/libexec/lib/sim-shim.dylib \
  SIMSHIM_OtestQueryBundlePath=/Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Products/Debug-iphonesimulator/AnAppTests.xctest \
  SIMSHIM___CFPREFERENCES_AVOID_DAEMON=YES \
  /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/sim \
    --arch=64 \
    --sdk=8.1 \
    --environment=merge \
    /Users/johng/Library/Developer/Xcode/DerivedData/AnApp-dkknystsulwqagabyqtdnkzjqosw/Build/Products/Debug-iphonesimulator/AnApp.app/AnApp
================================================================================
  [Info] Collecting info for testables... (1320 ms)
  run-test AnAppTests.xctest (iphonesimulator8.1, iPhone 6, application-test)
    [Info] Installed 'com.AnApp.mobile.ios.AdvertiserApp'. (746 ms)
    [Info] Launching test host and running tests ... (0 ms)
    [Info] The simulator failed to start, or the TEST_HOST application failed to run.; will retry 2 more times. (0 ms)
    [Info] Shut down iOS Simulator... (1120 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (114 ms)
    [Info] The simulator failed to start, or the TEST_HOST application failed to run.; will retry 1 more time. (0 ms)
    [Info] Shut down iOS Simulator... (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (114 ms)
    [Warning] The simulator failed to start, or the TEST_HOST application failed to run.. (0 ms)
      -[TEST_BUNDLE FAILED_TO_START]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
There was a problem starting the test bundle: The simulator failed to start, or the TEST_HOST application failed to run.
━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
    ☂ -[TEST_BUNDLE FAILED_TO_START] (0 ms) (0)
      -[NumberUtilsTest testPhoneNumber]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Test did not run: The simulator failed to start, or the TEST_HOST application failed to run.
━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
    ☂ -[NumberUtilsTest testPhoneNumber] (0 ms) (1)
    0 passed, 0 failed, 2 errored, 2 total (0 ms)


Failures:

  0) -[TEST_BUNDLE FAILED_TO_START] (AnAppTests.xctest)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
There was a problem starting the test bundle: The simulator failed to start, or the TEST_HOST application failed to run.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  1) -[NumberUtilsTest testPhoneNumber] (AnAppTests.xctest)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Test did not run: The simulator failed to start, or the TEST_HOST application failed to run.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

** RUN-TESTS FAILED: 0 passed, 0 failed, 2 errored, 2 total ** (15681 ms)

Answer 34 · 2014-11-06T14:54:57.000Z

@BobReid did you ever get things working with TeamCity? I'm also using TeamCity and not sure how to get things going since the LaunchAgent runs things.

Answer 35 · 2014-11-07T16:04:29.000Z

Still seeing this issue over SSH with the latest commit ver in xctool github: 106cb80

Running: xctool.sh -scheme <scheme> -sdk iphonesimulator -workspace <proj>/project.xcworkspace -destination 'name=iPhone 6,OS=8.1' test

And getting:

  run-test <name>Tests.xctest (8.1, iPhone 6, application-test)
    [Warning] Tried to install the test host app '<id>' but failed. (53 ms)
    [Info] Preparing test environment failed; will retry 2 more times (0 ms)
    [Info] Verified iOS Simulators... (0 ms)
    [Info] Shut down iOS Simulator... (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (124 ms)
    [Info] Reset iOS simulator content and settings at path "/Users/<user>/Library/Developer/CoreSimulator/Devices/0B256C97-65FF-4624-9A57-79C19B1DB17D/data" (1088 ms)
    [Warning] Tried to install the test host app '<id>' but failed. (19 ms)
    [Info] Preparing test environment failed; will retry 1 more time (0 ms)
    [Info] Verified iOS Simulators... (0 ms)
    [Info] Shut down iOS Simulator... (0 ms)
    [Info] Stopped any existing iOS simulator jobs to get a fresh simulator. (133 ms)
    [Info] Reset iOS simulator content and settings at path "/Users/<user>/Library/Developer/CoreSimulator/Devices/0B256C97-65FF-4624-9A57-79C19B1DB17D/data" (496 ms)
    [Warning] Tried to install the test host app '<id>' but failed. (19 ms)
    [Warning] Preparing test environment failed. (0 ms)
      -[TEST_BUNDLE FAILED_TO_START]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
There was a problem starting the test bundle: Simulator 'iPhone 6' was not prepared: Failed for unknown reason.
━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Answer 36 · 2014-11-15T01:51:47.000Z

@xfreebird @oscahie Guys! ur solution did help me launch the simulator, but I still got the app failed to install, any thoughts?

Answer 37 · 2014-11-15T02:24:24.000Z

I've personally given up on xctool, for now at least. After updating Xcode to 6.1, xctool was failing somewhere else (can't remember if it was the app installation or what). Anyway, I've switched back to xcodebuild, it's just less problematic (still need the user-launched ssh daemon though) plus I can now run the test cases on actual iOS devices (haven't done it yet but planning to).

Answer 38 · 2014-11-15T03:04:44.000Z

For reference, xcodebuild also fails to work inside of ssh.

xcodebuild -jobs 8 -parallelizeTargets -workspace AnApp.xcworkspace -scheme AnApp -destination 'OS=8.1,name=iPhone 6' test

Gives us the following epic fail:

2014-11-14 20:52:29.868 Beginning test session with Xcode 6A1052d
2014-11-14 20:52:29.868 /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -jobs 8 -parallelizeTargets -workspace AnApp.xcworkspace -scheme AnApp -destination OS=8.1,name=iPhone 6 test
2014-11-14 20:52:29.868 Testing on device: <DVTiPhoneSimulator: 0x7feb6c3379c0> {
                SimDevice: SimDevice : iPhone 6 (37D19143-24DD-4E77-9E1D-359066F7F38F) : state={ Shutdown } deviceType={ SimDeviceType : com.apple.CoreSimulator.SimDeviceType.iPhone-6 } runtime={ SimRuntime : 8.1 (12B411) - com.apple.CoreSimulator.SimRuntime.iOS-8-1 }
}
2014-11-14 20:52:29.868 Initializing test infrastructure.
2014-11-14 20:52:29.868 Writing testing status log to /var/folders/df/yyfqwnv92t36sh91b19pf8h00000gp/T/com.apple.dt.XCTest-status/Session-2014-11-14_20:52:29-rxlV4K.log.
2014-11-14 20:52:34.215 Launch session started, setting a disallow-finish-token on the run operation.
2014-11-14 20:52:34.217 Adding console adaptor for test process.
2014-11-14 20:54:29.872 Test operation failure: Timed out waiting 120 seconds for simulator to boot, current state is 1.
2014-11-14 20:54:29.873 _finishWithError:Error Domain=IDEUnitTestsOperationsObserverErrorDomain Code=3 "Timed out waiting 120 seconds for simulator to boot, current state is 1." UserInfo=0x7feb6dacd090 {NSLocalizedDescription=Timed out waiting 120 seconds for simulator to boot, current state is 1.} didCancel: 1

Answer 39 · 2014-11-17T15:31:14.000Z

Same issue with XCode 6.1.1 GM

Answer 40 · 2014-11-17T16:12:30.000Z

@xfreebird @oscahie I created a custom LaunchAgent-based sshd daemon, but still cannot launch the simulator, but launching JNLP manually works (not working from launchctl).

I'm curious if it's the version of OS X makes the difference, I'm still using Mavericks on my Jenkins slave. Have you upgraded to Yosemite?

Answer 41 · 2014-11-17T16:47:16.000Z

We're running 10.9.5 on our CI server.

Answer 42 · 2014-11-17T17:49:51.000Z

@oscahie The issue is not with xctool, xcodebuild (Xcode 6.1, Build version 6A1052d) cannot properly attach to the iOS Simulator over an SSH session. For now the workaround is to use a manually started JNLP session whether the OSX machine you're attempting to build on is Mavericks (10.9.5) or Yosemite (10.10).

Answer 43 · 2014-11-17T18:24:25.000Z

@kaeawc I'm aware of that. It can also work over an ssh session though if you start your own daemon from the logged-in user, as explained a couple of comments above. This way xcodebuild works well with Xcode 6.1, whereas xctool doesn't (at least it didn't last time I tried, things might have changed by now).

Answer 44 · 2014-11-27T12:22:04.000Z

@kaeawc I'm trying to set up the JNLP session to get xctool to work, but I'm getting permission errors -

Building remotely on iOS-Slave in workspace /workspace/Test Project
java.io.IOException: Failed to mkdirs: /workspace/Test Project

Would it be possible for you to walk through your setup or link to any resources you used? Cheers

Answer 45 · 2014-11-27T18:36:10.000Z

@scottknight The LaunchAgent used to work prior to XCode 6. LaunchAgent processes have GUI sessions so I am not sure what the actual problem is with xctool.

I did create a temporary patchwork solution until it gets figured out or we switch build tools. I wrote a small apple script that launches terminal and executes xctool from the terminal window and that works.
Still have a few edge cases where failures are reported as successes that I need to capture but for 99% of build it succeed / fails properly.

Answer 46 · 2014-12-16T19:51:14.000Z

Just ran into this issue. OS X 10.10.1 14B25. Xcode 6.1.1 6A2008a. Jenkins 1.594. Not using a slave config at all. Jenkins configured as a launch agent. iOS simulator will not launch for tests via xctool or xcodebuild. Even an AppleScript that tries to launch Terminal app to run the test shell script doesn't work.

It was working prior to upgrading the build system from OS X 10.9.5.

Answer 47 · 2014-12-17T03:59:19.000Z

Works for me on jenkins JNLP:

on OSX slave，connect with screen share or use Displays
in OSX slave terminal, Lanuch Jenkins JNLP

If connecting with JNLP still does not solve it, try the solution mentioned in this comment.
i.e.: Run these on command line:

DevToolsSecurity -enable
sudo dscl . -append /Groups/_developer GroupMembership jenkins
security authorizationdb write system.privilege.taskport is-developer

Thanks Michael 's answer

Apple follow up regarding Bug ID# 18001199:

The context provided by LaunchDaemons is not supported for running GUI applications. The SSH service, and the default setup for Jenkins, are both implemented as LaunchDaemons. In earlier versions of Xcode 5 xcodebuild could run tests on the iOS simulator in this context, but that was never a supported configuration, and as you have noted that is no longer working as of Xcode 6.

Unlike LaunchDaemons, LaunchAgents provide a context where you can run GUI applications - if the user is logged in at the time, with a window server / Aqua session. Converting your Jenkins configuration from being a LaunchDaemon to being a LaunchAgent would avoid the reported issue. You can also use launchd for running tests on the iOS simulator from a SSH session, either by crafting a LaunchAgent and manually loading / starting that, or by using "launchctl submit”.

Answer 48 · 2015-03-09T20:56:26.000Z

@BobReid, would you mind sharing that launch script that launches Web Start via a Terminal?

Answer 49 · 2015-08-05T18:02:27.000Z

Thank you for reporting this issue and appreciate your patience. We've notified the core team for an update on this issue. We're looking for a response within the next 30 days or the issue may be closed.

Answer 50 · 2015-08-05T18:39:12.000Z

Since it is more an Apple issue rather then xctool (and I doubt we can improve/fix it in xctool itself) I will close the issue. Hope you all found a workaround.

Feel free to send a PR with your workarounds. We can add all of them to a README, I guess, to help other engineers when they hit this issue.