ios-control/ios-deploy

DeveloperDiskImage.dmg for iOS 17 not found in Xcode 15

vashworth opened this issue · 17 comments

vashworth commented

As of Xcode 15 and iOS 17, DeveloperDiskImage.dmg is no longer found in the expect path, which is Xcode.app > Contents > Developer > Platforms > iPhoneOS.platform > DeviceSupport > 17.x.

This appears to be a permanent change:
https://developer.apple.com/forums/thread/730947
libimobiledevice/libimobiledevice#1452

Expected behavior

Debugging starts properly

Actual behavior.

Debugging phase fails due to missing DeveloperDiskImage.dmg.

------ Debug phase ------
Starting debug of xxxx (J617AP, J617AP, uknownos, unkarch, 17.0, 21A5248v) a.k.a. 'xxx' connected through USB...
2023-06-09 11:42:16.219 ios-deploy[45135:1129696] [ !! ] Unable to locate DeviceSupport directory with suffix 'DeveloperDiskImage.dmg'. This probably means you don't have Xcode installed, you will need to launch the app manually and logging output will not be shown!

Steps to reproduce the problem

  1. Download Xcode 15 beta
  2. Select new Xcode
xcrun xcode-select --switch /Applications/Xcode-beta.app/Contents/Developer
  1. Download iOS 17 on a physical device
  2. Run ios-deploy
ios-deploy --id xxxx --bundle path/to/Runner.app --app_deltas path/to/app-delta --debug --no-wifi

System Specs

Please run the commands below in your Terminal.app and include it in the issue. Check when done and include results below.

  • 1. system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: macOS 13.4 (22F66)
      Kernel Version: Darwin 22.5.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: xxx
      User Name: xxx
      Secure Virtual Memory: Enabled
      System Integrity Protection: Enabled
      Time since boot: 1 day, 6 hours, 59 minutes
  • 2. ios-deploy -V
    1.12.0
  • 3. xcodebuild -version
Xcode 15.0
Build version 15A5160n
  • 4. xcode-select --print-path
    /Applications/Xcode-beta.app/Contents/Developer
  • 5. gcc --version
Apple clang version 15.0.0 (clang-1500.0.28.1.1)
Target: arm64-apple-darwin22.5.0
Thread model: posix
InstalledDir: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
  • 6. lldb --version
lldb-1500.0.14.6
Apple Swift version 5.9 (swiftlang-5.9.0.114.6 clang-1500.0.27.1)
gabebear commented

I'm not sure if we want to have ios-deploy work with Xcode15; it's possible Xcode14 will be the final version supported. Apple has FINALLY created a non-hack mechanism with devicectl(e.g. try running xcrun devicectl --help). https://developer.apple.com/documentation/Updates/Xcode

  • 👀5
turbo commented

@gabebear Nice! Maybe worth adding to the readme 🙂

Summary of install & just launch with devicectl:

xcrun devicectl device install app --device 00008110-001XXXXXXXXXX ./xgen/Build/Products/Release-iphoneos/nilo.app
xcrun devicectl device process launch --device 00008110-001XXXXXXXXXX file:///private/var/containers/Bundle/Application/1604D2D5-35F3-4E43-8B47-1DEF5D778480/nilo.app

You get the second path from the output from the install command, or just use the bundle ID (sadly the app name isn't enough, unlike for xctrace)

  • 👍8
  • 👎1
  • 🚀1
steveames commented

Even using xcrun to install/launch there's still the issue of connecting to lldb and managing stderr/stdout. That's the majority of what I use ios-deploy for. I think that purpose is still legit in xcode15-land
unless there's a simple command for that as well that should go in the readme?

  • 👍1
gabebear commented

Ideally Apple will add a way to connect to lldb to devicectl.

Apple tracks feedback/radar reports to gauge interest in products. People who need iOS device debugging scripting support should file requests via https://feedbackassistant.apple.com/ . This could possibly get Apple to add this before the team that created devicectl moves to other things.

Saafo commented 
xcrun devicectl device install app --device 00008110-001XXXXXXXXXX ./xgen/Build/Products/Release-iphoneos/nilo.app
crun devicectl device process launch --device 00008110-001XXXXXXXXXX file:///private/var/containers/Bundle/Application/1604D2D5-35F3-4E43-8B47-1DEF5D778480/nilo.app

Seems Apple's devicectl is not easy to use, can we wrap it under the hood after Xcode 15?

steveames commented

Ideally Apple will add a way to connect to lldb to devicectl.
Yep. That'd be pretty nice. With CoreDevice there are IPv6 tunnels created to each device. You can point lldb directly at one of those and connect to it. The filehandle/socket forwarder is no longer required. The trick is figuring out the port that the debugserver is running on. That seems a tad convoluted currently. Doable but convoluted. I did take your advice and submit a feedback suggestion.

  • 👍2
  • 👎2
SachaTe commented 
xcrun devicectl device install app --device 00008110-001XXXXXXXXXX ./xgen/Build/Products/Release-iphoneos/nilo.app
xcrun devicectl device process launch --device 00008110-001XXXXXXXXXX file:///private/var/containers/Bundle/Application/1604D2D5-35F3-4E43-8B47-1DEF5D778480/nilo.app

devicectl is not as simple to use, we print some results to std::out and it seems like there is no way to capture this data like ios-deploy used to do. Also, we are stuck installing, launching, and then continuously polling the device to see if the application ran, as there is no way to attach to process launch

In addition with ios-deploy I was able to easily upload files to the bundle and it seems like with devicectl I keep getting errors

sk2212 commented 
xcrun devicectl device install app --device 00008110-001XXXXXXXXXX ./xgen/Build/Products/Release-iphoneos/nilo.app
xcrun devicectl device process launch --device 00008110-001XXXXXXXXXX file:///private/var/containers/Bundle/Application/1604D2D5-35F3-4E43-8B47-1DEF5D778480/nilo.app

devicectl is not as simple to use, we print some results to std::out and it seems like there is no way to capture this data like ios-deploy used to do. Also, we are stuck installing, launching, and then continuously polling the device to see if the application ran, as there is no way to attach to process launch

In addition with ios-deploy I was able to easily upload files to the bundle and it seems like with devicectl I keep getting errors

With the latest pymobiledevice3 version from https://github.com/doronz88/pymobiledevice3 I have found a method how to establish remote debugging with an iPhone with iOS 17:

Create a rsd-Tunnel:
sudo python3 -m pymobiledevice3 remote start-tunnel --script-mode
-> will output the rsd data ($rsd_output)

Start debugserver on iPhone
python3 -m pymobiledevice3 developer debugserver start-server --rsd $rsd_output

(e.g. python3 -m pymobiledevice3 developer debugserver start-server --rsd fdf0:721a:e29b::1 57537)

This will print something like this:

(lldb) platform select remote-ios
(lldb) target create /path/to/local/application.app
(lldb) script lldb.target.module[0].SetPlatformFileSpec(lldb.SBFileSpec('/private/var/containers/Bundle/Application/<APP-UUID>/application.app'))
(lldb) process connect connect://[fdf0:721a:e29b::1]:54437   <-- ACTUAL CONNECTION DETAILS!
(lldb) process launch

Using xcrun devicectl device install app command to install the App from an archive will let us parse out the data which is needed to setup lldb:

install@DE011-CMXXXX Scripts % xcrun devicectl device install app --device 00008110-xxxxxxx /Users/install/Projects/build_http_communication_Qt/Release-iphoneos/QtTest.app
12:04:26  Acquired tunnel connection to device.
12:04:26  Enabling developer disk image services.
12:04:26  Acquired usage assertion.
42%... 57%... 60%... 62%... 66%... 68%... 72%... 74%... 76%... 80%... 84%... 88%... 92%... 96%... Complete!
App installed:
? bundleID: de.bender.qtcomponenttest
? installationURL: file:///private/var/containers/Bundle/Application/8D43DD90-B138-4C42-8BE8-2752CAB6A259/QtTest.app/
? launchServicesIdentifier: unknown
? databaseUUID: 5D6368B1-5A96-462B-B5AD-B9F12163FEEA
? databaseSequenceNumber: 1508
? options:

With this output we can setup lldb:

$ lldb
(lldb) platform select remote-ios
(lldb) target create /Users/install/Projects/build_http_communication_Qt/Release-iphoneos/QtTest.app
(lldb) script lldb.target.module[0].SetPlatformFileSpec(lldb.SBFileSpec('/private/var/containers/Bundle/Application/8D43DD90-B138-4C42-8BE8-2752CAB6A259/QtTest.app'))
(lldb) process connect connect://[fdf0:721a:e29b::1]:54437
(lldb) process launch

Will result in an starting iOS app with debug output to lldb.

Next step is to pipe this output to a file, std_out or something else.

Maybe someone how knows ios-deploy is able to integrate this stuff.

I end up in creating a zsh script which is creating and parsing all the stuff to get the steps above automated.

  • 👍8
  • ❤️2
megabitsenmzq commented

I end up in creating a zsh script which is creating and parsing all the stuff to get the steps above automated.

Could you share your script? It will be so helpful.

sk2212 commented

@megabitsenmzq sure, but it is nothing special. Just some quick and dirty bash parsing. Feel free to improve.

#!/bin/bash
set -x


###############################################################################

function show_help()
{
  cat <<EOF
Usage:
  $(basename $0) [options]

  options
    -h              - show help and exit

  example:
    ./ios-ebee-deploy.sh      -b "/home/user/Projects/app_project/build_app/Release-iPhoneOS/iPhoneApp.app"
                              -i "iPhone UUID"
                              -p 'password' !!! <-- PLEASE KEEP CARE TO USE '' if password contains special chars --> !!!
                              -l "/home/user/logfile.out"

EOF
}

###############################################################################

while getopts b:i:p:l:h arg; do
  case $arg in
    h)
      show_help
      exit 0
      ;;
    b)
      app_path=${OPTARG}
      ;;
    i)
      device_id=${OPTARG}
      ;;
    p)
      sudo_pw=${OPTARG}
      ;;
    l)
      log_output=${OPTARG}
      ;;
    *)
      if [ $arg == ":" ]; then
        echo "ERROR: Option \"-$OPTARG\" requires an argument."
      else
        echo "ERROR: $arg Unknown option \"-$OPTARG\"."
      fi
      show_help
      exit 1
      ;;
  esac
done

if [ -z "$app_path" ]; then
 echo "Parameter -b is required. See -h for help."
 exit 1
fi

if [ -z "$device_id" ]; then
 echo "Parameter -i was not provided. Use devicectl to find first available iOS device."
 device_id=$(echo `xcrun devicectl list devices | awk '/available \(/ {print($6)}'`)
 if [ "$device_id" != "" ]; then
   echo "Using device id: $device_id"
 else
   echo "Warning: Cannot detect device id! Maybe XCode is running?"
 fi
fi

rm -rf tmp.o
rm -rf lldb.o
rm -rf install.o
rm -rf dynamic_data.sh
rm -rf lldb.commands


device_connected=$(echo `xcrun devicectl list devices | awk '/connected / {print($5)}' | awk -F'.' '{print($1)}'`)
if [ "$device_connected" != "" ]; then
 echo "ERROR: Possible opened XCode instance detected. Please close XCode an restart this script! Make sure selected iPhone is not in 'connected' state (xcrun devicectl list devices)."
 exit 1
fi


curl -sS "127.0.0.1:5555"
exit_code=$?
echo ""
if [ $exit_code -ne 0 ]; then
  if [ $exit_code -eq 56 ]; then
    echo "Tunnel-Server for getting rsd info is running but not reachable! Killing..."
    process_id=$(echo `echo ${sudo_pw} | sudo -S lsof -i:5555 | tail -n1 | awk -F' ' '{print ($2)}'`)
    echo ${sudo_pw} | sudo -S kill -9 ${process_id}
    sleep 5
  fi
fi


server_started=$(echo `nc -vz 127.0.0.1 5555 2>&1 | awk -F ' ' '{print $7}'`)
if [ ${server_started} == "succeeded!" ]; then
  echo "Server already running! Try to get rsd info string."
else
  echo ${sudo_pw} | sudo -S python3 -m pymobiledevice3 remote tunneld > server_start.o &
  sleep 10
fi

device_hostname=$(echo `xcrun devicectl list devices | awk '/available \(/ {print($5)}' | awk -F'.' '{print($1)}'`)
rsd_output="$(echo `curl -s -H "Accept: application/json" 127.0.0.1:5555 | jq -r --arg var "$device_hostname" '.[$var][0]'`) $(echo `curl -s -H "Accept: application/json" 127.0.0.1:5555 | jq -r --arg var "$device_hostname" '.[$var][1]'`)"

echo "Using $rsd_output for --rsd option"

xcrun devicectl device install app --device ${device_id}  ${app_path} > install.o
installation_url=$(cat install.o | grep 'installationURL' | awk -F 'file://' '{print $2}' | tr -d '\r\n' | sed 's/\/$//')

python3 -m pymobiledevice3 developer debugserver start-server --rsd ${rsd_output} > lldb.o &
sleep 5
connection_details=$(cat lldb.o  | grep -o 'connect://\[.*\]:[0-9]*')

echo "app_path=${app_path}" >> dynamic_data.sh
echo "remote_app_path=${installation_url}" >> dynamic_data.sh
echo "connection_details=${connection_details}" >> dynamic_data.sh

source $(dirname $0)/lldb.sh

if [ -z "$log_output" ]; then
 lldb -s lldb.commands

# | grep -v "\(lldb\)" &> ${app_path}/../../output.log
else
 lldb -s lldb.commands | grep -v "\(lldb\)" &> ${log_output}
fi

lldb.sh:

source dynamic_data.sh

rm -rf lldb.commands

echo "platform select remote-ios" >> lldb.commands
echo "target create $app_path" >> lldb.commands
echo "script lldb.target.module[0].SetPlatformFileSpec(lldb.SBFileSpec('$remote_app_path'))" >> lldb.commands
echo "script old_debug = lldb.debugger.GetAsync()" >> lldb.commands
echo "script lldb.debugger.SetAsync(True)" >> lldb.commands
echo "process connect $connection_details" >> lldb.commands
echo "script lldb.debugger.SetAsync(old_debug)" >> lldb.commands
echo "process launch" >> lldb.commands
echo "exit" >> lldb.commands
  • 👍2
laiyierjiangsu commented

@sk2212

hi, sk2212,

I have some questions about your script.

I don't understand the code "curl -sS "127.0.0.1:5555"" for . On my mac, failed to connect . where is 5555 coming from ?

`
++ echo

  • device_connected=
  • '[' '' '!=' '' ']'
  • curl -sS 127.0.0.1:5555
    curl: (7) Failed to connect to 127.0.0.1 port 5555 after 0 ms: Couldn't connect to server
  • exit_code=
    `
  1. echo ${sudo_pw} | sudo -S python3 -m pymobiledevice3 remote tunneld > server_start.o
    On my mac , there is nothing output to server_start.o . On the other hand , I found server_start.o have no further using in the whole script .how do you get the rst output ?

I am the first time to use mac real machine debugging, please forgive my little white.

laiyierjiangsu commented

Can anyone knows "127.0.0.1:5555" in sk2212's script? I really don't understand it's purpose. If someone knows,please share with me . I really appreciate.

sk2212 commented

Can anyone knows "127.0.0.1:5555" in sk2212's script? I really don't understand it's purpose. If someone knows,please share with me . I really appreciate.

curl -sS "127.0.0.1:5555" is just to check if there is a response provided by tunnel server instance started with pymobiledevice3. Do you have pymobiledevice3 installed?

laiyierjiangsu commented

Can anyone knows "127.0.0.1:5555" in sk2212's script? I really don't understand it's purpose. If someone knows,please share with me . I really appreciate.

curl -sS "127.0.0.1:5555" is just to check if there is a response provided by tunnel server instance started with pymobiledevice3. Do you have pymobiledevice3 installed?
@sk2212

Yes. I have installed pymobiledevice3, and following your instructions step by step, I am able to launch and debug apps installed on the phone using 'process launch'. However, when I try to use the script you provided to simplify the whole process, I find that 'server_start.o' is empty, resulting in an empty 'rsd_output'.

Given the presence of port 5555, I suspect this is a service launched locally by pymobiledevice3, but I couldn't find any documentation about this port in the pymobiledevice3 repository. I would be very grateful if you could provide me with the documentation for port 5555. Thank you very much.

sk2212 commented

@laiyierjiangsu yeah, it seems that the standard port '5555' was changed in 'pymobiledevice3':

doronz88/pymobiledevice3@6b0145e

laiyierjiangsu commented

@sk2212 Thank you for your help. I think I should read through the source code of pymobiledevice3 to thoroughly understand its mechanisms. It looks very interesting."

jswillard commented

For anyone still wanting to deploy and debug from the command line, Apple added support natively to LLDB in Xcode 16 Beta. See my comment on the flutter repo here: flutter/flutter#133465 (comment)

Hope this helps!

  • 👍2
  • ❤️2