Can't control the device from the UI
egegrsn opened this issue · 12 comments
Hi,
Thanks for the great effort you've put. I can successfully connect and reach my device from "http://127.0.0.1:8080/" port and everything works perfectly.
However, I can't control the device from the UI, its not responding to my mouse clicks and swipes at all. I hope you can help me to figure out this problem.
I have this error popping out in every 2 seconds
handle_events: error: libusb: interrupted [code -10]
Thanks in advance!
Hi. Did you finish WebDriverAgent setup?
If you set it up correctly, after you run it for test in Xcode you should see ServerURLHere->http://[SOME_IP]:8100<-ServerURLHere in the output.
Can you open DevTools before openning the steam page (copy link, open new tab, open devtools, paste link and hit the enter) and look at the messages in the first websocket connection (one without parameters)? If WDA was successfully started you will see something like this:
Screenshot.
Otherwise there will be a error code (e.g. code 65 - most likely problem with signing).
Hi,
I got the ServerURLHere output. I think I finished the setup. I am seeing something similar but not same as your Screenshot.
This is from my WDA test:
2020-11-17 11:08:16.801379+0300 WebDriverAgentRunner-Runner[12853:4096002] Running tests...
Test Suite 'All tests' started at 2020-11-17 11:08:27.392
Test Suite 'WebDriverAgentRunner.xctest' started at 2020-11-17 11:08:27.393
Test Suite 'UITestingUITests' started at 2020-11-17 11:08:27.394
t = nans Suite Set Up
2020-11-17 11:08:27.438227+0300 WebDriverAgentRunner-Runner[12853:4096002] [User Defaults] Couldn't write values for keys (
KeyboardAutocorrection
) in CFPrefsPlistSource<0x280ebe680> (Domain: com.apple.Preferences, User: kCFPreferencesCurrentUser, ByHost: No, Container: (null), Contents Need Refresh: No): setting preferences outside an application's container requires user-preference-write or file-write-data sandbox access
2020-11-17 11:08:27.466950+0300 WebDriverAgentRunner-Runner[12853:4096002] [User Defaults] Couldn't write values for keys (
KeyboardPrediction
) in CFPrefsPlistSource<0x280ebe680> (Domain: com.apple.Preferences, User: kCFPreferencesCurrentUser, ByHost: No, Container: (null), Contents Need Refresh: No): setting preferences outside an application's container requires user-preference-write or file-write-data sandbox access
2020-11-17 11:08:27.468758+0300 WebDriverAgentRunner-Runner[12853:4096002] [User Defaults] Couldn't write values for keys (
KeyboardShowPredictionBar
) in CFPrefsPlistSource<0x280ebe680> (Domain: com.apple.Preferences, User: kCFPreferencesCurrentUser, ByHost: No, Container: (null), Contents Need Refresh: No): setting preferences outside an application's container requires user-preference-write or file-write-data sandbox access
2020-11-17 11:08:27.472377+0300 WebDriverAgentRunner-Runner[12853:4096002] [User Defaults] Couldn't write values for keys (
DidShowGestureKeyboardIntroduction
) in CFPrefsPlistSource<0x280ebe680> (Domain: com.apple.Preferences, User: kCFPreferencesCurrentUser, ByHost: No, Container: (null), Contents Need Refresh: No): setting preferences outside an application's container requires user-preference-write or file-write-data sandbox access
Test Case '-[UITestingUITests testRunner]' started.
t = 0.00s Start Test at 2020-11-17 11:08:27.475
t = 0.00s Set Up
2020-11-17 11:08:27.488036+0300 WebDriverAgentRunner-Runner[12853:4096002] Built at Nov 17 2020 10:14:37
2020-11-17 11:08:27.511509+0300 WebDriverAgentRunner-Runner[12853:4096002] ServerURLHere->http://someip:8100<-ServerURLHere
2020-11-17 11:08:27.512519+0300 WebDriverAgentRunner-Runner[12853:4096380] Using singleton test manager
Also this is the message I got when I execute ./ws-qvh
INFO[0000] Starting WebSocket server
INFO[0024] Device 'xxx' USB connection ready, waiting for ping..
INFO[0024] AudioVideo-Stream has started
INFO[0025] command: "run-wda"
2020/11/17 11:03:37 handle_events: error: libusb: interrupted [code -10]
Command line invocation:
/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild test-without-building -project ./WebDriverAgent/WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -destination id=xxx-xcconfig ./wda-build.xcconfig
Build settings from configuration file './wda-build.xcconfig':
CODE_SIGN_IDENTITY = iPhone Developer
DEVELOPMENT_TEAM = xxx
2020/11/17 11:03:37 handle_events: error: libusb: interrupted [code -10]
2020/11/17 11:03:39 handle_events: error: libusb: interrupted [code -10]
2020/11/17 11:03:39 handle_events: error: libusb: interrupted [code -10]
2020/11/17 11:03:40 handle_events: error: libusb: interrupted [code -10]
I am missing something but I don't know what.. Thanks for your help again.
I am seeing something similar but not same as your Screenshot.
You need to select the response (Binary Message 261 B) to see its content.
handle_events: error: libusb: interrupted [code -10]
I don't think that this errors are produced by WDA. Looks like they came from gousb.
Did you try to run quicktime_video_hack directly? Does it work without these errors?
I don't know how to run quicktime_video_hack, I tried go run main.go and it gave me an error which we discussed in an other issue, everything works well besides controlling the device with touch/swipe events from the UI.
You have devtools opened for "device list" page, but it shoud be a "stream page".
I don't know how to run
quicktime_video_hack
Run go build, then ./quicktime_video_hack.
Qvh is working well, also I opened devtools in stream page
this is the console log from stream page:
Failed to run WebDriverAgent. Reason: failed, code: 66
I have a problem with signing probably.
This command is used to start WDA:
Lines 97 to 105 in a9a487a
You can try to run it from the same terminal where you run ws-qvh to see the output. Maybe you can find the reason.
OK. I solved the issue. Thanks a lot
What was it? In case someone else runs into this.
WebDriverAgent wasn't in the ws-qvh directory, I put it there, also I opened WBA project in xcode and changed the team to my team.
Also, what can be done to make the response times faster? Additionally, I think when inside an app, clicking coordinates are shifting, so when I click somewhere, its clicking a little bit higher than where I clicked. I am still testing it.
Huge work, thanks.
Also, what can be done to make the response times faster?
Unfortunately, nothing. This is how WDA works. And there is no other tools (as far as I know) for remote control (ui testing) for iOS devices.
Additionally, I think when inside an app, clicking coordinates are shifting, so when I click somewhere, its clicking a little bit higher than where I clicked.
There is a bug in the client's master branch (fixed in develop) with scale calculation. But it causes a huge mistake in the coordinates, not a little shifting.
Anyway, if you run into a problems with the client, please open new ticket here.
Thanks!