joshfarrant/shortcuts-js

Getting Shortcuts to "tap" items on the screen

SuePB59 opened this issue · 1 comments

I need to create shortcuts that will interact with elements on the screen of an app. For example if there is a menu icon I'd want the shortcut to "tap" that and then to "tap" a specific item on the menu. Is there no way to identify a location on the screen to touch? This can be done with the Accessibility>Voice Control>Show Grid if you are giving voice commands, so why isn't there a way to script it into a shortcut?

The short answer is that if you can do it with the Shortcuts app you can (in theory) do it with Shortcuts JS. If you can't, then it's a no-go I'm afraid.