csaroff/tryswift2024

tryswift2024

Sample code for try! Swift 2024 presentation

Chat Assistant

Sample code for retrieving text from any application using the Accessibility API.

Usage:

With the text selected, press the shortcut key Command + Control + A. A floating window with a text field will appear so you can enter any prompt to the AI assistant.

chat_assistant.mp4

assistant_safari.mp4

Inline Assistant

Sample code for rewriting text in any application using the Accessibility API.

Usage:

Launch TextEdit.app and enter text freely. You can enter a prompt to the AI assistant after /assist. The result returned by the AI assistant is directly inserted afterwards.

inlineassistant.mp4

Comic Translator

Translate comic lines.

Usage:

Launch any comic reader application. Press the crosshair button to enter capture mode. Click on a page in the comic reader application to capture a screenshot and let the AI assistant translate the extracted text using OCR.

comictranslator.mp4

AXUIElementInspector

Dumps the AXUIElement at the mouse cursor position. Learn how to use the basic accessibility API and how to convert values.

elementinspector.mp4

CarbonHotKey

Carbon API based global hotkey implementation.

Slide

• https://github.com/kishikawakatsumi/tryswift2024/blob/main/Enhancing%20Applications%20with%20Accessibility%20API.key
• https://github.com/kishikawakatsumi/tryswift2024/blob/main/Enhancing%20Applications%20with%20Accessibility%20API.pdf