tmandry/Swindler

Notification windows are not trackable and cause swindler to print errors

Opened this issue · 3 comments

thieso2 commented

When running an app that uses swindler I get the following whenever the NotoficationCenter displays a message:

Cannot convert property value nil to type String
Cannot convert property value nil to type Bool
Cannot convert property value nil to type Bool

Plus the windows created by NotoficationCenter are not visible/trackable in swindler.

Steps to reproduce - Start a swinder-enabled app and run

osascript -e 'display notification "Lorem ipsum dolor sit amet" with title "Title"'

to produce a notification from Terminal.

Expected behaviour would be not to get those error messages and being able to track the windows created by NotificationCenter.

tmandry commented

Thanks for reporting this. It looks like the issue comes from the window having a nil title. The booleans are probably also for the properties isMain/isFocused.

We could simply ignore these windows (with subrole AXNotificationCenterBanner). @thieso2 do you have a use case for tracking these windows? If so, I'm curious to know what it is. The windows cannot be moved or resized, to my knowledge.

thieso2 commented

Hey @tmandry. We use swindler to instrument the OS to notify our app (kind of a VCR of your digital life) of noteworthy screen-updates.

Our use-cases are:

  • we want to know when "something" worth recording happens on the screen
  • we allow for playback of screen-recording with pixelating windows that are not in focus.

So we really want to know all about all windows that are shown at any moment, including the notification.
Hope that makes our intention clear.

Thank you open-sourcing your work!

tmandry commented

Got it. This isn't the exact use case I had in mind for Swindler; a Window is generally supposed to be something that behaves like a regular window. I'd be open to looking for ways that your use case can be supported, assuming it doesn't make Swindler harder to use.

That said, perhaps the easiest way is for you to AXSwift directly to watch for these notification windows. You should be able to use this alongside Swindler without any problems. Its API is more verbose, but it gives you more control than Swindler.

See this example for how to observe a particular application (run this and watch the console as you create, destroy, and move Finder windows.)

Do you think this will work?