JohnCoates/Aerial

Screen Savers broken on 2+ monitor system in macOS Sonoma 14.0

xmddmx opened this issue Β· 131 comments

xmddmx commented

This is a follow-up thread to #1286

Initial testing with macOS Sonoma (first beta, 23a5257q) suggests the problem has worsened again:

  • with 2 screens, I am seeing the same problem we saw back in early 13.3 betas: screen 1 is blank, screen 1's content is showing up mis-sized on screen 2. And, it is now back to happening 100% of the time (20 out of 20 failures, using Aerial, using HotCorner)
  • similar problem with other third-party screensavers (such as https://iscreensaver.com ) so this is again not a bug in Aerial.
  • I'm also seeing the same problem with first-party Apple screensavers (such as Message and Shifting Tiles)!

Conclusion: the first beta of Sonoma 14.0 is broken, and seems to be as broken as early betas of Ventura 13.3.

glouel commented

Can confirm all of the same FYI.

glouel commented

Also, side note, while they added some "video wallpapers" based on Aerial, there's no screensaver. When you get out of the screensaver (whatever it is), the lock screen plays a video for 3 seconds and stays still.

That's... not what they showed with fanfare during the presentation (seemless playback between screensaver and desktop background, which is definitely an impressive syncing effort).

xmddmx commented

FYI, I've re-submitted this as FB12240576, referencing my prior feedbacks FB12023530 and FB12179420, and also fixed the TItle of this Issue (it said macOS 13.4, but should obviously be 14.0)

Version 14.0 Beta (23A5257q)

Aerial still does not function.

Screenshot 2023-06-06 at 6 33 05β€―PM

x86txt commented

I have also reported it to Apple via the Feedback Assistant. Even the stock included Aerial Video Wallpapers don't work, they just show up as still wallpaper images. Everyone should report it, as the more reports to Apple the more attention it will get. Use the Feedback Assistant that came with the Sonoma Beta. If you don't have it installed, here are instructions for it: https://support.apple.com/guide/feedback-assistant/intro-to-feedback-assistant-fba2e39e53f5/mac

glouel commented

Even the stock included Aerial Video Wallpapers don't work, they just show up as still wallpaper images.

But to be clear, that's how Apple implemented them. They are not video wallpapers as in, they are not intended to playback all the time (like with Companion). They just play when you login basically, then slowdown to "become" a fixed wallpaper. That's how Apple implemented them. There's not (at least not yet) a screensaver with those dynamic wallpapers... it's weird right now. You can animate them for like 5 seconds by going to your Apple menu top left and pressing "lock screen" (2nd option from the bottom).

Yet they clearly talk about screensavers here : https://www.apple.com/macos/sonoma-preview/

New slow-motion screen savers of breathtaking locations from around the world look beautiful on your large Mac display. When you log in, they seamlessly become your desktop wallpaper.

The log-in thing is here, as long as it's from boot up, or when you come back from display off (basically, a blank lock screen). If you somehow run any screen saver, the (traditional) lock screen will appear on top of the screensaver.

So it's - at the moment - a "lock screen screen saver". I doubt this is their full plan. If it is, I mean... I have a hard time processing this, not gonna lie !

glouel commented

I'll point to this update here for reference : #1286 (comment)

Can confirm all of the same FYI.

I am running 3 monitors and all 3 of mine are using screen savers. I have different ones on each screen.

glouel commented

I am running 3 monitors and all 3 of mine are using screen savers. I have different ones on each screen.

Do you mean different screensavers or different Aerial videos on each screen using Aerial ? Not sure I follow what you mean here.

xmddmx commented

Sonoma beta 2 is out, and the Screen Saver System Settings has changed a lot.

  1. I am now seeing Apple's "Aerial" screensavers showing up in the Screen Saver settings - I was not seeing any of these in beta 1.
  2. there is a new 'Show on all displays' option (which when Off, gives you a choice of which monitor to show the screensaver on.
  3. behavior is superbly buggy, both with Apple screensavers and third party. I'm seeing new bugs where the legacyScreenSaver engine seems to get stuck running in the background, screensavers are not playing on 2 monitors correctly, etc.

Here's what the System Settings pane looks like now:

image

xmddmx commented

If you choose one of the Apple "Aerial" screen savers, you get a new Show as wallpaper option:

image

xmddmx commented

A new bug I've never seen before: when using your Aerial (not Apple's version) the screensaver actually worked on both screens, but then I was unable to exit! No keypresses or mouse clicks or movement would exit.

About 30 seconds later, the screensaver finally exited (and I heard some 'beep' noises) suggesting the whole system was lagging. Now legacyScreenSaver appears to have hung:

image

In short - lots of new stuff, much of it broken.

xmddmx commented

And when running on a 2 monitor system, with 'Show on all displays' enabled, behavior is again erratic: sometimes it plays only on screen 1, sometimes only on screen 2 (but with screen 1's content), and on rare occasion it will play on 2 screens nor mally, but I've only seen that about 1 out of 10 times.

glouel commented

Can confirm all this, including the new weird bugginess. Also note that if you go into desktop backgrounds (is it now called wallpaper in settings ??) you have the invert switch, use as screen saver :

Capture d’écran 2023-06-22 aΜ€ 00 49 26

The important bit is, screensavers can now be run independently on each screen, and in theory you can run 2 different ones simultaneously on different screens (at least that's what they are going for). I think that the new ability to set a separate screensaver on different screen is likely the root cause of all our issues, they might have started implementing stuff around this a while back.

Right now if I set Aerial to all screens in 2 monitor configuration, I get it on either one of them, primary or secondary. The bug previously never displayed on primary. If I set "individually" on both screens, I get the same thing. It's insanely buggy in any case.

Regarding their implementation of aerial screensavers, I think they simply use the desktop window and animate/stop it. they don't create a new separate view. This is what they did previously with the lock screen thing, they just animate the desktop and stop it.

It's unlikely we'll get the ability to make "dynamic wallpaper/screen savers" soon as 3rd party, which obviously sucks.

Hey is it just me or has this thing not worked even close to right for more than a year now ?

glouel commented

@DSBlackHeart I first reported the bug february 23rd, this was 13.3 beta where they broke it.

Back on topic, if I set a dynamic background (but not as screensaver), and launch screensaver (in that case aerial), when I come back to the desktop, I get the slowed down animation and...

Capture d’écran 2023-06-22 aΜ€ 00 58 39

Did they really implement this in LEGACYSCREENSAVER πŸ˜΅β€πŸ’«

I can see how that would break legacyScreenSaver for "regular" uses. And that CPU usage for a still frame, seriously (and I have to force quit them if I switch back to regular wallpaper) !

xmddmx commented

Hey is it just me or has this thing not worked even close to right for more than a year now ?

More like 4 months - it's only been since 13.3 betas which started happening in March. Feels like a year though!

The important bit is, screensavers can now be run independently on each screen, and in theory you can run 2 different ones simultaneously on different screens (at least that's what they are going for).

Maybe? they don't really have a good UI for that, as it's only "Run on main screen" which if off, lets you choose 1 monitor to run on. I can't see any way to intentionally choose N arbitrary screensavers for N monitors, can you?

I think that the new ability to set a separate screensaver on different screen is likely the root cause of all our issues, they might have started implementing stuff around this a while back.

good theory

xmddmx commented

Did they really implement this in LEGACYSCREENSAVER πŸ˜΅β€πŸ’«

Yeah, I'm seeing that too. That's a real WTF.

glouel commented

The important bit is, screensavers can now be run independently on each screen, and in theory you can run 2 different ones simultaneously on different screens (at least that's what they are going for).

Maybe? they don't really have a good UI for that, as it's only "Run on main screen" which if off, lets you choose 1 monitor to run on. I can't see any way to intentionally choose N arbitrary screensavers for N monitors, can you?

It's very clunky but it works just like on the desktop backgrounds side, whatever you set is persisted, if I setup Aerial on one and switch to the other screen, set monterey there, if I pick the first screen back it shows Aerial.

And in that case I can confirm it runs both (and in that case, weirdly enough, fine).

xmddmx commented

I feel like with Sonoma beta 1, some people were talking about the new screensavers. But most of us were not seeing any new screensavers, only wallpaper. This was very confusing to me, and I was wondering if they were confused.

Now in beta 2 I'm seeing actual screensavers.

Could it be possible that poeple were seeing the screensavers in beta 1, but it was only showing up for some people? (a certain region, model of mac, GPU, CPU or something else?)

glouel commented

I'm 100% sure people just got confused with the lock screen and Craig saying screensavers ;)

In beta 1, if you set the desktop background, you got the animation at login, back from sleep, or pressing lock screen in your Mac menu bar top left. I doubt they A/B tested anything, just people not understanding the difference or reusing Apple's marketing.

9to5mac was infuriating about this for example.

xmddmx commented

It's very clunky but it works just like on the desktop backgrounds side, whatever you set is persisted, if I setup Aerial on one and switch to the other screen, set monterey there, if I pick the first screen back it shows Aerial.

OK, I see what you are saying, and it kinda works.

Unfortunately, this means third-party screensaver authors now have to test for the difference between

  • one screensaver, set to 'Show on all displays'
  • the same screensaver, set to 'Show on one display', but set to show on all displays using this confusing UI.

as well as other possible issues such as

  • two different versions of the same screensaver, both playing on different displays.
  • etc.

Yikes!

glouel commented

It's very clunky but it works just like on the desktop backgrounds side, whatever you set is persisted, if I setup Aerial on one and switch to the other screen, set monterey there, if I pick the first screen back it shows Aerial.

OK, I see what you are saying, and it kinda works.

Unfortunately, this means third-party screensaver authors now have to test for the difference between

  • one screensaver, set to 'Show on all displays'
  • the same screensaver, set to 'Show on one display', but set to show on all displays using this confusing UI.

Yeah, I think it may not change much, we already had different situations in macOS where your plugin was launched by one process and ran on 2 windows, or launched by 2 separate processes. I know I have code about that laying around in Aerial, I can never remember what the current situation is. The tiny preview in System Preferences was also sometimes sharing the same process (I think they no longer do that, if I had to guess, but not sure).

as well as other possible issues such as

  • two different versions of the same screensaver, both playing on different displays.

Usually this goes very wrong, it's worth trying putting one version in /Library and one in ~/ and set them both, see what happens. System Preferences for example couldn't handle this if you put 2 versions, itonly loaded the UI bundle from one, which lead to hilariously hard crashes to debug if you opened the second. Now I just don't let people put Aerial in /Library for that reason ;)

Adding a bit on the legacy thing, it's late and I'm a bit too tired to investigate but I'm starting to wonder if what they do is, run the unlock screen animation through legacyScreenSaver, and at the same time set the last frame of the animation as a "picture" desktop background proper, and hide (but not kill) the legacyScreenSaver process called Wallpaper. I'm saying this because force quitting it just results in nothing changing visually.

Edit : Thankfully all those changes are brilliantly documented... https://developer.apple.com/documentation/screensaver 😩

xmddmx commented

Yeah, I think it may not change much, we already had different situations in macOS where your plugin was launched by one process and ran on 2 windows, or launched by 2 separate processes. I know I have code about that laying around in Aerial, I can never remember what the current situation is. The tiny preview in System Preferences was also sometimes sharing the same process (I think they no longer do that, if I had to guess, but not sure).

So, with recent versions of iScreensaver, which are written in Swift, there's a nice multi-screen syncrhonization feature, so when transitioning to another asset (image, video, 3D GLTF file...) each screen would talk to the other one and try to wait until all were ready, so they can all run the next effect simultaneously. When it works it's pretty nice.

Because legacyScreensaver did run all instances in the same process, this was fairly easy to do, by just using Swift globals. Or at least it seemed easy and worked on Intel, but when Apple Silicon came around, the memory model changed and having mutliple threads accessing globals no longer worked, as described here: https://developer.apple.com/documentation/apple-silicon/addressing-architectural-differences-in-your-macos-code

In any case, it was solved by adding some protection around the memory access using the synchronized directive or similar, see https://stackoverflow.com/questions/24045895/what-is-the-swift-equivalent-to-objective-cs-synchronized

With Sonoma, this doesn't seem to be working, so I need to go back and check this assumption that all screens are running in different threads but the same process. Maybe it's not true any more?

glouel commented

Super interesting, I think it's quite possible they now run multiple instances of legacyScreenSaver, would be interesting to look at.

Thinking about it, I'm pretty sure I'm relying on sharing a playlist object on Aerial too, for the spanned effect where I play a single video across multiple screen (I don't use mutexes though). It's late so I'll try to check more tomorrow. Could be a huge mess indeed 😩

xmddmx commented

OK, I can clearly see one bug - here's a sample log from iScreensaver being launched on Sonoma beta 2 when set to Show on all displays

Process:Thread  Library         Message
1191: 0xaaf4	iScreensaver	########## iScreensaver.init frame=(0.0, 0.0, 1680.0, 1050.0) isPreview=false
1191: 0xaaf4	iScreensaver	########## iScreensaver.initalize 0x00007fca6c762830
1191: 0xaaf4	iScreensaver	########## iScreensaver.initalize isPreview=false previewMode=false debugFlag=false hudTestFlag=false
1191: 0xaaf4	iScreensaver	### getScreenNumber defaulting to 0
1191: 0xaaf4	iScreensaver	###  screen=[0] globalTestVar=1 myAddress=0x00007fca6c762830
1191: 0xaaf4	iScreensaver	#   [0] screenNumber = 0
1191: 0xaaf4	iScreensaver	#   [0] mainScreen = false
1191: 0xaaf4	iScreensaver	###  [0] hasCongfigureSheet 0x00007fca6c762830
1191: 0xaaf4	iScreensaver	###  [0] hasCongfigureSheet 0x00007fca6c762830
1191: 0xaaf4	iScreensaver	[0] # draw 0x00007fca6c762830
1191: 0xaaf4	iScreensaver	########## iScreensaver.init frame=(0.0, 0.0, 1440.0, 900.0) isPreview=false
1191: 0xaaf4	iScreensaver	########## iScreensaver.initalize 0x00007fca6c77d820
1191: 0xaaf4	iScreensaver	########## iScreensaver.initalize isPreview=false previewMode=false debugFlag=false hudTestFlag=false

Translating this gibberish:

  • the ScreenSaverView.Initialize call is being called once for each screen
  • screen[1] is being initialized first, which is unusual, but isn't a big problem, however...
  • the frame is wrong - 1680x1050 is my second monitor, and it's definitely not at coordinates 0,0
  • this incorrect X,Y coordinate is confusing our code (which figures out which monitor it's on based on the X,Y,W,H of the frame) so it's mis-detecting and defaults to Screen[0].
  • Next, the real Screen[0] is initialized, which has the correct coordinates (0.0, 0.0, 1440.0, 900.0) so it detects it's on screen zero
  • at that point, everythign is Fubar'd because there are two Screen[0]s running, and they step on each other.

Other thoughts:

  • I may be misreading this, but the debug logging shows that not only are the two screens being run from the same process (1191) they are also being run from the same thread (0xaaf4). This doesn't seem right to me, and perhaps could be part of the OS bug we are seeing? I would assume that each screen should be running in it's own thread, right?
  • is there another way to figure out which actual screen we are on? our code uses the x,y,w,h of the frame, but since x,y are wrong, this isn't working...
xmddmx commented

Another test, this time with the System Settings / Screen Saver pane open shows that the small screen saver Preview window is in fact run in a different process:

Process:Thread  Library         Message
1251: 0xc6cb	iScreensaver	########## iScreensaver.init frame=(0.0, 0.0, 143.0, 80.0) isPreview=true
1268: 0xc7c3	iScreensaver	########## iScreensaver.init frame=(0.0, 0.0, 1440.0, 900.0) isPreview=false

Note: when isPreview=true, the PID and ThreadID are different.

So, in Sonoma, screensavers on multiple monitors are run in the same process and thread (!?) during normal operation, but the screen saver settings Preview mode runs in its own process.

I've done some more testing and I think this behavior is similar in Ventura.

Is it possible that the main bug in Sonoma beta 2 is that it's giving us the wrong X,Y coordinates for Screen[1], Screen[2] etc.? Perhaps a workaround is possible.

xmddmx commented

Also, on Ventura on a 2 monitor system, screen[0] is initialized first, followed by screen[1] which seems more logical.

Also, both instances get the correct coordinates (the .frame.x and .frame.y are not zeros for screen[1] and higher...)

Ventura:

0x21642	6202	iScreensaver	########## iScreensaver.init frame=(0.0, 0.0, 1440.0, 900.0) isPreview=false	default	legacyScreenSaver
0x21642	6202	iScreensaver	###  screen=[0] globalTestVar=1 myAddress=0x00007fcaca005940	default	legacyScreenSaver
0x21642	6202	iScreensaver	########## iScreensaver.init frame=(1440.0, 0.0, 2560.0, 1440.0) isPreview=false	default	legacyScreenSaver
0x21642	6202	iScreensaver	###  screen=[1] globalTestVar=2 myAddress=0x00007fcaca008440	default	legacyScreenSaver

glouel commented

My understanding of the original bug was that as part of it, primary screen was misplaced on last screen so the messed up coordinates are part of that, but as far as I know we can’t change them. This is probably more complicated now I’ll try to dig a bit but I think it’s going to be a long wait for beta 3 🫣

glouel commented

Random observations :

  • I found, while not using the new desktop stuff, and only using Aerial, that legacyScreenSaver (Wallpaper) was actually Aerial that kept running in the background. My guess, 3rd party screen saver end up on the same path and they get "thrown" as wallpapers and kept running (which would be great) but that just doesn't work. I don't expect this to actually happen to be clear, it's just IMO their own implementation of mixed desktop background/screen saver, and we get caught in the middle. Reported as FB12428045

  • As you pointed out, we are initialised incorrectly with coordinates that are always zeroed in for origin. This can happen even when we end up running on a screen not zeroed in (I have 2 screens side to side, main is left, if we run on right, we still get it). As far as I can see, in my testing, most of the time we run on one screen or the other, if I set different sizes for my screens, we end up with various configurations, but mostly never the correct view on the correct screen. This is a variant of the original multimonitor bug, except we don't always end up on the last screen as before, now it's random. I'm still working on how to report those changes, but it's terrible.

  • Most of the time it shows the wrong preview in settings, I get the Ventura one running :

Capture d’écran 2023-06-23 aΜ€ 17 47 18

Funnily enough, when you press the preview button, things work correctly (on multiple screens even) ! A refreshing bug. Options button is not there though. Selecting another screen saver and selecting Aerial back fixes it. Reported as FB12428208

glouel commented

Adding this one which is probably what happens for bug 1 above :
Capture d’écran 2023-06-23 aΜ€ 18 24 04

I was pretty sure I was detecting saverwillstop events to pause the screensaver, and I did, yet we are playing. I have a hard time processing this but I believe we are somehow restarted as (unshown) wallpapers as Aerial keeps playing in the background, somehow, despite me pausing it when receiving the event.

xmddmx commented

More Sonoma Screensaver Bugs:

  • in Preview mode, mouse events are being received, which is not true in older OS's. Not necessarly a bug, but if your code assumes mouse events only happen while running fullscreen, this could lead to issues.
  • I confirm what you are seeing: the 'Preview' button to start the screensaver sometimes works differently than HotCorner or timeout.
  • the sequence of calls to startAnimation/stopAnimation (etc.) seems to have changed when using the Options panel, but I haven't diagnosed just what the changes are.
  • iScreensaver handles screensaver Options using a helper app. This launches OK, but now in Sonoma, if you hide the Helper app window, and then click the screensaver 'Options' button a second time, the .saver bundle tries to push the helper app frontmost using this code:
           let apps = NSRunningApplication.runningApplications(withBundleIdentifier: childBundleID)
            if let helperApp = apps.first {
                helperApp.activate(options: [ .activateIgnoringOtherApps ])
           }

This is no longer working, but I'm not sure why.

xmddmx commented

Most of the time it shows the wrong preview in settings, I get the Ventura one running :

I'm seeing something similar, or perhaps a variation?

  • if you have a screensaver selected, the correct screensaver shows up in the small Preview mode. If you click the 'Preview' button, that same screensaver runs fullscreen.
  • however, you can close system preferences, and activate the screensaver (using HotCorner, or Timeout...) and the OS will launch the wrong screensaver. In my case, looks like I have two screensavers with different filenames, but the same internal bundleID. I wonder if the bundleID is confusing the OS?

Version 14.0 Beta (23A5257q)

Aerial still does not function.

Screenshot 2023-06-06 at 6 33 05β€―PM

Dinesh that you with all the apple monitors?

Version 14.0 Beta (23A5257q)
Aerial still does not function.
Screenshot 2023-06-06 at 6 33 05β€―PM

Dinesh that you with all the apple monitors?

It is a lot of real estate LOL
:)
IMG_2552

Something I noticed:

I have found when Aerial tries to run, and it did this also under Ventura BTW

when Aerial starts my system fan and processor spin up, it sounds as if my I-Mac is going to take off like Air Force One or something, the fan speed maxes out, and it does this for 20 to 30 minutes.

I reported this but was given the canned response of "Rest The SMC" La Di Da by Apple

So before doing all of that I just turn Aerial and the other "Magic Window" off and uninstalled, does not work anyway for now.

Wow, I have not heard this much quite from my Imac in months, RPM is in the 1200 - 1500 range Normal
before it would spin at 2800+

Did anyone else notice? Just curious?
I have been using Macs Fans Control as a monitor for a long time.

glouel commented

Something I noticed:

In the future, please keep the discussion to the topic at hand as it's super hard for me and others who want to follow this precise issue if we talk about other stuff. I created another issue to discuss yours here #1310 and will reply to you there. Thanks !

Thanks for doing that,

glouel commented
  • iScreensaver handles screensaver Options using a helper app. This launches OK, but now in Sonoma, if you hide the Helper app window, and then click the screensaver 'Options' button a second time, the .saver bundle tries to push the helper app frontmost using this code:
           let apps = NSRunningApplication.runningApplications(withBundleIdentifier: childBundleID)
            if let helperApp = apps.first {
                helperApp.activate(options: [ .activateIgnoringOtherApps ])
           }

This is no longer working, but I'm not sure why.

Ok took me a bit to get to you but could it be the new sandboxed by default that denies you this ? It may be fine to launch, but not access the list of running apps in sandbox, for example ? That whole sandbox by default is just an epically unthought through disaster 😩

Companion gets the same prompt for daring to open my own screen saver plugin ! If only there was something like, hmm, an app extension system that would solve this πŸ˜…

Edit : See this session about the changes in sandboxing by default : https://developer.apple.com/wwdc23/10053?time=1066 The workaround is granting Full Disk Access, or individually making security bookmarks for external locations. Obviously, screen saver being on the fringe, there are many pitfalls for us (your app can install a screen saver without prompting apparently, which is nice, but you can't open the file or access the container of your screen saver from your app).

glouel commented

Extra notes about Sonoma implementation of Wallpaper/Screen Savers :

  • We can find a (new) WallpaperAgent.app that contains extensions for the various fixed/dynamic wallpapers /System/Library/CoreServices/WallpaperAgent.app/Contents/Extensions/, those files are here in Sonoma:
WallpaperAerialsExtension.appex		
WallpaperCAPackageExtension.appex	
WallpaperDynamicExtension.appex		
WallpaperGradientExtension.appex	
WallpaperImageExtension.appex		
WallpaperMontereyExtension.appex
WallpaperVenturaExtension.appex
wallpaper.appextensionpoint

As mentioned, this is all new, and all the wallpapers are .appex now basically. Went back to macOS 13.2 and nothing of the sort there.

  • If I set one of the new aerial wallpapers as a wallpaper + screensaver, I can't see any legacyScreenSaver.appex activity. Unlike what I previously assumed, they may not have implemented this as part of legacyScreenSaver.appex, but possibly as part of loginwindow+ something.

  • Some interesting events that show some "coordination" between loginwindow and WallpaperAgent as logged by Console.app just after my login was validated and before the screensaver exited :

15:45:18.295753+0200	loginwindow	7E87C2E5: BEGIN - Wallpaper Timeline: XPC Encode
15:45:18.295782+0200	loginwindow	665D8E61: BEGIN - Wallpaper Timeline: XPC JSON Encode
15:45:18.295846+0200	loginwindow	665D8E61: 🧸 - LoggedActivity 'Wallpaper Timeline: XPC JSON Encode' did not call end/error!
15:45:18.295866+0200	loginwindow	7E87C2E5: END - Wallpaper Timeline: XPC Encode
15:45:18.296943+0200	WallpaperAgent	958F9198: BEGIN - Wallpaper Timeline: XPC Decode
15:45:18.297121+0200	WallpaperAgent	EE7B0511: BEGIN - Wallpaper Timeline: XPC JSON Decode
15:45:18.297386+0200	WallpaperAgent	EE7B0511: END - Wallpaper Timeline: XPC JSON Decode

Now I want to see that JSON ! Convertly, I see the invert talking at start :

15:59:25.485346+0200	WallpaperAgent	692CBD1F: BEGIN - Wallpaper Timeline: XPC Encode
5:59:25.486282+0200	loginwindow	964C6BBB: END - Wallpaper Timeline: XPC Decode

So there's some XPC coordination, possibly about video position between wallpaper/loginwindow ?

  • Later on, I think this is really interesting :
15:45:18.300165+0200	WallpaperAgent	5D219CD5: END - [com.apple.wallpaper.extension.dynamic] wallpaper(CADCF9A4) set presentation mode to idle

This is the "slowdown" effect, is the wallpaper is a paused avplayer in the case of the WallpaperAerialExtension ? Or do they replace it with a fixed frame grab ?

  • So what plays the video during the screensaver ? I'm confused, I see this that makes it look a lot like WallpaperAerialsExtension is the one doing it
15:59:25.697273+0200	WallpaperAerialsExtension	<<<< VMC >>>> vmc2GMFigLogDumpStats: VMC(0x10f014600): client pid 12733; timebase time: 39.940 s rate: 0.00. recent stats: cfq empty: 46 times; frames decoded: 14, dropped: 0; decode: mean 0.000 s, max: 0.000 s; codec service tier: 0.
5:59:25.700139+0200	WallpaperAerialsExtension	[<private>]: minUpcomingOriginalPTS.seconds 39.939940 - self.firstPrerolledPTS!.seconds 39.939940 = 0.000000
15:59:25.704030+0200	WallpaperAerialsExtension	[<private>]: minUpcomingOriginalPTS.seconds 40.197940 - self.firstPrerolledPTS!.seconds 39.939940 = 0.258000
15:59:25.704063+0200	WallpaperAerialsExtension	[<private>]: Start timebase at 39.939940
15:59:25.704408+0200	WallpaperAerialsExtension	<<<< LayerSync >>>> figlayersync_setLayerTiming: (layer 0x600001467bc0) set layer speed: 1.000000
15:59:25.704514+0200	WallpaperAerialsExtension	<<<< FigVideoQueue >>>> vq_timebaseEffectiveRateChanged: (0x11d909ed0) Effective Timebase Rate of videoqueue changed to: 1.000 (pid 12733)

So basically :

  • the video is stopped (rate: 0)
  • the video moves to some point in time (start timebase)
  • the video is started (rate: 1)

If I had to guess how this all works :

  • when you login, the video plays
  • after login it stops playing at a given position, there's some XPC encoding that send that position to WallpaperAgent from loginwindow
  • a fixed frame grab is generated and used as the wallpaper to preserve resources
  • when you launch the screensaver, some XPC encoding from WallpaperAgent sends the time position of the current screenshot to loginwindow. loginwindow starts WallpaperAerialsExtension (???) at that position so it can animate the video. Playback starts
  • when you stop, loginwindow sends back the stop position to WallpaperAgent who will make a frame grab of the video again (maybe running or they are sharing WallpaperAerialsExtension ?? this is so weird).

I'll say this, I hope I've misunderstood this, I thought they would literally share the view between loginwindow/desktop/screensaver, and not just pass stuff around, try to coordinate video players, etc. This is certainly resource intensive (and basically was my first thought about how I would have implemented this between Aerial and Companion because I would have had no other choice). Weird.

  • Does this help anything ? Well outside the curiosity of how they did it, one thing is clear, at no point is legacyScreenSaver implicated. So why then, when we do set our savers do we see those legacyScreenSaver (Wallpaper) running ? Is it just loginwindow interacting with legacyScreenSaver as if it was WallpaperAerialExtensions ? Did Apple really intend for legacyScreenSaver to be ran as a wallpaper ? Who knows !

Edit : regarding the legacyScreenSaver (Wallpaper) we've all seen in Activity Monitor, if you use Ventura as a screen saver (which is an .appex), you end up with Ventura (Wallpaper) too. So I think that confirms they relaunch any appex as a wallpaper (perhaps mistakenly, or maybe they'll give it to use some day too!).

glouel commented

Also, something - for developers - that took me a while to understand : if legacyScreenSaver is still running in the background a version of your screen saver, and you install a new version, the new version will show up in System Settings... but not when running under ScreenSaverEngine.

Force quitting legacyScreenSaver (Wallpaper) will fix the issue and let you try your new build.

Edit : This also goes when updating Aerial for users.

glouel commented

Ok so, again trying to understand how to workaround the beta 2 situation, I kept wondering why Aerial didn't stop and noticed two things :

  • stopAnimation is never called on us (I want to say this is not particularly new, but usually we get killed)
  • the useful event I used as a workaround to pause video playback and patiently wait to be killed, com.apple.screensaver.willstop, is not sent when the screensaver is supposed to stop

After tracking down those events in Console, I noticed this :

par dΓ©faut	15:16:25.082087+0200	distnoted	register name: com.apple.screensaver.willstart object: kCFNotificationAnyObject token: 3f00000037 pid: 47665
par dΓ©faut	15:16:25.082693+0200	distnoted	register name: com.apple.screensaver.willstop object: kCFNotificationAnyObject token: 4000000036 pid: 47665

So those events (willstart and willstop) are just fired at the same time, which is pretty terrible for my workaround and Companion desktop mode (where I monitor those events to stop/start playback while the screensaver runs).

This is filed as FB12523305

Workaround, I found none at the time, I can't see any event firing up that I could use at the moment to correctly detect this situation but will keep looking (I do see a bunch of events fired every 10 seconds though!).

As a corollary to this, it keeps making the picture clearer, from our point of view, we just get never stopped, and maybe maybe, the appex may be "passed around" from ScreenSaverEngine to Wallpaper process, explaining what we see in Activity Monitor.

Edit : more stuff, per my previous post, legacyScreenSaver keeps us in memory, but this is how it tries to do it. In this scenario, I launcehd the screensaver once (two monitors), exited, launched again :

Here's what Aerial sees :

2023-07-05 15:49:16.208 : πŸ–ΌοΈ <AerialView: 0x13af2ace0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-05 15:49:16.209 : πŸ–ΌοΈ <AerialView: 0x13afe4a90> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)

note the views, acec0, 4a90, those are the "previously launched and kept around" views, they do receive a new startAnimation event each.

Yet... I see two new launched

2023-07-05 15:49:16.360 : πŸ–ΌοΈ AVinit (.saver) (0.0, 0.0, 2560.0, 1440.0) p: false o: false
2023-07-05 15:49:16.361 : πŸ–ΌοΈ <AerialView: 0x11d21d3c0> AV setup init (V3.2.7beta2) preview: false
2023-07-05 15:49:16.469 : πŸ–ΌοΈ AVinit (.saver) (0.0, 0.0, 2560.0, 1440.0) p: false o: false
2023-07-05 15:49:16.469 : πŸ–ΌοΈ <AerialView: 0x13ba28460> AV setup init (V3.2.7beta2) preview: false

So, are we just getting stacked and laying around running ? The answer seems to be yes, a few minutes later :

2023-07-05 15:54:07.407 : <AerialView: 0x13af2ace0> played did reach end

So ace0 is still here, playing in the background, never stopped. Sigh.

I can definitely confirm this with some new logging I added earlier, where each AerialView can register to listen to audio events :

2023-07-05 15:56:29.570 : 🎧 media remote callback
2023-07-05 15:56:29.574 : 🎧 audio info
2023-07-05 15:56:29.575 : 🎧 The Veronicas - Untouched (Hook Me Up (Bonus Track Version)) with artwork 
2023-07-05 15:56:29.575 : 🎧🟧 updateStatus
2023-07-05 15:56:29.575 : 🎧🟧 updateStatus
2023-07-05 15:56:29.575 : 🎧🟧 updateStatus
2023-07-05 15:56:29.576 : 🎧🟧 updateStatus
2023-07-05 15:56:29.576 : 🎧🟧 updateStatus
2023-07-05 15:56:29.576 : 🎧🟧 updateStatus

Note the 6 status updates, after launching the screensaver 3 times (two monitors)

And after a bunch more launches :

2023-07-05 15:59:09.605 : 🎧 media remote callback
2023-07-05 15:59:09.616 : 🎧 audio info
2023-07-05 15:59:09.617 : 🎧 Louisadonna - Γ€ deux (Fatigue) with artwork 
2023-07-05 15:59:09.617 : 🎧🟧 updateStatus
2023-07-05 15:59:09.617 : 🎧🟧 updateStatus
2023-07-05 15:59:09.617 : 🎧🟧 updateStatus
2023-07-05 15:59:09.618 : 🎧🟧 updateStatus
2023-07-05 15:59:09.618 : 🎧🟧 updateStatus
2023-07-05 15:59:09.618 : 🎧🟧 updateStatus
2023-07-05 15:59:09.618 : 🎧🟧 updateStatus
2023-07-05 15:59:09.618 : 🎧🟧 updateStatus
2023-07-05 15:59:09.619 : 🎧🟧 updateStatus
2023-07-05 15:59:09.619 : 🎧🟧 updateStatus
2023-07-05 15:59:09.619 : 🎧🟧 updateStatus
2023-07-05 15:59:09.619 : 🎧🟧 updateStatus
2023-07-05 15:59:09.619 : 🎧🟧 updateStatus
2023-07-05 15:59:09.619 : 🎧🟧 updateStatus

That's a lot of AerialViews running! If you use Sonoma beta2, if you see things slowing down, open Activity Monitor, filter for "wallpaper" and kill legacyScreenSaver (Wallpaper) or others. Surprisingly the CPU usage isn't that bad for Aerial. Ventura screensaver does way worse in that scenario.

Last edit : Maybe it's me or I just get lucky, but I have a feeling that after force quitting legacyScreenSaver, we get instantiated on the correct monitors (with caveats).

glouel commented

Beta3 is out, from a quick glance I see no change :

  • from a user perspective, it's still bugged for multimonitors when applied on all screens (you get one, the other, or both)
  • willstop event is still sent at launch just after willstart
  • we hang around as legacyScreenSaver (Wallpaper) and our views are never stopped or killed, they keep running in the background

As a user, I would recommend :

  • Setting Aerial on only one screen, set an apple screen saver on other monitors for the time being. At least this should ensure "proper" functionnality
  • Periodically kill legacyScreenSaver (Wallpaper) (and all other stuff with (Wallpaper) in the name, any screen saver you launch will be kept around, this is not just Aerial or 3rd party, this is all screen savers, including Apple's one)
glouel commented

New bug, manually launching ScreenSaverEngine used to properly launch the screen saver (from this link : https://stackoverflow.com/questions/4505198/how-can-i-start-the-screensaver-and-lock-the-screen-from-the-os-x-terminal ). This no longer works now, you get either unable to reach the lock screen or get in a lock screen loop. This affects Companion launch screen saver button, beta8 is incoming with a fix (which is to use a private API).

Reported as FB12542012

Thank You for all the updates, Looking forward to when this works again.

(Beginning to wonder if I need to break out the Flying Toasters)
Shows how old I am LOL

glouel commented

Ok so, slightly good news, I found a "kinda" workaround to our CPU woes under Sonoma. It's not a perfect fix but basically, instead of waiting for com.apple.screensaver.willstop, I now use com.apple.screenIsUnlocked to pause the video playback, which massively reduce background CPU usage.

This works even if you put a time exemption to your lock screen (say, don't ask password before 5 mins).

This trick will only work with screensavers that implement it, so if you use the Ventura saver alongside Aerial, the Ventura saver will still have major CPU usage (and it stacks every time you launch your screensavers).

This will be available in a new beta of Aerial today. Note: this also fix the pause/resume for Companion integration of the video wallpaper.

glouel commented

It's out here : https://github.com/JohnCoates/Aerial/releases/tag/v3.2.7beta3

If you install, make sure you kill legacyScreenSaver (Wallpaper) first.

Of note:

  • it's a workaround and not a great solution. The event is triggered technically before the screensaver is hidden so you will see Aerial pause during the transition to your desktop, it's not very smooth but it's better than high cpu usage for now
  • new beta3 doesn't seem to change anything from a quick look, multi monitor, this issue, I haven't seen much change if any
  • if you use companion desktop mode, it will now pause and resume correctly when you launch/stop the screensaver. But it will resume at full speed and not your chosen playback speed. This will be fixed in next Aerial beta
glouel commented

So beta 4 is here and a few early thoughts :

  • Multi monitor may be fixed ? At least for two monitors ?
  • Sometimes, screensavers just don't launch ? This seems to happen if I try to launch multiple times in a row quickly via a hot corner. It starts working again if I wait a few seconds.
  • legacyScreenSaver (Wallpaper) is still a thing, meaning, we are still running in the background. But Wallpaper is not localised (eg here in french) :
    Capture d’écran 2023-07-25 aΜ€ 19 50 05
  • System Settings showing the wrong preview, still a thing
  • I'm seeing this colorful things in my logs, which may mean that the com.apple.screensaver.willstop are now correctly fired when supposed to ? This needs double checking :
2023-07-25 20:08:38.718 : πŸ–ΌοΈ πŸ“’πŸ“’πŸ“’ πŸ–ΌοΈ πŸ“’πŸ“’πŸ“’ ☒️sonoma☒️ workaround IGNORING willStop
2023-07-25 20:08:38.720 : πŸ–ΌοΈ πŸ“’πŸ“’πŸ“’ πŸ–ΌοΈ πŸ“’πŸ“’πŸ“’ ☒️sonoma☒️ workaround IGNORING willStop
2023-07-25 20:08:38.749 : πŸ–ΌοΈ πŸ“’πŸ“’πŸ“’ ☒️sonoma☒️ workaround screenIsUnlocked
2023-07-25 20:08:38.750 : πŸ–ΌοΈ πŸ“’πŸ“’πŸ“’ ☒️sonoma☒️ workaround screenIsUnlocked
  • Just to clarify, our instances are still stacked on top of each other, so use Aerial betas for the workarounds for that.

I want to say progress. If you use 2+ monitors and see changes please let me know what you get.

xmddmx commented

Testing beta 4 with iScreensaver, I'm still seeing the bug where the screens are launched out of order: screen[1] before screen[0], and the coordinates are wrong (both screens claim to be located at 0,0 )

1131	default	11:43:22.400861-0700	iScreensaver	########## iScreensaver.init frame=(0.0, 0.0, 1680.0, 1050.0) isPreview=false
1131	default	11:43:22.425282-0700	iScreensaver	########## iScreensaver.init frame=(0.0, 0.0, 1440.0, 900.0) isPreview=false

When I test with Aerial, this doesn't seem to be a problem, and I am getting 2 monitor playback, which is an improvement. But as you say, there are still leftover legacyScreenSaver (Wallpaper) processes which need to be killed.

I'm also seeing failures when I try to launch the screensaver too quickly in succession: completely blank on both monitors.

xmddmx commented

I'm also having a heck of a time getting the right screensaver to run at all. I can build a debug screensaver using iScreensaver, install it, and it shows up in the Preview mode (and the log confirms the correct bundle is being run). But then when I try to run the screensaver, the Ventura screensaver runs instead.

However, if I go back to an earlier build, and select it in the Screen Savers window, and both Preview mode and full screen Hotcorner work normally (at least the correct screensaver runs, though it's buggy).

Are you having any trouble having more than one copy of Aerial installed and finding the OS won't launch the correct one?

glouel commented

I'm still seeing the bug where the screens are launched out of order: screen[1] before screen[0], and the coordinates are wrong (both screens claim to be located at 0,0 )

I think the 0,0 is sadly the new normal, which may make it a huge issue to detect which screen is which, assuming you have two same sized screens. I have to ask, since you have 2 screens with different resolutions, I assume you get the correct window size for each (so no black zones/scaled out stuff ?).

We may need new solutions to detecting which screen we are on, I had a halfway working prototype for that but haven't been able to touch it in a week (may still not for a couple more days).

Out of order launch, same, might be the new normal and not something we can rely on.

Are you having any trouble having more than one copy of Aerial installed and finding the OS won't launch the correct one?

I'm troubled by your description of "more than one copy", you can only get one copy of your screen saver installed at a time, unless you install one as a user and one for everyone, but that's always been a utter mess.

Overall I think this may be because you need for force quit legacyScreenSaver when you install a new version. legacyScreenSaver will keep the old version in memory and keep running it until it gets killed/relaunch. I can see that failing if it needs to load stuff, and revert to a system one in that case. System settings, being its own thing, launch the screensaver separately and will immediately load the new build. ScreenSaverEngine (or whatever that is now) won't.

So make sure you kill legacyScreenSaver before installing a build.

xmddmx commented

I'm troubled by your description of "more than one copy", you can only get one copy of your screen saver installed at a time, unless you install one as a user and one for everyone, but that's always been a utter mess.

Well, in prior OSs I used to be able to make a screensaver, "Screensaver build 1000", and then make a new build in XCode, name it "Screensaver build 1001" and have both installed, and switch between them at will. I do this when debugging, as it makes it very easy to A/B test a new feature or bugfix.

This technique is no longer working in Sonoma: Screensaver 1000 will work in Preview and HotCorner mode, but Screensaver 1001 only works in Preview, but fails in HotCorner mode (the OS runs 'Ventura' saver instead). I believe I've also seen cases where I thought 1001 was running but 1000 was.

I am copying these screensavers from one mac to another, and I'm wondering if I've got some issue with code-signing or notarization?

I'll test your idea about force-quitting legacyscreensaver as well.

I think the OS used to get confused if two installed .savers had the same bundleID, so I'll play around with changing that as well...

glouel commented

Well you must force quit legacyScreenSaver for it to reload in Sonoma, this is what I have to do right now when I make a new build with the same bundle id. You could install 2 screensavers with the same bundle id and different filenames alongside each other, but System Settings didn't load the correct XIBs when you did that for your UI. So try that.

glouel commented

TLDR : Don't use spanned mode in Aerial in Sonoma

Extra beta 4 thoughts :

  • Our stacked up instance DO receive a startAnimation in beta 4, this is new ! And can be fairly catastrophic :
2023-07-26 18:27:26.533 : πŸ–ΌοΈ <AerialView: 0x14a332ff0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.534 : πŸ–ΌοΈ <AerialView: 0x14a2ca0a0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.534 : πŸ–ΌοΈ <AerialView: 0x14cf0b1c0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.535 : πŸ–ΌοΈ <AerialView: 0x14a3ee570> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.535 : πŸ–ΌοΈ <AerialView: 0x148e1cea0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.535 : πŸ–ΌοΈ <AerialView: 0x14a071a00> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.535 : πŸ–ΌοΈ <AerialView: 0x14a4394c0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.535 : πŸ–ΌοΈ <AerialView: 0x14a444fc0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.536 : πŸ–ΌοΈ <AerialView: 0x14a3c89d0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.536 : πŸ–ΌοΈ <AerialView: 0x14a3e5d90> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.536 : πŸ–ΌοΈ <AerialView: 0x14ca22930> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.536 : πŸ–ΌοΈ <AerialView: 0x13a345be0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.537 : πŸ–ΌοΈ <AerialView: 0x13a345f40> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.537 : πŸ–ΌοΈ <AerialView: 0x14a272b50> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.537 : πŸ–ΌοΈ <AerialView: 0x14a4281f0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.537 : πŸ–ΌοΈ <AerialView: 0x14a2a5e30> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.538 : πŸ–ΌοΈ <AerialView: 0x13a306b10> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.538 : πŸ–ΌοΈ <AerialView: 0x14a2fcf30> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.538 : πŸ–ΌοΈ <AerialView: 0x14a2a9890> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.538 : πŸ–ΌοΈ <AerialView: 0x13a376d50> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.538 : πŸ–ΌοΈ <AerialView: 0x148f3bed0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.539 : πŸ–ΌοΈ <AerialView: 0x13a370cb0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.539 : πŸ–ΌοΈ <AerialView: 0x14ca36200> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.539 : πŸ–ΌοΈ <AerialView: 0x14a083ea0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.539 : πŸ–ΌοΈ <AerialView: 0x14a431520> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.540 : πŸ–ΌοΈ <AerialView: 0x13a376750> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.540 : πŸ–ΌοΈ <AerialView: 0x14a0c73c0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.540 : πŸ–ΌοΈ <AerialView: 0x12882fa50> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.540 : πŸ–ΌοΈ <AerialView: 0x14c93d2e0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.540 : πŸ–ΌοΈ <AerialView: 0x128a62fd0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.541 : πŸ–ΌοΈ <AerialView: 0x14cf7ce80> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.541 : πŸ–ΌοΈ <AerialView: 0x13a3a3930> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.541 : πŸ–ΌοΈ <AerialView: 0x14cf38ab0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.541 : πŸ–ΌοΈ <AerialView: 0x14a0ec970> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.541 : πŸ–ΌοΈ <AerialView: 0x128a9cf00> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.542 : πŸ–ΌοΈ <AerialView: 0x14cf3a850> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.542 : πŸ–ΌοΈ <AerialView: 0x14ca3eff0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.542 : πŸ–ΌοΈ <AerialView: 0x12c60dc40> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.542 : πŸ–ΌοΈ <AerialView: 0x12884be60> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.542 : πŸ–ΌοΈ <AerialView: 0x14cf2cdd0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.542 : πŸ–ΌοΈ <AerialView: 0x14cf95130> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.543 : πŸ–ΌοΈ <AerialView: 0x14cfe1090> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.543 : πŸ–ΌοΈ <AerialView: 0x14cf5d090> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.543 : πŸ–ΌοΈ <AerialView: 0x128afe410> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.543 : πŸ–ΌοΈ <AerialView: 0x14cfb3270> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.543 : πŸ–ΌοΈ <AerialView: 0x14cfcf9a0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.543 : πŸ–ΌοΈ <AerialView: 0x14cfc7f10> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.544 : πŸ–ΌοΈ <AerialView: 0x14a2b1310> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.544 : πŸ–ΌοΈ <AerialView: 0x1323d3090> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.544 : πŸ–ΌοΈ <AerialView: 0x12ede9e70> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.544 : πŸ–ΌοΈ <AerialView: 0x13080fd30> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.544 : πŸ–ΌοΈ <AerialView: 0x12c6e43f0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.545 : πŸ–ΌοΈ <AerialView: 0x11f253a70> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.545 : πŸ–ΌοΈ <AerialView: 0x11f253470> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.545 : πŸ–ΌοΈ <AerialView: 0x12c6f63b0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.545 : πŸ–ΌοΈ <AerialView: 0x148f7baa0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.553 : πŸ–ΌοΈ <AerialView: 0x14a332ff0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.553 : πŸ–ΌοΈ <AerialView: 0x14a2ca0a0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.554 : πŸ–ΌοΈ <AerialView: 0x14cf0b1c0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.554 : πŸ–ΌοΈ <AerialView: 0x14a3ee570> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.554 : πŸ–ΌοΈ <AerialView: 0x148e1cea0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.554 : πŸ–ΌοΈ <AerialView: 0x14a071a00> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.554 : πŸ–ΌοΈ <AerialView: 0x14a4394c0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.554 : πŸ–ΌοΈ <AerialView: 0x14a444fc0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.555 : πŸ–ΌοΈ <AerialView: 0x14a3c89d0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.555 : πŸ–ΌοΈ <AerialView: 0x14a3e5d90> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.555 : πŸ–ΌοΈ <AerialView: 0x14ca22930> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.555 : πŸ–ΌοΈ <AerialView: 0x13a345be0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.555 : πŸ–ΌοΈ <AerialView: 0x13a345f40> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.555 : πŸ–ΌοΈ <AerialView: 0x14a272b50> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.556 : πŸ–ΌοΈ <AerialView: 0x14a4281f0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.556 : πŸ–ΌοΈ <AerialView: 0x14a2a5e30> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.556 : πŸ–ΌοΈ <AerialView: 0x13a306b10> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.556 : πŸ–ΌοΈ <AerialView: 0x14a2fcf30> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.556 : πŸ–ΌοΈ <AerialView: 0x14a2a9890> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.556 : πŸ–ΌοΈ <AerialView: 0x13a376d50> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.557 : πŸ–ΌοΈ <AerialView: 0x148f3bed0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.557 : πŸ–ΌοΈ <AerialView: 0x13a370cb0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.557 : πŸ–ΌοΈ <AerialView: 0x14ca36200> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.557 : πŸ–ΌοΈ <AerialView: 0x14a083ea0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.557 : πŸ–ΌοΈ <AerialView: 0x14a431520> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.557 : πŸ–ΌοΈ <AerialView: 0x13a376750> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.558 : πŸ–ΌοΈ <AerialView: 0x14a0c73c0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.558 : πŸ–ΌοΈ <AerialView: 0x12882fa50> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.558 : πŸ–ΌοΈ <AerialView: 0x14c93d2e0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.558 : πŸ–ΌοΈ <AerialView: 0x128a62fd0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.558 : πŸ–ΌοΈ <AerialView: 0x14cf7ce80> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.558 : πŸ–ΌοΈ <AerialView: 0x13a3a3930> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.559 : πŸ–ΌοΈ <AerialView: 0x14cf38ab0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.559 : πŸ–ΌοΈ <AerialView: 0x14a0ec970> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.559 : πŸ–ΌοΈ <AerialView: 0x128a9cf00> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.559 : πŸ–ΌοΈ <AerialView: 0x14cf3a850> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.559 : πŸ–ΌοΈ <AerialView: 0x14ca3eff0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.559 : πŸ–ΌοΈ <AerialView: 0x12c60dc40> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.560 : πŸ–ΌοΈ <AerialView: 0x12884be60> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.560 : πŸ–ΌοΈ <AerialView: 0x14cf2cdd0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.560 : πŸ–ΌοΈ <AerialView: 0x14cf95130> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.560 : πŸ–ΌοΈ <AerialView: 0x14cfe1090> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.560 : πŸ–ΌοΈ <AerialView: 0x14cf5d090> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.560 : πŸ–ΌοΈ <AerialView: 0x128afe410> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.561 : πŸ–ΌοΈ <AerialView: 0x14cfb3270> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.561 : πŸ–ΌοΈ <AerialView: 0x14cfcf9a0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.561 : πŸ–ΌοΈ <AerialView: 0x14cfc7f10> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.561 : πŸ–ΌοΈ <AerialView: 0x14a2b1310> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.561 : πŸ–ΌοΈ <AerialView: 0x1323d3090> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.561 : πŸ–ΌοΈ <AerialView: 0x12ede9e70> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.562 : πŸ–ΌοΈ <AerialView: 0x13080fd30> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.562 : πŸ–ΌοΈ <AerialView: 0x12c6e43f0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.562 : πŸ–ΌοΈ <AerialView: 0x11f253a70> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.562 : πŸ–ΌοΈ <AerialView: 0x11f253470> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.562 : πŸ–ΌοΈ <AerialView: 0x12c6f63b0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)
2023-07-26 18:27:26.562 : πŸ–ΌοΈ <AerialView: 0x148f7baa0> startAnimation frame (0.0, 0.0, 2560.0, 1440.0) bounds (0.0, 0.0, 2560.0, 1440.0)

(that's a whole lot) Good news is, at least for Aerial, those instances are "marked as dead" so I don't start anything when receiving those events. So be mindful of that one as this is a significant change from b3.

  • Screen detection, at least mine, is broken. For users, that means that stuff like disabling screens, or using spanned mode, will currently fail in Sonoma. I'm hoping to fix that at some point. This is related to the fact that our view position is zeroed (0,0), instead of its real position, and I used the frame position to match to the correct monitor. This is annoying !
2023-07-26 18:27:26.760 : πŸ–ΌοΈ Using : Optional([id=9, width=2560, height=1440, bottomLeftFrame=(0.0, 0.0, 2560.0, 1440.0), topRightCorner=(2560.0, 1440.0), isMain=true, backingScaleFactor=2.0])
2023-07-26 18:27:26.780 : πŸ–ΌοΈ Using : Optional([id=9, width=2560, height=1440, bottomLeftFrame=(0.0, 0.0, 2560.0, 1440.0), topRightCorner=(2560.0, 1440.0), isMain=true, backingScaleFactor=2.0])

Again, this is annoying 😩 I need to test my workarounds for this. Because right now you can set a different screensaver PER space (not just per screen), I want to do screen + space detection but again, this is still WIP.

  • I'm going back to using com.apple.screensaver.willstop instead of com.apple.screenIsUnlocked as I believe it's the better event, though this may be wrong. It's again safe to use in beta4 but I will need to wait because of the outdated public beta...

  • I believe that the blank screen we get on multiple subsequent launches is just the whole "instance passing" mechanism being the issue, legacyScreenSaver goes from Screen Saver to Wallpaper mode, WallpaperAgent seems to now be handling it, waits a bit to try to generate a "fixed" wallpaper (again some mechanism that was built for their transition thing) and during that time, it seems that if you relaunch it the whole system is just "busy". Wait 30 secs and it works again. Eh.

glouel commented

@xmddmx

I've tried a few things to get the screen we're at but so far I have nothing. Do you currently have a working detection system ? I've seen you talk about screen initialization order and I'm wondering if you have any matching working.

The way I did it previously was to match the view's frame origin (which is now bugged to (0,0)).

I tried playing with self.window.screen but that just returns nil under legacyScreenSaver, independent of when its called. It does work however under System Settings, but that's not particularly helpful.

xmddmx commented

I also had the idea of using self.window.screen, but found that it was nil inside the intializer. I was hoping that later on (inside .Draw or AnimateOneFrame) it would be non-nil, but hadn't done those tests yet.

Sounds like you have and my idea won't work?

Have you filed a feedback case for this specific bug? Sometimes the really focused cases, such as "frame.origin is 0,0" seem to be more likely to get fixed.

glouel commented

I also had the idea of using self.window.screen, but found that it was nil inside the intializer. I was hoping that later on (inside .Draw or AnimateOneFrame) it would be non-nil, but hadn't done those tests yet.

Can confirm, I did try to check at a later time (after 30+ seconds) and still nil. Init was nil for when running in System Settings but showed results a tiny bit afterwards. But yeah, useless in the end.

Have you filed a feedback case for this specific bug? Sometimes the really focused cases, such as "frame.origin is 0,0" seem to be more likely to get fixed.

Not yet but will do tomorrow ! At this point I'm out of ideas, I did look at private APIs but didn't find anything that would help. So let's try and get that one fixed.

glouel commented

@xmddmx took me a while as I got busy but finally made a very detailed report of the frame origin issue as FB12831305

Initial testing of Sonoma beta 5:

  • the initialize() call is still being passed the wrong screen coordinates (all screens claim to be at 0,0).
  • It feels like I'm having an easier time switching between various debug builds of iScreensaver - on beta 4 the wrong code was often running
  • still seeing many dead/hung processes, you really have to force-quit the Legacy Screensaver process after each test run.

I have added some code to try to work around the screen coordinates issue: basically, it ignores the X,Y coordiante and matches on Width,Height instead. This is working OK on my test system (which has 2 different-sized monitors) but would fail on a system with 2 identical monitors.

So far in testing, it does seem like screen[1] is always initialized before screen[0] in Sonoma. If this is reliable, then it may be possible to come up with an algorithm to figure out which screen is which based on this reverse order, combined with the width x height values?

  • another issue affecting iScreensaver: if your ScreenSaverView has a WKWebView, it seems to matter when you show the webView: we were doing something where the webview was hidden, then loads in, and only once loaded do we set .isHidden = false (we do this to avoid a flash before the HTML has loaded). In Sonoma this is not working properly. I believe Aerial doesn't use any WebViews so this is not an issue?

I only spent 10 minutes on b5 but broadly agree.

Regarding the last chance matching thing, I think there's a backing scale factor property that can be used too to differentiate (it's the retina factor, not 100% sure how it's called, it's 2 for retina, 1 for regular screens), I think that may be an extra layer (I personally use 2 2560 monitors, one from Studio Display with retina and one old Asus thing without so that would work).

I think the order thing is also a last last layer of fallback, I'm wondering if it's main, secondary, secondary or left, middle, right, if you see what I mean ? (reversed in that case).

Worth investigating. I'll try to have another look tomorrow.

I've never had this much frustration testing screensavers in macOS, it seems like Sonoma is really trying to make our lives hard. For example, is there any way our processes can detect when they've been "converted" (?) into the (Wallpaper) process?

Would it be enough to get the ProcessInfo and look for "(Wallpaper)" and terminate if so? Or is that string translated into various languages...

I've never had this much frustration testing screensavers in macOS, it seems like Sonoma is really trying to make our lives hard.

I guess I'm getting numb to it at this point but yeah, they are trying hard to break our spirits ;)

For example, is there any way our processes can detect when they've been "converted" (?) into the (Wallpaper) process?

Well there's the willStop event now that works pretty well if you just want to detect that (I use that to "stop" my view).

Would it be enough to get the ProcessInfo and look for "(Wallpaper)" and terminate if so? Or is that string translated into various languages...

So I can be 100% wrong on this but I don't believe that we, the screensaver, can terminate ourselves, we are just plugins to some code and I don't think we can kill ourselves from the saver itself. I mean, I could be wrong, but basically it's :

ScreenSaverEngine.app -> legacyScreenSaver.appex -> us

And at some point we (legacyScreenSaver) get passed around, so even if we could kill ourselves, what would we kill ? ScreenSaverEngine, WallpaperAgent ? Who knows ? From an external app though ? 100% you could check for that but yeah, that string is 100% translated in other languages since beta 2 (I think). It says "Fond d'Γ©cran" here for example.

An external app though could look for screensaver.willStop event, wait a bit, kill legacyScreenSaver*, and be done with it. It seems a lot of work for something they'll 100% have to address on their end before the end of the beta cycle though, as it makes intel macs 100% unusable with 1st party screen savers. But if you are frustrated enough by it, sure, that could be the way to go.

Remember though that a user can set different screen saver per screen, and if you set one of the new Apple ones and yours on different screens, legacyScreenSaver does kinda have a purpose in that case. So wait like 10/30s before killing maybe is safe (from my understanding of how they do things, it's fine to kill at that point).

Still, too much work for something they'll fix IMO. I wouldn't be so confident on the screen coord issue though.

@xmddmx very quick update I spent the afternoon trying to make a workaround for the coordinates thing, some notes :

  • I implemented something in the (probably common) case where people have monitors with different resolution. This works fine (instead of comparing the ScreenSaverView/NSScreen frame as I previously did, I compare frame.size).
  • In the case where a user has 2 screens with similar resolutions, but maybe one is retina and one isn't (which just happens to be my personal setup), I tried using the backingScale factor idea I had mentioned above. This is a bust at least for me, it looks like when I check for this, ScreenSaverView.window still returns nil, so I cant' read the backingScale. This is annoying.
  • Based on our impressions that screens were initialised "backwards", I tried doing a "first match" from a reverse screen array. This didn't work on my setup (a M1 Ultra Mac Studio), either I messed something, or the order in which we are initialised is random, at least on my setup. Annoying again !

At this point I still don't see a way to make a 100% coverage fix but I'll take the better than nothing one.

The other hot issue for me is setting the screen saver as default. Because you can now set a different screen saver per screen/space, the old way to set a screen saver by editing a plist doesn't work anymore. I tracked the new plist, I don't remember if I talked about it here, but the relevant configuration keys are a binary blob which is not helpful. You can check if you are set as default still, but not set yourself default (the old plist is still written by System Settings, but ignored in the end). Annoying.

Bugs are filled, finger are crossed, and other stuff was done. We'll see if it bears fruit.

but the relevant configuration keys are a binary blob which is not helpful

That sucks, but I do love me a reverse-engineering binary blob puzzle! But I'm guessing the blob is not writeable by us mortals...

but the relevant configuration keys are a binary blob which is not helpful

That sucks, but I do love me a reverse-engineering binary blob puzzle! But I'm guessing the blob is not writeable by us mortals...

I believe it should be, it's in the user directory, but in such a weird place :

~/Library/Application Support/com.apple.wallpaper/Store/Index_v2.plist

From my understanding each screen is represented twice, there's a key provider that either says wallpaper or screen saver basically. If you use spaces, you'll have 2 entries per screen/space. Looks roughly like this :

Content = {
    Choices = (
        {
            Configuration = {length = 130, bytes = 0x62706c69 73743030 d1010256 6d6f6475 ... 00000000 0000005d };
            Files = ();
            Provider = "com.apple.wallpaper.choice.screen-saver";
        }
    );
    Shuffle = "$null";
};

They may change the whole thing but if you want to have a go, go ahead ;) It may be a simple encoding, I admit I didn't even try the basic things.

So I can be 100% wrong on this but I don't believe that we, the screensaver, can terminate ourselves, we are just plugins to some code and I don't think we can kill ourselves from the saver itself. I mean, I could be wrong, but basically it's :

ScreenSaverEngine.app -> legacyScreenSaver.appex -> us

And at some point we (legacyScreenSaver) get passed around, so even if we could kill ourselves, what would we kill ?

For fun, I tried adding a call to

exit(0)
within the
com.apple.screensaver.willstop
notification handler, and it actually seems to work to terminate the LegacyScreensaver process.

Is this a bad thing to do? Not sure, but it seems like it will sure make testing easier for the time being.

@xmddmx I had no idea this would actually work... this is completely awesome !

Because of the passing thing, is it good, is it bad ? is there a better time (screenIsUnlocked could either be better or too late) ? I don't know, and I want to say who cares ! I'm completely amazed that it works, thanks for giving it a go πŸ˜…

I'll give it a shot locally too and let you know if I see bad things happening, but yeah this will make :

  1. dev/testing much simpler
  2. possibly fix many many many issues

Kudos ! I'll add it locally later today and report back if I see weird stuff. I mean maybe some stuff will complain in console, but it's not like it's not complaining already anyway !

It works, it looks pretty clean, I tried doing it a bit later, but it looks like we may not be able to exit if we are already passed to the wallpaper thing, so I think you have it right here. I'll report if there's an issue.

This is interesting, I added some debug info to look at the NSView's.Window property:

iScreensaver	[0] ########## .frame = (10000.0, 10000.0, 1440.0, 900.0)
iScreensaver	[0] ########## .windowNumber = 814
iScreensaver	[1] ########## .frame = (10000.0, 10000.0, 1680.0, 1050.0)
iScreensaver	[1] ########## .windowNumber = 815

  • Each instance does appear to have its own NSWIndow (windowNumbers are diffrerent)
  • However, the "10000,10000" dummy XY coordinates are interesting, suggesting this is not a normal window. Perhaps it's some sort of offscreen window whose content is getitng piped over to the display monitor?

Unfortunately, it seems that .windowNumber is set in the order of creation, and I'm finding that sometimes screen[1] initliazes before screen[0] so we can't use windowNumber to figure out which screen is which.

I also noticed this:

NSWindow.deviceDescription:

screen[0] ########## .deviceDescription = [__C.NSDeviceDescriptionKey(_rawValue: NSDeviceBitsPerSample): 8, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceResolution): NSSize: {144, 144}, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceIsScreen): YES, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceSize): NSSize: {1440, 900}, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceColorSpaceName): NSCalibratedRGBColorSpace]


screen[1] ########## .deviceDescription = [__C.NSDeviceDescriptionKey(_rawValue: NSDeviceResolution): NSSize: {144, 144}, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceColorSpaceName): NSCalibratedRGBColorSpace, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceSize): NSSize: {1680, 1050}, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceIsScreen): YES, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceBitsPerSample): 8]

Interesting, but doesn't seem to be anything that would help determine which screen is which.

@xmddmx how soon are you getting this ? I can't seem to read ScreenSaverView.window during my init process. Interesting nonetheless, maybe they park those windows out of view before putting them in place, legacyScreenSaver.appex seems like a huge pile of hack it's so crazy...

I'm checking a few seconds later.

Also, my setup is probably different than yours, in that I have a ScreenSaverView, which adds a NSView, which adds a WKWebView. I'm seeing a lot of weird bugs in the WKWebView with HTML/JavaScript/CSS video and animations not working properly, but I don't think Aerial uses any of that.

Maybe the presence of a WebView is changing things?

It might be worth for you to try adding a dummy NSView to your ScreenSaverView and see if perhaps that view will have a useable .window value?

Good tips, I'll have a look and let you know, thanks !

The exit() fix is broadly great. It appears the fix doesn't work in the following scenario:

  1. Let screen saver activate
  2. Let system fall asleep
  3. Wake system with touchID (such that you don't see the screen saver)

There may be another notification that we can hook onto that would catch this situation.

Yep, I think maybe this notification, willSleepNotification, can be used :

NSWorkspace.shared.notificationCenter.addObserver(
        self, selector: #selector(onSleepNote(note:)),
        name: NSWorkspace.willSleepNotification, object: nil)

Made a quick build we'll see tomorrow how it goes. There's an IORegister thing also mentionned where I found that notification (bottom of the page) but I'd rather not use that : https://stackoverflow.com/questions/9247710/what-event-is-fired-when-mac-is-back-from-sleep

@xmddmx so the trick seems to work for system going to sleep, patch is here : 0fe3461

I insta-exit when I get the notification, I think it's safer this way (we want to act before the system is put to sleep). Also logging is useless here (unless your logging is atomic and that's probably bad, we want to go fast).

As pointed to me earlier by @zeromhz the bug that we saw since beta 4 on quick subsequent launches behaves slightly differently. We get a black screen in that case.

This is more or less what we got earlier but I had noticed that it actually showed back the old (unkilled) instance in those cases (I could still see my uncleared overlays from previous launch).

It still seems fine in any case and I'll roll with both those changes after a tiny bit of cleanup in next beta, it should provide a very good improvement as even on my Mac Studio each view still took half a percent cpu with me tearing down everything I could think of.

Beta 6 came and went - very brief testing showed no major changes.
Beta 7 is out today. Downloading it now.

In Beta 7, I'm not seeing any functional changes to the big issues:

  • the Initialize call still has all screens at X,Y=0,0
  • unless your screensaver calls exit(), the legacyScreenSaver processes get stuck and never exit
  • WKWebView does not work properly inside LegacyScreenSaver (this bug doesn't affect Aerial, but does affect iScreensaver, and has been reported as FB13094564 )

Yeah, I just tried a fresh reinstall of the latest beta after updating to Sonoma DP Beta 7 and just get the black screen and then need to killl LegacyScreenSaver.

I've bothered the developer(s) about this a few times and I know they're aware of the issue. I'm assuming that they're waiting for the Sonoma GM to drop so they don't waste dev energy and time on a potentially moving goalpost. I'm excited though, as I finally caved and got an OLED TV after decades of using small monitors only. But I know it will be worth the wait!

  • the Initialize call still has all screens at X,Y=0,0

@xmddmx Sooooo, I think there may be a workaround but I'm not 100% sure of how to get there. Someone pointed me to this https://www.jwz.org/blog/2023/08/xscreensaver-6-07-out-now/

In the release notes he talks about the 13.3 horror show and mentions a callback I wasn't looking at yet, viewDidMoveToWindow.

Here's the good news, if I override that thing on ScreenSaverView, from there I can read window.screen and it actually works correctly.

2023-09-03 19:00:02.060 : πŸ₯¬πŸŒΎ window.screen Optional("Optional(<NSScreen: 0x135a14e70; name=\"Studio Display\"; backingScaleFactor=2.000000; frame={{0, 0}, {2560, 1440}}; visibleFrame={{47, 0}, {2513, 1415}}>)")

(this is self.window?.screen.debugDescription, self.window?.screen is correctly initialised here)

The bad news is, this seems to get triggered by "something" (maybe something just before startAnimation), and I'm loading the video way too early in Aerial, so, right now, I can't use that trick to use the correct video on the correct screen. But I'm 100% sure there's a path in there. We just need to asynchronously grab the screen from that call, and only make the decision on what to display after we got it. I tried delaying things with some thread sleep but that's rarely the way to go, and it didn't work.

I'm not sure any of that makes sense, but I thought I'd give you a heads up as you may not have the same constraints I do and grabbing the screen there may work for you.

@xmddmx I looked at his code to fix the Ventura bug and I feel extremely dumb now, we were so, so close...

The important bit I missed is that you can use NSApplication.shared.windows to list every window that belongs to legacyScreenSaver and that changes everything 😭

From then on you can just walk those, check if there's a view attached, if not attach an unattached one, then reorder if needed... 🀦

I haven't checked yet when exactly he triggers this, I'll dig a bit more tomorrow. If you feel like it, I suggest you look at the code (get the linux distribution on his site, then look for VENTURA_KLUDGE in OSX).

Edit : Interestingly now those windows are of a weird type, that might be the new thing they introduced in legacyScreenSaver :

2023-09-03 19:37:40.562 : window : <NSServiceViewControllerWindow: 0x15530e780>
2023-09-03 19:37:40.562 : window : <NSServiceViewControllerWindow: 0x151a47480>

This is great - I found a pretty simple workaround:

  • when ScreensaverView.init(frame, isPreview) is called, don't do much, just set a timer callback to a delayedInitialize() function
  • before the delayedInitialize() is called, hasConfigureSheet(), viewDidMoveToWindow(), and draw() will be called - since your instance will not be initalized yet, you may need add some guard code to these functions.
  • then, when the delayedInitialize() is called, you can then use the following code to figure out which screen you are actually on:
private func delayedInitialize() {
  [...]
  let screenNumber = NSScreen.getScreenNumber(view:self)
  [...]
}


extension NSScreen {
    class func getScreenNumber(view:NSView) -> Int {
        let sa = NSScreen.screens
        for i in 0..<sa.count {
            let s = sa[i]
            let s2 = view.window?.screen
            if s == s2 {
                return i
            }
        }
       return -1    
    }
}

I believe this solves the Sonoma "which screen am I on" issue. It does not solve the Ventura 13.4 problems (where Views are on the wrong screen or not on a window at all) which is what the VENTURA_KLUDGE fix is all about in XSaver. Since that's mostly fixed in Ventura 13.5, I'm not sure if it's worth worrying about that extra complexity...

@xmddmx awesome !

I'll give it a shot tomorrow ! how long do you set your timer for, 1s ?

I did give it a quick shot earlier by trying to delay things but at a much later point, and it just gets stupid because there's so so so many legacy workaround for every single bug that we've been facting that I couldn't make anything proper. Delaying early is definitely a better idea πŸ’ͺ

I'm using .01 seconds and it seem fine.

I'm using .01 seconds and it seem fine.

Yeah looks like that's plenty, you were right !

2023-09-04 15:12:05.104 : πŸ–ΌοΈ AVinit (.saver) (0.0, 0.0, 2560.0, 1440.0) p: false o: false
2023-09-04 15:12:05.105 : πŸ–ΌοΈ <AerialView: 0x127914650> viewDidMoveToWindow frame: (0.0, 0.0, 2560.0, 1440.0) window: Optional(<NSServiceViewControllerWindow: 0x136f32d10>)

Btw, I completely messed up my attempt yesterday, using thread.sleep instead of dispatchqueue 🀦I'll blame that on the shock that 13.3 was fixable, still haven't digested this one πŸ˜…

Still need to button up things but will have a new Aerial today with that fix.

Right now to recap the issues that we haven't workarounded :

  • WKWebView does not work properly inside LegacyScreenSaver (this bug doesn't affect Aerial, but does affect iScreensaver, and has been reported as FB13094564 )
  • Can't set a screen saver programatically (they changed the file from Index_v2 to Index, we had good luck unscrambling the data, but modifying it doesn't seem to work from someone who tested it)

There's probably other annoyances I missed but those are the big ones I think ?

In Ventura, I was never able to get keyboard events working, but I was able to find a way to capture mouse events (move, click) which allows us to keep a onscreen UI (HUD: heads-up-display) that could be used while the screensaver was running, though I could only get it to work on a single monitor.

Given that legacyScreensaver in Sonoma appears to be totally rewitten from scratch, I though "why not try again?"

Here's what I've found:

  • In Ventura, you can capture mouse events with a local event monitor:
    NSEvent.addLocalMonitorForEvents(matching: [.leftMouseDown, .leftMouseUp, .leftMouseDragged], handler:mouseEventMonitor)
    You can, by returning nil from the event monitor, acutally block events from the OS seeing them.

This technique doesn't work in Sonoma - the events are never received.

  • in Sonoma, you can get events via a global event monitor:
    NSEvent.addGlobalMonitorForEvents(matching: [.mouseMoved, .leftMouseDown, .leftMouseUp, .leftMouseDragged], handler:mouseEventMonitor)
    In Sonoma, the events are received. However, this global event monitor works differently: for example, you can't return 'nil' to block an event.

When I log these events, event.window == nil, however the windowNumber is not 0, and in fact the windowNumber is a different windowNumber than self.window.

I suspect what's going on here is that in Sonoma, our screensaver is being run in some weird offscreen NSServiceViewControllerWindow, which is somehow displaying its contents to a different window which is a real window.

Unfortuantely, this real window does not show up in NSApplication.shared.windows so there doesn't seem an easy way to hack into it.

So at the moment, it looks like getting mouseMove events is possible (which could be somewhat useful for an onscreen UI/HUD) but any sort of clicking always exits the screensaver.

I will keep investigating.

When I log these events, event.window == nil, however the windowNumber is not 0, and in fact the windowNumber is a different windowNumber than self.window.

This is definitely weird.

I suspect what's going on here is that in Sonoma, our screensaver is being run in some weird offscreen NSServiceViewControllerWindow, which is somehow displaying its contents to a different window which is a real window.

This seems quite probable and would explains a lot of the weird stuff we've seen.

Unfortuantely, this real window does not show up in NSApplication.shared.windows so there doesn't seem an easy way to hack into it.

I'm wondering why they tried to isolate all this but I would guess those ServiceViews are like super locked down for security reasons, could explain why you saw 10000 coordinates a while back.

So at the moment, it looks like getting mouseMove events is possible (which could be somewhat useful for an onscreen UI/HUD) but any sort of clicking always exits the screensaver.

Do you think we can get trackpad gestures ? Like swipe left/right ? They may be the closest thing we can get to an actual button press if we can't cancel events (which I think they absolutely don't want us to do and will fight us endlessly on).

I will keep investigating.

Keep me posted (I realise I got swamped last year when you emailed me about this, super sorry again).

Re: WKWebView issues - I've reported this to apple as FB13094564, and I also submitted a TSI ticket today. THere's another third party screensaver that suffers from this as well so I submitted an Issue with that project also: liquidx/webviewscreensaver#75

(Since Aerial doesn't use WebKit, I feel kind of bad mentioning those issues here, though it certainly is handy to have 'all the sonoma bugs' in one place. )

Also, in quick testing of today's Sonoma RC release, I don't see any changes to the bugs we've found in legacyScreenSaver: screens are still misreported as (0,0), the legacyScreenSaver processes get stuck (someimtes running at 100% CPU), etc.

Tested today's Sonoma RC2 release: no changes that I can see, still buggy as before.

Testing in 14.1 beta 1 and today's 14.1 beta 2: I don't see any changes to the bugs we've found in legacyScreenSaver: screens are still misreported as (0,0), the legacyScreenSaver processes get stuck (someimtes running at 100% CPU), etc.

Yeah at this point I have little expectations they'll fix those.

I did find (through users) a few extra things :

  • Locations services are unusable as a screen saver (even with legacyScreenSaver being allowed to use location services)
  • You can no longer run command line stuff (this breaks my nightshift integration, and script running)
  • I've had weird reports on 13.5.2 that I never confirmed but they may have backported some Sonoma bugs there, this wouldn't be the first time

I'm starting to get slightly annoyed at this point πŸ˜…

  • You can no longer run command line stuff (this breaks my nightshift integration, and script running)

I'm not seeing this, in fact I just solved an issue using the command line. My screensaver uses a standalone helper app to run the 'Options" UI, and although the .saver could easily launch the app (by using Process.launch() ) a subsequent press of the 'Options' button was no longer able to activate the app window, because NSRunningApplication.runningApplications(withBundleIdentifier: was no longer finding the app.

The solution was to simply issue a shell command:
open Helpers/Helper.app

which brings the app frontmost.

Note: my helper app lives inside the .saver bundle - pehaps this is important?

p.s. here's my "Shell" function which uses Process() also:

func shell(_ command:String ) -> String
{
    let task = Process()
    var arguments = ["-c"]
    arguments.append(command)
    task.launchPath = "/bin/bash"
    task.arguments = arguments
    
    let pipe = Pipe()
    task.standardOutput = pipe
    task.launch()
    
    let data = pipe.fileHandleForReading.readDataToEndOfFile()
    let output = String(data: data, encoding: String.Encoding.utf8)!
    if output.count > 0 {
        //remove newline character.
        let lastIndex = output.index(before: output.endIndex)
        return String(output[output.startIndex ..< lastIndex])
    }
    return output
}

Ha interesting, thanks for the info!

I'll double check how I'm doing it and see if I can find the root cause.

@xmddmx

Do you run terminal programs by any chance, or a shell script ? Here's what I used to do that no longer works :

Process.launch() with

  1. directly launching a shell script (test.sh is chmod +x and a basic bash script)
task.launchPath = "/Users/Shared/test.sh"

This fails as "permission denied" (it didn't prior to 14.0, and works outside the screensaver sandbox)

  1. calling a terminal utility
task.launchPath = "/usr/libexec/corebrightnessdiag"
task.arguments = ["nightshift-internal"]

Fails as permission denied too (again this worked fine before, and works outside the sandbox)

I looked a bit at using open instead, but this is mostly for launching apps, not command line stuff. I tried launching Terminal.app but it no longer appears to be in /Applications/Utilities/ (I guess it's aliased from somewhere else...)

I think open is probably the way to go, but it's apparently a mess to be able to pass parameters around and get the output if you go with open. I'll keep digging a bit, but thanks for the info.

Edit : Some extra stuff I tried

  • Terminal.app is now in /System/Applications/Utilities/Terminal.app
  • Using executableURL instead of launchPath doesn't change anything
  • For the script, the error to be precise is : /Users/Shared/test.sh: line 2: /bin/ps: Operation not permitted

Where my test.sh has :

#!/bin/bash
ps -A -o %cpu | awk '{s+=$1} END {print s "%"}'

(it's a quick way to get cpu usage)

So I think it's more likely that any system command is locked down, is the exact issue.

Replacing the script with :

#!/bin/bash
echo "hello"

Works fine (but is pretty useless obviously).

See #1305 (comment) for my actual code. The only command I'm using so far is "open /path/to/my/helper.app" which works fine - but maybe because the helper app is inside my .saver bundle, the sandbox allows it?

Also, I just tested this from within the .saver

# get display sleep time
/usr/bin/pmset -g | /usr/bin/grep displaysleep | /usr/bin/awk '{ print $2 }'

And I tested it, and it does work fine in Sonoma.

Ha, I didn't see your edit, thanks a whole lot I'll give it a shot !

Ok this is getting crazy, your script does work as is with my old code πŸ˜…

May I ask you try ps or /usr/libexec/corebrightnessdiag ? I have a hard time understanding how pmset is ok but not ps, makes 0 sense... I'll keep digging, thanks for your help.

in my .saver in 14.1 beta 2 on a M1 macbook air

using the shell() function above,

shell("/usr/libexec/corebrightnessdiag")

returns

corebrightnessdiag [nightshift] [sunschedule] [nightshift-internal] [status-info]

The command 'ps' however, returns nothing.

Ok I think your /bin/bash trick seems to work, I'm looking at passing arguments right now and I'm not there yet, but I think I'm close. Thanks a whole lot !

To be precise, in a terminal, this is what I need to run :

/usr/libexec/corebrightnessdiag nightshift-internal

I'm having issues passing the nightshift-internal right now but I'll keep digging.