Animate window move/resize/swap/warp

[dominiklohmann]

Discussion / Feature request

This is a long shot, and probably far off in the future. This has been previously discussed in https://github.com/koekeishiya/chunkwm/issues/144.

I've dug up some stuff regarding this.

• How Chromium resizes and moves windows using CGS* functions https://github.com/ChromiumWebApps/chromium/blob/c7361d39be8abd1574e6ce8957c8dbddd4c6ccf7/chrome/browser/ui/cocoa/constrained_window/constrained_window_animation.mm
• A (very old) blog post explaining how CGSSetWindowWarp works http://kevin.sb.org/2006/07/23/cgssetwindowwarp-explained/
• An working example for using CGSSetWindowWarp and CGSSetWindowTranslation for custom animations https://github.com/sailesha/CGSSetWindowWarp-Sample/blob/master/windowAnimation.mm

Obviously all the above examples require access to the NSWindow instance for async animations, which is inaccessible from the Dock process—which is the only process yabai injects code into.

Simply calling SLSSetWindowTransformation with a CGAffineTransform (e..g CGAffineTransformMakeTranslation(100, 100) to move a window 100 right and 100 down) will move the window, but this is not animated. Calling this repeatedly will likely cause lag.

And now here's the long shot that I'd love to see investigated in the future: There are additional functions called SLSTransactionCreate and SLSTransactionCommit, with variants of the transform and warp functions named SLSTransactionWindowTransform and so on. Assuming this works similar to CATransaction on iOS, this could then be used to animate multiple windows with a duration and at the same time.

These SLSTransaction* functions are available (on 10.15 dev beta 4 currently):

nm /System/Library/PrivateFrameworks/SkyLight.framework/SkyLight | grep SLSTransaction

0000000000200d1e T _SLSTransactionAddWindowToSpace
0000000000200f3a T _SLSTransactionAddWindowToSpaceAndRemoveFromSpaces
0000000000201f5c T _SLSTransactionBindSurface
0000000000200129 T _SLSTransactionClearMenuBarSystemOverrideAlphas
00000000001ffce4 T _SLSTransactionClearWindowLockedBounds
00000000001ff834 T _SLSTransactionClearWindowSystemLevel
0000000000202cb6 T _SLSTransactionCommit
0000000000202ee0 T _SLSTransactionCommitCoalescing
00000000001fd88b T _SLSTransactionCreate
0000000000201a22 T _SLSTransactionDeferWindowMoveEvents
00000000002013c1 T _SLSTransactionDestroySpace
00000000001fd788 T _SLSTransactionGetTypeID
000000000031b9d0 b _SLSTransactionGetTypeID.once
000000000031b9d8 b _SLSTransactionGetTypeID.typeID
000000000020109f T _SLSTransactionHideSpace
00000000001ffd7d T _SLSTransactionMoveWindowWithGroup
0000000000201457 T _SLSTransactionMoveWindowsToManagedSpace
000000000020205f T _SLSTransactionOrderSurface
00000000001fe097 T _SLSTransactionOrderWindow
00000000001fe196 T _SLSTransactionOrderWindowGroup
0000000000201c0a T _SLSTransactionOverrideAppSleepNotifications
0000000000201eb2 T _SLSTransactionPostBroadcastNotification
0000000000202dd4 t _SLSTransactionPostCommit
0000000000201d2a T _SLSTransactionPostNotificationToConnection
0000000000200dd1 T _SLSTransactionRemoveWindowFromSpace
0000000000200e84 T _SLSTransactionRemoveWindowFromSpaces
000000000020132b T _SLSTransactionResetSpaceMenuBar
00000000001fff5c T _SLSTransactionResetWindow
00000000001fef10 T _SLSTransactionResetWindowSubLevel
0000000000202922 T _SLSTransactionSetBackdropChameleonContribution
000000000020283c T _SLSTransactionSetChameleonUpdatesEnabled
0000000000201ec6 T _SLSTransactionSetClientAdvisory
0000000000201950 T _SLSTransactionSetEventCapture
0000000000201654 T _SLSTransactionSetManagedDisplayCurrentSpace
0000000000201875 T _SLSTransactionSetManagedDisplayIsAnimating
00000000002001a9 T _SLSTransactionSetMenuBarBounds
00000000001ffff5 T _SLSTransactionSetMenuBarSystemOverrideAlpha
0000000000200a84 T _SLSTransactionSetSpaceAbsoluteLevel
0000000000200bea T _SLSTransactionSetSpaceAlpha
0000000000200b37 T _SLSTransactionSetSpaceOrderingWeight
0000000000200957 T _SLSTransactionSetSpaceShape
000000000020054f T _SLSTransactionSetSpaceTransform
0000000000202175 T _SLSTransactionSetSurfaceBounds
0000000000202563 T _SLSTransactionSetSurfaceOpacity
0000000000202666 T _SLSTransactionSetSurfaceResolution
00000000001fe295 T _SLSTransactionSetWindowAlpha
00000000001fe3c6 T _SLSTransactionSetWindowBrightness
00000000001ff8cd T _SLSTransactionSetWindowGlobalClipShape
00000000001fe4f7 T _SLSTransactionSetWindowLevel
00000000001ff9fd T _SLSTransactionSetWindowLockedBounds
00000000001fe66b T _SLSTransactionSetWindowOpaqueShape
00000000001fe79b T _SLSTransactionSetWindowProperty
00000000001fefa9 T _SLSTransactionSetWindowReleasesBackingOnOrderOut
00000000001feace T _SLSTransactionSetWindowResolution
00000000001fed2a T _SLSTransactionSetWindowShape
00000000001fee5a T _SLSTransactionSetWindowSubLevel
00000000001ff08f T _SLSTransactionSetWindowSystemAlpha
00000000001ff77e T _SLSTransactionSetWindowSystemLevel
00000000001ff1c0 T _SLSTransactionSetWindowTransform
00000000001ff5e9 T _SLSTransactionSetWindowWarp
0000000000201009 T _SLSTransactionShowSpace
000000000020158b T _SLSTransactionSpaceTileMoveToSpaceAtIndex
0000000000201af2 T _SLSTransactionUpdateRegion
000000000020120c T _SLSTransactionWillSwitchSpaces
00000000001fe615 t __SLSTransactionCommitAction
00000000001fdfe4 t __SLSTransactionDecode_CFString
0000000000052ef0 t __SLSTransactionDecode_CGSSpaceIDArray
00000000001fdc6b t __SLSTransactionDecode_packed64
00000000001fdc9e t __SLSTransactionDecode_packed64_tag
0000000000201135 t __SLSTransactionEncode_CFArray
00000000001fdd41 t __SLSTransactionEncode_CFString
00000000002016fd t __SLSTransactionEncode_UUIDString
00000000001fdba3 t __SLSTransactionEncode_packed64
00000000001fd994 t __SLSTransactionEncode_resize
00000000001fd7cd t __SLSTransactionFinalize
00000000001fd925 t __SLSTransactionPopWriteStream
0000000000201d3a t __SLSTransactionPostNotification
0000000000202a53 t __SLSTransactionPushWriteStream
0000000000202bf6 t __SLSTransactionResetWriteStream
000000000031b9e8 b __SLSTransactionWriteStreamKey.key
000000000031b9e0 b __SLSTransactionWriteStreamKey.once
000000000020300f t ___SLSTransactionCommitCoalescing_block_invoke
00000000001fd7b4 t ___SLSTransactionGetTypeID_block_invoke
0000000000201507 t ___SLSTransactionMoveWindowsToManagedSpace_block_invoke
0000000000202ed8 t ___SLSTransactionPerformCommitAction
0000000000202520 t ___SLSTransactionSetSurfaceBounds_block_invoke
0000000000202808 t ___SLSTransactionSetSurfaceResolution_block_invoke
00000000001fe5ef t ___SLSTransactionSetWindowLevel_block_invoke
00000000001feca3 t ___SLSTransactionSetWindowResolution_block_invoke
00000000002012a5 t ___SLSTransactionWillSwitchSpaces_block_invoke
00000000001fdad3 t ____SLSTransactionWriteStreamKey_block_invoke

Why? Look at this example from i3. Looks nice, doesn't it?

[koekeishiya]

I'm not particularly interested in doing this, but would not be against a PR where this is optional.

[danielfalbo]

Another animation idea:
it would be really cool to animate the opening of the terminal in a sci-fi style, kinda like edex-ui
.
Of course it would be cool for other windows too but especially for the terminal because it's usually really flexible in terms of size and is the app that feels the most like a sci-fi thing 🤪.

Alacritty, for example, is completely resizable.

In the gif I put an Alacritty window in a floating space, and, after trying to make it as small and centered as possible, I first do yabai -m window --resize left:-20:0 and yabai -m window --resize right:20:0 multiple times until the window horizontally fills the screen, and then the same thing vertically with yabai -m window --resize bottom:0:20 and yabai -m window --resize top:0:-20

[ghost]

Any update on this?

[albert-ying]

Just saw this on reddit, it looks so smooth

https://www.reddit.com/r/unixporn/comments/vtpwhc/hyprland_dynamic_wallpapers_and_settings_with_a/if9iaoy/?context=3

[koekeishiya]

Just to note that the mentioned SLSTransaction function family does not work in Monterey 12.5. (Maybe there are entitlements or whatever you can somehow add to your binary to make them work, but they no longer work OOTB as they do in Big Sur).

[itaysharir]

Did anyone manage to implement it and has a compiled version of it?

[ghost]

im wondering of this too

[FelixKratz]

anim.mp4

I figured it is not a big problem to call SLSSetWindowTransformation frequently to create the intermediate frames. I have hardcoded there to be 100 frames in any resize/move animation.

[ghost]

brilliant.

anim.mp4

I figured it is not a big problem to call SLSSetWindowTransformation frequently to create the intermediate frames. I have hardcoded there to be 100 frames in any resize/move animation.

brilliant.

[koekeishiya]

@FelixKratz

Did you only apply the transform in that POC, or does it also synchronise said transform with the desired window frame change? This is important because macOS itself applies transforms to windows in various contexts, and that will undo the operation.

[FelixKratz]

@FelixKratz

Did you only apply the transform in that POC, or does it also synchronise said transform with the desired window frame change? This is important because macOS itself applies transforms to windows in various contexts, and that will undo the operation.

It is not polished at all and very hacked together, this is all I did:
https://github.com/FelixKratz/yabai/commit/2ba29b73df0c9e71cd3d34bf0c97ef2e59cfa7bf
I am only setting the transform for as long as I am animating and then I do the actual proper move/resize. These animations block yabai, which is why I have made them so short.

[koekeishiya]

I'll probably take a shot at this for the next release, as I'm working on smoothing out the experience. I have a fairly high bar for what I consider acceptable though, so not sure if it is feasible to ship in the end. A dealbreaker for me is the ability to animate all windows that are modified in the same operation simultaneously, so that they start/end at the same time.

[FelixKratz]

I'll probably take a shot at this for the next release, as I'm working on smoothing out the experience. I have a fairly high bar for what I consider acceptable though, so not sure if it is feasible to ship in the end. A dealbreaker for me is the ability to animate all windows that are modified in the same operation simultaneously, so that they start/end at the same time.

Great to hear! Animating the windows in parallel would be very nice, but quite a bit of restructuring of the code base I believe. It would probably also be much better to animate from the SA directly. I only chose to not do that because then the SA would have to notify yabai when the animation is finished or unlock some kind of mutex that the SA and yabai share and that would go too far for a simple POC. This would probably also benefit greatly from your new ipc approach via shmem + mutex. When those animations are done "proper" (parallel and less flickering) it will feel very nice I am sure.

What I stumbled over while implementing the POC was that once the affine transform is reset to the original transform before moving and resizing the window via AX there would sometimes be a noticeable flickering and I didn't find a way to resize the window through SLS functions, i.e. SLSSetWindowShape does not seem to be respected when set via SA. So I was stuck with the slow resize via AX.

[koekeishiya]

Pushed a proof of concept myself: https://github.com/koekeishiya/yabai/tree/animate 269d29d

Remember to reinstall the scripting addition after building the above branch.
Add yabai -m config window_animation_duration 0.35 to enable animations; adjust the value as you like.
This does not work with window borders yet, so keep borders disabled.

This still only animates a single window at a time.
Multiple windows are animated simultaneously: 45e18e0

What I stumbled over while implementing the POC was that once the affine transform is reset to the original transform before moving and resizing the window via AX there would sometimes be a noticeable flickering and I didn't find a way to resize the window through SLS functions, i.e. SLSSetWindowShape does not seem to be respected when set via SA. So I was stuck with the slow resize via AX.

I don't think it is possible to get around this; I have attempted to do so in my version, but no guarantees that it is a universal solution.

[koekeishiya]

Transactional:

sample showing repeated calls to yabai -m space --rotate 90.

The flicker is caused by the issue mentioned above; when resetting the transform at the end of the animation to apply the change in size using the AX API. Hopefully there will be a workaround to that...

[koekeishiya]

Resizing using the AX API is just too slow. Need to figure out some other way to resize a window.. When resizing a window using the native (mouse) handle at the edges, the operation appears to be rather snappy. I wonder if we could somehow hook into that.

[FelixKratz]

When looking at the animation happening when clicking the green fullscreen button on any windows handle I got a (janky?) idea:

• Spawn a new window showing a capture of the surface of the window to be animated
• Position the "fake" window exactly where the "real" window is located
• Make the "real" window invisible (alpha -> 0)
• Resize and reposition the "real" window (while invisible)
• Animate the transition via the "fake" window
• Make the "real" window visible again (at correct position and size already)
• Remove the "fake" window

I don't know if this is ok to do performance wise, but I suspect something similar is happening in the system animation to fullscreen. They additionally transition between the scaled versions of the unresized and the resized window through the animation, making it look a bit smoother.

[koekeishiya]

I had an idea that it would maybe be ok to animate the actual windows as is done, but taking a screen capture of the space at the final frame; displaying this screen capture as a topmost window and destroy it (probably some kind of blurred fade) while removing the transforms and applying the resize animation.

Thanks to your comment now I recorded my screen while doing the enter/exit fullscreen application, and it actually looks like Apple themselves are doing something like this, surprisingly enough.

[FelixKratz]

I tried my idea by misusing the border to temporarily draw the capture of the window. The scripting addition is not needed for the animation at all (only for changing the opacity) and it looks very smooth. https://github.com/FelixKratz/yabai/commit/18eb2ca9d2caf1fffb8097e0378efd9826ff54c7

[koekeishiya]

Are you able to animate the windows simultaneously using that approach though?

Edit:

I guess you could iterate through all windows, grab a capture of the window, and set the opacity to 0. Set the size and position using the AX API.

Animate all the captures.

Iterate through the list of windows, setting the opacity back to the previous value.

[FelixKratz]

Are you able to animate the windows simultaneously using that approach though?

Edit:

I guess you could iterate through all windows, grab a capture of the window, and set the opacity to 0. Set the size and position using the AX API.

Animate all the captures.

Iterate through the list of windows, setting the opacity back to the previous value.

In my current implementation it is single window, but if I pass a window list to the animation function (same as you have done) it will be possible to animate simultaneously. Maybe I will try to rebase to your POC and implement this method there, but I think the digest of this is that we can definitely circumvent the flickering. Maybe it would be good to use the faster SLSHW functions for window capture instead of what I am using currently.

[koekeishiya]

It works pretty well, except that the window order is not the same as it is pre-screenshot, and sometimes the screenshot colours look different from the window itself. e.g Safari is darker than it is in the screenshot. This also breaks the window fading option if an animation is played on window create/destroy/focus change.

Edit: So the issue appears to be that the screenshot does not capture vibrancy effects among other things

[ubuntudroid]

That's an awesome feature! Thanks a lot! ❤️

However, I also have problems with SIP enabled: the animations are only playing very rarely when swapping or moving windows. Only about every 7th (or even more) command triggers an animation. Not predictably though.

yabai -m config window_animation_duration    0.35

UPDATE
Actually it looks like that's actually the same thing @dominiklohmann reported, just that our assumptions on how we think it should work differ. 😏 I thought every swap/move etc. should trigger an animation. But maybe I'm wrong here.

[koekeishiya]

Found a bug with the animations: With SIP enabled (and thus no scripting addition loaded), and window_animation_duration set to a non-zero value, there sometimes is a very wonky animation happening:

So this is happening if you perform some action that would cancel an in-progress animation for a window and start a new animation towards a new state. If you set the animation duration to 1.0 seconds, and do yabai -m window --swap west followed by yabai -m window --swap east shortly after, you will trigger that behaviour.

What's happening is that the first swap command will trigger the start of an animation, however we need the scripting addition to hide the real window while we animate a proxy; performing a transactional switch so that it isn't actually obvious to the user that this is what's happening. Because the scripting-addition is not loaded, the swap doesn't occur, and there is no animation showing.

When an animation is cancelled because a new one is started, the proxy is updated and swaps with the old proxy. This swap does not rely on the scripting addition, which is why those windows suddenly become visible.

I do believe I stated that this particular config option requires SIP to be disabled. However, I agree that maybe there should be some limitation in place preventing the value to be changed from 0.0 on systems where SIP is enabled.

There is no way to allow this functionality with SIP enabled, unfortunately.

[ubuntudroid]

I do believe I stated that this particular config option requires SIP to be disabled. However, I agree that maybe there should be some limitation in place preventing the value to be changed from 0.0 on systems where SIP is enabled.

There is no way to allow this functionality with SIP enabled, unfortunately.

That makes sense. Think I should have read the docs first - just saw it in the changelog and jumped right into it. 😅 My bad!

[koekeishiya]

Added a guard that only allows window_animation_duration to be set if the appropriate SIP flags are disabled.

[FelixKratz]

The animation can get fairly stuttery at times. My testing shows that this is connected to disabling the update on border resize and border redraw. Even though animations use a different SLS connection, they seem to affect each other still.

This patch https://github.com/FelixKratz/yabai/commit/a20e36e017e4e3257b9904625a6b0f080d9e1136 makes the animations (especially noticeable on large displays and short animation durations) dramatically smoother for my configuration. It however introduces redraw flickering since the update has not been disabled on redraw.

[jakenvac]

Is it normal for there to be a delay before the animations play? I'm using the latest version on macOS 12.6 apple silicon.

After using yabai -m window --warp west there's about a 500ms delay before the windows will actually swap.

[koekeishiya]

Is it normal for there to be a delay before the animations play?

This delay is caused by actually setting the change in size/position using the accessibility API, and there is nothing that can be done to speed up that operation. The accessibility API is janky and that's really it - until Apple either improves the API/performance of it, or implements a different system for third-party software to interact with windows/applications.

[albert-ying]

can window open and close animations be added?

It might be possible to create a window opening animation; depends on how much delay there is between the window spawning and yabai getting notified, I guess this would likely be implemented as a fade+scale transition.

Animating on window close is pretty much impossible. The window is already inaccessible from system APIs by the time we are notified that a window has been destroyed.

What if we let the yabai to send the window closing signal? Say implement a command like yabai -m close_current_window and assign cmd+w to it using shkd

[koekeishiya]

What if we let the yabai to send the window closing signal? Say implement a command like yabai -m close_current_window and assign cmd+w to it using shkd

That would probably work, but I'd prefer something more universal. I often use cmd+q instead of cmd+w if it is the last window of an application, and that would not trigger the animation with this solution.

Not that I am fully opposed to experimenting with something like what you mentioned.

[FelixKratz]

I have pondered a bit further with the animation system and came up with a new POC for a fade between the scaled window and the unscaled window at the end of an animation. I believe this makes animations a lot more smooth https://github.com/FelixKratz/yabai/commit/de13dbca8d62a8a453d15455c7f3adeb527a8e09:

anim.mp4

[koekeishiya]

How does that look when you hit windows that have min/max size constraints?

[FelixKratz]

How does that look when you hit windows that have min/max size constraints?

A bit funny:

constraint.mp4

[PrayagS]

I have pondered a bit further with the animation system and came up with a new POC for a fade between the scaled window and the unscaled window at the end of an animation. I believe this makes animations a lot more smooth FelixKratz@de13dbc:

@FelixKratz I'm on the latest release v5.0.2 and enabled animation but noticed a different behavior on my terminal windows.

The text resizes after the window is set to its new location. In your video, the text resizing and window movement animation happen simultaneously. What could be the reason for this? Does your branch still have a different code?

I'm using the kitty terminal. And frame rate is set to 120 in yabai.

[koekeishiya]

Is it normal for there to be a delay before the animations play?

. This delay is caused by actually setting the change in size/position using the accessibility API, and there is nothing that can be done to speed up that operation. The accessibility API is janky and that's really it - until Apple either improves the API/performance of it, or implements a different system for third-party software to interact with windows/applications.

This started to annoy me and I got a little bit creative, and found a way to improve the situation.

#2060

[net]

Would love a way to just disable animations for any resize.

[koekeishiya]

window_animation_duration 0.0 ?

[net]

I mean disable only for resize operations, but not for move operations. So any animations that are compromised by the stretching effect, while keeping the animations that look perfect.

[koekeishiya]

I'm not sure if that level of complexity can be handled reasonably in the animation system. I agree that the abrupt end from final animated state to displaying the actual window is not ideal.

[koekeishiya]

@net
#2137

Please try the latest master; is it better or worse in your opinion? Make sure to uninstall the scripting addition and reinstall it after compiling.

Edit: I think it is a lot nicer actually.

[net]

I think it's better, but still not something I'd leave enabled. The stretching itself is the issue. I just noticed the stretching causes window corners to deform too when their ratio changes.

The only way I can think to do it, given the constraints, in an aesthetically pleasing way—and this way admittedly would take a lot more work—is to apply a strong gaussian blur to the overlay so you can't really see the window content deforming, and mask the corners so they look like standard macOS corners throughout the full animation.

Screen.Recording.2024-03-03.at.00.54.53.mov

[net]

Actually, here's another aesthetically pleasing option, this one even more complicated, but perhaps with the best possible effect:

1. For resizes narrowing a window, do not deform the window image, but rather animate the width down by masking in from the right side.
2. For resizes shortening a window, do the same as above, but masking up from the bottom.
3. For resizes widening a window, take the color of a pixel on the rightmost edge of the window image, preferable near the top just before the corner radius of top right corner begins (to increase the chance of capturing the window's UI background color). Use that color as a background fill color to animate a mask out from the right side of the window image, revealing the background fill as it animates.
4. For resizes lengthening a window, do the same as above, but masking down from the bottom.

A resize may use various combinations of the above options. After the resize, possibly alpha-fade to the window's true UI. The effect of this should be to appear as if the window doesn't relayout its UI until after the animation, but that the window frame itself still cleanly animates.

[koekeishiya]

I think that is too much work for the actual processor to do for this to feel snappy, with the way this has to work. Preparing animation state already takes between 60-130ms depending on a variety of conditions.

I think we looked at how Apple performs transitions a bit further up this thread, and they fade transition between the old and new state. Admittedly their version looks better as they have access to more info than yabai does.

[koekeishiya]

FelixKratz did a more invasive alpha fade POC a couple of comments above. The window transitions approximately in the middle of the animation, so the stretch is less pronounced. Do you think such a solution would be acceptable? I guess it would be hard to tell without trying it yourself and observe the results directly. Looking at a gif is not always the same/as accurate.

[net]

Specifically, Apple does it by performing the alpha fade during the first two thirds of the animation, with some easing function that performs most of the fade within the first few frames. As you noted, this has the effect that by the time the resize animation is slowing down towards the end, the window has already faded to the new state, and so the warping is much less noticeable. They don't even do any corner masking.

I think using the same timings as Apple is as good as anyone can ask for 😄