Add Wayland support

[q2dg]

Sorry if it's a silly question.

[phil294]

Not a silly question at all. Currently, all Hotkey logic etc. is heavily based on Xorg, so it would significant work, if possible at all. But I guess it will (have to) at some point.

I'm clueless about Wayland, I guess this thread provides some useful development info though https://news.ycombinator.com/item?id=32654500

Edit:
Just to make this clear, even though Wayland is pretty important for the project's longevity, I personally won't work on it very soon. PR always welcome.

[aaaa2aaaa]

For wayland, it currently depends on whether or not the compositors have implemented protocols to do some of these actions that may be considered a security risk and are by some compositors like gnome/kde. Currently I think only wlroots-based compositors have the protocols implemented to do some of this. In the future however, this kind of functionality would be permitted through the use of portals which would ask the user to allow permission first.

So basically you would either have to use wlroots specific compositor protocols which might be removed in the future or give your support for the ability to do this in portals.

Currently I think there's portals for input/reading input in development but you would have to open issues for other functionality (and you may have to argue with people to convince them that the functionality is needed)

This might sound ridiculous since on xorg this stuff "just works" but at least when it's done only programs you choose can manipulate input and do all this other stuff when you give them permission to do it.

Hope this helps

[phil294]

if anybody wants to give it a go, espanso seems to have solved these problems (?). I think anything that works there, should be applicable to this project too.

[aaaa2aaaa]

That requires root access and you can only read input and write input, a very small subset of AHK's functionality.

[aaaa2aaaa]

Currently the only reasonable way to properly implement something like AHK on wayland would be to make your own wayland protocols then add support for them in compositors then compile the compositors yourself, if you just want to have programmable macros on wayland there is other tools that can do that. https://github.com/ReimuNotMoe/ydotool

[phil294]

oh so it does require root, okay. Thanks for your input, this will surely come handy some day.

[jf2048]

In the future however, this kind of functionality would be permitted through the use of portals which would ask the user to allow permission first.

The project to get input emulation working with portals is called libei, you can see some blog posts about it here:

https://who-t.blogspot.com/2020/08/libei-library-to-support-emulated-input.html

https://who-t.blogspot.com/2021/08/libei-status-update.html

http://who-t.blogspot.com/2022/03/libei-adding-support-for-passive.html

Those should have links to some of the portal PRs.

Currently the only reasonable way to properly implement something like AHK on wayland would be to make your own wayland protocols then add support for them in compositors then compile the compositors yourself

Well no, an interested party should give feedback on the drafts for the portal and help get those finished and stabilized. Hacking the compositors is unlikely to fix anything, as the real issue is how there is a lack of good API.

[aaaa2aaaa]

AHK doesn't just do input emulation, it also has stuff like manipulating windows. Is there any portal drafted to do that stuff? Right now I don't think there's even a standardized way to see the coordinates of a window.

[jf2048]

I have not seen any proposals for that. Someone may attempt to do it at some point, but my personal opinion is that efforts to do so will fail. Window managers on Linux are just too fragmented, even something as innocuous-seeming as window coordinates is unreliable and means different things depending on the window manager. This is a problem even on X11...

[phil294]

I have read a few bits about this now and wow, the situation is dire. I think the sanest way is to go the evdev / uinput route here. It's true that e.g. the portal route would be cleaner, but also working on and pushing proposals and working with non-standardized hacks around such fundamental functionality is not a path I want to take, and also completely out of scope for this project. Still, thank you for your explanations, and perhaps one day we can do this properly.

Since executables compiled by ahk_x11 must be independent and portable, it cannot depend on any kind of managed background service, installer script or manual setup. Instead, I suggest that every script that uses any keyboard related command and runs on a non-x11 system spawns its own little child process and exchanges data via socket. That process attempts to hook into the raw keyboard events at script start. If it fails to do so, the end user is prompted to choose from three buttons:

Insufficient permission to read your keyboard (Wayland). Please choose:

1. [Recommended] Insert password to continue: Needs to be done for every program start
2. Re-run the script with root permissions: Not recommended
3. Add user to input group: Insecure and potentially dangerous, not recommended, but solves this for all future scripts

Those are essentially one-click actions and run nicely interactive with sudo --askpass. It gives non-technical users a recommendation while being clear and configurable to power users. Edit: maybe also https://espanso.org/docs/install/linux/#adding-the-required-capabilities

From what I have seen espanso and ydotool do, uinput code is no rocket science (in contrast to X11-targetting code), so this would be a clear path towards a realistic goal of supporting any graphical system.

All this display server specific logic like hotkeys, window management, key automation needs to be extracted into dedicated DisplayAdapter classes. This would also open up the possibility to add Windows support, but there is really no point in doing so.

Edit: Granted, window management (WinMaximize and so forth) may be tricky or impossible to achieve this route, but I don't really see an alternative way until Wayland pulls itself together

edit2: some notes @ https://news.ycombinator.com/item?id=34253521

edit3: possible to use https://github.com/bendahl/uinput? i don't know if Go libs can be linked into crystal code just like that, but if so, this could make developing this a breeze

edit4: I have now slowly started investigating this feature with the help of https://github.com/nickbclifford/evdev.cr, looks doable

[phil294]

FYI I'm working on this currently but it is an awful lot of work unfortunately, so I don't know yet when it will be finished. A lot of stuff seems to work, in parts even better than on X11, and others not at all, esp. Win* commands.

[aaaa2aaaa]

It'll probably be a while before you can even get some win* commands working. Look how long this protocol has been stuck:
https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/59
https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/155
https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/187

[phil294]

oh wow, but at least something is happening. It's admittedly a bit hard to follow though.

Yes, I'm not even taking that route. But thanks to AtSpi, the accessibility protocol, it seems that matching windows by name and pid is still possible, as is interacting with elements inside windows, so it looks like e.g. ControlSend, , keys, Firefox still works. But window matching options A and ahk_class are gone.

Hotkeys/Hotstrings and Send and MouseClick also works fine, including absolute positioning.

[phil294]

Hi!

here is a first demo version of ahk_x11 with Wayland support. I don't really recommend yet to use it, but if you feel adventurous, you can now. Several things don't work properly yet, so check out the Readme from the Wayland branch https://github.com/phil294/AHK_X11/tree/evdev-wayland#wayland-evdev

I've figured that at-spi, the a11y protocol under Linux, is more capable than I thought and I need to read up onto a lot of things. This will still take a while. I don't have super much time available currently, so I can't tell for sure when I can continue to work on all of this. Because of that, I'm simply sharing what I have built so far.

[mercifulboss]

In KDE it seems to be possible to get window coordinates, and various other properties on Wayland when establishing window rules. In fact quite a lot of information can be obtained abt windows, including titles, children, etc. perhaps this might be something to look into?

[phil294]

@mercifulboss is this compositor-specific api? because I'd like to stay in wayland land as much as possible and avoid implementing things for all compositors in parallel.

[mercifulboss]

@mercifulboss is this compositor-specific api? because I'd like to stay in wayland land as much as possible and avoid implementing things for all compositors in parallel.

This is compositor independent api.

My understanding (and I may be wrong) is that it is a daemon that creates a virtual keyboard using DBUS which will both detect keypresses and overwrites text when it detects certain user defined (in a configuration file) keywords. In this way it entirely bypasses the compositor. Again, this is my very basic understanding of how this works, but it works great on wayland.