Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In-home Streaming + Wayland #6148

Open
Mushoz opened this issue Mar 17, 2019 · 195 comments
Open

In-home Streaming + Wayland #6148

Mushoz opened this issue Mar 17, 2019 · 195 comments
Assignees

Comments

@Mushoz
Copy link

Mushoz commented Mar 17, 2019

Your system information

  • Steam client version (build number or date): February 18th build
  • Distribution (e.g. Ubuntu): Arch 5.0.2 Kernel
  • Opted into Steam client beta?: No
  • Have you checked for system updates?: Yes, system is fully up-to-date

Please describe your issue in as much detail as possible:

Whenever I run my desktop with Wayland, and connect my steamlink to my Desktop, the screen on the TV will be rubbish with every color of the rainbow all over the screen. The screencapture of the desktop is clearly not going as planned. Actions still work fine, and I can navigate via the controller connected to the steamlink by simply watching what I am doing on my computer screen. Once I launch a game, the corrupted mess disappears and I can properly see the game being streamed to the steamlink.

Steps for reproducing this issue:

  1. Use Wayland (I am using Gnome on Wayland in case that matters)
  2. Start steam
  3. Start steamlink
  4. Connect to desktop

What happens:
I see a corrupted mess on my TV

What should happen:
I should be seeing steam big-picture mode, as my computer display is showing.

Workaround:
Use gnome on Xorg.

@MayeulC
Copy link

MayeulC commented Nov 26, 2019

I am affected as well. Steam Remote play doesn't work, neither does remote play together. Input and audio works perfectly.

The computers used for testing were:

  • Archlinux desktop (AMD R9 Fury, Mesa, RADV), native steam (and flatpak)
  • Archlinux laptop (Intel integrated graphics), via flatpak

Clients tested:

  • Android application, LAN/LTE
  • laptop with sway (with and without hardware decoding)
  • desktop with sway (with and without hardware decoding)

Servers tested:

  • ❌ desktop with sway (with flatpak, without, with and without HW encoding)
  • ❌ laptop with sway (with and without HW encoding)
  • ❌ desktop with plasma wayland (with HW encoding)
  • ✔️ desktop with i3 (with HW encoding)
  • ✔️ desktop with plasma on X11 (with HW encoding)

I am thus reasonably certain that the issue is with Wayland or XWayland (I would need to test plasma without HW encoding to be sure there are not two separate symptoms here). Screen sharing from applications like Firefox do not work on Wayland either.
A possible future solution might leverage pipewire for capture.

I haven't seen anything on the client besides corruption (like uninitialized memory, with a few textures from former apps), even after a game starts. I used simple games for my tests (FTL: Faster than light, Crypt of the necrodancer), and only tried 3D games like Portal 2 on X11 to confirm it still worked. Do 3D games use a different capture method? (Vulkan layer, OpenGL lib injection). Which ones should I try?

@ghost
Copy link

ghost commented Jan 13, 2020

Same issue on Plasma Wayland (tested on Arch Linux). I assume Steam would have to ask pipewire to stream the desktop.

@tgunnoe
Copy link

tgunnoe commented Apr 28, 2020

Same issue on Sway

@PMaynard
Copy link

PMaynard commented May 6, 2020

Same on Fedora 32, with RPMFusion steam, and Sway.
Audio works. But no visual on the steam link device iPhone current iOS.

@sinedoOo
Copy link

Same issue on Fedora 32 + Gnome wayland

@PMaynard
Copy link

anyone tried the flatpak version?

@fuzunspm
Copy link

same issue on Arch Linux swaywm/wayland + iPad setup

@MayeulC
Copy link

MayeulC commented May 18, 2020

anyone tried the flatpak version?

@PMaynard: Re-read my comment, I am using the flatpak version.

This is pretty much a known issue for every wayland environment at that point, so I don't think there's much point in adding pages of comments that everyone has the same issue, and it would be more productive for everyone to upvote the issue.

Valve seems to be working on a wayland compositor, I believe the capture functionality could be integrated into it, and then have the compositor act as a client to also display stuff on the screen (if needed). Not sure if that's the way they want to go forward with, but that's what I would do. Pipewire and waypipe are other, more general options.

@yokem55
Copy link

yokem55 commented Jan 3, 2021

Playing around with this a bit more - in a sway session at least it seems that only the big picture interface has trouble being streamed. Starting a game while looking at the normal monitor with the keyboard launches the game, and the game will then appear over the steam link (android app or standalone device) properly. This is only a partial work around, but it does mean xwayland has what it needs to capture and stream the game.

@MayeulC
Copy link

MayeulC commented Jan 3, 2021

@yokem55, that's interesting, it never worked for me. Neither steam nor browsers are able to capture other Xwayland windows, only displaying a black background. @emersion hinted that it should work already, so I am not sure what's going on, I will re-test soon. Some of it could be due to multi-monitor specificities.

@awmath
Copy link

awmath commented Feb 3, 2021

@emersion hinted that it should work already, so I am not sure what's going on

I think there has been a missunderstanding as @emersion was talking about Steam Play, which works fine on wayland, whereas this topic is about Steam Remote Play, which doesn't.

@maweki
Copy link

maweki commented Feb 3, 2021

@emersion hinted that it should work already, so I am not sure what's going on

I think there has been a missunderstanding as @emersion was talking about Steam Play, which works fine on wayland, whereas this topic is about Steam Remote Play, which doesn't.

I think you misunderstood. For me as well, steam link/remote play works on wayland for a game but the big picture mode window is just black with a few artifacts.

That's exactly what I'm seeing. So streaming demonstrably works. Just unusable if you can't select a game.

@maweki
Copy link

maweki commented Feb 3, 2021

I don't want to create additional noise, but to document what I think we're all talking about.

1 My Steam Link Android Phone screen before connecting
Screenshot_20210203-094800.png

2 Pressing Play on the phone, trying to stream big picture mode
Screenshot_20210203-094820.png

3 Quick-Selecting dicey dungeons below, skipping big picture streaming
Screenshot_20210203-094909.png

And as OP posted, even for the arrifacted big picture streaming, commands from the phone get relayed just fine. You just can't see it.

@awmath
Copy link

awmath commented Feb 3, 2021

@maweki True, I should have been more specific. The Problem is not streaming the games on Wayland via Steam Remote Play, but anything desktop related.
It still holds true, that @emersion was talking about Steam Play, which has nothing to do with Steam Remote Play.
Furthermore, at least for me it is not the Big Picture mode itself which is the problem. You can go ahead and in the Steam Link settings choose "Launch Mode: Desktop". This won't bring up the Big Picture Mode but still won't stream anything useful until you start a game.

@awmath
Copy link

awmath commented Feb 3, 2021

I finally found a workaround!
The problem seems to be the wayland compatibility of steam. This seems counter-intuitive at first, but seemingly all screen capture software has trouble with wayland (just do a quick search for wayland screen capture black screen on the net)
So steam has to run under x11 to support remote play. Luckily we have a way of using x11 applications under wayland with Xwayland. Now we "just" have to force steam to run under Xwayland

First we start a Xwayland server with a seperate display Xwayland :11. The display number should be any display number which is currently unused. Then start steam with DISPLAY=:11 steam.
Done.

You can fiddle with the Xwayland settings until you are satisfied
Now just to make it a little bit more comfortable...

@maweki
Copy link

maweki commented Feb 3, 2021

I finally found a workaround!
First we start a Xwayland server with a seperate display Xwayland :11. The display number should be any display number which is currently unused. Then start steam with DISPLAY=:11 steam.
Done.

So what you're basically saying is, while Steam or Linux (who is responsible for that?) seems to run most games in XWayland already, the steam client proper does not and is therefore not capturable? This would imply that games that work with and use wayland would not be streamable either, yeah?

@awmath
Copy link

awmath commented Feb 4, 2021

So what you're basically saying is, while Steam or Linux (who is responsible for that?) seems to run most games in XWayland already, the steam client proper does not and is therefore not capturable? This would imply that games that work with and use wayland would not be streamable either, yeah?

That's correct. In fact that's how I stumbled upon this "solution". I tried adding non-steam applications to see which could be captured and which not. And even some steam games couldn't be captured (Wasteland 2 for example).
By default all applications should start under wayland and only legacy applications shouldn't.
By running steam inside the Xwayland server, steam will start all other applications in this instance as well. So all games should be streamable from there on.

Edit:
One more observation. Starting Xwayland with the -rootless argument (which would be desirable) won't work. Which leads me to believe that maybe it's not that steam isn't running in a Xwayland session, but rather the default Xwayland implementations in Gnome/Sway/"you name it" all use the rootless Xwayland path. While this is definitly the right choice I wonder if there is a way around this.

Edit2:
For gnome I can confirm that Xwayland runs as rootless which causes the issue with remote play.
So one fix could be to disable Xwayland for Gnome alltogether by adding --no-x11 to the gnome-shell command in the systemd service file, and autostarting a Xwayland server with a custom command and without -rootless.

Also for all wlroots based wayland compositors (sway, cage, gamescope, ... ) this is a bigger Problem as it is hardcoded in https://github.com/swaywm/wlroots/blob/6c08fe979615ac88648eb91c87ea4c41fa1d7bdf/xwayland/server.c#L65.
Though there has been a merged pull request in october which could help here.
If we create a new file with the content:

#!/bin/bash
Xwayland ${@##-rootless}

make it executable and pass it to the compositor via WLR_XWAYLAND=... the Xwayland server should be started without the rootless argument. Given the used compositor uses the required version of wlroots (>0.12.0)

@Tzigamm
Copy link

Tzigamm commented Jun 14, 2021

Any progress on this? Wayland is finding its way into more and more systems with Fedora, Ubuntu and others enabling it by default. Some players (me included) switch to wayland for gamer-oriented features such as a better multi-munitor support, better high refresh rate support natively and native HiDPI options, as of now I cannot stream any game to my Steam Link nor to any of my friends through Remote Play Together without switching first to X11 (which is really not an option in my case). Now that multiple Wayland streaming alternatives exists (most browsers support screen sharing on wayland, OBS works fine too) I hope Valve can do something about it before too many of us are stuck without a very cool and useful feature

@michaelnew
Copy link

So here's what I think is happening, plus 2/3 of a workaround (maybe someone else can figure out the rest of this):

Most games run in an Xwayland window, and Remote Play is able to stream the contents of that window just fine. Anything that's running in a native Wayland window doesn't work (DOOM Eternal, for example), because Remote Play isn't using any capture methods that work on Wayland.

Even though Steam itself runs in Xwayland (it's definitely not Wayland native), it doesn't try and stream the contents of that window. Instead it tries to capture the entire desktop, which won't work if you're using Wayland. It's probably a sensible choice since a lot of games will try and show little launchers or popups or whatever and you'd want to be able to click on them or do whatever you need to do to get the game to launch. But regardless; it breaks streaming for Steam itself.

If you do something like this: Xephyr :1 -screen 1920x1080 & DISPLAY=:1 steam it will put steam into it's own little X11 box, and when it streams the "desktop" it's really just streaming the contents of that X instance. The problem is that games try to launch inside of that X instance, and at least the games that I tried didn't work (missing Vulkan libs or somesuch). But you can make games launch outside of Steam's little sandox by setting the launch options to DISPLAY=:0 -- %command%.

However... Remote Play doesn't switch to streaming the contents of that Xwayland window. It just sits there streaming the Big Picture interface. So if there's a way to convince Steam to stream that window I think we'd be in business.

Obviously the real solution would be for Steam to be able to capture the desktop on Wayland, probably with xdg-desktop-portal. OBS did this recently. There's a plugin here that was merged into OBS proper. So it's definitely doable now.

@minikN
Copy link

minikN commented Jul 24, 2021

Is this being actively worked on by Valve?

@maweki
Copy link

maweki commented Jul 24, 2021

Is this being actively worked on by Valve?

I imagine it must. At least the odd guy. The plan for SteamOS and the new hardware can not be to stay on X for the rest of eternity. It would be unusual for Valve to drop the ball on this long-term architecture stuff.

@cinerea0
Copy link

cinerea0 commented Sep 7, 2021

Has anyone come up with any other solutions for this problem? Neither the Xephyr not Xwayland solutions listed above have worked for me.

@kode54
Copy link

kode54 commented Sep 9, 2021

Gamescope now has a PipeWire stream, which was just implemented less than a month ago. The issue is linked above.

@Mushoz
Copy link
Author

Mushoz commented Sep 9, 2021

@kode54 What exactly do you mean? What issue is linked above? And what exactly does Gamescope have to do with in-home streaming not working properly under Wayland?

@mwpow3ll
Copy link

I'm running stock Fedora 39 (Gnome) with Steam installed from DNF.

Everything works great on X11, but when I switch to a Wayland session, All I get streaming to my Steamlink hardware is a black screen with the default mouse cursor in the center of it. I know Steam BPM is running because while the mouse cursor doesn't move, I'm able to blindly move through Steam BPM UI with my controller. I can even launch a game, and I will hear the game audio as though it is running and sitting on the title screen. Oddly enough, the default mouse cursor will even switch to the mouse cursor used by the game, but it remains fixed in the center of a black screen.

I tried installing Xwaylandvideobridge, but this doesn't work...

To my knowledge, pipewire and pipewire-dmabuf are default with Steam, so I haven't tried those...

@perroboc
Copy link

perroboc commented Mar 27, 2024

Replying to #6148 (comment)

Have you tested Steam Beta?

@User8395
Copy link

User8395 commented Mar 28, 2024

Replying to #6148 (comment)

Have you tested Steam Beta?

@mwpow3ll You should. It fixed everything for me plus it added a Please Stand By message that pops up when the Big Picture window is out of focus.

@User8395
Copy link

My only problem was switching back to the Steam UI from the game. Holding the Steam button and Tab did nothing, and I had to intervene myself on the computer.

@YoinkerBoinker
Copy link

It sorta works on my End. However if i "minimize" steam which i do regulary to stream movies for example i'll get "live broadcast please stand by" on my TV. And i can't seem to do anything. Which makes it pretty much unusable for me. Any clue why is that? I also can't press a "steam button" since i'm using a small keyboard for my TV.

@Nowa-Ammerlaan
Copy link

Any clue why is that?

It's because an xwayland application such as steam is not allowed to just capture the wayland desktop. It has to go through pipewire, and then the user will be prompted to select if and what to share. There is some experimental support for this which can be enabled by launching steam with the -pipewire argument. However when I tried this on my system it worked rather poorly, it was slow, crashed, and there were a bunch of graphical artifacts.

@YoinkerBoinker
Copy link

YoinkerBoinker commented Mar 29, 2024

Ty that's really sad. So i'm pretty much stuck to x11. Is there actually anything that can be done when seeing the "stand by" message? It seems like everytime i see it i have to go to the computer to get rid of it? Maybe i should try to add firefox as a non-steam game and see if that works lol.

@tgurr
Copy link

tgurr commented Mar 29, 2024

However if i "minimize" steam which i do regulary to stream movies for example i'll get "live broadcast please stand by" on my TV. And i can't seem to do anything.

@YoinkerBoinker For your usecase a look at Sunshine/Moonlight might be worth a try as it offers to select streaming the desktop in addition to e.g. individual applications like Steam (Big Picture) and also works on wayland (utilizing dmabuf/kmsgrab as far as I understand):

image

@Wgelyjr
Copy link

Wgelyjr commented Mar 31, 2024

Replying to #6148 (comment)

Have you tested Steam Beta?

@mwpow3ll You should. It fixed everything for me plus it added a Please Stand By message that pops up when the Big Picture window is out of focus.

Also confirmed fixed for me, Fedora 39 plasma Wayland, 6700xt. Steam link is working flawlessly on the most recent beta client.

@OSCrustacean
Copy link

Same as the above, Fedora 39 with Plasma 5.27 has Steam Link working on Steam Client Beta (Flatpak) on an RX 5700 XT. The only thing I've noticed is that launching big picture takes a considerable amount of time when launching from the button in the client for the first time of a session, but it is fully functional if you wait a minute or so. Launching "flatpak run com.valvesoftware.Steam -bigpicture" or selecting stream in the Steam Link app allows big picture to launch almost immediately, not sure what the difference is.

@d3vilguard
Copy link

d3vilguard commented May 24, 2024

Plasma, Wayland. Before I was able to get games going using gamescope, now I can only get portal streaming.. SteamUI streams also. Switching to X is not an option!
ps, beta steam doesn't fix anything. Don't fall for the comments.
ps2, it's terrible.

@User8395
Copy link

My setup is that I normally use Steam Link on Wayland, however if something happens, then I use Moonlight to fix any problems then I go back to Steam Link

@User8395
Copy link

User8395 commented Jun 4, 2024

Pipewire should have an option where you can permanently give screen access to a specific program.

@YellowOnion
Copy link

Capture apps request access to a opaque capture device, pipewire facilitates the communication and routing of video and audio streams, but like with sound, it doesn't actually produce or consume a video stream, that's for pipewire clients, It's up to your Wayland window manager via a xdg-desktop-portal to handle requests and create a capture source, the protocol supports the ability reuse old sessions, it's up to your portal and Steam to handle session reuse correctly though.

common portals:
GNOME xdg-desktop-portal-gnome
KDE xdg-desktop-portal-kde

@User8395
Copy link

User8395 commented Jun 6, 2024

The reason I still use Xorg is because when I'm running Minecraft: Java Edition from Steam Wayland, Steam Link will stay at "Please Stand By" until I select a game from the Minecraft Launcher.

@TheJackiMonster
Copy link

Pipewire should have an option where you can permanently give screen access to a specific program.

It's definitely supported in GNOME. Works with OBS for me.

@User8395

This comment was marked as off-topic.

@alterNERDtive

This comment was marked as off-topic.

@User8395

This comment was marked as off-topic.

@LuNeder
Copy link

LuNeder commented Jun 6, 2024

Pipewire should have an option where you can permanently give screen access to a specific program.

That’s your xdg-desktop-portal’s job. Open an issue (or even better: pull request) to the portal you use requesting this feature.

Depending on your portal you might also be able to make a script for that. For instance, wlroots’ xdg-desktop-portal-wlr let’s you change the selection menu from dmenu to literally anything else - such as your own script that makes some checks before asking or automatically allowing and automatically choosing a monitor. I was initially planning to do that, but wlroots sucks and works like shit so I just stopped using it and migrated back to x11 and compiz before getting to it (at least until I can do something with smithay myself or something). Still, if you can get the program that’s requesting your screen then the script would be pretty straightforward. Not sure how this works on other portals, but you could always open an issue to yours if one doesn’t exist already.

@perroboc
Copy link

perroboc commented Jun 6, 2024

Back on topic: Current stable release works fine using Wayland on Plasma 6. I was able to stream Hades just fine.

There's no need to use Beta version.

Using Fedora 40, Plasma 6, Wayland, AMDGPU.

Steam Beta Branch:  Stable Client
Steam Version:  1716584667
Steam Client Build Date:  Fri, May 24 4:48 PM UTC -08:00
Steam Web Build Date:  Fri, May 24 4:31 PM UTC -08:00
Steam API Version:  SteamClient021
Operating System: Fedora Linux 40
KDE Plasma Version: 6.0.5
KDE Frameworks Version: 6.2.0
Qt Version: 6.7.1
Kernel Version: 6.8.11-300.fc40.x86_64 (64-bit)
Graphics Platform: Wayland
Processors: 32 × AMD Ryzen 9 7950X 16-Core Processor
Memory: 31,0 GiB of RAM
Graphics Processor: AMD Radeon RX 7900 XTX
Manufacturer: ASUS

@LuNeder
Copy link

LuNeder commented Jun 6, 2024

That said, are we sure that the original problem in this issue even still exists?

Admittedly I haven’t used wayland in a while, but when I did I never had a problem that was caused by Steam. Everything worked fine for me from the Steam side, as long as it was launched with -pipewire.
Of course, wlroots was (and still is) a problem and you have to apply a hack/patch in order for it to work (and good luck opening an issue about that on their tracker), but then again that’s not a Steam issue. Just use a window manager that’s based on an actually good library and you’ll be fine. There are some really cool wms coming out using smithay these days.

What I mean is that this issue miiiiiight be ready to be closed imo, if no one is having a problem actually caused by Steam.

@VoodaGod
Copy link

VoodaGod commented Jun 13, 2024

With this setup:

Steam Beta Branch:  Stable Client
Steam Version:  1716584667
Steam Client Build Date:  Fri, May 24 22:48 UTC -08:00
Steam Web Build Date:  Fri, May 24 22:31 UTC -08:00
Steam API Version:  SteamClient021
Operating System: Manjaro Linux 
KDE Plasma Version: 6.0.5
KDE Frameworks Version: 6.3.0
Qt Version: 6.7.1
Kernel Version: 6.9.3-3-MANJARO (64-bit)
Graphics Platform: Wayland
Processors: 16 × AMD Ryzen 7 5800X3D 8-Core Processor
Memory: 62,7 GiB of RAM
Graphics Processor: AMD Radeon RX 6700 XT
Manufacturer: Gigabyte Technology Co., Ltd.
Product Name: X570 AORUS PRO
System Version: -CF

running steam without pipewire, i can stream the big picture UI fine, but as soon as focus shifts to a started game or i alt tab to anything else, i just get the "please stand by" screen until i refocus the steam window

interestingly, with pipewire i still always get the screen capture select popup, but the permission seems to be remembered, because streaming works fine without interacting with the popup

@Nicceboy
Copy link

Nicceboy commented Jul 13, 2024

Replying to #6148 (comment)

In last year and early this year, I played Elden Ring about 200 hours without problems by streaming it with Steam Link, by using Wayland + sway and pipewire. Now, with new UI and this "broadcast" window, it just stopped working. It just wants to stream the content of this "broadcast" window, which has just the abort game button. BigUI seems to still work, but it no more wants to stream anything else.

@LuNeder
Copy link

LuNeder commented Nov 17, 2024

Does anyone happen to know how I can make KDE automatically accept steam's request for screen sharing? I can't really accept it if I'm not at home, and if i wasn'þ at home I wouldn't be using steam link lmao

In KDE's popup there's an option that says like "allow this to be restored in next sessions" or something, but that changes nothing.

@LuNeder
Copy link

LuNeder commented Nov 17, 2024

Uh, but actually now (at least on NixOS) if I launch steam with -pipewire it will crash as soon as I try to connect to Steam Link
(this also happens on X11 idf I launch with -pipewire, but on X11 without -pipewire Steam Link works fine. On Wayland -pipewire is needed, and with -pipewire steam will crash as soon as I connect to Steam Link)

steam-pipewire-coredump-wayland.txt
steam-pipewire-coredump-x11.txt
steam-logs-pipewire-segfault.tar.gz

@rhasselbaum
Copy link

rhasselbaum commented Nov 18, 2024

Uh, but actually now (at least on NixOS) if I launch steam with -pipewire it will crash as soon as I try to connect to Steam Link (this also happens on X11 idf I launch with -pipewire, but on X11 without -pipewire Steam Link works fine. On Wayland -pipewire is needed, and with -pipewire steam will crash as soon as I connect to Steam Link)

I'm running Steam on NixOS (unstable) and I don't have this problem. Wayland without -pipewire works fine for me, although I do get the KDE Plasma prompt to approve screen sharing. (Not a problem for me, since I only use this to stream from office PC to living room TV.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests