-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
steamtinkerlaunch is incompatible with storing compatdata on the Steam Deck's SD card, because symlinked compatdata folders are read as proton version "None" and overwritten with brand new prefix #692
Comments
Thanks for opening an issue!
This is actually a Steam Client bug on Steam Deck and, as far as I know, not intended behaviour. This was not the case circa July-August this year when I got my Steam Deck as I still have several compatdata folders on the microSD card's library folder. This is just some extra background I wanted to give that this is not intended behaviour. I reported this issue upstream to Steam for Linux, and another user did shortly thereafter under a "save data issue" presumably for when Windows games store game save data in the prefix.
This is quite interesting. SteamTinkerLaunch shouldn't be creating a new prefix as far as I know. Also, the title of the issue is a little incorrect.
This is not what happens. All games start with Proton version "none" as far as I know for first-time launches. So this phrasing is not really correct, but I am not sure how else to phrase it better 😅 I'm more interested with the prefix being overwritten. As far as I understand this isn't something SteamTinkerLaunch should be doing. I recall that Steam does this if you use symlinked prefixes. It will also update a prefix if the version of Proton you run a game with changes, so if you went from 7.0-4 to 7.0-5, the prefix would be reset basically. A user in another issue ended up factory resetting their Steam Deck because using symlinked prefixes caused them lots of headaches. Please try the latest SteamTinkerLaunch-git and see if you can reproduce the problem, though I don't think this would've been fixed in the latest git as I don't recall this happening at all before. For development purposes though it's easier if you're trying from the latest git (v11.11 is 3 months old at this point). You'll also need to attach a full log please :-) If SteamTinkerLaunch is overwriting prefixes that would be strange behaviour. I know SteamTinkerLaunch tries to figure out where a game's prefix is based on where it is installed, or more accurately based on the app manifest in the library folder where it is installed (the I'll take a look into this but could you please verify that this happens only with SteamTinkerLaunch? And also, just once more, please attach a full log. Also, is this causing any immediate issues? Just wondering :-) Thanks! |
Update: Turns out that the log line you quoted isn't anywhere in SteamTinkerLaunch... Did you definitely get it from |
Actually, that log message is from Proton, when it performs prefix upgrading. This seems like an action performed by Proton and not SteamTInkerLaunch: https://github.com/ValveSoftware/Proton/blob/proton_7.0/proton#L502 - If you're curious, when Proton is used to run a game, it's actually a Python script that gets executed. A few different commands can be passed to this You did a lot of background research on this issue which is very highly appreciated, so I'm still happy to discuss the issue if you have more background to give on why it could be a SteamTinkerLaunch issue. And likewise, I'll still take a look through the code to make sure there isn't anything on the STL side that could be messing up. Although, technically, the issue of prefixes being created in the wrong place on Steam Deck is more than likely a bug on Valve's end. If it's causing major problems though a workaround could be added to try and find the prefix on the main SSD if we're on Steam Deck and we fail to find it in the same library folder as the game. That is, if this is required on the STL side. EDIT: I had a quick glance over that Python script and it seems to be responsible for the behaviour you're seeing. However, that doesn't automatically mean STL can wipe its hands clean here, it's still possible STL is doing something to incorrectly trigger the behaviour. I'm having a look through the launch code and associated functions ( I haven't taken a very in-depth look at how the The logic after that gets, well, convoluted at least to my sleepy brain. But I would guess somewhere along the way it sets the prefix up and that's why you see it creating a brand new prefix. I'm not certain that SteamTinkerLaunch is doing anything here and hopefully my info here helps illustrate the investigation I've been through so far :-) |
Oh. Thanks for letting me know--that's good news!
This isn't a first-time launch though. It's happening repeatedly, every time.
Yeah, I spent all day thinking it was Steam. Except it's not happening if I just launch the game through Steam. It's only happening specifically when steamtinkerlaunch is the compatibility tool used, and I hit the PLAY button. But you're probably somewhat onto something, because I saw that line in the game launch log rather than the steamtinkerlaunch log. I'm just not sure why Steam would only be doing this when steamtinkerlaunch is involved.
The .acf file is definitely on the SD card, so this fact makes this issue even weirder. The bottom line is, yes, this is only happening when steamtinkerlaunch is used. So this is a very odd issue, indeed. I'll install the git version and post full logs probably in the next few days. Thanks for the quick response! And you immediately figured out so much! |
I first ran into this when adding support for Hedge Mod Manager, so I reported it in Steam for Linux issue 8941, another user reported it with Steam for Linux issue 8963. It has been a real pain and I hope it's fixed soon :-)
That's very interesting. Have you selected a Proton version from the SteamTinkerLaunch game menu? The impression I got from your earlier reply saying that all settings were left at default seems to me that the game Proton version was not touched, though usually that means a game would fail to launch. Still I'd be interested to know what it's set to. The Proton log line itself says it's going from I guess indeed the problem here is related to the Proton version passed to the game prefix? Maybe it's something to do with the folders being symlinked but I am not totally sure. The log should have this sort of information so I can more easily trace and look for suspicious log lines, which I can then trace back in the code (for example if the prefix is being incorrectly grabbed somehow). Though I still don't fully understand what would be going wrong, whether SteamTinkerLaunch looks on the microSD card or the boot SSD for the prefix, it should be able to find it and there shouldn't really be any issues. If the game is on the microSD card it should look in This is not to say that it isn't an STL issue (you've outlined that it's something that only happens with STL), moreso I'm just thinking out loud a little bit.
Actually that's ok, the
Thanks for clearing it up! It's quite odd for sure. Just wondering, is this causing any major immediate problems? As in, is this is super-critical bug that is potentially breaking things on SteamOS? If so it'll absolutely need fixed and thoroughly tested before STL v12 goes out, because if it is a critical issue it's something I would want to be absolutely sure is fixed before v12 goes out (which is loosely estimated to be released next month so plenty of time for investigation and testing anyway). Since the way Proton handles prefix upgrading seems to involve copying lots of files I'm not sure that it should be causing major problems (e.g. there shouldn't be any locally-stored save file data loss or anything). Probably the biggest pain here I would guess is the time it takes to re-create the prefix, hopefully there isn't anything more 😅 My time is a little bit limited coming up to the holidays but I'll look into what I can in the meantime waiting on the logs, this issue will remain open in the meantime :-) The issue template had a comment but just a reminder that the log for the latest execution is stored at Thanks! |
Moreso a note to myself: seems like STL should just get the compat data path from Steam:
It also symlinks the prefix in the STL config folder but this shouldn't be used for a game launch afaik, just other misc things (OSCDP = Original Steam Compat Data Path). Does not appear to be used for game launch and should not be changed. |
Yeah, that's correct. STL is set to Proton 7.0 for this game.
In my case it's certainly very annoying, because it means if I want to easily mod Fallout 4 right now then I'm going to have to hold off on storing its compatdata on the SD card with the game until this is fixed. But I wouldn't call that critical. Please do relax for the holidays! I'll get those logs up. Maybe it's a bigger issue for any 64GB Deck users since it's generally going to be a big barrier to storing the compatdata of any ModOrganizer or Vortex game on the SD card. But I haven't even seen anyone else bring this issue up, which makes me wonder if it's just me. I'm not doing anything weird or non-default, but I do tend to just magically have issues no one else does even when there's no explanation--I have that kind of curse. |
Oh, this is interesting. I didn't know the game's compatdata made a difference here, I'm fairly sure STL stores ModOrganizer and Vortex mods outside of the game's prefix. At least for MO2 the default should be in the MO2 prefix at I asked this because I wanted to know, is this preventing STL from working entirely without any workarounds. That wasn't to say the issue isn't important though!
I know how that is! But I think in this instance it's something that's worth taking a look into. From what I can gather STL just takes the compatdata path from an environment variable which is set by Steam. I'll check to see if STL is potentially incorrectly changing this path but from looking at some of my own logs at least from a PC launch, STL seems to just be taking the compatdata path sent from Steam. Once I can see more of what STL is doing I'll know what to look for when I try to recreate the scenario on my own Steam Deck to see what STL is expecting vs what the reality is and how to best fix the problem :-) |
Update: I can re-create the problem on my PC. Just to ensure I understand correctly:
If that is correct, then I was able to re-create the issue on my desktop PC with the latest STL from git (11.12.20221215-2). Very interesting! The last version of Proton used to run a game is stored in a text file in the prefix, at Well at least I can reproduce the issue, so it isn't just some strange curse :-) |
Ah I have some good news! It seems this is "intentional" SteamTinkerLaunch behaviour, in that the code is taking a deliberate action here. I found these lines in my log:
I'm trying to learn more about this So it seems like this is something STL is doing intentionally. The question here is why is it doing this, and is this a setting which should be enabled by default on Steam Deck I wonder. I am hoping this is a checkbox or dropdown option that can be selected to fix this behaviour. The other issue is should this be default Steam Deck behaviour. Thanks for bringing this to my attention, I had no idea STL did this! The next step is to figure out why and hopefully give you a good answer that is as simple as a menu toggle :-) Sorry for the inconvenience caused here and hopefully there should be a resolution soon. Also, I think I forgot to mention, but if you want to use STL from git (which you shouldn't have to for this specific issue) you can install it from ProtonUp-Qt as well if you enable "Advanced Mode" from its About menu :-) |
Okay, so when STL uses this compatdata redirect feature, it creates its own symlink. When a symlink exists and the feature is disabled, it assumes it has to "clean up" after itself and remove an existing prefix. Based on other parts of how this feature works, this makes sense. However, STL creates a specific file in prefixes it creates symlinks for, which means we can differentiate between symlinks created by a user and symlinks created by STL. It creates a file in the prefix called As a result, I made some code changes when I didn't expect to. I thought maybe it would be a case of changing a config option but it seems not. Instead I changed the logic to this:
This should hopefully resolve the issue. In my testing it seems to work without breaking the Once you have some time, feel free to install SteamTinkerLaunch-git either manually or with ProtonUp-Qt. You'll want to make sure you're on Fingers crossed this fixes it! :-) |
I see! I figured that was indeed the default behavior, but I was hoping that, by using STL as a compatibility tool, I could have per-game installations of MO inside each game's compatdata, though I hadn't tested yet if STL is willing to work that way. At the very least, I know I can tell each instance of MO's global installation to store all its files in that game's compatdata. I'd still like to do that.
It's actually the other way around. The compatdata folder is in the microSD, and the symlink to the microSD is in ~/.local/share. But if you're testing this on a PC, where that Steam Deck bug isn't occurring, consequently meaning that Steam is actually using the microSD location to find that game's compatdata, then I don't see why you wouldn't get the same behavior testing it swapped around like that.
Yup! Thanks for reproducing. You're very impressive for being just 2 months into the lead role.
Good tip, didn't know that.
Oh!!! That explains everything. I had learned about that setting in the wiki, then found it disabled, and assumed that it couldn't have anything to do with the issue even though it was so very adjacent, because I figured it wouldn't be doing anything while disabled. Thanks for tracking that down. OK, I look forward to testing this out. It really seems like it'll all be solved! |
Ahh sorry for my misunderstanding here, but I agree the behaviour should be the same even with them swapped. At least I hope so 😅 If all is well the issue should be solved! If not, I'll see about doing some more testing and seeing if I can track it down further. The log message I saw with This is kinda unrelated but
Is there a specific reason for this? I'm far from a MO2 power-user but STL uses symlinks and updates some MO2 files to make that single installation of MO2 work with various prefixes, and I think some files may be symlinked when necessary from discussions I remember. Just wondering if there's a specific advantage to having MO2 installed per-instance, because this idea was actually discussed a little bit iirc several months back. At least other users haven't brought this idea up or mentioned any problems that doing this would fix, but that doesn't mean it's without merit so I'd like to hear a little bit more about this. I don't know when it would be gotten around to but if you feel it's something that would bring significant benefit please feel free to open a feature request for it too! :-) |
Tested it on v11.12.20221217-1 and it's solved. Cheers! As for the MO thing, it's so that I can hot swap this SD card between multiple devices. I'd especially love to have the Steam Deck as the home base for the actual game files, keeping them up to date, and it travels with me, and then I can plug those same files into a more powerful computer if I want to use Remote Play to get better graphics/performance or a bigger screen. But yeah, as you said, if nothing else I should be able to accomplish that with symlinks. |
Ahh I think I understand. You want MO2 on each install so you always have MO2 I guess? Symlinks can do it and STL does some magic symlinking. It makes sense and sounds like quite an interesting use-case for your Steam Deck. Also, super glad to hear the issue is solved!! I like quick resolutions :-) Closing Happy tinkering! |
It looks like it actually works the way I thought it might and wanted it to. If I run MO2 from the STL instance that runs when I launch Fallout 4 with it as the compatibility tool, its AppData folder shows up in FO4's compatdata prefix in my Steam library, rather than the master MO2 prefix in ~/.config/steamtinkerlaunch/mo2. One thing I'm going to have to do manually now (it doesn't seem to have been done automatically), just to keep things seamless, is create symlink ~/.config/steamtinkerlaunch/mo2/compatdata/pfx/drive_c/users/steamuser/AppData/Local/ModOrganizer/Fallout 4 -> /run/media/mmcblk0p1/steamapps/compatdata/377160/pfx/drive_c/users/steamuser/AppData/Local/ModOrganizer/Fallout 4 |
Special notes to anyone else using their Steam Deck MO2 setups portably on an SD card like me:
|
Wow, thanks for detailing this! Would you mind if I added this (as a straight copy/paste + preserving formatting) to the Modding wiki and credited you and this comment? You've done a lot of legwork here and it would be a shame for this to get lost. Your comment would stay, the idea is just to boost visibility and better document this process. Also:
Afaik there indeed is no way to do this, but this isn't a bad idea. I'm not too sure where it would go in the UI, but I could add a command that would do this I think. There is already the ability to use one-time Winetricks and one-time Winecfg from the UI for the current game's prefix, so the idea I have right now is to extend this to the command line and allow a user to pass an AppID, a keyword like |
Sure, why not. It's not a very good writeup and might not even be the best way to do this, but it helps. |
System Information
Background Info
I've decided to store some games on my SD card on my Steam Deck. Turns out the Steam Deck is hardcoded to use the internal storage's library to store compatdata prefixes, rather than whatever library the game is installed on as per Steam's behavior on any other Linux device. I'm using symlinks to store those folders on the SD card anyway so that I can have all of a game's data in one place, so that I can swap SD cards between devices.
Issue Description
For certain games I have the compatdata folder symlinked to another Steam library, for example for Fallout 4,
~/.local/share/Steam/steamapps/compatdata/377160
is symlinked to/run/media/mmcblk0p1/steamapps/compatdata/377160
. If I launch the game without forcing any particular compatibility tool, the symlink works as intended. However, if I use steamtinkerlaunch as the compatibility tool, then upon hitting the PLAY button on the steamtinkerlaunch window, the symlink is replaced with a brand new auto-generated prefix folder in compatdata.I assume this is happening right when the log outputs the following line:
Proton: Upgrading prefix from None to 7.0-100 (/home/deck/.local/share/Steam/steamapps/compatdata/377160/)
All of the proton and symlink settings for this game are on their defaults--disabled.
The text was updated successfully, but these errors were encountered: