-
Notifications
You must be signed in to change notification settings - Fork 310
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
msrdc.exe spams .etl files in RdClientAutoTrace directory #978
Comments
I also have this issue. My linux distribution is Ubuntu 22.04 |
@nevmerzhitsky, thanks for reporting the issue, do you observe same issue when WSL is launched without docker? such as start WSL by @ligq, do you also use Doccker desktop? And if you can share us a few of those *.etl, it would help us to diagnostic the issue, thanks! |
No, I am not using Docker. Every 30 seconds a .etl file was created. |
Yes, files are create even when Docker Desktop stopped. But today I was noticed that JetBrains Toolbox (1.27.2.13801, Windows 10.0) related to the issue: I see a new |
I have not installed anything about JetBrains. |
Today I updated the wsl to the latest version, still got this issue. |
@ligq, would you please clarify what do you mean by "wsl terminal" ? If you close "wsl terminal" (and keep new files being created), what do you see when you type |
"wsl terminal" means I run wsl in the windows terminal. PS C:\Users\ligq> wsl --list -v |
Hi, Any updates regarding this issue? |
I am waiting for a solution to this issue... |
May relate to microsoft/WSL#8486 |
I just discovered that my system is generating these log files every time File Explorer scans the system for drives. This happens whenever I open Explorer to the Home location and again every time it refreshes the tab. The log retention is exactly one week, and currently I have a few hundred of the logs totaling 3 GB. More information in the related discussion linked above. Hope this helps you with troubleshooting or corrective action. |
File explorer definitely isn't the only thing creating these log entries, because a couple dozen have shown up in my logs tonight while I was watching videos in Chrome. |
First, please allow me to clarify a few things. 1: One log file is always created every time when WSL/WSLg is launched. Thus, if WSL is terminated (aka wsl --shutdown) and restarted "repeatedly", it can end up large number of files being created. 2: Log files older than one week will be deleted, but there is no limit by number of files or total size of all files in the folder. 3: When Linux GUI applications are used, the size of one currently active log file can grow over time, but it shouldn't be significant, it should be less then 2~300mb even active WSL session is running more than 1 month. Above 3 are currently expected behavior, but we are looking into to add limit by number of files. Second, if WSL/WSLg is not terminated and relaunched repeatedly (like 1), but new log files are being created, or log files keep growing more than 500mb, this is not what's expected, thus, if you are experiencing such issues, please share a few log files with us, we would like to see as many as possible cases, and please make sure you are on the latest version of WSL, thanks! |
@nevmerzhitsky, thanks for sharing log file
From provided log file, it's very likely your case is fixed by microsoft/weston-mirror#138, which will be included in next WSL update, thanks! |
If I have an active WSL terminal open, I only see one log entry for the session, and opening File Explorer is instantaneous. However, if I close the WSL terminal, then any refresh of the File Explorer "Home" view will start a fresh WSL process. There's a noticeable delay (with progress bar) while File Explorer starts WSL, which I can see in Task Manager. If I leave the File Explorer window open, it'll start WSL again every few minutes when the view auto-refreshes. That still doesn't explain what happened yesterday, though, as I wasn't actively using WSL or File Explorer. I presume that some other background process must have scanned the WSL drive. I'm attaching some of the logfiles from last night, including at least one log caused by File Explorer and a few more from unknown sources. RdClientAutoTrace-WppAutoTrace-20230321.zip Altogether there were about 2 dozen logfiles accumulated last night. Right in the middle of them, I can see this entry in Event Viewer:
This is the first time I've noticed an error like that, no idea whether it's related. What's the recommended way to check WSL for updates? |
Please try |
Thanks! The update command says that I'm already running the latest version. |
I'm now seeing this as well, started randomly possibly after an update to Windows (on Beta channel insider builds) It's creating about one file every second all of which are around 8MB and it's filling up my drive extremely quickly and with tons of unnecessary writes. I've attached a random one here if that helps at all, couldn't see anything noteworthy using tracerpt but I have also no idea what I'm looking at. It happens whenever WSL is running at all, have to use |
@MattJeanes, thank you very much for sharing the log, the log indicates the socket is disconnected and it seems triggered from Linux VM side, do you see any dumps at /mnt/wslg/dumps? if not, please share /mnt/wslg/*.log with us, (note, this folder is cleaned up at every WSL start up, thus, dumps/logs need to be collected while active WSL instance is experiencing the issue) thanks! [for internal reference] |
@hideyukn88 thank you for looking into this. I was not able to find any dumps in the
I'm sure there is other useful stuff in the logs too! Please let me know if you need anything else or wish to reach out directly to me to investigate, I really appreciate it. |
Oops, didn't actually attach the file - here it is! |
@MattJeanes, thank you very much for log, as you pointed, below error is the fatal and this causes the disconnection of remote desktop protocol.
While error message indicates it's failing to convert "Domain" string, but based on FreeRDP source code, it can be either "Domain name", "User name", "Password", "AlternateShell" or "WorkingDirectory". What it does here is, Windows sends these strings in Unicode, and FreeRDP is converting to UTF-8, and one of these conversions was failed. But based on the RdClientAutoTrace you provided which logs what Windows is sending to Linux side; these are simply NULL (below screen capture). There is no logging for "Password" for security purpose, but this should be as well. At my environment, these are all NULL. Thus, I can't tell why conversion has been failed at your end. So, just in case, I believe you have multple RdClientAutoTrace files can be found on your local disk, would you please share a few more of them? And is it possible you can run privately build version of WSLg with extended logging? if yes, I can send something to you. thank you very much for your help! |
@hideyukn88 yes I am happy to run debug versions of WSLg if you can share how to do that. If you would like to reach out over email you can find it on my GitHub profile, although it is also useful to be talking in this public thread as well in case anyone else is seeing similar issues. Here is a zip with 73 of those logs from when I booted WSL up earlier to grab those logs for you 🙂 They compress very well, 600MB down to 3MB! |
@MattJeanes, would you please try attached private system distro VHD (at below link, unzip it first). You can install it by following steps at https://github.com/microsoft/wslg/blob/main/CONTRIBUTING.md#installing-a-private-version-of-the-wslg-system-distro. Once the issue occurs again, please share /mnt/wslg/wlog.log with us, the log contains some additional information including the "Password" field mentioned above, it should be nil, but just in case, please double check, if it's exposing your real password or other than nil, please simply remove it and let us know, thanks for helping! |
@hideyukn88 I have done that and the much more verbose log is attached below! Password is indeed showing nil as you said it should. The issue happens immediately after booting WSL and I have also noticed the mouse cursor often showing the 'working in background' symbol (pic below) and flickering back to the normal mouse seemingly every time it writes one of these files. Not sure if that's relevant but thought worth noting! |
Oh also, some extra system information in case it is useful: Windows build: 22H2 (OS Build 22631.1900) I have docker desktop installed but the issue was also happening without it installed, and happens with only the primary Ubuntu WSL runnning:
As I mentioned I also tried completely uninstalling WSL as much as I could and re-installing it to no avail. |
@MattJeanes, for diagnostics purpose, I rebuilt system distro private VHD without ICU library (which in charge of Unicode conversion) and switch to WinPR's inhouse conversion, would you please try this? |
@hideyukn88 I have tried Arch WSL from the Microsoft Store and with only that one running it still shows the same issue:
I have also tried your new system VHD (back on Ubuntu now) and it unfortunately has not helped the issue. Attached below is a zip containing one of the RdClientAutoTrace files and the wlog.log file. I really appreciate your help with this by the way, hope we can get to the bottom of this! |
@MattJeanes, would you please try below VHD again and please share /mnt/wslg/wlog.log with us, thanks! |
@hideyukn88 sure, have attached logs below. |
@MattJeanes, I've updated to address possible buffer handling, please try again. |
@hideyukn88 I think you're onto something - it looks to be working 😄 I can now open graphical apps successfully! There is one file it has created in the DiagOutputDir but it is no longer spamming. I have attached it and the wlog.log again, very promising results! Still some errors in the wlog.log but not sure if they are normal. |
@MattJeanes, thanks for checking, it's good to hear it's working!
These are not really error but I logged them in that way (easy to tell), so all looks good, thank you very much for helping us to solve this! thanks! |
@hideyukn88 Ah okay, great! No problem at all, I am super happy that you've been able to fix this and so quickly as well 🙂 I was a little worried that I would be shouting into the wind by commenting here originally so I am very glad to be proven wrong. I'll keep running the custom Again really appreciate you looking into this for me, hope you have a great rest of the week! |
(edit: had some incorrect config. Made some progress and updated this comment to reflect that) I'm having the exact same issue as MattJeanes (down to the flickering cursor), I tried using the latest custom
I tried running Some details:
Setting Here's a zip containing a dump of some Let me know if I can provide any more info, |
@camerondubas, using my private VHD shouldn't cause such network issue, just in case, would you please re-download the VHD and try again? If persist, would you please take WSL log following https://github.com/microsoft/WSL/blob/master/CONTRIBUTING.md#8-collect-wsl-logs-recommended-method, thanks! |
Having the same configuration in my system. Uploading the trace etl file. I have tried doing the registry tweak, re-install etc. Did not help. RdClientAutoTrace-WppAutoTrace-20230710-034045-804.zip PS C:\Users\Dell> wsl -l -v
|
Yes, you can. |
I am getting about 140 new such files per minute. |
@lexavey, thanks for sharing your experience, it is unfortunate that this issue is still not addressed. While the log is deleted automatically time to time, but it was not enough in this case. I have added some mitigation for this by #1082, so that it will not take out large amount of space on user's disk. If you can share a few of those files, it will help us to understand more about this issue or your specific situation, thanks! |
@hideyukn88 So RdClientAutoTrace etl files are error/crash logs? Due to their frequency I previously assumed they were debug logs. It seems to be if they are error/crash logs, we need to find out the root causes of error/crash right? Otherwise limiting process restart etc.. is just a band aid. If these are crash logs, can you please share how to open/read them? That way users can maybe help with diagnosis? |
Dupe of microsoft/WSL#10216 ? |
@sarim, no, they are not error/crash log, those logs are being created even everything is perfectly working, and unfortunately user can not read these logs in meaningful way. The log can be opened by wpa.exe (Windows Performance Analyzer) which tool is available publicly, but in order to access important logging information, it requires full private debug symbols of msrdc which is not available publicly, thanks! |
@lexavey, the behavior you observed is expected, the new log file is created every fresh WSL startup, and the log you provided does not show any errors, everything is working as expected, then WSL is terminated after roughly 50 secs of idle time. In this case, #1082 will not prevent from new log file being created, thus we are also working on to limit the number of files being created (or any other possible limitation), thanks! |
I just deleted 200GB+ of these files under |
Hello! I just had 23.3 GB worth of disk space consumed by these etl logs until I stopped WSL manually. It is after I stopped it that I realised I had lost so much space. This hasn't happened to me before - at least at this scale. I have attached one of these files below. |
To whoever experiencing this issue, would you please try the system.vhd from below link? @MattJeanes, thanks for helping us with previous drop, but if you don't mind, would you please try this new one again and let us know if this still works? thanks! https://1drv.ms/u/s!AqQ2qEh_9iIOhqk8zLxXTvCerpopWg?e=pHYU7d Here is how you can configure WSL to side-load private system-distro VHD. And if it didn't work, please share RdClientAutoTrace files and /mnt/wslg/wlog.log, thanks! |
@hideyukn88 I have tried the new VHD and it looks good to me! It's created one file and appears to be holding it open which I believe is intended behaviour |
I had 32 874 files, total size 205 GiB |
Exactly the same issue here 240GB of log files deleted. I am running WSL2 Ubuntu 20.04 LTS and Docker desktop. |
I can also confirm this fix works 🙌. I was unable to use the PC as constant low diskspace. |
Exactly the same issue here77GB of log files deleted. I am running WSL2 Ubuntu 20.04 LTS and Docker desktop. |
Not reproduced for me anymore, thanks!
Docker Desktop (Windows), version 4.25.0 (126437) |
I just deleted 50gb of logs. This thread was last commented over a year ago. I am pretty sure my WSL install isn't even that old, maybe 8 months? So as of 8 months back, there was no update to the default install that prevented this behaviour, I assume. Is there now? I only have a 1Tb SSD, I can't be losing 100gb to logs every week or so. |
Version
Microsoft Windows [Version 10.0.19045.2546]
WSL Version
Kernel Version
5.15.83.1-microsoft-standard-WSL2
Distro Version
Ubuntu 20.04
Other Software
Docker Desktop (Windows), version 4.16.3
JetBrains Toolbox 1.27.2.13801, Windows 10.0
Repro Steps
Start Docker Desktop and work with some containers
Expected Behavior
No new
.etl
files inC:\Users\username\AppData\Local\Temp\DiagOutputDir\RdClientAutoTrace
Actual Behavior
I see the
msrdc.exe
process in my Windows which creates spam of.etl
files in theC:\Users\username\AppData\Local\Temp\DiagOutputDir\RdClientAutoTrace
directory from time to time. Full path of themsrdc
process:C:\Program Files\WindowsApps\MicrosoftCorporationII.WindowsSubsystemForLinux_1.1.2.0_x64__8wekyb3d8bbwe\msrdc.exe /v:1DA2E607-905F-45FE-8912-D4B1F8019ABF /hvsocketserviceid:8C4086BF-FACB-11E6-BD58-64006A7986D3 /silent /wslg /plugin:WSLDVC_PACKAGE /wslgsharedmemorypath:WSL\1DA2E607-905F-45FE-8912-D4B1F8019ABF\wslg "C:\Program Files\WindowsApps\MicrosoftCorporati...
Diagnostic Logs
No response
The text was updated successfully, but these errors were encountered: