WSL 2 should automatically release disk space back to the host OS #4699
Comments
|
+1 for being able to move the .vhdx to a different partition, either via config file or the "Move" feature in the Control Panel of Windows (currently getting a message that it's disabled) |
|
+1 here too |
|
+1 also add the ability to specify the initial size limit. |
|
I also would like to see a setting for initial partition size limit. It looks like all distros are installed with a 250GB partition size. I see no (easy) way to extend this. My yocto builds require slightly more space than this and they have to be on an ext3/4 partition so I can not use the /mnt/c or other NTFS partitions. |
|
In Home you don't even have the The default partition size should have some connection to the size of the Windows host partition. What is the use of a limit that exceeds that of the host (unless actual magic is involved)? Also, artificially limiting the maximum size to a fraction of that of the host seems a bit unnecessary. I think you really want an experience that is as close to that of WSL1 as possible, but with the incredible perf and compat gains, of course. That is, the WSL2 file system shouldn't feel like it was a static volume bound to a file on the host (although it is). Ideally, even the WSL partition/volume size should shrink and grow with available space on the host in a somewhat timely fashion. Now there is a stretch goal for you! |
|
FWIW, since these commands looked benign I ran them to reclaim some disk space: This broke my docker. I had to reset to factory defaults & reinstall docker and restart the pc a few times along the way to fix. All docker data is gone (doesn't matter in my case because it's just dev environment). YMMV. Microsoft Windows 19041.1 with WSL2 |
|
I've tried the commands as well. I could find an issue that docker engine (docker.sock) is not available after starting. But after restarting Docker Desktop and changing the settings / resources / wsl integration to off + apply, then to on again + apply did the trick. Looks like the sock I ran the test a second time, but now my whole Ubuntu 18.04 distro is broken, the terminal closes either immediately or after some seconds without any text in it. |
|
Maybe disabling wsl integration first, then quitting docker and then using the optimize-vhd command can make it work. But I'm not going to take that risk now. |
|
It's back alive, got the Ubuntu distro working again. Seem like I just have to wait a little bit. Terminal opens and I have the bash prompt. The WSL integration was disabled, after enabling again docker also works again in Ubuntu. |
|
If you are looking for a way how to reclaim your disk space from Docker using WSL2, then there is a button for this if you are using Docker for Desktop (Edge 2.3.0.0). In the Dashboard / Troubleshoot - there is a
It does not solve the original issue but it's a quick way to a fresh start. |
|
For Windows 10 Home (alternative wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
exitThanks to @davidwin for the tip #4699 (comment). |
|
I just got no disc space, I have not even installed much. I added a symbolic link that was no problem on my old system. |
|
I'm on Windows 10 Pro build 2004 here (stable 2020 spring edition) and I just noticed I lost 60GB on my primary SSD from this because I started moving my source code into WSL 2's file system and forgot one of my podcast sites has a bunch of wave files in a private directory. When I run How can I get around this error? This put my SSD at a dangerously low level of space and I'd rather not have to reinstall the entire WSL 2 instance from scratch and reconfigure Docker every time I accidentally copy an unused file into WSL 2's file system. |
|
@nickjj You need to be in the directory where the distro's vhdx resides. In my case with Ubuntu20.04 I had to wsl.exe --shutdown
cd C:\Users\onoma\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\
optimize-vhd -Path .\ext4.vhdx -Mode full |
|
Thanks, I was able to find it there too. That and what @StefanScherer mentioned about restarting Docker, along with disabling / enabling the distro in Docker Desktop again got things back to normal without any system level restart. |
|
My contribution, bringing together knowledge from above. It still requires a little manual work to get the DistroFolder variable set for the distro you want to target (you could have more than one, find them with Cheers! ### Optimize (shrink) WSL 2 .vhdx
## Must be run in PowerShell as Administrator user
# DistroFolder found at: $env:LOCALAPPDATA\Packages\
# Examples:
# CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc
# CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc
cd $env:LOCALAPPDATA\Packages\REPLACE_ME_WITH_TARGET_DISTRO_FOLDERNAME\LocalState\
wsl --shutdown
optimize-vhd -Path .\ext4.vhdx -Mode full
#Run `wsl` or your favorite terminal to resume useBELOW UNTESTED!! ### Resize WSL 2 .vhdx
## WARNING!! UNTESTED!! Read and make note of the MaxSizeGB variable and functionality before running
## Must be run in PowerShell as Administrator user
# DistroFolder found at: $env:LOCALAPPDATA\Packages\
# Examples:
# CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc
# CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc
Set-Variable -Name "MaxSizeGB" -Value "32GB"
cd $env:LOCALAPPDATA\Packages\REPLACE_ME_WITH_TARGET_DISTRO_FOLDERNAME\LocalState\
wsl --shutdown
resize-vhd -Path .\ext4.vhdx -SizeBytes $MaxSize
#Run `wsl` or your favorite terminal to resume use |
|
Both cmd and powershell can't find optimize-vhd command even in admin preveliege |
|
@eromoe optimize-vhd is only available in Windows 10 Pro with Hyper-v feature installed. Otherwise you will need to use the compact option in Diskpart. |
|
The WSL2 disk management is definitely a pig. Developing some (legacy app) Docker containers, after a week my "working" WSL2 .vhdx is 110G (with only 14 GB of actual data) and my docker-desktop-data .vhdx is 40G (with only 3 GB of actual data). |
|
Has anyone ever seen this when trying to run At this point my drive has almost no space and I'm not finding anything on Google on how to compact it, short of destroying the entire WSL 2 instance and remaking it from scratch. I've stopped Docker Desktop and everything. There's no hint on what's using it. |
|
@nickjj With |
|
The second alternative mentioned is proving the fastest and most reliable for me, Exporting/Importing WSL 2 to move it to another disk. The other thing I have tried is converting between Fixed and Dynamic volumes in combination with this. My use is for docker's WSL2 integration, and the need to export and restore large data volumes (>250GB). This is similar to what most of the respondents are referring to, yet they do not always say whether they are using WSL1 vs. WSL2. WSL1 sets a fixed disk size as specified in your Docker settings. WSL2 uses windows default setting of 256GB max volume size. I could not bash into or ssh into the docker-data distro shown or access its ext4.vhdx volume through various methods. What would be really nice is if WSL would adopt a docker overlay2 driver or some other more friendly HyperV linux filesystem, faster than vhdx on ntfs, with an API to control allocation. Even better would be some type of cloud filesystem integration that could rival local dev speeds. Here are my notes in gist |
|
I just symlinked $ErrorActionPreference = "Stop"
$newLocation = "E:\VMs\WSL2\"
cd "~\AppData\Local\Docker\wsl\data"
wsl --shutdown
Optimize-VHD .\ext4.vhdx -Mode Full
mkdir $newLocation -Force
mv ext4.vhdx $newLocation
cd ..
rm "data"
New-Item -ItemType SymbolicLink -Path "data" -Target $newLocation |
|
@mikemaccana That's where Docker Desktop with WSL2 backend saves the docker images. Use the Troubleshoot option in the Dashboard to clean them. |
|
OK, ran:
To get rid of the docker images that had been accumulating. I have my hard drive back, and Docker is now pulling down images (good) but it's failing with: Which I'm now troubleshooting. Edit: since the IP addresses are local, poking around in Docker networking config and restarting docker fixed it. |
|
For further Docker Desktop issues is better to ask them in https://github.com/docker/for-win/issues |
This issue is also logged here indeed docker/for-win#244 , go there for Docker-specific elements; as mentioned by onomatopellan above. |
|
@igortas this seems extremely peculiar. WSL2 was a game changing (supposedly) feature in an attempt to make Windows more developer friendly. But the way WSL2 runs, it is not usable (in a safe way), for so many. Which is why I posted this comment earlier.
If not Mac, atleast Linux. There is something very fundamental that we don't understand, or something that Microsoft cannot share about this issue. |
|
@qwertynik the easiest way is to reinstall the |
|
Yeah. That's a good idea @igortas. Easy to forget about such simple solutions when remaining deep into problems. But this has its own demands. One needs to remember the steps that were taken to configure the distro powered by WSL2. Or, rather one should completely rely on a docker like service alone when working with different distros - this atleast will make reinstallations easier. Because, when using docker it is easier to build the habit of having all config in code. While these options are great, the best would have been to run a command and free up the space that is not being utilised. There is something about virtualization technology that should be making this difficult to solve. |
|
With the The Sources:
|
|
I didn't think that Hyper-V and the (Hyper-V–based, but distinct,) Windows Virtual Machine Platform that WSL2 uses could be installed at the same time, though? ' |
@RandomDSdevel I had to turn on
|
|
I have Windows 10 (Home) with WSL and I have tried both options suggested for Home here but either nothing is happening or I keep losing memory on my hard disk. After I have shut down the WSL and tried to run: the message I got was: DISKPART> detach vdisk DiskPart has encountered an error: The requested operation could not be completed due to a limitation of the virtual disk system. Virtual hard disk files must not be compressed or encrypted and must not be fragmented. The files I work with are mostly gzip files containing large biological data. Of course, I have either transferred them in USB sticks or deleted them, and yet no space is freed on my hard disk. Can I unzip them somehow or do something so that these commands can work? Any help could be extremely useful! Thank you! |
I can also confirm that having the Virtual Machine Runtime installed, then installing they Hyper-V Module for Windows PowerShell and Hyper-V Services, but not the Hyper-V Hypervisor, works. The sequence of commands you gave still doesn't work directly with VHDX image files that have NTFS compression applied to them, but it still helps streamline things as compared to having to use |
|
Just like @prasanthcakewalk I was not able to use optimize-vhd until after installing Hyper-V. Anyways - I prefer a non GUI method og getting the tool (well Hyper-V) installed, so I use this PS cmd (in an elevated prompt):
And yes - you DO have to perform the reboot to get the tool...! My VHD then got reduced from 126 to 86 GB Using diskpart->compact seems to perform the same action as performing that made no further reduction (that makes sense I think) I am only using 48 GB So I guess that now it's time to look into defragging the disk to reclaim space (and getting a bigger disk)... Hmm Made no change - still 48 GB usage |
|
Before support for sparse disk images got added and maybe still for non-sparse disk images, WSL would pin deleted files' ' |
I have the same problem, did you find a solution? |
|
I've done the Googling doesn't help too much. Would it be possible to repair it? |
|
Olá, tenho um Windows 11 pro e WSL2 e consegui resolver meu problema da seguinte forma. 1° passo: Se você não está com o Hyper-V ativado, ative-o da seguinte forma:
2° passo: Se seu VHD já está com espaço em disco alto e você programou para espaçar automaticamente com Obs: Caso você ainda não tenha programado para para espaçar automaticamente, pule o passo 2. 3° passo: 3.2 - Identifique o arquivo de disco virtual: O arquivo de disco virtual geralmente está localizado em: Obs 3.2: Substitua <your_username> e <distro_package_name> pelos valores apropriados. Obs 3.2: Certifique-se de NÃO clicar no arquivo “ext4.vhdx” para não gerar erro que o arquivo esteja sendo usado em outra instância, certifique-se de que nada relacionado ao wsl esteja aberto. 3.3 Para Windows Pro ou superior: • Use o Optimize-VHD: > Abra o PowerShell como administrador > Navegue até o diretório que contém o arquivo “ext4.vhdx” com: 3.4 Execute o comando: 4° passo: ative o espaçamento automático com Pronto, seu VHD foi compactado para o tanto de espaço de fato utilizado, e o armazenamento espaçado voltou para o seu disco rígido C: Funcionou para mim, eu estava com 105G no VHD sendo que de fato eu só estava usando 19G no meu \home. Isso levou ele para 20G e meu SSD (C:) que estava só com 68GB livre, voltou para 152GB livre. |
I've transitioned from utilizing a virtual machine to implementing a dual-boot setup on my PC, i.e. I now have both Windows and Linux operating systems installed separately. |
For me, this error was caused because of the "sparse" configuration of wsl set to true. Then I ran |
Thanks, that resolved it for me too! |
|
So, i'm successfully able to perform the compacting operation -- but it seems the solution is only temporary, as the disk re-expands shortly after I begin interacting in WSL again. There a way to set a limit on the size of the virtual disk that anyone's figured out? |
well, yes and no. three tips beforehand: calling so, the long answer: someone made the maybe questionable decision of both giving the virtual harddisk inside the VHD the size of a full terabyte as well as giving the filesystem partition of that virtual harddisk the full size. so, the limitation is basically a terabyte, and the virtual harddisk file will never grow beyond that - much. some metadata is required, and some space is used for (thin provisioning) housekeeping, but essentially, when all space is used, its size is 1TB plus some headers/wrappers, same as with thick provisioning. if you use getting to the 'no' for reducing/limiting: getting to a smaller disk is similar, but unreasonably harder. you'll have to the tricky bit is getting the reduced size right, otherwise you'll lose data. making the resized partition somewhat bigger than the resized filesystem should give you some marging, same as making the resized VHD somewhat bigger than the resized partition. you can grow partition and filesystem after to fit the space after the VHD resize to reclaim whatever safety marking you did. the hard bit is that it'll work, but .. the fact that so far I couldn't be bothered myself and instead relocated the VHD to a volume where I have more diskspace (and can afford the 1TB used) should tell you how annoying it is. I hope this explanation provides some additional information for a few people, maybe you'll get to further ideas on how to improve the situation :) |
|
Based on @karolineWss with a bit translation, and previous people's contributions, summarize what worked for my machine (Windows 11 Pro). Reduced from 300+GB to 78 GB in my case.
Solution with
|
|
Do you have any updates for resolving this issue? I was using the WSL Ubuntu-22.04 Distribution and tried everything you said. At last, I was able to recover just 10 GB from the 380 GB used. However, I was unable to recover more memory. This is so frustrating. Ultimately, I have to run the command |
I have had no issues, although I updated WSL2 to the beta version with the sparse disk support (caution, downgrade/rollback doesn't seem to be supported well), which I think is now out as a stable release or will be soon? I used
Here are the powershell terminal commands I use: wslcompact -c docker-desktop-data
wslcompact -c UbuntuThen I set the disks to be sparse: wsl.exe --manage docker-desktop-data --set-sparse true
wsl.exe --manage Ubuntu --set-sparse trueNow when I delete stuff from either vdisk (eg: for Docker it's References of my earlier comments that are now buried (they are all very verbose but probably contain useful insights for most):
I also documented this above in those reference links, but this is an issue with memory reclaim.
Eventually over time WSL2 would release that memory on the host side, otherwise you need to force it. In a a WSL2 terminal like for Ubuntu, run If that still doesn't do it then you may need to shutdown WSL with Good news is there is settings to opt-in for better memory reclaim, although I think it had some caveats especially with Docker so I didn't bother with it. Not sure if that got fixed since, and if the update would have that enabled by default or not.
Well now you know how to do it properly 👍 If it helps this is what I am running: wsl --version
WSL version: 2.0.0.0
Kernel version: 5.15.123.1-1
WSLg version: 1.0.57That is the beta/pre-release WSL2 2.0 release IIRC, if there is a new stable release out perhaps give that a try too. |
I ended up trying it, and was successful in resizing & size limiting my Ubuntu:
so now the rootfs container on the host system can't grow much (metadata!) beyond the size given, and my Ubuntu can't run into a stall due to host file system full. |
Have you verified that? Even if your Ubuntu has free space, if the host disk is full (0 bytes left), then it would be the same problem with no extra space to allocate. Reducing the vdisk max size only limits the disk growing too large and wasting disk space on the host, but sparse vdisk makes that mostly unnecessary. If I am low on disk space and use WSL2 to build a project that uses a few GB of RAM, this memory usage would also use similar amount of disk space on the host. That risks running out of host disk space and WSL2 becomes unresponsive, not even As reclaiming disk space on the host does not recover from that issue, you have to reboot the host AFAIK, it's a major drawback vs VMware guest. |
@polarathene, many thanks for your clarification. Pretty clear!!! Appreciate that! |
I really love the Dynamic-Wallpaper software. If there were a Windows or Linux version, I would be willing to purchase it for a lifetime. |
I really love the Dynamic-Wallpaper software. If there were a Windows or Linux version, I would be willing to purchase it for a lifetime. |
|
I have been searching for a clean solution that:
I figured out that you can restrict the size of a VHDX using defaultVhdSize parameter in .wslconfig :
Note that once this parameter is setup, you'll need to create a new WSL environment from scratch, so that the VHDX file is re-created with this maximum size. In other words, it won't affect existing VHDX. Here is the procedure (replace Ubuntu-24.04 by whatever you are using) :
If you eventually fill completely you WSL drive, you'll see that it won't be able to grow beyond 40 GB. Thus avoiding the underlying Windows HD to be full. Hope this will help. |
|
It's possible to use this batch file to do the compact steps in one click, when not using sparse: |
and others
Is your feature request related to a problem? Please describe.
I occasionally have workloads that consume large amounts of disk space for temporary use in
/tmp. For example, transcoding large video files or processing large datasets exported from a database. Since WSL2 stores its filesystem on a.vhdx, that file grows when the dataset is processed and never releases that space. Also, it doesn't appear to re-use released disk space in the.vhdxwhen files are deleted... it seems to prefer to grow the file rather than re-use existing empty space. (I haven't explicitly tested that theory, but my workload deletes temporary files as they are used and I never had enough of them existing simultaneously to reach 250GB, but the size of the.vhdxeventually expanded to that size.When I'm done with that workload, I have to:
This is annoying since it basically means that my system backups include a huge
.vhdxfile that is mostly empty. Also, until about 2 weeks ago, it was consuming the entire remainder of my system disk. I had only 50GB free, which was enough to handle the workload, but since it continued to grow even after files were removed, that didn't matter. (I figured it was time to upgrade the disk size anyway)Describe the solution you'd like
Automatic compaction of the
.vhdx, or a way to do so while WSL2 is still running so that I can schedule it for frequent cleanups.Describe alternatives you've considered
it would probably be fine for my desktop, but not really an option for my laptop. Not really worth the time either since that entire process would probably take about 30 minutes, but compacting is usually under 3.
The text was updated successfully, but these errors were encountered: