Does virtual memory keep increasing when switching models?

[s4130]

Each time I switch models, my virtual memory usage increases, which continuously consumes C drive space. Eventually, this causes the program to stop running and even led to a blue screen once.
The models are cached in virtual memory, but when switching to another model, the cached previous model isn't unloaded. Even when switching back to a previously cached model, it will still be cached again in virtual memory.Like below.Am I understanding correctly?

First, I use Model A to generate an image:

Then, I switch to Model B to generate another image:

Next, I switch back to Model A and generate an image:

Finally, I switch to Model B once more to generate an image:

[WhiteZeroX]

Isn't there a Setting for number of cached models to keep? Set it to 0.

[s4130]

Is it these settings? They don’t seem to be working now. #2176

[s4130]

The reason I'm bringing up this issue is that very few people have reported encountering it, so I'm wondering if this is a common situation or just an uncommon one.

[Juqowel]

Model type? Any extensions is on?

[s4130]

The model used above is 1.5.
I tried disabling external extensions and using the XL model.

Here is Model A (XL) loaded::

Switched to Model B (XL):

Switched back to Model A (XL):

At this point, my laptop can no longer handle switching XL models:

[Juqowel]

This looks like a normal behavior of my extension for forge, but if it's not any extension - that's strange. I'm on 9a698e2 now and cannot reproduce this issue.

[s4130]

This is the console output when testing the xl model in the above reply:

Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Version: f2.0.1v1.10.1-previous-613-gf10214e35
Commit hash: f10214e358e730b495ce7bfb0edc4846caecf1b2
Launching Web UI with arguments: --disable-ipex-hijack --theme light --xformers --listen --enable-insecure-extension-access --no-gradio-queue --no-download-sd-model --skip-python-version-check
Total VRAM 4096 MB, total RAM 16215 MB
pytorch version: 2.5.1+cu124
xformers version: 0.0.28.post3
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce GTX 1650 Ti : native
VAE dtype preferences: [torch.float32] -> torch.float32
CUDA Using Stream: False
Using xformers cross attention
Using xformers attention for VAE
*** "Disable all extensions" option was set, will only load built-in extensions ***
ControlNet preprocessor location: D:\sd-webui-aki-v4.5\models\ControlNetPreprocessor
Loading additional modules ... done.
2024-11-10 15:55:07,962 - ControlNet - INFO - ControlNet UI callback registered.
Model selected: {'checkpoint_info': {'filename': 'D:\\sd-webui-aki-v4.5\\models\\Stable-diffusion\\tPonynai3_v65.safetensors', 'hash': '109fc8e7'}, 'additional_modules': [], 'unet_storage_dtype': None}
Using online LoRAs in FP16: True
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 53.3s (prepare environment: 4.4s, import torch: 19.1s, initialize shared: 0.3s, other imports: 0.6s, load scripts: 4.1s, initialize google blockly: 15.0s, create ui: 4.0s, gradio launch: 5.9s).
Environment vars changed: {'stream': False, 'inference_memory': 1638.0, 'pin_shared_memory': False}
[GPU Setting] You will use 60.00% GPU memory (2457.00 MB) to load weights, and use 40.00% GPU memory (1638.00 MB) to do matrix computation.
Environment vars changed: {'stream': False, 'inference_memory': 1024.0, 'pin_shared_memory': False}
[GPU Setting] You will use 74.99% GPU memory (3071.00 MB) to load weights, and use 25.01% GPU memory (1024.00 MB) to do matrix computation.
Model selected: {'checkpoint_info': {'filename': 'D:\\sd-webui-aki-v4.5\\models\\Stable-diffusion\\tPonynai3_v65.safetensors', 'hash': '109fc8e7'}, 'additional_modules': [], 'unet_storage_dtype': None}
Using online LoRAs in FP16: False
Environment vars changed: {'stream': False, 'inference_memory': 1638.0, 'pin_shared_memory': False}
[GPU Setting] You will use 60.00% GPU memory (2457.00 MB) to load weights, and use 40.00% GPU memory (1638.00 MB) to do matrix computation.
Model selected: {'checkpoint_info': {'filename': 'D:\\sd-webui-aki-v4.5\\models\\Stable-diffusion\\tPonynai3_v65.safetensors', 'hash': '109fc8e7'}, 'additional_modules': [], 'unet_storage_dtype': None}
Using online LoRAs in FP16: True
Loading Model: {'checkpoint_info': {'filename': 'D:\\sd-webui-aki-v4.5\\models\\Stable-diffusion\\tPonynai3_v65.safetensors', 'hash': '109fc8e7'}, 'additional_modules': [], 'unet_storage_dtype': None}
[Unload] Trying to free all memory for cuda:0 with 0 models keep loaded ... Done.
StateDict Keys: {'unet': 1680, 'vae': 248, 'text_encoder': 197, 'text_encoder_2': 518, 'ignore': 0}
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
K-Model Created: {'storage_dtype': torch.bfloat16, 'computation_dtype': torch.float32}
Model loaded in 17.8s (unload existing model: 0.3s, forge model load: 17.4s).
All loaded to GPU.
Moving model(s) has taken 0.02 seconds
token_merging_ratio = 0.3
[Unload] Trying to free 8004.16 MB for cuda:0 with 0 models keep loaded ... Current free memory is 3271.72 MB ... Done.
[Memory Management] Target: KModel, Free GPU: 3271.72 MB, Model Require: 4897.05 MB, Previously Loaded: 0.00 MB, Inference Require: 1638.00 MB, Remaining: -3263.33 MB, CPU Swap Loaded (blocked method): 3641.84 MB, GPU Loaded: 1255.21 MB
Moving model(s) has taken 2.78 seconds
[Unload] Trying to free 2592.85 MB for cuda:0 with 0 models keep loaded ... Current free memory is 1994.72 MB ... Current free memory is 1994.72 MB ... Unload model KModel Done.
[Memory Management] Target: IntegratedAutoencoderKL, Free GPU: 3251.47 MB, Model Require: 319.11 MB, Previously Loaded: 0.00 MB, Inference Require: 1638.00 MB, Remaining: 1294.36 MB, All loaded to GPU.
Moving model(s) has taken 4.85 seconds
Model selected: {'checkpoint_info': {'filename': 'D:\\sd-webui-aki-v4.5\\models\\Stable-diffusion\\noobaiXLNAIXL_epsilonPred10Version.safetensors', 'hash': 'bef5b673'}, 'additional_modules': [], 'unet_storage_dtype': None}
Using online LoRAs in FP16: True
Loading Model: {'checkpoint_info': {'filename': 'D:\\sd-webui-aki-v4.5\\models\\Stable-diffusion\\noobaiXLNAIXL_epsilonPred10Version.safetensors', 'hash': 'bef5b673'}, 'additional_modules': [], 'unet_storage_dtype': None}
[Unload] Trying to free all memory for cuda:0 with 0 models keep loaded ... Current free memory is 2930.85 MB ... Current free memory is 2930.85 MB ... Unload model IntegratedAutoencoderKL Done.
StateDict Keys: {'unet': 1680, 'vae': 248, 'text_encoder': 196, 'text_encoder_2': 518, 'ignore': 0}
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
K-Model Created: {'storage_dtype': torch.bfloat16, 'computation_dtype': torch.float32}
Model loaded in 28.0s (unload existing model: 7.1s, forge model load: 20.8s).
All loaded to GPU.
Moving model(s) has taken 0.02 seconds
token_merging_ratio = 0.3
[Unload] Trying to free 8004.16 MB for cuda:0 with 0 models keep loaded ... Current free memory is 3251.60 MB ... Current free memory is 3251.60 MB ... Done.
[Memory Management] Target: KModel, Free GPU: 3251.60 MB, Model Require: 4897.05 MB, Previously Loaded: 0.00 MB, Inference Require: 1638.00 MB, Remaining: -3283.45 MB, CPU Swap Loaded (blocked method): 3657.46 MB, GPU Loaded: 1239.59 MB
Moving model(s) has taken 1.97 seconds
[Unload] Trying to free 2592.85 MB for cuda:0 with 0 models keep loaded ... Current free memory is 2011.21 MB ... Current free memory is 2011.21 MB ... Current free memory is 2011.21 MB ... Unload model KModel Done.
[Memory Management] Target: IntegratedAutoencoderKL, Free GPU: 3251.47 MB, Model Require: 319.11 MB, Previously Loaded: 0.00 MB, Inference Require: 1638.00 MB, Remaining: 1294.36 MB, All loaded to GPU.
Moving model(s) has taken 1.34 seconds
Model selected: {'checkpoint_info': {'filename': 'D:\\sd-webui-aki-v4.5\\models\\Stable-diffusion\\tPonynai3_v65.safetensors', 'hash': '109fc8e7'}, 'additional_modules': [], 'unet_storage_dtype': None}
Using online LoRAs in FP16: True
Loading Model: {'checkpoint_info': {'filename': 'D:\\sd-webui-aki-v4.5\\models\\Stable-diffusion\\tPonynai3_v65.safetensors', 'hash': '109fc8e7'}, 'additional_modules': [], 'unet_storage_dtype': None}
[Unload] Trying to free all memory for cuda:0 with 0 models keep loaded ... Current free memory is 2930.85 MB ... Current free memory is 2930.85 MB ... Current free memory is 2930.85 MB ... Unload model IntegratedAutoencoderKL Done.
StateDict Keys: {'unet': 1680, 'vae': 248, 'text_encoder': 197, 'text_encoder_2': 518, 'ignore': 0}
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
K-Model Created: {'storage_dtype': torch.bfloat16, 'computation_dtype': torch.float32}
Model loaded in 23.3s (unload existing model: 4.6s, forge model load: 18.7s).
All loaded to GPU.
Moving model(s) has taken 0.02 seconds
token_merging_ratio = 0.3
[Unload] Trying to free 8004.16 MB for cuda:0 with 0 models keep loaded ... Current free memory is 3251.60 MB ... Current free memory is 3251.60 MB ... Current free memory is 3251.60 MB ... Done.
[Memory Management] Target: KModel, Free GPU: 3251.60 MB, Model Require: 4897.05 MB, Previously Loaded: 0.00 MB, Inference Require: 1638.00 MB, Remaining: -3283.45 MB, CPU Swap Loaded (blocked method): 3657.46 MB, GPU Loaded: 1239.59 MB
Moving model(s) has taken 2.40 seconds
[Unload] Trying to free 2592.85 MB for cuda:0 with 0 models keep loaded ... Current free memory is 2011.21 MB ... Current free memory is 2011.21 MB ... Current free memory is 2011.21 MB ... Current free memory is 2011.21 MB ... Unload model KModel Done.
[Memory Management] Target: IntegratedAutoencoderKL, Free GPU: 3251.47 MB, Model Require: 319.11 MB, Previously Loaded: 0.00 MB, Inference Require: 1638.00 MB, Remaining: 1294.36 MB, All loaded to GPU.
Moving model(s) has taken 6.62 seconds

[DenOfEquity]

Please take a look at #2315.
It's a simple change, only 3 (+1 comment) lines added to backend/memory_management.py. Needs testing on more setups than just my old laptop.