Problem when setting CPU memory limit

[BoomBoomPY]

Describe the bug

When trying to set a CPU memory limit, I get this error:

line 85, in convert_file_size_to_int

raise ValueError(err_msg)
ValueError: size 64000MiBGiB is not in a valid format. Use an integer for bytes, or a string with an unit (like ‘5.0GB’).

Is there an existing issue for this?

• I have searched the existing issues

Reproduction

Change the CPU memory and try to load a model

Screenshot

Logs

Traceback (most recent call last):

File “/home/chungus/Downloads/LocalGPT/installer_files/env/lib/python3.10/site-packages/accelerate/utils/modeling.py”, line 70, in convert_file_size_to_int

mem_size = int(float(size[:-3]) * (2**30))
ValueError: could not convert string to float: ‘64000MiB’

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File “/home/chungus/Downloads/LocalGPT/text-generation-webui/modules/ui_model_menu.py”, line 195, in load_model_wrapper

shared.model, shared.tokenizer = load_model(shared.model_name, loader)
File “/home/chungus/Downloads/LocalGPT/text-generation-webui/modules/models.py”, line 79, in load_model

output = load_func_map[loader](model_name)
File “/home/chungus/Downloads/LocalGPT/text-generation-webui/modules/models.py”, line 320, in AutoGPTQ_loader

return modules.AutoGPTQ_loader.load_quantized(model_name)
File “/home/chungus/Downloads/LocalGPT/text-generation-webui/modules/AutoGPTQ_loader.py”, line 57, in load_quantized

model = AutoGPTQForCausalLM.from_quantized(path_to_model, **params)
File “/home/chungus/Downloads/LocalGPT/installer_files/env/lib/python3.10/site-packages/auto_gptq/modeling/auto.py”, line 108, in from_quantized

return quant_func(
File “/home/chungus/Downloads/LocalGPT/installer_files/env/lib/python3.10/site-packages/auto_gptq/modeling/_base.py”, line 859, in from_quantized

max_memory = accelerate.utils.get_balanced_memory(
File “/home/chungus/Downloads/LocalGPT/installer_files/env/lib/python3.10/site-packages/accelerate/utils/modeling.py”, line 767, in get_balanced_memory

max_memory = get_max_memory(max_memory)
File “/home/chungus/Downloads/LocalGPT/installer_files/env/lib/python3.10/site-packages/accelerate/utils/modeling.py”, line 654, in get_max_memory

max_memory[key] = convert_file_size_to_int(max_memory[key])
File “/home/chungus/Downloads/LocalGPT/installer_files/env/lib/python3.10/site-packages/accelerate/utils/modeling.py”, line 85, in convert_file_size_to_int

raise ValueError(err_msg)
ValueError: size 64000MiBGiB is not in a valid format. Use an integer for bytes, or a string with an unit (like ‘5.0GB’).

System Info

Most recent Ubuntu, RTX 4070 (gpu0) and RTX 3070 (gpu1)

[8skull]

*I'm in the same boat with the same problem.

I just avoided using CPU regarding sliders

[github-actions]

This issue has been closed due to inactivity for 6 weeks. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.