allocate 80% for cpu is unset in max_memory

[SunMarc]

What does this PR do ?

This PR modifies max_memory by allocating 80% of cpu memory of "cpu" is not passed in max_memory. The idea was in case the user did not specify the cpu allocation using max_memory arg, we should just infer from the current system available memory minus some reserved pool for the OS to continue to work. Otherwise, we will move directly to disk offload.
cc @mfuntowicz as you suggested.

Before this PR:

max_memory = {"0:"15GiB"}
get_max_memory(max_memory)
# returns {0: 16106127360}

After this PR

max_memory = {"0:"15GiB"}
get_max_memory(max_memory)
# returns {0: 16106127360,  'cpu': 384541448601.60004}

TODO:

• check accelerate tests
• check transformers tests

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[muellerzr]

LG2M, slight nit on the docstring

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.