covert SVDLinear dtype

[PHOSPHENES8]

If the data types are not converted appropriately, the following errors may occur:

File "/home/user/miniconda3/lib/python3.11/site-packages/transformers/models/llama/modeling_llama.py", line 691, in forward
    query_states = self.q_proj(hidden_states)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/miniconda3/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/miniconda3/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/miniconda3/lib/python3.11/site-packages/peft/tuners/adalora/layer.py", line 175, in forward
    result += (dropout(x) @ (lora_A * lora_E).T @ lora_B.T) * scaling / ranknum
               ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
RuntimeError: expected mat1 and mat2 to have the same dtype, but got: c10::Half != float```

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[younesbelkada]

In principle this looks good ! thanks a lot for fixing !
what do you think @pacman100 @BenjaminBossan ?

[pacman100]

Thank you for the fixes!