feat: HuggingFaceAPIChatGenerator

[anakin87]

Related Issues

• part of Rethink TGI Generators and TEI embedders #7279
• part of HF TGI generators are restricting too much the available models #7384
• based on feat: HuggingFaceAPIGenerator #7464 (please do not merge before the base has been merged)

Proposed Changes:

As discussed in #7279 (comment),

• introduce a new better HuggingFaceAPIChatGenerator
• deprecate HuggingFaceTGIChatGenerator

How did you test it?

CI, new tests, extensive manual tests: with HF Inference API, local TGI container and paid HF Inference Endpoint

Notes for the reviewer

Our implementation is based on huggingface_hub.
Since huggingface_hub recently introduced a Chat-completion API, I used this instead of the text_generation:

• the chat template is rendered server-side when models are powered by TGI (most of the cases)
• otherwise, the templating happens client-side (requires minijinja) -> we can get rid of transformers in this component
• they aim at rendering all templates server-side in the future.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[vblagoje]

Found some minor issues, great work @anakin87 🚀

[vblagoje]

Found some minor issues, great work @anakin87 🚀