Fine-Tune APIs for LLM Documentation

[StefanoFioravanzo]

This issue tracks the Kubeflow 1.9 documentation deliverables for the new Fine-Tune APIs for LLMs.

• Write intro doc with high-level presentation and user stories. @StefanoFioravanzo
  Draft to be turned into a docs webpage https://docs.google.com/document/d/18PuuaDRISj5mlrBn1GJrxwuB6Z5zTtXKpVbLUIeLx-8/edit?usp=sharing
  Architecture doc. @andreyvelich
  API docs

  Everything tracked here Training: Add Fine-Tune API Docs website#3718

• Tutorial

cc @kubeflow/wg-training-leads @deepanker13
cc release team docs leads @diegolovison @hbelmiro

[StefanoFioravanzo]

This feature is based on this work:

• Create TFJob and PyTorchJob from Function APIs in the Training SDK #1659

do we have supporting documentation for TFJob and PyTorchJob Function APIs, should we track it separately of include it in the architecture/API doc above?

[deepanker13]

@StefanoFioravanzo I can help with the tutorial. Also do you have any reference for api documentation?

[andreyvelich]

Example Notebooks can be found here:

• https://github.com/kubeflow/training-operator/blob/master/examples/pytorch/text-classification/Fine-Tune-BERT-LLM.ipynb
• https://github.com/kubeflow/training-operator/blob/master/examples/pytorch/language-modeling/train_api_hf_dataset.ipynb

[StefanoFioravanzo]

@andreyvelich @deepanker13 are we writing a tutorial based on these APIs eventually?

[andreyvelich]

@andreyvelich @deepanker13 are we writing a tutorial based on these APIs eventually?

We already have this Notebook to try out this feature: https://github.com/kubeflow/training-operator/blob/master/examples/pytorch/text-classification/Fine-Tune-BERT-LLM.ipynb
I think, initially we can just link this Notebook in the website.

[StefanoFioravanzo]

Ok! Where do you suggest we link it?

[andreyvelich]

Ok! Where do you suggest we link it?

I already linked it here: https://deploy-preview-3718--competent-brattain-de2d6d.netlify.app/docs/components/training/user-guides/fine-tuning/#next-steps

[hbelmiro]

@StefanoFioravanzo is there any pending work to close this issue?

[andreyvelich]

This should be complete.
Thanks @StefanoFioravanzo for your help!