Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add LoRA configuration support for fine-tuning module #169

Open
4 tasks
gkumbhat opened this issue Sep 5, 2023 · 5 comments · May be fixed by #212
Open
4 tasks

Add LoRA configuration support for fine-tuning module #169

gkumbhat opened this issue Sep 5, 2023 · 5 comments · May be fixed by #212
Assignees
@ibm-peach-fish
Milestone
Add support for LoRA

Comments

@gkumbhat
Copy link
Collaborator

gkumbhat commented Sep 5, 2023

Description

After refactoring #163 PT to take out prompt tuning related configuration to separate module. We want to add support for prompt tuning (specifically LoRA) in fine-tuning module, i.e text_generation. Changes:

  1. Add LoRA configuration support in the toolkit functionality created in Refactor peft module to take out common peft config functionality #163
  2. Add LoRA configuration and training from text-generation module (train).
  3. Expose parameter required for LoRA configuration via .train function
  4. Add support for saving LoRA models with "merged weights". This is to be done in .train function itself, that way, the model that we configure to __init__ function will look like any other transformers model.

Acceptance Criteria

  • Unit tests cover new/changed code
  • Examples build against new/changed code
  • README updated here
  • Example for training lora added in example script
@gkumbhat gkumbhat added this to the Add support for LoRA milestone Sep 5, 2023
@gkumbhat gkumbhat changed the title Add LoRA configuration support for PEFT module Add LoRA configuration support for fine-tuning module Sep 5, 2023
@rawkintrevo rawkintrevo self-assigned this Sep 12, 2023
@chakrn chakrn moved this from ToDo to In Progress in caikit ecosystem Sep 19, 2023
@ibm-peach-fish
Copy link

@gkumbhat please confirm that for description items 2,3,4 you want https://github.com/caikit/caikit-nlp/blob/main/caikit_nlp/modules/text_generation/text_generation_local.py#L172 updated only ( I didn't see a train function in text_generation_tgis.py )

@ibm-peach-fish
Copy link

ibm-peach-fish commented Sep 29, 2023 •
edited
Loading

@gkumbhat re: point 2.

Add LoRA configuration and training from text-generation module (train).

is from a typo that should be to?

@ibm-peach-fish
Copy link

From slack conversation- add flag about weather model is a lora or not to the config.yaml

@ibm-peach-fish
Copy link

also per slack conversation- FYI if anyone is doing something later and happens to read this (good for you).

Merged models have lower latency than base + LoRA

src: https://huggingface.co/docs/peft/conceptual_guides/lora

@ibm-peach-fish
Copy link

@gkumbhat could i get more specification on what readme.md you want updated, I'm having a hard time following the link

@ibm-peach-fish ibm-peach-fish linked a pull request Oct 6, 2023 that will close this issue
Open
8 tasks
@ibm-peach-fish ibm-peach-fish linked a pull request Oct 9, 2023 that will close this issue
[caikitnlp-169] Add LoRA configuration support for fine-tuning module #212
Open
8 tasks
@chakrn chakrn moved this from In Progress to Ready for Review in caikit ecosystem Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ibm-peach-fish ibm-peach-fish

Labels
None yet
Projects
caikit ecosystem
Status: Ready for Review
Milestone
Add support for LoRA
Development

Successfully merging a pull request may close this issue.

[caikitnlp-169] Add LoRA configuration support for fine-tuning module ibm-peach-fish/caikit-nlp
3 participants
@rawkintrevo @gkumbhat @ibm-peach-fish