v0.4.0 release tracker

[joecummings]

Estimated release date: Tuesday, November 5th
Estimated branch cut date (aka code freeze): Tuesday, October 29th
Actual branch cut date: Thursday, October 31 🎃
Release owner: @joecummings

New features:

• [Feat] Activation offloading for distributed lora recipe #1645
• Activation offloading for fullfinetuning + fix tied embedding #1847 (@felipemello1)
• Add vqa_dataset, update docs #1820
• Add KD distributed recipe #1631 (@lindawangg, @ebsmothers)
• QLoRA with bias + Llama 3.2 Vision QLoRA configs #1726 (@ebsmothers)
• [Feat] Add Llama3.2V 90B #1837 (@felipemello1)
• Toggling KV-caches #1763
• Qwen2.5 #1863 (@calvinpelletier)

Docs

• Cacheing doc nits #1876
• Point Github link at top right of documentation to pytorch/torchtune #1843 (@ebsmothers)
• Fix contradictory information in InstructDataset and PromptTemplate #1844 (@joecummings)
• Unpin install instructions are hardcoded from a specific PyTorch version #1845 (@RdoubleA)
• TextCompletionDataset load_dataset_kwargs should point to Hugging Face documentation #1846 (@felipemello1)
• Add built-in prompt templates to the bottom of PromptTemplate doc #1851 (@SalmanMohammadi)
• [docs] Custom components page #1910 (@RdoubleA)
• Fixing DoRA docs, adding to mem opt tutorial #1918
• Updating memory optimization overview #1916

QoL improvements:

• Adds __repr__ to Message #1757

Bug fixes:

• Fixing quantization in eval recipe #1777
• [BUG] Include system prompt in Phi3 by default #1778
• Fix save adapter weights only #1764
• Metaformers <> torchtune discrepancies (@pbontrager, @acisseJZhong)
• qwen2 is not supported by QAT #1818 (@joecummings)
• Grad acc "fix" #1842 (@ebsmothers)
• How to use finetuned lora adapter in a huggingface-like pipeline #1779 (@pbontrager)
• [BUG] Llama3.2 vision eleuther eval recipe RuntimeError: stack expects each tensor to be equal size, but got [259, 6404] #1874 (@pbontrager)
• [bug] fix sharding multimodal #1889 (@felipemello1)
• Restore backward after each batch for grad accum #1917 (@ebsmothers)

Deprecations -> Removal

• Delete deprecated ChatDataset and InstructDataset #1781
• Remove deprecated _converters.py file #1839 (@RdoubleA)
• Remove deprecated torchtune/models/gemma/transformer.py #1841 (@SalmanMohammadi)
• Delete chat_formats.py and instruct_formats.py #1849 (@RdoubleA)

New deprecations

• Deprecating TiedEmbeddingTransformerDecoder #1815
• Move cosine LR scheduler with warmup from modules/ to training/ #1587

[krammnic]

@joecummings We actually deprecated lr_schedulers in modules(moved to training)

[RdoubleA]

can we also add #1767

[joecummings]

can we also add #1767

Yep! Feel free to update the tracker with this once #1767 has a dedicated owner. (Assign one please)

[joecummings]

Completed https://github.com/pytorch/torchtune/releases/tag/v0.4.0