Generative AI with LLMs

This comprehensive course provides a deep dive into Generative AI and Large Language Models (LLMs), equipping you with practical knowledge and skills to leverage this transformative technology.

Course Highlights

• Foundational Knowledge: Gain a thorough understanding of generative AI fundamentals, including model architectures, training methodologies, and deployment strategies.

• Practical Skills: Learn hands-on techniques for training, fine-tuning, and deploying LLMs in real-world applications.

• Industry Expertise: Learn directly from AWS AI practitioners who actively build and deploy AI solutions for business use-cases.

• Cutting-Edge Research: Stay current with the latest developments in Generative AI and understand how companies are creating value with this technology.

Course Content

Week 1: Generative AI use cases, project lifecycle, and model pre-training

Videos (116 minutes total)

• Course Introduction (6 min)
• Introduction to Week 1 (5 min)
• Generative AI & LLMs (4 min)
• LLM use cases and tasks (2 min)
• Text generation before transformers (2 min)
• Transformers architecture (7 min)
• Generating text with transformers (5 min)
• Prompting and prompt engineering (5 min)
• Generative configuration (7 min)
• Generative AI project lifecycle (4 min)
• Introduction to AWS labs (5 min)
• Lab 1 walkthrough (14 min)
• Pre-training large language models (9 min)
• Computational challenges of training LLMs (10 min)
• Optional: Efficient multi-GPU compute strategies (8 min)
• Scaling laws and compute-optimal models (8 min)
• Pre-training for domain adaptation (5 min)

Readings (52 minutes total)

• Contributor Acknowledgments (10 min)
• Forum Guidelines and Information (2 min)
• Transformers: Attention is all you need (10 min)
• Lab Guidelines (5 min)
• Domain-specific training: BloombergGPT (10 min)
• Week 1 resources (10 min)
• Lecture Notes Week 1 (5 min)

Hands-on Learning

• Week 1 Assessment (60 min)
• Lab: Generative AI Use Case - Dialogue Summarization (120 min)
• Intake Survey (1 min)

Week 2: Fine-tuning and evaluating large language models

Videos (77 minutes total)

• Introduction - Week 2 (4 min)
• Instruction fine-tuning (7 min)
• Fine-tuning on a single task (3 min)
• Multi-task instruction fine-tuning (8 min)
• Model evaluation (10 min)
• Benchmarks (5 min)
• Parameter efficient fine-tuning (PEFT) (4 min)
• PEFT techniques 1: LoRA (8 min)
• PEFT techniques 2: Soft prompts (7 min)
• Lab 2 walkthrough (17 min)

Readings (25 minutes total)

• Scaling instruct models (10 min)
• Week 2 Resources (10 min)
• Lecture Notes Week 2 (5 min)

Hands-on Learning

• Week 2 Assessment (60 min)
• Lab: Fine-tune a generative AI model for dialogue summarization (120 min)

Week 3: Reinforcement Learning and LLM-Powered Applications

Videos (141 minutes total)

• Introduction - Week 3 (4 min)
• Aligning models with human values (3 min)
• Reinforcement learning from human feedback (RLHF) (8 min)
• RLHF: Obtaining feedback from humans (6 min)
• RLHF: Reward model (2 min)
• RLHF: Fine-tuning with reinforcement learning (3 min)
• Optional video: Proximal policy optimization (13 min)
• RLHF: Reward hacking (6 min)
• Scaling human feedback (5 min)
• Lab 3 walkthrough (18 min)
• Model optimizations for deployment (7 min)
• Generative AI Project Lifecycle Cheat Sheet (2 min)
• Using the LLM in applications (9 min)
• Interacting with external applications (4 min)
• Helping LLMs reason and plan with chain-of-thought (5 min)
• Program-aided language models (PAL) (7 min)
• ReAct: Combining reasoning and action (9 min)
• LLM application architectures (5 min)
• Optional video: AWS Sagemaker JumpStart (5 min)
• Responsible AI (9 min)
• Course conclusion (3 min)

Readings (43 minutes total)

• KL divergence (10 min)
• [IMPORTANT] Reminder about end of access to Lab Notebooks (2 min)
• ReAct: Reasoning and action (10 min)
• Week 3 resources (10 min)
• Lecture Notes Week 3 (5 min)
• Acknowledgments (1 min)
• (Optional) Opportunity to Mentor Other Learners (5 min)

Hands-on Learning

• Week 3 Assessment (60 min)
• Lab: Fine-tune FLAN-T5 with reinforcement learning to generate more-positive summaries (120 min)
• Reinforcement Learning from Human Feedback (RLHF)
• Chain-of-thought prompting
• Knowledge augmentation techniques
• Real-world application strategies

Course Completion & Certification

After completing all the labs, quizzes, and assessments across the three weeks, you will receive a verified certificate from Coursera demonstrating your proficiency in Generative AI and LLMs. This certificate validates your understanding of:

• Foundational concepts in Generative AI
• Advanced model training and fine-tuning techniques
• Practical application of LLMs in real-world scenarios

View my course completion certificate here