Generative AI with Large Language Models

• Foundation Models
• A brief history of how transformers were born
• The new role of AI professionals
• The rise of seamless transformer APIs
• The rise of the Transformer: Attention Is All You Need
• Training and performance
• Hugging Face transformer models
• The paradigm shift: What is an NLP task?
• Investigating the potential of downstream tasks
• Running downstream tasks
• Matching datasets and tokenizers
• Exploring sentence and WordPiece tokenizers to u...fficiency of subword tokenizers for transformers
• Designing a universal text-to-text model
• The rise of text-to-text transformer models
• A prefix instead of task-specific formats
• The T5 model
• Text summarization with T5
• From text-to-text to new word predictions with OpenAI ChatGPT
• Part I: Defining generative ideation
• Part II: Automating prompt design for generative image design
• Part III: Automated generative ideation with Stable Diffusion
• The future is yours!

7 LiveLab in this lesson — see the labs panel →

• Why Mixture of Experts?
• Architecture & Algorithmic Design
• System Engineering Challenges
• Why Knowledge Distillation?
• Comparison between Knowledge Distillation and Traditional Approaches
• Distillation Strategies: Architectures of Transfer
• Training a tokenizer and pretraining a transformer
• Building KantaiBERT from scratch
• Pretraining a Generative AI customer support model on X data
• Getting Started Training Vision Models without Coding
• Uploading the dataset
• Training models with AutoTrain
• Deploying a model
• Running our models for inference

8 LiveLab in this lesson — see the labs panel →

• GPTs as GPTs
• The architecture of OpenAI GPT transformer models
• OpenAI models as assistants
• Getting started with the GPT-4 API
• Retrieval Augmented Generation (RAG) with GPT-4
• Transformer visualization with BertViz
• Interpreting Hugging Face transformers with SHAP
• Transformer visualization via dictionary learning
• Other interpretable AI tools

5 LiveLab in this lesson — see the labs panel →

• The architecture of BERT
• Fine-tuning BERT
• Building a Python interface to interact with the model
• Risk management
• Fine-tuning a GPT model for completion (generative)
• Preparing the dataset
• Fine-tuning an original model
• Running the fine-tuned GPT model
• Managing fine-tuned jobs and models
• Before leaving
• LLM embeddings as an alternative to fine-tuning
• Fundamentals of text embedding with NLTK and Gensim
• Implementing question-answering systems with embedding-based search techniques
• Transfer learning with Ada embeddings

3 LiveLab in this lesson — see the labs panel →

• The emergence of functional AGI
• Cutting-edge platform installation limitations
• Auto-BIG-bench
• WandB
• When will AI agents replicate?
• Risk management
• Risk mitigation tools with RLHF and RAG

3 LiveLab in this lesson — see the labs panel →