Adding Intro to LLMs

Author: bansalkanav

Module 9 - GenAI (LLMs and Prompt Engineering)/1. Intro to LLMs/Introduction to LLMs.ipynb

@@ -0,0 +1,162 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "b323bf88-cf1d-43b4-abb0-0b7c35bb5f42",
+   "metadata": {},
+   "source": [
+    "# **Introduction to LLMs**\n",
+    "\n",
+    "**Credits:** The following short notes on LLMs are created after reading a phenomenal **\"Quick Start Guide To LLMs\"** by **Sinan Ozdemir**.\n",
+    "\n",
+    "### **A little bit about Transformers:**\n",
+    "<img style=\"float: right;\" width=\"400\" height=\"400\" src=\"data/images/transformer.jpeg\">\n",
+    "\n",
+    "1. Sequence to Sequence Model.\n",
+    "2. Has two main components: Encoder and Decoder\n",
+    "3. An **encoder** which is tasked with taking in raw text, splitting them up into its core components, convert them into vectors and using **self-attention** to understand the context of the text.\n",
+    "4. Transformer's self attention mechanism allows each word to \"attend to\" al other words in the sequence which enables it to capture long-term dependencies and contextual relationships between words. The goal is to understand each word as it relates to the other tokens in the input text.\n",
+    "5. A **decoder** excels at generating text by using a modified type of attention (i.e. **cross attention**) to predict the next best token.\n",
+    "6. Transformers are **trained** to solve a specific NLP task called as **Language Modeling**.\n",
+    "7. **Limitation:** Transformers are still limited to an input context window (i.e. maximum length og text it can process at any given moment)\n",
+    "\n",
+    "### **Attention**\n",
+    "1. It is a mechanism that assigns different weights to different parts of the input allowing the model to prioritize and emphasize the most important information while performing tasks like translation or summarization.\n",
+    "2. Attention allows a model to focus on different parts of the input dynamically, leading to improved performance.\n",
+    "\n",
+    "### **What is Language Modeling?**\n",
+    "1. Language Modeling involves creation of statistical/deep learning models for predicting the likelyhood of a sequence of tokens in a specified vocabulary.\n",
+    "2. Two types of Language Modeling Tasks are:  \n",
+    "    a. Autoencoding Task  \n",
+    "    b. Autoregressive Task  \n",
+    "3. **Autoregressive Language Models** are trained to predict the next token in a sentence, based on the previous tokens in the phrase. These models correspond to the **decoder** part of the transformer model. A mask is applied on the full sentence so that the attention head can only see the tokens that came before. These models are ideal for text generatation. For eg: **GPT**\n",
+    "4. **Autoencoding Language Models** are trained to reconstruct the original sentence from a corrupted version of the input. These models correspond to the **encoder** part of the transformer model. Full input is passed. No mask is applied. Autoencoding models create a bidirectional representation of the whole sentence. They can be fine-tuned for a variety of tasks, but their main application is sentence classification or token classification. For eg: **BERT**\n",
+    "5. **Combination of autoregressive and autoencoding language models** are more versatile and flexible in generating text. It has been shown that the combination models can generate more diverse and creative text in different context compared to pure decode-based autoregressive models due to their ability to capture additional context using the encoder. For eg: **T5**\n",
+    "\n",
+    "### **LLMs are:**\n",
+    "1. Usually derived from Transformer architecture (but nor necesserily) by training on large amount of text data.\n",
+    "2. Designed to understand and generate human language, code, and much more.\n",
+    "3. Highly parallelized and scalable.\n",
+    "4. Example: BERT, GPT and T5\n",
+    "5. Techniques like: Stop word removal, stemming, and truncation are not used nor are they necessary for LLMs. LLMs are designed to handle the inherent complexity and variability of human language, including the use of stop words and variations in word forms like tenses and misspellings.\n",
+    "6. Every LLM on the market has been **pre-trained** on a large corpus of the text data and on a specific language modeling related tasks.\n",
+    "7. **Remember:** How an LLM is **pre-trained** and **fine-tuned** makes all the difference.\n",
+    "\n",
+    "### **Pre-Training, Transfer Learning and Fine-Tuning**\n",
+    "<img style=\"float: right;\" width=\"400\" height=\"400\" src=\"data/images/transfer_learning.jpeg\">\n",
+    "\n",
+    "1. **Pre-training** of an LLM happens on a large corpus of text data and on a specific language modeling related task. During this phase LLM tries to learn and understand general language and relationships between words.\n",
+    "2. **Transfer Learning** is a technique used in machine learning to leverage the knowledge gained from one task to improve performance on another related task. Understand that pre-trained model has already learned a lot of information about the language and the relationships between words, and this information can be used as a starting point to improve performance on a new task.  \n",
+    "    **a.** Transfer Learning for LLMs involves taking an LLM that has been pre-trained on one corpus of text data and then fine-tuning it for a specific downstream task, such as text classification or text generation, by updating the model's parameter with task-specific data.  \n",
+    "    **b.** Transfer Learning allows LLMs to be **fine-tuned** for specific tasks with much smaller amounts of task-specific data than it would require if the model were trained from scratch. This greatly reduces the amount of time and resources required to train LLMs.  \n",
+    "<img style=\"float: right;\" width=\"400\" height=\"400\" src=\"data/images/fine_tuning_loop.jpeg\">\n",
+    "3. **Fine-tuning** involves training the LLM on a smaller, task-specific dataset to adjust its parameters for the specific task at hand. The basic fine-tuning loop is more or less same.  \n",
+    "    **a.** Define a model you want to fine-tune as well as fine-tuning parameters (eg: learning rate)  \n",
+    "    **b.** Aggregate some training data.  \n",
+    "    **c.** Compute loss and gradients.  \n",
+    "    **d.** Update the model via backpropogation.  \n",
+    "4. The Transformers package from Hugging Face provides a neat and clean interface for training and fine-tuning LLMs.\n",
+    "\n",
+    "### **Alignment in LLMs**\n",
+    "1. Alignment in Language Model refers to how well the model can respond to the input prompts that match the user's expectations. Put another way, an aligned LLM has an objective that matches a human's objective.\n",
+    "2. A popular method of aligning language model is through the incorporation of Reinforcement Learning into the training loop.\n",
+    "3. Reinforcement Learning with Human Feedback (RLHF) is a popular method of aligning pre-trained LLMs that uses human feedback to enhance their performance.\n",
+    "\n",
+    "### **Popular Modern LLMs**\n",
+    "\n",
+    "#### **1. BERT (Bidirectional Encoder Representation from Transformers)**\n",
+    "<img style=\"float: right;\" width=\"300\" height=\"300\" src=\"data/images/bert_oov.jpeg\">\n",
+    "\n",
+    "1. By Google - Autoencoding Language Model\n",
+    "2. Pretrained on:  \n",
+    "    **a.** English Wikipedia - At the time 2.5 billion words  \n",
+    "    **b.** Book Corpus - 800 million words  \n",
+    "3. BERT's tokenizer handles OOV tokens (out of vocabulary / previously unknown) by breaking them up into smaller chunks of known tokens.\n",
+    "4. Trained on two language modeling specific tasks:  \n",
+    "    **a.** **Masked Language Modeling (MLM) aka Autoencoding Task** - Helps BERT recognize token interaction within the sentence.    \n",
+    "    **b.** **Next Sentence Prediction (NSP) Task** - Helps BERT to understand how tokens interact with each other between sentences.  \n",
+    "<img style=\"float: right;\" width=\"300\" height=\"300\" src=\"data/images/bert_language_model_task.jpeg\">\n",
+    "5. BERT uses three layer of token embedding for a given piece of text: Token Embedding, Segment Embedding and Position Embedding.\n",
+    "6. BERT uses the encoder of transformer and ignores the decoder to become exceedingly good at processing/understanding massive amounts of text very quickly relative to other slower LLMs that focus on generating text one token at a time.\n",
+    "7. BERT itself doesn't classify text or summarize documents but it is often used as a pre-trained model for downstream NLP tasks. \n",
+    "<img style=\"float: right;\" width=\"300\" height=\"300\" src=\"data/images/bert_classification.jpeg\">\n",
+    "8. 1 year later RoBERTa by Facebook AI shown to not require NSP task. It matched and even beat the original BERT model's performance in many areas.\n",
+    "\n",
+    "\n",
+    "#### **2. GPT (Generative Pre-Trained Transformer)**\n",
+    "\n",
+    "1. By OpenAI - Autoregressive Language Model\n",
+    "2. Pretrained on: Proprietary Data (Data for which the rights of ownership are restricted so that the ability to freely distribute the is limited)\n",
+    "3. Autoregressive Language Model that uses attention to predict the next token in a sequence based on the previous tokens.\n",
+    "4. GPT relies on the decoder portion of the Transformer and ignores the encoder to become exceptionally good at generating text one token at a time.\n",
+    "\n",
+    "#### **3. T5 (Text to Text Transfer Transformer)**\n",
+    "<img style=\"float: right;\" width=\"400\" height=\"400\" src=\"data/images/t5.jpeg\">\n",
+    "\n",
+    "1. By Google - Combination of Autoencoder and Autoregressor Language Model.\n",
+    "2. T5 uses both encoder and decoder of the Transformer to become highly versatile in both processing and generating text.\n",
+    "3. T5 based models can generate wide range of NLP tasks, from text classification to generation.\n",
+    "\n",
+    "#### **4. Domain Specific LLMs**\n",
+    "\n",
+    "1. BioGPT - Trained on large scale biomedical literature (more than 2 million articles). Developed by the AI healthcare company, Owkin, in collaboration with Hugging Face.\n",
+    "2. SciBERT\n",
+    "3. BlueBERT\n",
+    "\n",
+    "### **Applications:**\n",
+    "#### **1. Medical Domain**\n",
+    "1. Electronic Medical Record (EMR) Processing\n",
+    "2. Clinical Trial Matching\n",
+    "3. Drug Discovery\n",
+    "\n",
+    "#### **2. Finance**\n",
+    "1. Fraud Detection\n",
+    "2. Sentiment Analysis of Financial News\n",
+    "3. Trading Strategies\n",
+    "4. Customer Service Automation via Chatbots and Virtual Assistants\n",
+    "\n",
+    "#### **3. And many more**\n",
+    "1. Text Classification\n",
+    "2. Text Summarization\n",
+    "3. Chatbots\n",
+    "4. Information Retreival\n",
+    "\n",
+    "### **Quick Summary**\n",
+    "1. What really sets the Transformers appart from other deep learning architectures is its ability to capture long-term dependencies and relationships between tokens using attention mechanism.\n",
+    "2. Attention is the crucial component of Transformer.\n",
+    "3. Factor behind transformer's effectiveness as a language model is it is highly parallelizable, allowing for faster training and efficient processing of text.\n",
+    "4. LLMs are pre-trained on large corpus and fine-tuned on smaller datasets for specific tasks.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "79ed93c8-4db0-4cef-bcaf-3e5a209aaf49",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}