diff --git a/colab notebook/Agents_01.ipynb b/jupyter notebooks/Agents Fast Start/Agents_01.ipynb similarity index 94% rename from colab notebook/Agents_01.ipynb rename to jupyter notebooks/Agents Fast Start/Agents_01.ipynb index d2bef9c7..f27d5c9a 100644 --- a/colab notebook/Agents_01.ipynb +++ b/jupyter notebooks/Agents Fast Start/Agents_01.ipynb @@ -1,47 +1,36 @@ { - "nbformat": 4, - "nbformat_minor": 0, - "metadata": { - "colab": { - "provenance": [] - }, - "kernelspec": { - "name": "python3", - "display_name": "Python 3" - }, - "language_info": { - "name": "python" - } - }, "cells": [ { "cell_type": "markdown", + "metadata": { + "id": "StkY5oHGU-iN" + }, "source": [ "# LLMWare Model Exploration\n", "\n", "## This is the 'entrypoint' example that provides a general introduction of llmware models.\n", "\n", "This notebook provides an introduction to LLMWare Agentic AI models and demonstrates their usage." - ], - "metadata": { - "id": "StkY5oHGU-iN" - } + ] }, { "cell_type": "code", - "source": [ - "# install dependencies\n", - "!pip3 install llmware" - ], + "execution_count": null, "metadata": { "collapsed": true, "id": "KyaEnPzOVTJe" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "# install dependencies\n", + "!pip3 install llmware" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "mcOxXgs1XTjD" + }, "source": [ "If you have any dependency install issues, please review the README, docs link, or raise an Issue.\n", "\n", @@ -58,38 +47,41 @@ "The second script `\"welcome_to_llmware.sh\"` will install all of the dependencies.\n", "\n", "If using Windows, then use the `\"welcome_to_llmware_windows.sh\"` script." - ], - "metadata": { - "id": "mcOxXgs1XTjD" - } + ] }, { "cell_type": "code", - "source": [ - "# Import Library\n", - "from llmware.models import ModelCatalog" - ], + "execution_count": null, "metadata": { "id": "n4aKjcEiVjYE" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "# Import Library\n", + "from llmware.models import ModelCatalog" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "ePtRGBIlZEkP" + }, "source": [ "## GETTING STARTED WITH AGENTIC AI\n", "All LLMWare models are accessible through the ModelCatalog generally consisting of two steps to access any model\n", "\n", "- Step 1 - load the model - pulls from global repo the first time, and then automatically caches locally\n", "- Step 2 - use the model with inference or function call" - ], - "metadata": { - "id": "ePtRGBIlZEkP" - } + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true, + "id": "D0xL5WOgVlGX" + }, + "outputs": [], "source": [ "# 'Standard' Models use 'inference' and take a general text input and provide a general text output\n", "\n", @@ -97,16 +89,16 @@ "response = model.inference(\"My son is 21 years old.\\nHow old is my son?\")\n", "\n", "print(\"\\nresponse: \", response)" - ], - "metadata": { - "collapsed": true, - "id": "D0xL5WOgVlGX" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true, + "id": "1AkSZ3Z_VqWt" + }, + "outputs": [], "source": [ "# Optional parameters can improve results\n", "model = ModelCatalog().load_model(\"bling-phi-3-gguf\", temperature=0.0,sample=False, max_output=200)\n", @@ -120,16 +112,13 @@ "response = model.inference(prompt,add_context=text_passage)\n", "\n", "print(\"\\nresponse: \", response)" - ], - "metadata": { - "collapsed": true, - "id": "1AkSZ3Z_VqWt" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", + "metadata": { + "id": "OuNEktB-aPVw" + }, "source": [ "## Models we have and support\n", "Inference models can also be integrated into Prompts - which provide advanced handling for integrating with knowledge retrieval, managing source information, and providing fact-checking\n", @@ -140,13 +129,16 @@ "- we do **include other popular models** such as `phi-3`, `qwen-2`, `yi`, `llama-3`, `mistral`\n", "- it is easy to extend the model catalog to **include other 3rd party models**, including `ollama` and `lm studio`.\n", "- we do **support** `open ai`, `anthropic`, `cohere` and `google api` models as well." - ], - "metadata": { - "id": "OuNEktB-aPVw" - } + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true, + "id": "erEHenbjaYqi" + }, + "outputs": [], "source": [ "all_generative_models = ModelCatalog().list_generative_local_models()\n", "print(\"\\n\\nModel Catalog - load model with ModelCatalog().load_model(model_name)\")\n", @@ -156,28 +148,28 @@ " model_family = model[\"model_family\"]\n", "\n", " print(\"model: \", i, model)" - ], - "metadata": { - "collapsed": true, - "id": "erEHenbjaYqi" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", + "metadata": { + "id": "tLCuxZcYdTHn" + }, "source": [ "## Slim Models\n", "Slim models are 'Function Calling' Models that perform a specialized task and output python dictionaries\n", "- by design, slim models are specialists that **perform single function**.\n", "- by design, slim models generally **do not require any specific** `'prompt instructions'`, but will often accept a `\"parameter\"` which is passed to the function." - ], - "metadata": { - "id": "tLCuxZcYdTHn" - } + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true, + "id": "1ZS2wo8zdDOd" + }, + "outputs": [], "source": [ "model = ModelCatalog().load_model(\"slim-sentiment-tool\")\n", "response = model.function_call(\"That was the worst earnings call ever - what a disaster.\")\n", @@ -186,16 +178,15 @@ "print(\"\\nresponse: \", response)\n", "print(\"llm_response: \", response['llm_response'])\n", "print(\"sentiment: \", response['llm_response']['sentiment'])" - ], - "metadata": { - "collapsed": true, - "id": "1ZS2wo8zdDOd" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Ohp-shGjkDkz" + }, + "outputs": [], "source": [ "# here is one of the slim model applied against a common earnings extract\n", "\n", @@ -212,15 +203,15 @@ "response = model.function_call(text_passage,function=\"extract\",params=[\"revenue\"])\n", "\n", "print(\"\\nextract response: \", response)" - ], - "metadata": { - "id": "Ohp-shGjkDkz" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "w13LLW_OdxCm" + }, + "outputs": [], "source": [ "# Function calling models generally come with a test set that is a great way to learn how they work\n", "# please note that each test can take a few minutes with 20-40 test questions\n", @@ -232,39 +223,50 @@ "ModelCatalog().tool_test_run(\"slim-summary-tool\")\n", "ModelCatalog().tool_test_run(\"slim-xsum-tool\")\n", "ModelCatalog().tool_test_run(\"slim-boolean-tool\")" - ], - "metadata": { - "id": "w13LLW_OdxCm" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", + "metadata": { + "id": "kOPly8bfdnan" + }, "source": [ "## Agentic AI\n", "Function calling models can be integrated into Agent processes which can orchestrate processes comprising multiple models and steps - most of our use cases will use the function calling models in that context\n", "\n", "## Last note:\n", "Most of the models are packaged as `\"gguf\"` usually identified as GGUFGenerativeModel, or with `'-gguf'` or `'-tool` at the end of their name. These models are optimized to run most efficiently on a CPU-based laptop (especially Mac OS). You can also try the standard Pytorch versions of these models, which should yield virtually identical results, but will be slower." - ], - "metadata": { - "id": "kOPly8bfdnan" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "rvLVgWYMe6RO" + }, "source": [ "## Journey is yet to start!\n", "Loved it?? This is just an example of our models. Please check out our other Agentic AI examples with every model in detail here: https://github.com/llmware-ai/llmware/tree/main/fast_start/agents\n", "\n", "Also, if you have more interest in RAG, then please go with our RAG examples, which you can find here: https://github.com/llmware-ai/llmware/tree/main/fast_start/rag\n", "\n", - "If you liked it, then please **star our repo https://github.com/llmware-ai/llmware** ⭐" - ], - "metadata": { - "id": "rvLVgWYMe6RO" - } + "If you liked it, then please **star our repo https://github.com/llmware-ai/llmware** ⭐\n", + "\n", + "Any doubts?? Join our **discord server: https://discord.gg/GN49aWx2H3** 🫂" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" } - ] -} \ No newline at end of file + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/jupyter notebooks/Agents Fast Start/Agents_02.ipynb b/jupyter notebooks/Agents Fast Start/Agents_02.ipynb new file mode 100644 index 00000000..af663fa4 --- /dev/null +++ b/jupyter notebooks/Agents Fast Start/Agents_02.ipynb @@ -0,0 +1,375 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "XpQMDcKy-fa0" + }, + "source": [ + "# Welcome to BLING & DRAGON Model Zoo Sampler by LLMWare.ai 🦾\n", + "This is the 2nd tutorial of our Agents Fast Start Series. So please fasten your seat belts and get ready to dive in Bling and Dragon models.\n", + "\n", + "\n", + "## Get started with local inferences in minutes...\n", + "\n", + "This example is a \"hello world\" model zoo sampler for LLMWare BLING and DRAGON models, and shows a simple epeatable recipe for prompting models locally using a provided set of sample context passages and questions.\n", + "\n", + "It is designed to be easy to select among different LLMWARE models in the BLING (Best Little Instruct No-GPU) and DRAGON (Delivering RAG On ...) model families." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "kWJY0nLC-fzM" + }, + "source": [ + "## But what are Bling and Dragon models??\n", + "Both Bling and Dragon models have been fine-tuned on complex business, financial and legal materials, with specific focus on accurate fact-based question-answering for a context passage.\n", + "\n", + "**BLING models** vary between 0.5B - 3.8B parameters and have been specifically designed to run on a CPU, including on local laptops.\n", + "\n", + "**DRAGON models** vary between 6-9B parameters, and when quantized, can generally run OK on most CPU-based laptops, but are designed for optimal use on a GPU or inference server. These models operate on the same principles as the BLING models, making it easy to 'test' with BLING, and then 'upgrade' to DRAGON in shifting into production environment for greater accuracy.\n", + "\n", + "### Key training objectives:\n", + "- **Fact Reliance**: Answers are grounded in the provided context; without it, the model often returns \"Not Found,\" reducing hallucinations and making it ideal for RAG and workflow tasks.\n", + "\n", + "- **Concise Answers**: Responses are short and factual—optimized for speed, integration, and automation rather than chat-like dialogue.\n", + "\n", + "- **Negative Sampling**: Trained to respond with \"Not Found\" when the context doesn’t contain the answer, aiding in multi-passage retrieval and filtering.\n", + "\n", + "- **Instruction-Free**: No system prompts or role instructions needed—just context and a question." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "yU-iCADREXj4" + }, + "outputs": [], + "source": [ + "# install dependencies\n", + "!pip3 install llmware # if you're getting error, then try upgrading the pip by runnnig this command : python -m pip install --upgrade pip" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "MbFpA7pWEZok" + }, + "source": [ + "If you have any dependency install issues, please review the README, docs link, or raise an Issue.\n", + "\n", + "Usually, if there is a missing dependency, the code will give the warning - and a clear direction like `pip install transformers'` required for this example, etc.\n", + "\n", + "As an alternative to pip install ... if you prefer, you can also clone the repo from github which provides a benefit of having access to 100+ examples.\n", + "\n", + "To clone the repo:\n", + "```\n", + "git clone \"https://www.github.com/llmware-ai/llmware.git\"\n", + "sh \"welcome_to_llmware.sh\"\n", + "```\n", + "\n", + "The second script `\"welcome_to_llmware.sh\"` will install all of the dependencies.\n", + "\n", + "If using Windows, then use the `\"welcome_to_llmware_windows.sh\"` script." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "otqo-DNMItUc" + }, + "outputs": [], + "source": [ + "# Now's let jump into the code part!\n", + "import time # To see how much time a model takes to load\n", + "from llmware.prompts import Prompt # A class (or function) inside the prompts module that helps you build or manage prompts for LLMs." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "DpwHOs_ILM5d" + }, + "source": [ + "Now below is a function that contains a few test cases. You can add your own test cases as well.\n", + "\n", + "To adapt this test set, just create your own list with dictionary entries and keys 'query', 'answer' and 'context'." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "I8-bR172JT1K" + }, + "outputs": [], + "source": [ + "def hello_world_questions():\n", + "\n", + " \"\"\"\n", + " Representative test set - we would recommend running this script as a 'hello world' test on the first\n", + " try that you use a model, and then adapt the content to your own set of context and questions.\n", + "\n", + " There is nothing special about these questions, and in fact, you will note that many of the models will get\n", + " a couple of answers wrong (especially the ~1B parameter models). The errors are important insights\n", + " as you evaluate which models to consider for your use case.\n", + " \"\"\"\n", + "\n", + " test_list = [\n", + "\n", + " {\"query\": \"What is the total amount of the invoice?\",\n", + " \"answer\": \"$22,500.00\",\n", + " \"context\": \"Services Vendor Inc. \\n100 Elm Street Pleasantville, NY \\nTO Alpha Inc. 5900 1st Street \"\n", + " \"Los Angeles, CA \\nDescription Front End Engineering Service $5000.00 \\n Back End Engineering\"\n", + " \" Service $7500.00 \\n Quality Assurance Manager $10,000.00 \\n Total Amount $22,500.00 \\n\"\n", + " \"Make all checks payable to Services Vendor Inc. Payment is due within 30 days.\"\n", + " \"If you have any questions concerning this invoice, contact Bia Hermes. \"\n", + " \"THANK YOU FOR YOUR BUSINESS! INVOICE INVOICE # 0001 DATE 01/01/2022 FOR Alpha Project P.O. # 1000\"},\n", + "\n", + " {\"query\": \"What was the amount of the trade surplus?\",\n", + " \"answer\": \"62.4 billion yen ($416.6 million)\",\n", + " \"context\": \"Japan’s September trade balance swings into surplus, surprising expectations\"\n", + " \"Japan recorded a trade surplus of 62.4 billion yen ($416.6 million) for September, \"\n", + " \"beating expectations from economists polled by Reuters for a trade deficit of 42.5 \"\n", + " \"billion yen. Data from Japan’s customs agency revealed that exports in September \"\n", + " \"increased 4.3% year on year, while imports slid 16.3% compared to the same period \"\n", + " \"last year. According to FactSet, exports to Asia fell for the ninth straight month, \"\n", + " \"which reflected ongoing China weakness. Exports were supported by shipments to \"\n", + " \"Western markets, FactSet added. — Lim Hui Jie\"},\n", + "\n", + " {\"query\": \"What was Microsoft's revenue in the 3rd quarter?\",\n", + " \"answer\": \"$52.9 billion\",\n", + " \"context\": \"Microsoft Cloud Strength Drives Third Quarter Results \\nREDMOND, Wash. — April 25, 2023 — \"\n", + " \"Microsoft Corp. today announced the following results for the quarter ended March 31, 2023,\"\n", + " \" as compared to the corresponding period of last fiscal year:\\n· Revenue was $52.9 billion\"\n", + " \" and increased 7% (up 10% in constant currency)\\n· Operating income was $22.4 billion \"\n", + " \"and increased 10% (up 15% in constant currency)\\n· Net income was $18.3 billion and \"\n", + " \"increased 9% (up 14% in constant currency)\\n· Diluted earnings per share was $2.45 \"\n", + " \"and increased 10% (up 14% in constant currency).\\n\"},\n", + "\n", + " {\"query\": \"When did the LISP machine market collapse?\",\n", + " \"answer\": \"1987.\",\n", + " \"context\": \"The attendees became the leaders of AI research in the 1960s.\"\n", + " \" They and their students produced programs that the press described as 'astonishing': \"\n", + " \"computers were learning checkers strategies, solving word problems in algebra, \"\n", + " \"proving logical theorems and speaking English. By the middle of the 1960s, research in \"\n", + " \"the U.S. was heavily funded by the Department of Defense and laboratories had been \"\n", + " \"established around the world. Herbert Simon predicted, 'machines will be capable, \"\n", + " \"within twenty years, of doing any work a man can do'. Marvin Minsky agreed, writing, \"\n", + " \"'within a generation ... the problem of creating 'artificial intelligence' will \"\n", + " \"substantially be solved'. They had, however, underestimated the difficulty of the problem. \"\n", + " \"Both the U.S. and British governments cut off exploratory research in response \"\n", + " \"to the criticism of Sir James Lighthill and ongoing pressure from the US Congress \"\n", + " \"to fund more productive projects. Minsky's and Papert's book Perceptrons was understood \"\n", + " \"as proving that artificial neural networks approach would never be useful for solving \"\n", + " \"real-world tasks, thus discrediting the approach altogether. The 'AI winter', a period \"\n", + " \"when obtaining funding for AI projects was difficult, followed. In the early 1980s, \"\n", + " \"AI research was revived by the commercial success of expert systems, a form of AI \"\n", + " \"program that simulated the knowledge and analytical skills of human experts. By 1985, \"\n", + " \"the market for AI had reached over a billion dollars. At the same time, Japan's fifth \"\n", + " \"generation computer project inspired the U.S. and British governments to restore funding \"\n", + " \"for academic research. However, beginning with the collapse of the Lisp Machine market \"\n", + " \"in 1987, AI once again fell into disrepute, and a second, longer-lasting winter began.\"},\n", + "\n", + " {\"query\": \"When will employment start?\",\n", + " \"answer\": \"April 16, 2012.\",\n", + " \"context\": \"THIS EXECUTIVE EMPLOYMENT AGREEMENT (this “Agreement”) is entered \"\n", + " \"into this 2nd day of April, 2012, by and between Aphrodite Apollo \"\n", + " \"(“Executive”) and TestCo Software, Inc. (the “Company” or “Employer”), \"\n", + " \"and shall become effective upon Executive’s commencement of employment \"\n", + " \"(the “Effective Date”) which is expected to commence on April 16, 2012. \"\n", + " \"The Company and Executive agree that unless Executive has commenced \"\n", + " \"employment with the Company as of April 16, 2012 (or such later date as \"\n", + " \"agreed by each of the Company and Executive) this Agreement shall be \"\n", + " \"null and void and of no further effect.\"}\n", + " ]\n", + "\n", + " return test_list" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Rczsud2kOhqp" + }, + "source": [ + "Now, let's try to run these test cases with different models of Dragon and Bling series.\n", + "\n", + "We will be printing:\n", + "- LLM Response\n", + "- Gold Answer\n", + "- LLM Usage" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "UpcI5QIjOzSQ" + }, + "outputs": [], + "source": [ + "def llmware_bling_dragon_hello_world (model_name):\n", + "\n", + " \"\"\" Simple inference loop that loads a model and runs through a series of test questions. \"\"\"\n", + "\n", + " t0 = time.time()\n", + " test_list = hello_world_questions()\n", + "\n", + " print(f\"\\n > Loading Model: {model_name}...\")\n", + "\n", + " # please note that by default, we recommend setting temperature=0.0 and sample=False for fact-based RAG\n", + " prompter = Prompt().load_model(model_name, temperature=0.0, sample=False)\n", + "\n", + " t1 = time.time()\n", + " print(f\"\\n > Model {model_name} load time: {t1-t0} seconds\")\n", + "\n", + " for i, entries in enumerate(test_list):\n", + " print(f\"\\n{i+1}. Query: {entries['query']}\")\n", + "\n", + " # run the prompt\n", + " output = prompter.prompt_main(entries[\"query\"],context=entries[\"context\"], prompt_name=\"default_with_context\")\n", + "\n", + " llm_response = output[\"llm_response\"].strip(\"\\n\")\n", + " print(f\"LLM Response: {llm_response}\")\n", + " print(f\"Given Answer: {entries['answer']}\")\n", + " print(f\"LLM Usage: {output['usage']}\")\n", + "\n", + " t2 = time.time()\n", + " print(f\"\\nTotal processing time: {t2-t1} seconds\")\n", + "\n", + " return 0" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "KALiWrOtO7N9" + }, + "source": [ + "Below is the main function with a series of models listed. Select the models one by one by just replacing the name of `my_model = bling_gguf[1]` to the one you want to use with the index i.e. the place where the model you want to test is present.\n", + "\n", + "For example, if I want to run dragon_gguf -> \"dragon-mistral-answer-tool\" model which is present at position 3 or index 2 (indexing starts from 0), therefore we will use `my_model = dragon_gguf[2]`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "tuNXtu4GPDoV" + }, + "outputs": [], + "source": [ + "if __name__ == \"__main__\":\n", + "\n", + " bling_pytorch = [\n", + "\n", + " # pytorch models - will run fast on GPU, and smaller ones good for CPU only\n", + " # note: you will need to install pytorch and transformers to pull and access these models\n", + "\n", + " \"llmware/bling-1b-0.1\",\n", + " \"llmware/bling-tiny-llama-v0\",\n", + " \"llmware/bling-1.4b-0.1\",\n", + " \"llmware/bling-falcon-1b-0.1\",\n", + " \"llmware/bling-cerebras-1.3b-0.1\",\n", + " \"llmware/bling-sheared-llama-1.3b-0.1\",\n", + " \"llmware/bling-sheared-llama-2.7b-0.1\",\n", + " \"llmware/bling-red-pajamas-3b-0.1\",\n", + " \"llmware/bling-stable-lm-3b-4e1t-v0\",\n", + " \"llmware/bling-phi-3\",\n", + " \"llmware/bling-phi-3.5\"\n", + " ]\n", + "\n", + " dragon_pytorch = [\n", + "\n", + " # pytorch models - intended for GPU server use - will require pytorch, transformers, and in some cases,\n", + " # other dependencies (einops, flash_attn).\n", + "\n", + " \"llmware/dragon-mistral-7b-v0\",\n", + " \"llmware/dragon-yi-6b-v0\",\n", + " \"llmware/dragon-qwen-7b\",\n", + " \"llmware/dragon-llama-7b-v0\",\n", + " \"llmware/dragon-mistral-0.3\",\n", + " \"llmware/dragon-llama-3.1\",\n", + " \"llmware/dragon-deci-7b-v0\"\n", + " ]\n", + "\n", + " bling_gguf = [\n", + "\n", + " # smaller cpu-oriented models - optimal for running on a CPU\n", + "\n", + " \"bling-phi-3.5-gguf\", # **NEW** - phi-3.5 (3.8b)\n", + " \"bling-answer-tool\", # this is quantized bling-tiny-llama (1.1b)\n", + " \"bling-qwen-0.5b-gguf\", # **NEW** - qwen2 (0.5b)\n", + " \"bling-qwen-1.5b-gguf\", # **NEW** - qwen2 (1.5b)\n", + " \"bling-stablelm-3b-tool\", # quantized bling-stablelm-3b (2.7b)\n", + " \"bling-phi-3-gguf\", # quantized phi-3 (3.8b)\n", + " \"bling-phi-2-gguf\", # quantized phi-2 (2.7b)\n", + " ]\n", + "\n", + " dragon_gguf = [\n", + "\n", + " # larger models - 6b - 9b\n", + "\n", + " \"dragon-yi-answer-tool\", # quantized yi-6b (v1) (6b)\n", + " \"dragon-llama-answer-tool\",\n", + " \"dragon-mistral-answer-tool\",\n", + " \"dragon-qwen-7b-gguf\", # **NEW** qwen2-7b (7b)\n", + " \"dragon-yi-9b-gguf\", # **NEW** yi-9b (8.8b)\n", + " \"dragon-llama-3.1-gguf\",\n", + " \"dragon-mistral-0.3-gguf\"\n", + "\n", + " ]\n", + "\n", + " # for most use cases, we would recommend using the GGUF for faster inference\n", + " # NEW - if you are running on a Windows machine, then try substituting for one of the following:\n", + " # -- \"bling-tiny-llama-ov\" -> uses OpenVino model version - requires `pip install openvino` and `pip install openvino_genai`\n", + " # -- \"bling-tiny-llama-onnx\" -> uses ONNX model version - requires `pip install onnxruntime_genai`\n", + "\n", + " my_model = bling_gguf[1]\n", + "\n", + " llmware_bling_dragon_hello_world(my_model)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7EM24n7gMxWw" + }, + "source": [ + "# Final Notes 🌪️\n", + "\n", + "We are always updating the BLING and DRAGON model collection with new models, including improvements in the training techniques and experimenting with new base models, and try to keep this script updated as a good 'hello world' sandbox entry point for testing and evaluation.\n", + "\n", + "We have trained these models on a wide range of base foundation models to also support preferences among specific users and clients for a particular base model.\n", + "\n", + "We score each of these models on a RAG benchmark test for accuracy and a number of specialized metrics. Please see the example \"`get_model_benchmarks`\" for a view of this.\n", + "\n", + "Loved it?? This is just an example of our models. Please check out our other Agentic AI examples with every model in detail here: https://github.com/llmware-ai/llmware/blob/main/fast_start/agents/agents-2-llmware_model_sampler_bling_dragon.py\n", + "\n", + "Also, if you have more interest in RAG, then please go with our RAG examples, which you can find here: https://github.com/llmware-ai/llmware/tree/main/fast_start/rag\n", + "\n", + "If you liked it, then please **star our repo** https://github.com/llmware-ai/llmware ⭐\n", + "\n", + "Any doubts?? Join our **discord server: https://discord.gg/GN49aWx2H3** 🫂" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/jupyter notebooks/Agents Fast Start/Agents_03.ipynb b/jupyter notebooks/Agents Fast Start/Agents_03.ipynb new file mode 100644 index 00000000..a2f86d03 --- /dev/null +++ b/jupyter notebooks/Agents Fast Start/Agents_03.ipynb @@ -0,0 +1,294 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "source": [ + "# Welcome to Slim Extract Fast Start by LLMWare.ai 🦾\n", + "This is the 3rd tutorial of our Agents Fast Start Series. In this tutorial, we will dive into the use of Sliim Extract model.\n", + "\n", + "\n", + "## Slim Extract Model ...\n", + "\n", + "SLIMs are **Structured Language Instruction Models**, which are small, specialized 1-3B parameter LLMs, finetuned to generate structured outputs (Python dictionaries and lists, JSON and SQL) that can be handled programmatically, and stacked together in multi-step, multi-model Agent workflows - all running on a local CPU.\n", + "\n", + "The Slim-extract model are used to **extract custom keys from selected text**.\n" + ], + "metadata": { + "id": "7nv_BZnr7MUU" + } + }, + { + "cell_type": "code", + "source": [ + "# install dependencies\n", + "!pip3 install llmware # if you're getting error, then try upgrading the pip by runnnig this command : python -m pip install --upgrade pip" + ], + "metadata": { + "id": "K68hEcgu78my" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "If you have any dependency install issues, please review the README, docs link, or raise an Issue.\n", + "\n", + "Usually, if there is a missing dependency, the code will give the warning - and a clear direction like `pip install transformers'` required for this example, etc.\n", + "\n", + "As an alternative to pip install ... if you prefer, you can also clone the repo from github which provides a benefit of having access to 100+ examples.\n", + "\n", + "To clone the repo:\n", + "```\n", + "git clone \"https://www.github.com/llmware-ai/llmware.git\"\n", + "sh \"welcome_to_llmware.sh\"\n", + "```\n", + "\n", + "The second script `\"welcome_to_llmware.sh\"` will install all of the dependencies.\n", + "\n", + "If using Windows, then use the `\"welcome_to_llmware_windows.sh\"` script." + ], + "metadata": { + "id": "TjpSZbx179pB" + } + }, + { + "cell_type": "code", + "source": [ + "# Import the ModelCatalog\n", + "from llmware.models import ModelCatalog" + ], + "metadata": { + "id": "DFpu9BOd8Ax6" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "We have included a set of sample earnings releases (comprising lines ~10 - ~385 of this script), and run a simple loop through the earnings releases, showing how to create an extract prompt to identify 'revenue growth' from these examples.\n", + "\n", + "here are several function-calling models in the slim-extract family, fine-tuned on multiple leading small model base foundations - full list and options are below in the code." + ], + "metadata": { + "id": "7U3N9zlV8PkI" + } + }, + { + "cell_type": "code", + "source": [ + "earnings_releases = [\n", + "\n", + " {\"context\": \"Adobe shares tumbled as much as 11% in extended trading Thursday after the design software maker \"\n", + " \"issued strong fiscal first-quarter results but came up slightly short on quarterly revenue guidance. \"\n", + " \"Here’s how the company did, compared with estimates from analysts polled by LSEG, formerly known as Refinitiv: \"\n", + " \"Earnings per share: $4.48 adjusted vs. $4.38 expected Revenue: $5.18 billion vs. $5.14 billion expected \"\n", + " \"Adobe’s revenue grew 11% year over year in the quarter, which ended March 1, according to a statement. \"\n", + " \"Net income decreased to $620 million, or $1.36 per share, from $1.25 billion, or $2.71 per share, \"\n", + " \"in the same quarter a year ago. During the quarter, Adobe abandoned its $20 billion acquisition of \"\n", + " \"design software startup Figma after U.K. regulators found competitive concerns. The company paid \"\n", + " \"Figma a $1 billion termination fee.\"},\n", + "\n", + " {\"context\": \"Dick’s Sporting Goods raised its dividend by 10% on Thursday as the company posted its largest sales \"\n", + " \"quarter in its history and projected another year of growth. The company’s shares jumped more than \"\n", + " \"15% in intraday trading. CEO Lauren Hobart said on an earnings call Thursday that Dick’s sales \"\n", + " \"growth came from bigger tickets — either higher prices or more expensive items — as its transactions \"\n", + " \"were flat. Many retailers benefited from a 53rd week in fiscal 2023, but Dick’s said it still broke \"\n", + " \"records during its fiscal fourth quarter even without those extra days. Here’s how the athletic \"\n", + " \"apparel retailer did compared with what Wall Street was anticipating, based on a survey of \"\n", + " \"analysts by LSEG, formerly known as Refinitiv: Earnings per share: $3.85 adjusted vs. $3.35 expected \"\n", + " \"Revenue: $3.88 billion vs. $3.80 billion expected The company’s reported net income for the three-month \"\n", + " \"period that ended Feb. 3 was $296 million, or $3.57 per share, compared with $236 million, or $2.60 a \"\n", + " \"share, a year earlier. Excluding one-time items related to impairment charges and inventory write-offs, \"\n", + " \"Dick’s reported earnings per share of $3.85. Sales rose to $3.88 billion, up about 8% from $3.60 billion \"\n", + " \"a year earlier. “With our industry-leading assortment and strong execution, we capped off the year \"\n", + " \"with an incredibly strong fourth quarter and holiday season,” Hobart said in a statement. “We are \"\n", + " \"guiding to another strong year in 2024. We plan to grow both our sales and earnings through \"\n", + " \"positive comps, higher merchandise margin and productivity gains,” she added. During the quarter, \"\n", + " \"same-store sales rose 2.8%, well ahead of the 0.8% lift that analysts had expected, according to \"\n", + " \"StreetAccount. “Growth in transactions” and market share gains drove the increase, said Executive \"\n", + " \"Chairman Ed Stack.\"},\n", + "\n", + " {\"context\": \"Comcast topped both revenue and profit estimates in the fourth quarter as it lost fewer broadband \"\n", + " \"subscribers than expected, and it raised its dividend 7%, the company said Thursday. \"\n", + " \"Here’s how Comcast performed, compared with estimates from analysts surveyed by LSEG, \"\n", + " \"formerly known as Refinitiv. Earnings per share: 84 cents adjusted vs. 79 cents expected \"\n", + " \"Revenue: $31.25 billion vs. $30.51 billion expected For the quarter ended Dec. 31, net \"\n", + " \"income rose 7.8% to $3.26 billion, or 81 cents a share, compared with $3.02 billion, or \"\n", + " \"70 cents a share, a year earlier. Revenue increased 2.3% compared with the prior-year period. \"\n", + " \"Adjusted earnings before interest, taxes, depreciation and amortization (EBITDA) was flat year \"\n", + " \"over year at about $8 billion. 'For the third consecutive year, we generated the highest revenue, \"\n", + " \"adjusted EBITDA and adjusted EPS in our company’s history', Comcast Chief Executive Officer Brian \"\n", + " \"Roberts said in a statement. 'We also reported the highest adjusted EBITDA on record at Theme Parks; \"\n", + " \"were the #1 studio in worldwide box office for the first time since 2015; and maintained Peacock’s \"\n", + " \"position as the fastest growing streamer in the U.S.'\"},\n", + "\n", + " {\"context\": \"Dollar General forecast annual sales above Wall Street estimates on Thursday, banking on higher \"\n", + " \"demand from inflation-hit customers buying groceries and essentials from the discount retailer’s stores. \"\n", + " \"Shares of the company rose about 6% in early trading, after falling nearly 45% in 2023 on rising costs \"\n", + " \"and stiff competition from bigger retailers. But higher prices and borrowing costs have prompted \"\n", + " \"budget-conscious consumers to cook more meals at home, helping Dollar General record stronger \"\n", + " \"footfall at its outlets as shoppers hunt for lower-margin, needs-based goods, over pricier general \"\n", + " \"merchandise. “With customer traffic growth and market share gains during the quarter, we believe our \"\n", + " \"actions are resonating with customers,” CEO Todd Vasos said in a statement. Vasos’s strategy - to focus \"\n", + " \"on the basics, like more employee presence at stores, greater customer engagement and expanding \"\n", + " \"private-label brands - has helped stabilize Dollar General’s business. Over the last few quarters, \"\n", + " \"Dollar General and rival Dollar Tree have struggled with rising costs linked to their supply \"\n", + " \"chains, labor and raw materials, while facing tough competition from retailers like Walmart \"\n", + " \"and Chinese ecommerce platform Temu. Dollar Tree’s shares fell more than 15% on Wednesday, after it \"\n", + " \"forecast weak sales and profit for 2024 and laid out plans to shutter 970 of its Family Dollar \"\n", + " \"stores. “Dollar General has a much rosier outlook than Dollar Tree... Dollar Tree’s challenges \"\n", + " \"with Family Dollar were years in the making, while Dollar General has embarked on an aggressive \"\n", + " \"effort to add more frozen, refrigerated and fresh produce,” eMarketer senior analyst Zak Stambor said. \"\n", + " \"Dollar General forecast 2024 sales to grow between 6.0% and 6.7%, above analysts’ estimate of 4.4% \"\n", + " \"growth to $40.33 billion, according to LSEG data. It still sees annual per-share profit between \"\n", + " \"$6.80 and $7.55, compared with estimates of $7.55. Its fourth-quarter net sales of $9.86 billion \"\n", + " \"surpassed estimates of $9.78 billion. It also reported an estimate-beating profit of $1.83 per share.\"},\n", + "\n", + " {\"context\": \"Best Buy surpassed Wall Street’s revenue and earnings expectations for the holiday quarter on \"\n", + " \"Thursday, even as the company navigated through a period of tepid consumer electronics demand. \"\n", + " \"But the retailer warned of another year of softer sales and said it would lay off workers and \"\n", + " \"cut other costs across the business. CEO Corie Barry offered few specifics, but said the \"\n", + " \"company has to make sure its workforce and stores match customers’ changing shopping habits. \"\n", + " \"Cuts will free up capital to invest back into the business and in newer areas, such as artificial \"\n", + " \"intelligence, she added. “This is giving us some of that space to be able to reinvest into \"\n", + " \"our future and make sure we feel like we are really well positioned for the industry to \"\n", + " \"start to rebound,” she said on a call with reporters. For this fiscal year, Best Buy anticipates \"\n", + " \"revenue will range from $41.3 billion to $42.6 billion. That would mark a drop from the most \"\n", + " \"recently ended fiscal year, when full-year revenue totaled $43.45 billion. It said comparable \"\n", + " \"sales will range from flat to a 3% decline. The retailer plans to close 10 to 15 stores \"\n", + " \"this year after shuttering 24 in the past fiscal year. One challenge that will affect sales \"\n", + " \"in the year ahead: it is a week shorter. Best Buy said the extra week in the past fiscal \"\n", + " \"year lifted revenue by about $735 million and boosted diluted earnings per share by about \"\n", + " \"30 cents. Shares of Best Buy closed more than 1% higher Thursday after briefly touching \"\n", + " \"a 52-week high of $86.11 earlier in the session. Here’s what the consumer electronics \"\n", + " \"retailer reported for its fiscal fourth quarter of 2024 compared with what Wall Street was \"\n", + " \"expecting, based on a survey of analysts by LSEG, formerly known as Refinitiv: \"\n", + " \"Earnings per share: $2.72, adjusted vs. $2.52 expected Revenue: $14.65 billion vs. $14.56 \"\n", + " \"billion expected A dip in demand, but a better-than-feared holiday Best Buy has dealt \"\n", + " \"with slower demand in part due to the strength of its sales during the pandemic. Like \"\n", + " \"home improvement companies, Best Buy saw outsized spending as shoppers were stuck at \"\n", + " \"home. Plus, many items that the retailer sells like laptops, refrigerators and home \"\n", + " \"theater systems tend to be pricier and less frequent purchases. The retailer has cited other \"\n", + " \"challenges, too: Shoppers have been choosier about making big purchases while dealing \"\n", + " \"with inflation-driven higher prices of food and more. Plus, they’ve returned to \"\n", + " \"splitting their dollars between services and goods after pandemic years of little \"\n", + " \"activity. Even so, Best Buy put up a holiday quarter that was better than feared. \"\n", + " \"In the three-month period that ended Feb. 3, the company’s net income fell by 7% to \"\n", + " \"$460 million, or $2.12 per share, from $495 million, or $2.23 per share in the year-ago \"\n", + " \"period. Revenue dropped from $14.74 billion a year earlier. Comparable sales, a metric that \"\n", + " \"includes sales online and at stores open at least 14 months, declined 4.8% during the \"\n", + " \"quarter as shoppers bought fewer appliances, mobile phones, tablets and home theater \"\n", + " \"setups than the year-ago period. Gaming, on the other hand, was a strong sales \"\n", + " \"category in the holiday quarter.\"}\n", + "\n", + "]" + ], + "metadata": { + "id": "I49eI2UI8RaX" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "# Available Slim Extract Models\n", + "slim_extract_models = [\"slim-extract-tool\", # original - stablelm-3b (2.7b)\n", + " \"slim-extract-tiny-tool\", # tiny-llama 1.1b\n", + " \"slim-extract-qwen-1.5b-gguf\", # **NEW** qwen 1.5b\n", + " \"slim-extract-phi-3-gguf\", # **NEW** phi-3 (3.8b)\n", + " \"slim-extract-qwen-0.5b-gguf\"] # **NEW** qwen 0.5b" + ], + "metadata": { + "id": "wbtHQKMP8rDY" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "# Load the Model\n", + "model = ModelCatalog().load_model(\"slim-extract-tool\",sample=False,temperature=0.0, max_output=100) # You can change the model from any of the above one and test it yourself." + ], + "metadata": { + "id": "5STiKpjk83_s" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "# iterate through the earnings release samples above\n", + "for i, sample in enumerate(earnings_releases):\n", + "\n", + " # key line: execute function_call on selected model with 'custom_key' = \"revenue growth\"\n", + " response = model.function_call(sample[\"context\"], function=\"extract\", params=[\"revenue growth\"])\n", + "\n", + " # display the response on the screen\n", + " print(\"extract response: \", i, response[\"llm_response\"])" + ], + "metadata": { + "id": "SW_J08CB88LC" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "# Final Notes 🐛\n", + "SLIM or Structured Language Instruction Models are powerful small models and have a great use case.\n", + "\n", + "In this fast start series, we experimented with the SLIM-EXTRACT MODEL that has a great use case in extracting custom keys from selected text.\n", + "\n", + "The available SLIM EXTRACT MODELS are:\n", + "- slim-extract-tool [original - stablelm-3b (2.7b)]\n", + "- slim-extract-tiny-tool [tiny-llama 1.1b]\n", + "- slim-extract-qwen-1.5b-gguf [**NEW** qwen 1.5b]\n", + "- slim-extract-phi-3-gguf [**NEW** phi-3 (3.8b)]\n", + "- slim-extract-qwen-0.5b-gguf [**NEW** qwen 0.5b]\n", + "\n", + "HomeWork:\n", + "Test all these on your own with your cutsom test cases!" + ], + "metadata": { + "id": "lXq7-toh9jwR" + } + }, + { + "cell_type": "markdown", + "source": [ + "## Journey is yet to start!\n", + "Loved it?? This is just an example of our models. Please check out our other Agentic AI examples with every model in detail here: https://github.com/llmware-ai/llmware/tree/main/fast_start/agents\n", + "\n", + "Also, if you have more interest in RAG, then please go with our RAG examples, which you can find here: https://github.com/llmware-ai/llmware/tree/main/fast_start/rag\n", + "\n", + "If you liked it, then please **star our repo https://github.com/llmware-ai/llmware** ⭐\n", + "\n", + "Any doubts?? Join our **discord server: https://discord.gg/GN49aWx2H3** 🫂" + ], + "metadata": { + "id": "HoVSbPiX9bwG" + } + } + ], + "metadata": { + "colab": { + "gpuType": "T4", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file