AI-Travel-Agent/AI_Travel_Agent.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b9b2a01c-1c1d-4b21-b03b-0f33d4412053",
   "metadata": {},
   "source": [
    "# AI Travel Agent"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18b167f6",
   "metadata": {},
   "source": [
    "This notebook showcases on deploying local LLM agents using the Langchain tools on Intel® Core™ Ultra Processors. The aim is to deploy an Travel Agent on the iGPU (integrated GPU) of the AIPC. For this, Llamacpp GPU backend is setup and the agent created using the local LLM model. The agent makes use of langchain toolkits and tools for travel related queries."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c023e7b9",
   "metadata": {},
   "source": [
    "#### Table of Contents\n",
    "1. Initial setup and installations\n",
    "      - Set the secret keys\n",
    "      - Login to Huggingface and download the huggingface models\n",
    "      - Select Local LLM Model\n",
    "      - Initialize LlamaCpp Model\n",
    "2. Create the agent\n",
    "      - Langchain Tools\n",
    "      - Prompt Template\n",
    "      - Agent\n",
    "3. Run the agent\n",
    "      - Agent Executor\n",
    "      - Testing scenarios\n",
    "4. Deploying with Streamlit"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d12ec791",
   "metadata": {},
   "source": [
    "### 1. Initial setup"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e140d37",
   "metadata": {},
   "source": [
    "#### Set the secret keys\n",
    "Load the secret API keys ([Amadeus toolkit](https://developers.amadeus.com/get-started/get-started-with-self-service-apis-335), [SerpAPI](https://serpapi.com/), [GoogleSearchAPIWrapper](https://serper.dev/)) from a `.env` file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7b22bc24-27fa-4a09-bc20-1a5ea554da66",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "\"\"\"\n",
    "Loading the secret API keys from a .env file into the environment.\n",
    "\"\"\"\n",
    "try:\n",
    "    load_dotenv()\n",
    "except Exception as e:\n",
    "    print(f\"An error occurred: {str(e)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22795f4a-a55a-4576-8174-b41eea6ad1be",
   "metadata": {},
   "source": [
    "#### Login to Huggingface and download the huggingface models\n",
    "This step is optional if you've already logged into the Huggingface and dowloaded the models in the terminal using huggingface-cli as outlined in the README.md."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "da69bede-41e6-4fc5-b45b-48ba6c017b11",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "051ada26bbab4c4a892e17048e2bcbaa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(HTML(value='<center> <img\\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.sv…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from huggingface_hub import login\n",
    "login()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8d47dfb0-dca0-46c0-9842-9f264a24e14b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'models\\\\Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from huggingface_hub import hf_hub_download\n",
    "\n",
    "hf_hub_download(\n",
    "    repo_id=\"bartowski/Meta-Llama-3.1-8B-Instruct-GGUF\",\n",
    "    filename=\"Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf\",\n",
    "    local_dir=\"./models\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b51b7e22-8e14-4501-987f-46898667dd8d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'models\\\\Qwen2.5-7B-Instruct-Q4_K_S.gguf'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hf_hub_download(\n",
    "    repo_id=\"bartowski/Qwen2.5-7B-Instruct-GGUF\",\n",
    "    filename=\"Qwen2.5-7B-Instruct-Q4_K_S.gguf\",\n",
    "    local_dir=\"./models\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "400750b2",
   "metadata": {},
   "source": [
    "#### Select Local LLM Model\n",
    "Select a Local Large language model from the dropdown list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c9ac0ce3-9a99-4c11-af51-3b920f8177dc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3f60489ff22b4fabb3474b6c8b23f346",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Dropdown(description='Model:', options=('Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf', 'Qwen2.5-7B-Instruct-Q4_K_S.…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import ipywidgets as widgets\n",
    "from IPython.display import display\n",
    "\n",
    "def find_gguf_files(directory):\n",
    "    \"\"\"\n",
    "    This function can be used to find the GGUF models present in the directory.\n",
    "    If the filename ends with .gguf then the new model name will be appended to gguf_files list.\n",
    "\n",
    "    Raises:\n",
    "\t\tException: If there is any error during finding the GGUF models, an error is displayed.\n",
    "    \n",
    "    \"\"\"\n",
    "    try:\n",
    "        gguf_files = []\n",
    "        for file in os.listdir(directory):\n",
    "            if file.endswith('.gguf'):\n",
    "                gguf_files.append(file)\n",
    "\n",
    "        return gguf_files\n",
    "            \n",
    "    except Exception as e:\n",
    "        print(f\"Error: Finding the GGUF models: {str(e)}\")\n",
    "\n",
    "gguf_files = find_gguf_files(\"./models\")\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "Download the models under `./models` folder to get the model name in the widgets dropdown options and for the model usage.\n",
    "Select a local LLM model from the dropdown list.\n",
    "If not selected explicitly from the dropdown list, as mentioned in the value Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf selected automatically. \n",
    "\n",
    "Raises:\n",
    "\t\tException: If there is any error during the model an error is displayed.\n",
    "\"\"\"\n",
    "\n",
    "if len(gguf_files) == 0:\n",
    "    print(f\"No GGUF model was found in this directory.\")\n",
    "\n",
    "if len(gguf_files):\n",
    "    try:\n",
    "        selected_model = widgets.Dropdown(\n",
    "            options=gguf_files,\n",
    "            value='Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf',\n",
    "            description='Model:',\n",
    "            disabled=False\n",
    "        )\n",
    "    \n",
    "        display(selected_model) \n",
    "    except Exception as e:\n",
    "        print(f\"Error: Model not selected:{str(e)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a74ad9f",
   "metadata": {},
   "source": [
    "#### Initialize LlamaCpp Model\n",
    "\n",
    "LlamaCpp is a high-performance C++ backend designed for efficient inference and deployment of LLM models. The python wrapper for this is Llamacpp-Python which integrates these optimizations into Python, allowing developers to deploy LLaMA models efficiently with enhanced language understanding and generation capabilities.\n",
    "\n",
    "**Note**: Please make sure that [LlamaCpp installation process](./README.md#setting-up-environment-and-llamacpp-python-gpu-backend) is completed before proceeding to the next step as outlined in the README.md.\n",
    "\n",
    "#### Setting up environment and LlamaCPP-python GPU backend\n",
    "\n",
    "Open a new terminal as administrator (right-click the terminal icon and select 'Run as administrator') and perform the following steps:\n",
    "\n",
    "1. **Create and activate the conda environment**\\\n",
    "    `conda create -n gpu_llmsycl python=3.11 -y`\\\n",
    "    `conda activate gpu_llmsycl`\n",
    "   \n",
    "2. **Initialize oneAPI environment**\\\n",
    "   *On Windows:*\\\n",
    "     `@call \"C:\\Program Files (x86)\\Intel\\oneAPI\\setvars.bat\" intel64 --force`\\\n",
    "   *On Linux:*\\\n",
    "     `source /opt/intel/oneapi/setvars.sh --force`\n",
    "\n",
    "3. **Set the environment variables and install Llamacpp-Python bindings**\\\n",
    "   *On Windows:*\\\n",
    "   `set CMAKE_GENERATOR=Ninja`\\\n",
    "   `set CMAKE_C_COMPILER=cl`\\\n",
    "   `set CMAKE_CXX_COMPILER=icx`\\\n",
    "   `set CXX=icx`\\\n",
    "   `set CC=cl`\\\n",
    "   `set CMAKE_ARGS=\"-DGGML_SYCL=ON -DGGML_SYCL_F16=ON -DCMAKE_CXX_COMPILER=icx -DCMAKE_C_COMPILER=cl\"`\\\n",
    "   `pip install llama-cpp-python==0.3.8 -U --force --no-cache-dir --verbose`\\\n",
    "   *On Linux:*\\\n",
    "   `CMAKE_ARGS=\"-DGGML_SYCL=on -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx\" pip install llama-cpp-python==0.3.8 -U --force --no-cache-dir --verbose`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "98b28742-6799-481b-a3d4-7963164ddc5b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_model_load_from_file_impl: using device SYCL0 (Intel(R) Graphics) - 16838 MiB free\n",
      "llama_model_loader: loaded meta data with 33 key-value pairs and 292 tensors from models/Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf (version GGUF V3 (latest))\n",
      "llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.\n",
      "llama_model_loader: - kv   0:                       general.architecture str              = llama\n",
      "llama_model_loader: - kv   1:                               general.type str              = model\n",
      "llama_model_loader: - kv   2:                               general.name str              = Meta Llama 3.1 8B Instruct\n",
      "llama_model_loader: - kv   3:                           general.finetune str              = Instruct\n",
      "llama_model_loader: - kv   4:                           general.basename str              = Meta-Llama-3.1\n",
      "llama_model_loader: - kv   5:                         general.size_label str              = 8B\n",
      "llama_model_loader: - kv   6:                            general.license str              = llama3.1\n",
      "llama_model_loader: - kv   7:                               general.tags arr[str,6]       = [\"facebook\", \"meta\", \"pytorch\", \"llam...\n",
      "llama_model_loader: - kv   8:                          general.languages arr[str,8]       = [\"en\", \"de\", \"fr\", \"it\", \"pt\", \"hi\", ...\n",
      "llama_model_loader: - kv   9:                          llama.block_count u32              = 32\n",
      "llama_model_loader: - kv  10:                       llama.context_length u32              = 131072\n",
      "llama_model_loader: - kv  11:                     llama.embedding_length u32              = 4096\n",
      "llama_model_loader: - kv  12:                  llama.feed_forward_length u32              = 14336\n",
      "llama_model_loader: - kv  13:                 llama.attention.head_count u32              = 32\n",
      "llama_model_loader: - kv  14:              llama.attention.head_count_kv u32              = 8\n",
      "llama_model_loader: - kv  15:                       llama.rope.freq_base f32              = 500000.000000\n",
      "llama_model_loader: - kv  16:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010\n",
      "llama_model_loader: - kv  17:                          general.file_type u32              = 14\n",
      "llama_model_loader: - kv  18:                           llama.vocab_size u32              = 128256\n",
      "llama_model_loader: - kv  19:                 llama.rope.dimension_count u32              = 128\n",
      "llama_model_loader: - kv  20:                       tokenizer.ggml.model str              = gpt2\n",
      "llama_model_loader: - kv  21:                         tokenizer.ggml.pre str              = llama-bpe\n",
      "llama_model_loader: - kv  22:                      tokenizer.ggml.tokens arr[str,128256]  = [\"!\", \"\\\"\", \"#\", \"$\", \"%\", \"&\", \"'\", ...\n",
      "llama_model_loader: - kv  23:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...\n",
      "llama_model_loader: - kv  24:                      tokenizer.ggml.merges arr[str,280147]  = [\"Ġ Ġ\", \"Ġ ĠĠĠ\", \"ĠĠ ĠĠ\", \"...\n",
      "llama_model_loader: - kv  25:                tokenizer.ggml.bos_token_id u32              = 128000\n",
      "llama_model_loader: - kv  26:                tokenizer.ggml.eos_token_id u32              = 128009\n",
      "llama_model_loader: - kv  27:                    tokenizer.chat_template str              = {{- bos_token }}\\n{%- if custom_tools ...\n",
      "llama_model_loader: - kv  28:               general.quantization_version u32              = 2\n",
      "llama_model_loader: - kv  29:                      quantize.imatrix.file str              = /models_out/Meta-Llama-3.1-8B-Instruc...\n",
      "llama_model_loader: - kv  30:                   quantize.imatrix.dataset str              = /training_dir/calibration_datav3.txt\n",
      "llama_model_loader: - kv  31:             quantize.imatrix.entries_count i32              = 224\n",
      "llama_model_loader: - kv  32:              quantize.imatrix.chunks_count i32              = 125\n",
      "llama_model_loader: - type  f32:   66 tensors\n",
      "llama_model_loader: - type q4_K:  217 tensors\n",
      "llama_model_loader: - type q5_K:    8 tensors\n",
      "llama_model_loader: - type q6_K:    1 tensors\n",
      "print_info: file format = GGUF V3 (latest)\n",
      "print_info: file type   = Q4_K - Small\n",
      "print_info: file size   = 4.36 GiB (4.67 BPW) \n",
      "init_tokenizer: initializing tokenizer for type 2\n",
      "load: control token: 128098 '<|reserved_special_token_90|>' is not marked as EOG\n",
      "load: control token: 128191 '<|reserved_special_token_183|>' is not marked as EOG\n",
      "load: control token: 128130 '<|reserved_special_token_122|>' is not marked as EOG\n",
      "load: control token: 128119 '<|reserved_special_token_111|>' is not marked as EOG\n",
      "load: control token: 128136 '<|reserved_special_token_128|>' is not marked as EOG\n",
      "load: control token: 128155 '<|reserved_special_token_147|>' is not marked as EOG\n",
      "load: control token: 128196 '<|reserved_special_token_188|>' is not marked as EOG\n",
      "load: control token: 128101 '<|reserved_special_token_93|>' is not marked as EOG\n",
      "load: control token: 128138 '<|reserved_special_token_130|>' is not marked as EOG\n",
      "load: control token: 128181 '<|reserved_special_token_173|>' is not marked as EOG\n",
      "load: control token: 128034 '<|reserved_special_token_26|>' is not marked as EOG\n",
      "load: control token: 128209 '<|reserved_special_token_201|>' is not marked as EOG\n",
      "load: control token: 128031 '<|reserved_special_token_23|>' is not marked as EOG\n",
      "load: control token: 128050 '<|reserved_special_token_42|>' is not marked as EOG\n",
      "load: control token: 128244 '<|reserved_special_token_236|>' is not marked as EOG\n",
      "load: control token: 128148 '<|reserved_special_token_140|>' is not marked as EOG\n",
      "load: control token: 128198 '<|reserved_special_token_190|>' is not marked as EOG\n",
      "load: control token: 128229 '<|reserved_special_token_221|>' is not marked as EOG\n",
      "load: control token: 128165 '<|reserved_special_token_157|>' is not marked as EOG\n",
      "load: control token: 128246 '<|reserved_special_token_238|>' is not marked as EOG\n",
      "load: control token: 128017 '<|reserved_special_token_9|>' is not marked as EOG\n",
      "load: control token: 128216 '<|reserved_special_token_208|>' is not marked as EOG\n",
      "load: control token: 128161 '<|reserved_special_token_153|>' is not marked as EOG\n",
      "load: control token: 128224 '<|reserved_special_token_216|>' is not marked as EOG\n",
      "load: control token: 128082 '<|reserved_special_token_74|>' is not marked as EOG\n",
      "load: control token: 128004 '<|finetune_right_pad_id|>' is not marked as EOG\n",
      "load: control token: 128249 '<|reserved_special_token_241|>' is not marked as EOG\n",
      "load: control token: 128107 '<|reserved_special_token_99|>' is not marked as EOG\n",
      "load: control token: 128079 '<|reserved_special_token_71|>' is not marked as EOG\n",
      "load: control token: 128225 '<|reserved_special_token_217|>' is not marked as EOG\n",
      "load: control token: 128175 '<|reserved_special_token_167|>' is not marked as EOG\n",
      "load: control token: 128223 '<|reserved_special_token_215|>' is not marked as EOG\n",
      "load: control token: 128182 '<|reserved_special_token_174|>' is not marked as EOG\n",
      "load: control token: 128068 '<|reserved_special_token_60|>' is not marked as EOG\n",
      "load: control token: 128252 '<|reserved_special_token_244|>' is not marked as EOG\n",
      "load: control token: 128178 '<|reserved_special_token_170|>' is not marked as EOG\n",
      "load: control token: 128221 '<|reserved_special_token_213|>' is not marked as EOG\n",
      "load: control token: 128052 '<|reserved_special_token_44|>' is not marked as EOG\n",
      "load: control token: 128122 '<|reserved_special_token_114|>' is not marked as EOG\n",
      "load: control token: 128151 '<|reserved_special_token_143|>' is not marked as EOG\n",
      "load: control token: 128121 '<|reserved_special_token_113|>' is not marked as EOG\n",
      "load: control token: 128158 '<|reserved_special_token_150|>' is not marked as EOG\n",
      "load: control token: 128096 '<|reserved_special_token_88|>' is not marked as EOG\n",
      "load: control token: 128090 '<|reserved_special_token_82|>' is not marked as EOG\n",
      "load: control token: 128238 '<|reserved_special_token_230|>' is not marked as EOG\n",
      "load: control token: 128139 '<|reserved_special_token_131|>' is not marked as EOG\n",
      "load: control token: 128176 '<|reserved_special_token_168|>' is not marked as EOG\n",
      "load: control token: 128077 '<|reserved_special_token_69|>' is not marked as EOG\n",
      "load: control token: 128214 '<|reserved_special_token_206|>' is not marked as EOG\n",
      "load: control token: 128171 '<|reserved_special_token_163|>' is not marked as EOG\n",
      "load: control token: 128112 '<|reserved_special_token_104|>' is not marked as EOG\n",
      "load: control token: 128180 '<|reserved_special_token_172|>' is not marked as EOG\n",
      "load: control token: 128060 '<|reserved_special_token_52|>' is not marked as EOG\n",
      "load: control token: 128000 '<|begin_of_text|>' is not marked as EOG\n",
      "load: control token: 128152 '<|reserved_special_token_144|>' is not marked as EOG\n",
      "load: control token: 128116 '<|reserved_special_token_108|>' is not marked as EOG\n",
      "load: control token: 128072 '<|reserved_special_token_64|>' is not marked as EOG\n",
      "load: control token: 128059 '<|reserved_special_token_51|>' is not marked as EOG\n",
      "load: control token: 128094 '<|reserved_special_token_86|>' is not marked as EOG\n",
      "load: control token: 128187 '<|reserved_special_token_179|>' is not marked as EOG\n",
      "load: control token: 128103 '<|reserved_special_token_95|>' is not marked as EOG\n",
      "load: control token: 128127 '<|reserved_special_token_119|>' is not marked as EOG\n",
      "load: control token: 128023 '<|reserved_special_token_15|>' is not marked as EOG\n",
      "load: control token: 128037 '<|reserved_special_token_29|>' is not marked as EOG\n",
      "load: control token: 128228 '<|reserved_special_token_220|>' is not marked as EOG\n",
      "load: control token: 128002 '<|reserved_special_token_0|>' is not marked as EOG\n",
      "load: control token: 128006 '<|start_header_id|>' is not marked as EOG\n",
      "load: control token: 128091 '<|reserved_special_token_83|>' is not marked as EOG\n",
      "load: control token: 128044 '<|reserved_special_token_36|>' is not marked as EOG\n",
      "load: control token: 128218 '<|reserved_special_token_210|>' is not marked as EOG\n",
      "load: control token: 128211 '<|reserved_special_token_203|>' is not marked as EOG\n",
      "load: control token: 128073 '<|reserved_special_token_65|>' is not marked as EOG\n",
      "load: control token: 128168 '<|reserved_special_token_160|>' is not marked as EOG\n",
      "load: control token: 128183 '<|reserved_special_token_175|>' is not marked as EOG\n",
      "load: control token: 128234 '<|reserved_special_token_226|>' is not marked as EOG\n",
      "load: control token: 128235 '<|reserved_special_token_227|>' is not marked as EOG\n",
      "load: control token: 128067 '<|reserved_special_token_59|>' is not marked as EOG\n",
      "load: control token: 128039 '<|reserved_special_token_31|>' is not marked as EOG\n",
      "load: control token: 128106 '<|reserved_special_token_98|>' is not marked as EOG\n",
      "load: control token: 128250 '<|reserved_special_token_242|>' is not marked as EOG\n",
      "load: control token: 128173 '<|reserved_special_token_165|>' is not marked as EOG\n",
      "load: control token: 128126 '<|reserved_special_token_118|>' is not marked as EOG\n",
      "load: control token: 128047 '<|reserved_special_token_39|>' is not marked as EOG\n",
      "load: control token: 128240 '<|reserved_special_token_232|>' is not marked as EOG\n",
      "load: control token: 128045 '<|reserved_special_token_37|>' is not marked as EOG\n",
      "load: control token: 128195 '<|reserved_special_token_187|>' is not marked as EOG\n",
      "load: control token: 128078 '<|reserved_special_token_70|>' is not marked as EOG\n",
      "load: control token: 128137 '<|reserved_special_token_129|>' is not marked as EOG\n",
      "load: control token: 128186 '<|reserved_special_token_178|>' is not marked as EOG\n",
      "load: control token: 128048 '<|reserved_special_token_40|>' is not marked as EOG\n",
      "load: control token: 128076 '<|reserved_special_token_68|>' is not marked as EOG\n",
      "load: control token: 128029 '<|reserved_special_token_21|>' is not marked as EOG\n",
      "load: control token: 128013 '<|reserved_special_token_5|>' is not marked as EOG\n",
      "load: control token: 128197 '<|reserved_special_token_189|>' is not marked as EOG\n",
      "load: control token: 128056 '<|reserved_special_token_48|>' is not marked as EOG\n",
      "load: control token: 128123 '<|reserved_special_token_115|>' is not marked as EOG\n",
      "load: control token: 128095 '<|reserved_special_token_87|>' is not marked as EOG\n",
      "load: control token: 128089 '<|reserved_special_token_81|>' is not marked as EOG\n",
      "load: control token: 128057 '<|reserved_special_token_49|>' is not marked as EOG\n",
      "load: control token: 128163 '<|reserved_special_token_155|>' is not marked as EOG\n",
      "load: control token: 128011 '<|reserved_special_token_3|>' is not marked as EOG\n",
      "load: control token: 128053 '<|reserved_special_token_45|>' is not marked as EOG\n",
      "load: control token: 128160 '<|reserved_special_token_152|>' is not marked as EOG\n",
      "load: control token: 128222 '<|reserved_special_token_214|>' is not marked as EOG\n",
      "load: control token: 128035 '<|reserved_special_token_27|>' is not marked as EOG\n",
      "load: control token: 128162 '<|reserved_special_token_154|>' is not marked as EOG\n",
      "load: control token: 128205 '<|reserved_special_token_197|>' is not marked as EOG\n",
      "load: control token: 128109 '<|reserved_special_token_101|>' is not marked as EOG\n",
      "load: control token: 128185 '<|reserved_special_token_177|>' is not marked as EOG\n",
      "load: control token: 128114 '<|reserved_special_token_106|>' is not marked as EOG\n",
      "load: control token: 128159 '<|reserved_special_token_151|>' is not marked as EOG\n",
      "load: control token: 128179 '<|reserved_special_token_171|>' is not marked as EOG\n",
      "load: control token: 128115 '<|reserved_special_token_107|>' is not marked as EOG\n",
      "load: control token: 128087 '<|reserved_special_token_79|>' is not marked as EOG\n",
      "load: control token: 128113 '<|reserved_special_token_105|>' is not marked as EOG\n",
      "load: control token: 128054 '<|reserved_special_token_46|>' is not marked as EOG\n",
      "load: control token: 128030 '<|reserved_special_token_22|>' is not marked as EOG\n",
      "load: control token: 128170 '<|reserved_special_token_162|>' is not marked as EOG\n",
      "load: control token: 128012 '<|reserved_special_token_4|>' is not marked as EOG\n",
      "load: control token: 128064 '<|reserved_special_token_56|>' is not marked as EOG\n",
      "load: control token: 128118 '<|reserved_special_token_110|>' is not marked as EOG\n",
      "load: control token: 128206 '<|reserved_special_token_198|>' is not marked as EOG\n",
      "load: control token: 128099 '<|reserved_special_token_91|>' is not marked as EOG\n",
      "load: control token: 128133 '<|reserved_special_token_125|>' is not marked as EOG\n",
      "load: control token: 128190 '<|reserved_special_token_182|>' is not marked as EOG\n",
      "load: control token: 128097 '<|reserved_special_token_89|>' is not marked as EOG\n",
      "load: control token: 128086 '<|reserved_special_token_78|>' is not marked as EOG\n",
      "load: control token: 128120 '<|reserved_special_token_112|>' is not marked as EOG\n",
      "load: control token: 128193 '<|reserved_special_token_185|>' is not marked as EOG\n",
      "load: control token: 128049 '<|reserved_special_token_41|>' is not marked as EOG\n",
      "load: control token: 128242 '<|reserved_special_token_234|>' is not marked as EOG\n",
      "load: control token: 128142 '<|reserved_special_token_134|>' is not marked as EOG\n",
      "load: control token: 128188 '<|reserved_special_token_180|>' is not marked as EOG\n",
      "load: control token: 128144 '<|reserved_special_token_136|>' is not marked as EOG\n",
      "load: control token: 128247 '<|reserved_special_token_239|>' is not marked as EOG\n",
      "load: control token: 128065 '<|reserved_special_token_57|>' is not marked as EOG\n",
      "load: control token: 128117 '<|reserved_special_token_109|>' is not marked as EOG\n",
      "load: control token: 128033 '<|reserved_special_token_25|>' is not marked as EOG\n",
      "load: control token: 128184 '<|reserved_special_token_176|>' is not marked as EOG\n",
      "load: control token: 128040 '<|reserved_special_token_32|>' is not marked as EOG\n",
      "load: control token: 128204 '<|reserved_special_token_196|>' is not marked as EOG\n",
      "load: control token: 128210 '<|reserved_special_token_202|>' is not marked as EOG\n",
      "load: control token: 128245 '<|reserved_special_token_237|>' is not marked as EOG\n",
      "load: control token: 128135 '<|reserved_special_token_127|>' is not marked as EOG\n",
      "load: control token: 128071 '<|reserved_special_token_63|>' is not marked as EOG\n",
      "load: control token: 128153 '<|reserved_special_token_145|>' is not marked as EOG\n",
      "load: control token: 128194 '<|reserved_special_token_186|>' is not marked as EOG\n",
      "load: control token: 128177 '<|reserved_special_token_169|>' is not marked as EOG\n",
      "load: control token: 128236 '<|reserved_special_token_228|>' is not marked as EOG\n",
      "load: control token: 128248 '<|reserved_special_token_240|>' is not marked as EOG\n",
      "load: control token: 128241 '<|reserved_special_token_233|>' is not marked as EOG\n",
      "load: control token: 128212 '<|reserved_special_token_204|>' is not marked as EOG\n",
      "load: control token: 128207 '<|reserved_special_token_199|>' is not marked as EOG\n",
      "load: control token: 128001 '<|end_of_text|>' is not marked as EOG\n",
      "load: control token: 128003 '<|reserved_special_token_1|>' is not marked as EOG\n",
      "load: control token: 128005 '<|reserved_special_token_2|>' is not marked as EOG\n",
      "load: control token: 128007 '<|end_header_id|>' is not marked as EOG\n",
      "load: control token: 128010 '<|python_tag|>' is not marked as EOG\n",
      "load: control token: 128014 '<|reserved_special_token_6|>' is not marked as EOG\n",
      "load: control token: 128015 '<|reserved_special_token_7|>' is not marked as EOG\n",
      "load: control token: 128016 '<|reserved_special_token_8|>' is not marked as EOG\n",
      "load: control token: 128018 '<|reserved_special_token_10|>' is not marked as EOG\n",
      "load: control token: 128019 '<|reserved_special_token_11|>' is not marked as EOG\n",
      "load: control token: 128020 '<|reserved_special_token_12|>' is not marked as EOG\n",
      "load: control token: 128021 '<|reserved_special_token_13|>' is not marked as EOG\n",
      "load: control token: 128022 '<|reserved_special_token_14|>' is not marked as EOG\n",
      "load: control token: 128024 '<|reserved_special_token_16|>' is not marked as EOG\n",
      "load: control token: 128025 '<|reserved_special_token_17|>' is not marked as EOG\n",
      "load: control token: 128026 '<|reserved_special_token_18|>' is not marked as EOG\n",
      "load: control token: 128027 '<|reserved_special_token_19|>' is not marked as EOG\n",
      "load: control token: 128028 '<|reserved_special_token_20|>' is not marked as EOG\n",
      "load: control token: 128032 '<|reserved_special_token_24|>' is not marked as EOG\n",
      "load: control token: 128036 '<|reserved_special_token_28|>' is not marked as EOG\n",
      "load: control token: 128038 '<|reserved_special_token_30|>' is not marked as EOG\n",
      "load: control token: 128041 '<|reserved_special_token_33|>' is not marked as EOG\n",
      "load: control token: 128042 '<|reserved_special_token_34|>' is not marked as EOG\n",
      "load: control token: 128043 '<|reserved_special_token_35|>' is not marked as EOG\n",
      "load: control token: 128046 '<|reserved_special_token_38|>' is not marked as EOG\n",
      "load: control token: 128051 '<|reserved_special_token_43|>' is not marked as EOG\n",
      "load: control token: 128055 '<|reserved_special_token_47|>' is not marked as EOG\n",
      "load: control token: 128058 '<|reserved_special_token_50|>' is not marked as EOG\n",
      "load: control token: 128061 '<|reserved_special_token_53|>' is not marked as EOG\n",
      "load: control token: 128062 '<|reserved_special_token_54|>' is not marked as EOG\n",
      "load: control token: 128063 '<|reserved_special_token_55|>' is not marked as EOG\n",
      "load: control token: 128066 '<|reserved_special_token_58|>' is not marked as EOG\n",
      "load: control token: 128069 '<|reserved_special_token_61|>' is not marked as EOG\n",
      "load: control token: 128070 '<|reserved_special_token_62|>' is not marked as EOG\n",
      "load: control token: 128074 '<|reserved_special_token_66|>' is not marked as EOG\n",
      "load: control token: 128075 '<|reserved_special_token_67|>' is not marked as EOG\n",
      "load: control token: 128080 '<|reserved_special_token_72|>' is not marked as EOG\n",
      "load: control token: 128081 '<|reserved_special_token_73|>' is not marked as EOG\n",
      "load: control token: 128083 '<|reserved_special_token_75|>' is not marked as EOG\n",
      "load: control token: 128084 '<|reserved_special_token_76|>' is not marked as EOG\n",
      "load: control token: 128085 '<|reserved_special_token_77|>' is not marked as EOG\n",
      "load: control token: 128088 '<|reserved_special_token_80|>' is not marked as EOG\n",
      "load: control token: 128092 '<|reserved_special_token_84|>' is not marked as EOG\n",
      "load: control token: 128093 '<|reserved_special_token_85|>' is not marked as EOG\n",
      "load: control token: 128100 '<|reserved_special_token_92|>' is not marked as EOG\n",
      "load: control token: 128102 '<|reserved_special_token_94|>' is not marked as EOG\n",
      "load: control token: 128104 '<|reserved_special_token_96|>' is not marked as EOG\n",
      "load: control token: 128105 '<|reserved_special_token_97|>' is not marked as EOG\n",
      "load: control token: 128108 '<|reserved_special_token_100|>' is not marked as EOG\n",
      "load: control token: 128110 '<|reserved_special_token_102|>' is not marked as EOG\n",
      "load: control token: 128111 '<|reserved_special_token_103|>' is not marked as EOG\n",
      "load: control token: 128124 '<|reserved_special_token_116|>' is not marked as EOG\n",
      "load: control token: 128125 '<|reserved_special_token_117|>' is not marked as EOG\n",
      "load: control token: 128128 '<|reserved_special_token_120|>' is not marked as EOG\n",
      "load: control token: 128129 '<|reserved_special_token_121|>' is not marked as EOG\n",
      "load: control token: 128131 '<|reserved_special_token_123|>' is not marked as EOG\n",
      "load: control token: 128132 '<|reserved_special_token_124|>' is not marked as EOG\n",
      "load: control token: 128134 '<|reserved_special_token_126|>' is not marked as EOG\n",
      "load: control token: 128140 '<|reserved_special_token_132|>' is not marked as EOG\n",
      "load: control token: 128141 '<|reserved_special_token_133|>' is not marked as EOG\n",
      "load: control token: 128143 '<|reserved_special_token_135|>' is not marked as EOG\n",
      "load: control token: 128145 '<|reserved_special_token_137|>' is not marked as EOG\n",
      "load: control token: 128146 '<|reserved_special_token_138|>' is not marked as EOG\n",
      "load: control token: 128147 '<|reserved_special_token_139|>' is not marked as EOG\n",
      "load: control token: 128149 '<|reserved_special_token_141|>' is not marked as EOG\n",
      "load: control token: 128150 '<|reserved_special_token_142|>' is not marked as EOG\n",
      "load: control token: 128154 '<|reserved_special_token_146|>' is not marked as EOG\n",
      "load: control token: 128156 '<|reserved_special_token_148|>' is not marked as EOG\n",
      "load: control token: 128157 '<|reserved_special_token_149|>' is not marked as EOG\n",
      "load: control token: 128164 '<|reserved_special_token_156|>' is not marked as EOG\n",
      "load: control token: 128166 '<|reserved_special_token_158|>' is not marked as EOG\n",
      "load: control token: 128167 '<|reserved_special_token_159|>' is not marked as EOG\n",
      "load: control token: 128169 '<|reserved_special_token_161|>' is not marked as EOG\n",
      "load: control token: 128172 '<|reserved_special_token_164|>' is not marked as EOG\n",
      "load: control token: 128174 '<|reserved_special_token_166|>' is not marked as EOG\n",
      "load: control token: 128189 '<|reserved_special_token_181|>' is not marked as EOG\n",
      "load: control token: 128192 '<|reserved_special_token_184|>' is not marked as EOG\n",
      "load: control token: 128199 '<|reserved_special_token_191|>' is not marked as EOG\n",
      "load: control token: 128200 '<|reserved_special_token_192|>' is not marked as EOG\n",
      "load: control token: 128201 '<|reserved_special_token_193|>' is not marked as EOG\n",
      "load: control token: 128202 '<|reserved_special_token_194|>' is not marked as EOG\n",
      "load: control token: 128203 '<|reserved_special_token_195|>' is not marked as EOG\n",
      "load: control token: 128208 '<|reserved_special_token_200|>' is not marked as EOG\n",
      "load: control token: 128213 '<|reserved_special_token_205|>' is not marked as EOG\n",
      "load: control token: 128215 '<|reserved_special_token_207|>' is not marked as EOG\n",
      "load: control token: 128217 '<|reserved_special_token_209|>' is not marked as EOG\n",
      "load: control token: 128219 '<|reserved_special_token_211|>' is not marked as EOG\n",
      "load: control token: 128220 '<|reserved_special_token_212|>' is not marked as EOG\n",
      "load: control token: 128226 '<|reserved_special_token_218|>' is not marked as EOG\n",
      "load: control token: 128227 '<|reserved_special_token_219|>' is not marked as EOG\n",
      "load: control token: 128230 '<|reserved_special_token_222|>' is not marked as EOG\n",
      "load: control token: 128231 '<|reserved_special_token_223|>' is not marked as EOG\n",
      "load: control token: 128232 '<|reserved_special_token_224|>' is not marked as EOG\n",
      "load: control token: 128233 '<|reserved_special_token_225|>' is not marked as EOG\n",
      "load: control token: 128237 '<|reserved_special_token_229|>' is not marked as EOG\n",
      "load: control token: 128239 '<|reserved_special_token_231|>' is not marked as EOG\n",
      "load: control token: 128243 '<|reserved_special_token_235|>' is not marked as EOG\n",
      "load: control token: 128251 '<|reserved_special_token_243|>' is not marked as EOG\n",
      "load: control token: 128253 '<|reserved_special_token_245|>' is not marked as EOG\n",
      "load: control token: 128254 '<|reserved_special_token_246|>' is not marked as EOG\n",
      "load: control token: 128255 '<|reserved_special_token_247|>' is not marked as EOG\n",
      "load: special tokens cache size = 256\n",
      "load: token to piece cache size = 0.7999 MB\n",
      "print_info: arch             = llama\n",
      "print_info: vocab_only       = 0\n",
      "print_info: n_ctx_train      = 131072\n",
      "print_info: n_embd           = 4096\n",
      "print_info: n_layer          = 32\n",
      "print_info: n_head           = 32\n",
      "print_info: n_head_kv        = 8\n",
      "print_info: n_rot            = 128\n",
      "print_info: n_swa            = 0\n",
      "print_info: n_embd_head_k    = 128\n",
      "print_info: n_embd_head_v    = 128\n",
      "print_info: n_gqa            = 4\n",
      "print_info: n_embd_k_gqa     = 1024\n",
      "print_info: n_embd_v_gqa     = 1024\n",
      "print_info: f_norm_eps       = 0.0e+00\n",
      "print_info: f_norm_rms_eps   = 1.0e-05\n",
      "print_info: f_clamp_kqv      = 0.0e+00\n",
      "print_info: f_max_alibi_bias = 0.0e+00\n",
      "print_info: f_logit_scale    = 0.0e+00\n",
      "print_info: f_attn_scale     = 0.0e+00\n",
      "print_info: n_ff             = 14336\n",
      "print_info: n_expert         = 0\n",
      "print_info: n_expert_used    = 0\n",
      "print_info: causal attn      = 1\n",
      "print_info: pooling type     = 0\n",
      "print_info: rope type        = 0\n",
      "print_info: rope scaling     = linear\n",
      "print_info: freq_base_train  = 500000.0\n",
      "print_info: freq_scale_train = 1\n",
      "print_info: n_ctx_orig_yarn  = 131072\n",
      "print_info: rope_finetuned   = unknown\n",
      "print_info: ssm_d_conv       = 0\n",
      "print_info: ssm_d_inner      = 0\n",
      "print_info: ssm_d_state      = 0\n",
      "print_info: ssm_dt_rank      = 0\n",
      "print_info: ssm_dt_b_c_rms   = 0\n",
      "print_info: model type       = 8B\n",
      "print_info: model params     = 8.03 B\n",
      "print_info: general.name     = Meta Llama 3.1 8B Instruct\n",
      "print_info: vocab type       = BPE\n",
      "print_info: n_vocab          = 128256\n",
      "print_info: n_merges         = 280147\n",
      "print_info: BOS token        = 128000 '<|begin_of_text|>'\n",
      "print_info: EOS token        = 128009 '<|eot_id|>'\n",
      "print_info: EOT token        = 128009 '<|eot_id|>'\n",
      "print_info: EOM token        = 128008 '<|eom_id|>'\n",
      "print_info: LF token         = 198 'Ċ'\n",
      "print_info: EOG token        = 128008 '<|eom_id|>'\n",
      "print_info: EOG token        = 128009 '<|eot_id|>'\n",
      "print_info: max token length = 256\n",
      "load_tensors: loading model tensors, this can take a while... (mmap = true)\n",
      "load_tensors: layer   0 assigned to device SYCL0\n",
      "load_tensors: layer   1 assigned to device SYCL0\n",
      "load_tensors: layer   2 assigned to device SYCL0\n",
      "load_tensors: layer   3 assigned to device SYCL0\n",
      "load_tensors: layer   4 assigned to device SYCL0\n",
      "load_tensors: layer   5 assigned to device SYCL0\n",
      "load_tensors: layer   6 assigned to device SYCL0\n",
      "load_tensors: layer   7 assigned to device SYCL0\n",
      "load_tensors: layer   8 assigned to device SYCL0\n",
      "load_tensors: layer   9 assigned to device SYCL0\n",
      "load_tensors: layer  10 assigned to device SYCL0\n",
      "load_tensors: layer  11 assigned to device SYCL0\n",
      "load_tensors: layer  12 assigned to device SYCL0\n",
      "load_tensors: layer  13 assigned to device SYCL0\n",
      "load_tensors: layer  14 assigned to device SYCL0\n",
      "load_tensors: layer  15 assigned to device SYCL0\n",
      "load_tensors: layer  16 assigned to device SYCL0\n",
      "load_tensors: layer  17 assigned to device SYCL0\n",
      "load_tensors: layer  18 assigned to device SYCL0\n",
      "load_tensors: layer  19 assigned to device SYCL0\n",
      "load_tensors: layer  20 assigned to device SYCL0\n",
      "load_tensors: layer  21 assigned to device SYCL0\n",
      "load_tensors: layer  22 assigned to device SYCL0\n",
      "load_tensors: layer  23 assigned to device SYCL0\n",
      "load_tensors: layer  24 assigned to device SYCL0\n",
      "load_tensors: layer  25 assigned to device SYCL0\n",
      "load_tensors: layer  26 assigned to device SYCL0\n",
      "load_tensors: layer  27 assigned to device SYCL0\n",
      "load_tensors: layer  28 assigned to device SYCL0\n",
      "load_tensors: layer  29 assigned to device SYCL0\n",
      "load_tensors: layer  30 assigned to device SYCL0\n",
      "load_tensors: layer  31 assigned to device SYCL0\n",
      "load_tensors: layer  32 assigned to device SYCL0\n",
      "load_tensors: tensor 'token_embd.weight' (q4_K) (and 0 others) cannot be used with preferred buffer type CPU_AARCH64, using CPU instead\n",
      "load_tensors: offloading 32 repeating layers to GPU\n",
      "load_tensors: offloading output layer to GPU\n",
      "load_tensors: offloaded 33/33 layers to GPU\n",
      "load_tensors:        SYCL0 model buffer size =  4185.99 MiB\n",
      "load_tensors:   CPU_Mapped model buffer size =   281.81 MiB\n",
      "......................................................................................\n",
      "llama_init_from_model: n_seq_max     = 1\n",
      "llama_init_from_model: n_ctx         = 4096\n",
      "llama_init_from_model: n_ctx_per_seq = 4096\n",
      "llama_init_from_model: n_batch       = 512\n",
      "llama_init_from_model: n_ubatch      = 512\n",
      "llama_init_from_model: flash_attn    = 0\n",
      "llama_init_from_model: freq_base     = 10000.0\n",
      "llama_init_from_model: freq_scale    = 1\n",
      "llama_init_from_model: n_ctx_per_seq (4096) < n_ctx_train (131072) -- the full capacity of the model will not be utilized\n",
      "Running with Environment Variables:\n",
      "  GGML_SYCL_DEBUG: 0\n",
      "  GGML_SYCL_DISABLE_OPT: 0\n",
      "Build with Macros:\n",
      "  GGML_SYCL_FORCE_MMQ: no\n",
      "  GGML_SYCL_F16: no\n",
      "Found 1 SYCL devices:\n",
      "|  |                   |                                       |       |Max    |        |Max  |Global |                     |\n",
      "|  |                   |                                       |       |compute|Max work|sub  |mem    |                     |\n",
      "|ID|        Device Type|                                   Name|Version|units  |group   |group|size   |       Driver version|\n",
      "|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|\n",
      "| 0| [level_zero:gpu:0]|                         Intel Graphics|  12.70|     64|    1024|   32| 17656M|            1.6.32413|\n",
      "SYCL Optimization Feature:\n",
      "|ID|        Device Type|Reorder|\n",
      "|--|-------------------|-------|\n",
      "| 0| [level_zero:gpu:0]|      Y|\n",
      "llama_kv_cache_init: kv_size = 4096, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 32, can_shift = 1\n",
      "llama_kv_cache_init: layer 0: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 1: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 2: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 3: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 4: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 5: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 6: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 7: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 8: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 9: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 10: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 11: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 12: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 13: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 14: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 15: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 16: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 17: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 18: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 19: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 20: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 21: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 22: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 23: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 24: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 25: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 26: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 27: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 28: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 29: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 30: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init: layer 31: n_embd_k_gqa = 1024, n_embd_v_gqa = 1024\n",
      "llama_kv_cache_init:      SYCL0 KV buffer size =   512.00 MiB\n",
      "llama_init_from_model: KV self size  =  512.00 MiB, K (f16):  256.00 MiB, V (f16):  256.00 MiB\n",
      "llama_init_from_model:  SYCL_Host  output buffer size =     0.49 MiB\n",
      "llama_init_from_model:      SYCL0 compute buffer size =   296.00 MiB\n",
      "llama_init_from_model:  SYCL_Host compute buffer size =    16.01 MiB\n",
      "llama_init_from_model: graph nodes  = 1030\n",
      "llama_init_from_model: graph splits = 2\n",
      "CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | LLAMAFILE = 1 | OPENMP = 1 | AARCH64_REPACK = 1 | \n",
      "Model metadata: {'general.name': 'Meta Llama 3.1 8B Instruct', 'general.architecture': 'llama', 'general.type': 'model', 'llama.block_count': '32', 'general.basename': 'Meta-Llama-3.1', 'general.finetune': 'Instruct', 'general.size_label': '8B', 'general.license': 'llama3.1', 'llama.context_length': '131072', 'llama.embedding_length': '4096', 'llama.feed_forward_length': '14336', 'llama.attention.head_count': '32', 'tokenizer.ggml.eos_token_id': '128009', 'general.file_type': '14', 'llama.attention.head_count_kv': '8', 'llama.rope.freq_base': '500000.000000', 'quantize.imatrix.entries_count': '224', 'llama.attention.layer_norm_rms_epsilon': '0.000010', 'llama.vocab_size': '128256', 'llama.rope.dimension_count': '128', 'tokenizer.ggml.model': 'gpt2', 'tokenizer.ggml.pre': 'llama-bpe', 'general.quantization_version': '2', 'tokenizer.ggml.bos_token_id': '128000', 'tokenizer.chat_template': '{{- bos_token }}\\n{%- if custom_tools is defined %}\\n    {%- set tools = custom_tools %}\\n{%- endif %}\\n{%- if not tools_in_user_message is defined %}\\n    {%- set tools_in_user_message = true %}\\n{%- endif %}\\n{%- if not date_string is defined %}\\n    {%- set date_string = \"26 Jul 2024\" %}\\n{%- endif %}\\n{%- if not tools is defined %}\\n    {%- set tools = none %}\\n{%- endif %}\\n\\n{#- This block extracts the system message, so we can slot it into the right place. #}\\n{%- if messages[0][\\'role\\'] == \\'system\\' %}\\n    {%- set system_message = messages[0][\\'content\\']|trim %}\\n    {%- set messages = messages[1:] %}\\n{%- else %}\\n    {%- set system_message = \"\" %}\\n{%- endif %}\\n\\n{#- System message + builtin tools #}\\n{{- \"<|start_header_id|>system<|end_header_id|>\\\\n\\\\n\" }}\\n{%- if builtin_tools is defined or tools is not none %}\\n    {{- \"Environment: ipython\\\\n\" }}\\n{%- endif %}\\n{%- if builtin_tools is defined %}\\n    {{- \"Tools: \" + builtin_tools | reject(\\'equalto\\', \\'code_interpreter\\') | join(\", \") + \"\\\\n\\\\n\"}}\\n{%- endif %}\\n{{- \"Cutting Knowledge Date: December 2023\\\\n\" }}\\n{{- \"Today Date: \" + date_string + \"\\\\n\\\\n\" }}\\n{%- if tools is not none and not tools_in_user_message %}\\n    {{- \"You have access to the following functions. To call a function, please respond with JSON for a function call.\" }}\\n    {{- \\'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.\\' }}\\n    {{- \"Do not use variables.\\\\n\\\\n\" }}\\n    {%- for t in tools %}\\n        {{- t | tojson(indent=4) }}\\n        {{- \"\\\\n\\\\n\" }}\\n    {%- endfor %}\\n{%- endif %}\\n{{- system_message }}\\n{{- \"<|eot_id|>\" }}\\n\\n{#- Custom tools are passed in a user message with some extra guidance #}\\n{%- if tools_in_user_message and not tools is none %}\\n    {#- Extract the first user message so we can plug it in here #}\\n    {%- if messages | length != 0 %}\\n        {%- set first_user_message = messages[0][\\'content\\']|trim %}\\n        {%- set messages = messages[1:] %}\\n    {%- else %}\\n        {{- raise_exception(\"Cannot put tools in the first user message when there\\'s no first user message!\") }}\\n{%- endif %}\\n    {{- \\'<|start_header_id|>user<|end_header_id|>\\\\n\\\\n\\' -}}\\n    {{- \"Given the following functions, please respond with a JSON for a function call \" }}\\n    {{- \"with its proper arguments that best answers the given prompt.\\\\n\\\\n\" }}\\n    {{- \\'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.\\' }}\\n    {{- \"Do not use variables.\\\\n\\\\n\" }}\\n    {%- for t in tools %}\\n        {{- t | tojson(indent=4) }}\\n        {{- \"\\\\n\\\\n\" }}\\n    {%- endfor %}\\n    {{- first_user_message + \"<|eot_id|>\"}}\\n{%- endif %}\\n\\n{%- for message in messages %}\\n    {%- if not (message.role == \\'ipython\\' or message.role == \\'tool\\' or \\'tool_calls\\' in message) %}\\n        {{- \\'<|start_header_id|>\\' + message[\\'role\\'] + \\'<|end_header_id|>\\\\n\\\\n\\'+ message[\\'content\\'] | trim + \\'<|eot_id|>\\' }}\\n    {%- elif \\'tool_calls\\' in message %}\\n        {%- if not message.tool_calls|length == 1 %}\\n            {{- raise_exception(\"This model only supports single tool-calls at once!\") }}\\n        {%- endif %}\\n        {%- set tool_call = message.tool_calls[0].function %}\\n        {%- if builtin_tools is defined and tool_call.name in builtin_tools %}\\n            {{- \\'<|start_header_id|>assistant<|end_header_id|>\\\\n\\\\n\\' -}}\\n            {{- \"<|python_tag|>\" + tool_call.name + \".call(\" }}\\n            {%- for arg_name, arg_val in tool_call.arguments | items %}\\n                {{- arg_name + \\'=\"\\' + arg_val + \\'\"\\' }}\\n                {%- if not loop.last %}\\n                    {{- \", \" }}\\n                {%- endif %}\\n                {%- endfor %}\\n            {{- \")\" }}\\n        {%- else  %}\\n            {{- \\'<|start_header_id|>assistant<|end_header_id|>\\\\n\\\\n\\' -}}\\n            {{- \\'{\"name\": \"\\' + tool_call.name + \\'\", \\' }}\\n            {{- \\'\"parameters\": \\' }}\\n            {{- tool_call.arguments | tojson }}\\n            {{- \"}\" }}\\n        {%- endif %}\\n        {%- if builtin_tools is defined %}\\n            {#- This means we\\'re in ipython mode #}\\n            {{- \"<|eom_id|>\" }}\\n        {%- else %}\\n            {{- \"<|eot_id|>\" }}\\n        {%- endif %}\\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\\n        {{- \"<|start_header_id|>ipython<|end_header_id|>\\\\n\\\\n\" }}\\n        {%- if message.content is mapping or message.content is iterable %}\\n            {{- message.content | tojson }}\\n        {%- else %}\\n            {{- message.content }}\\n        {%- endif %}\\n        {{- \"<|eot_id|>\" }}\\n    {%- endif %}\\n{%- endfor %}\\n{%- if add_generation_prompt %}\\n    {{- \\'<|start_header_id|>assistant<|end_header_id|>\\\\n\\\\n\\' }}\\n{%- endif %}\\n', 'quantize.imatrix.chunks_count': '125', 'quantize.imatrix.file': '/models_out/Meta-Llama-3.1-8B-Instruct-GGUF/Meta-Llama-3.1-8B-Instruct.imatrix', 'quantize.imatrix.dataset': '/training_dir/calibration_datav3.txt'}\n",
      "Available chat formats from metadata: chat_template.default\n",
      "Using gguf chat template: {{- bos_token }}\n",
      "{%- if custom_tools is defined %}\n",
      "    {%- set tools = custom_tools %}\n",
      "{%- endif %}\n",
      "{%- if not tools_in_user_message is defined %}\n",
      "    {%- set tools_in_user_message = true %}\n",
      "{%- endif %}\n",
      "{%- if not date_string is defined %}\n",
      "    {%- set date_string = \"26 Jul 2024\" %}\n",
      "{%- endif %}\n",
      "{%- if not tools is defined %}\n",
      "    {%- set tools = none %}\n",
      "{%- endif %}\n",
      "\n",
      "{#- This block extracts the system message, so we can slot it into the right place. #}\n",
      "{%- if messages[0]['role'] == 'system' %}\n",
      "    {%- set system_message = messages[0]['content']|trim %}\n",
      "    {%- set messages = messages[1:] %}\n",
      "{%- else %}\n",
      "    {%- set system_message = \"\" %}\n",
      "{%- endif %}\n",
      "\n",
      "{#- System message + builtin tools #}\n",
      "{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n",
      "{%- if builtin_tools is defined or tools is not none %}\n",
      "    {{- \"Environment: ipython\\n\" }}\n",
      "{%- endif %}\n",
      "{%- if builtin_tools is defined %}\n",
      "    {{- \"Tools: \" + builtin_tools | reject('equalto', 'code_interpreter') | join(\", \") + \"\\n\\n\"}}\n",
      "{%- endif %}\n",
      "{{- \"Cutting Knowledge Date: December 2023\\n\" }}\n",
      "{{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n",
      "{%- if tools is not none and not tools_in_user_message %}\n",
      "    {{- \"You have access to the following functions. To call a function, please respond with JSON for a function call.\" }}\n",
      "    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n",
      "    {{- \"Do not use variables.\\n\\n\" }}\n",
      "    {%- for t in tools %}\n",
      "        {{- t | tojson(indent=4) }}\n",
      "        {{- \"\\n\\n\" }}\n",
      "    {%- endfor %}\n",
      "{%- endif %}\n",
      "{{- system_message }}\n",
      "{{- \"<|eot_id|>\" }}\n",
      "\n",
      "{#- Custom tools are passed in a user message with some extra guidance #}\n",
      "{%- if tools_in_user_message and not tools is none %}\n",
      "    {#- Extract the first user message so we can plug it in here #}\n",
      "    {%- if messages | length != 0 %}\n",
      "        {%- set first_user_message = messages[0]['content']|trim %}\n",
      "        {%- set messages = messages[1:] %}\n",
      "    {%- else %}\n",
      "        {{- raise_exception(\"Cannot put tools in the first user message when there's no first user message!\") }}\n",
      "{%- endif %}\n",
      "    {{- '<|start_header_id|>user<|end_header_id|>\\n\\n' -}}\n",
      "    {{- \"Given the following functions, please respond with a JSON for a function call \" }}\n",
      "    {{- \"with its proper arguments that best answers the given prompt.\\n\\n\" }}\n",
      "    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n",
      "    {{- \"Do not use variables.\\n\\n\" }}\n",
      "    {%- for t in tools %}\n",
      "        {{- t | tojson(indent=4) }}\n",
      "        {{- \"\\n\\n\" }}\n",
      "    {%- endfor %}\n",
      "    {{- first_user_message + \"<|eot_id|>\"}}\n",
      "{%- endif %}\n",
      "\n",
      "{%- for message in messages %}\n",
      "    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n",
      "        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n",
      "    {%- elif 'tool_calls' in message %}\n",
      "        {%- if not message.tool_calls|length == 1 %}\n",
      "            {{- raise_exception(\"This model only supports single tool-calls at once!\") }}\n",
      "        {%- endif %}\n",
      "        {%- set tool_call = message.tool_calls[0].function %}\n",
      "        {%- if builtin_tools is defined and tool_call.name in builtin_tools %}\n",
      "            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n",
      "            {{- \"<|python_tag|>\" + tool_call.name + \".call(\" }}\n",
      "            {%- for arg_name, arg_val in tool_call.arguments | items %}\n",
      "                {{- arg_name + '=\"' + arg_val + '\"' }}\n",
      "                {%- if not loop.last %}\n",
      "                    {{- \", \" }}\n",
      "                {%- endif %}\n",
      "                {%- endfor %}\n",
      "            {{- \")\" }}\n",
      "        {%- else  %}\n",
      "            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n",
      "            {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n",
      "            {{- '\"parameters\": ' }}\n",
      "            {{- tool_call.arguments | tojson }}\n",
      "            {{- \"}\" }}\n",
      "        {%- endif %}\n",
      "        {%- if builtin_tools is defined %}\n",
      "            {#- This means we're in ipython mode #}\n",
      "            {{- \"<|eom_id|>\" }}\n",
      "        {%- else %}\n",
      "            {{- \"<|eot_id|>\" }}\n",
      "        {%- endif %}\n",
      "    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n",
      "        {{- \"<|start_header_id|>ipython<|end_header_id|>\\n\\n\" }}\n",
      "        {%- if message.content is mapping or message.content is iterable %}\n",
      "            {{- message.content | tojson }}\n",
      "        {%- else %}\n",
      "            {{- message.content }}\n",
      "        {%- endif %}\n",
      "        {{- \"<|eot_id|>\" }}\n",
      "    {%- endif %}\n",
      "{%- endfor %}\n",
      "{%- if add_generation_prompt %}\n",
      "    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n",
      "{%- endif %}\n",
      "\n",
      "Using chat eos_token: <|eot_id|>\n",
      "Using chat bos_token: <|begin_of_text|>\n"
     ]
    }
   ],
   "source": [
    "\"\"\" \n",
    "Below shows how to load a local LLM using Llamacpp-python GPU backend for SYCL.\n",
    "\"\"\"\n",
    "\n",
    "from langchain_community.llms import LlamaCpp\n",
    "from langchain_core.callbacks import CallbackManager, StreamingStdOutCallbackHandler\n",
    "\n",
    "\"\"\"\n",
    "Download and copy the models under `./models` folder. Create and initialize the LlamaCpp with the selected model. Model and hyperparameters can be changed based on the end user requirements. \n",
    "Here we are using Meta Llama 3.1(Q4_K_S) model which is configured using some hyperparameters, such as GPU Layers to be offloaded on all the layers for GPU-accelerated inference, Context Length of 4096 tokens.\n",
    "Temperature set as 0 for deterministic output, Top-P Sampling as 0.95 for controlled randomness and Batch Size as 512 for parallel processing\n",
    "\n",
    "Raises:\n",
    "    Exception: If there is any error during model loading an error is displayed. \n",
    "\"\"\"\n",
    "try:\n",
    "    callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])\n",
    "    llm = LlamaCpp(\n",
    "        model_path=\"models/\" + selected_model.value,   # Path to the Llama model file\n",
    "        n_gpu_layers=-1,                               # Number of layers to be loaded into gpu memory (default: 0)\n",
    "        seed=512,                                      # Random number generator (RNG) seed (default: -1, -1 = random seed)\n",
    "        n_ctx=4096,                                    # Token context window (default: 512)\n",
    "        f16_kv=True,                                   # Use half-precision for key/value cache (default: True)\n",
    "        callback_manager=callback_manager,             # Pass the callback manager for output handling\n",
    "        verbose=True,                                  # Print verbose output (default: True)\n",
    "        temperature=0,                                 # Temperature controls the randomness of generated text during sampling (default: 0.8)\n",
    "        top_p=0.95,                                    # Top-p sampling picks the next token from top choices with a combined probability ≥ p (default: 0.95)\n",
    "        n_batch=512,                                   # Number of tokens to process in parallel (default: 8)\n",
    "    )\n",
    "    \n",
    "    # llm.client.verbose = False                       # Print verbose state information (default: True). Uncomment to Disable the verbose client output here\n",
    "except Exception as e:\n",
    "    print(f\"Model loading error: {str(e)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e37f64ca",
   "metadata": {},
   "source": [
    "### 2. Create the agent"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcef1e11",
   "metadata": {},
   "source": [
    "#### Langchain Tools\n",
    "We use [**GoogleSerperAPIWrapper**](https://python.langchain.com/docs/integrations/tools/google_serper/), [**serpapi**](https://python.langchain.com/docs/integrations/providers/serpapi/), [**Amadeus Toolkit**](https://python.langchain.com/docs/integrations/tools/amadeus/) tools for the agent to access for answering user queries.\n",
    "Please click on the below links to get more information on [*langchain tools*](https://python.langchain.com/docs/concepts/tools/):\\\n",
    "[**Langchain - GoogleSerperAPIWrapper**](https://python.langchain.com/api_reference/community/utilities/langchain_community.utilities.google_serper.GoogleSerperAPIWrapper.html)\\\n",
    "[**Langchain - serpapi**](https://python.langchain.com/api_reference/community/utilities/langchain_community.utilities.serpapi.SerpAPIWrapper.html)\\\n",
    "[**Langchain - Amadeus Toolkit**](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.amadeus.toolkit.AmadeusToolkit.html)\\\n",
    "These tools setup allows us to perform web searches and retrieve flight-related data efficiently."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d107c404-a288-4c91-a306-0e594015914b",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "Using Langchain GoogleSerperAPIWrapper Tool which queries the Google Search API and returns result.\n",
    "\n",
    "Raises:\n",
    "    Exception: If there is any error during the loading of the GoogleSerperAPIWrapper tool, an error is displayed.\n",
    "\"\"\"\n",
    "from langchain_community.utilities import GoogleSerperAPIWrapper\n",
    "from langchain_core.tools import Tool\n",
    "\n",
    "try:\n",
    "    search = GoogleSerperAPIWrapper()       # Initialize the search wrapper to perform Google searches\n",
    "    google_search_tool = Tool(\n",
    "        name=\"Google Search tool\",\n",
    "        func=search.run,\n",
    "        description=\"useful for when you need to ask with search\",\n",
    "    )\n",
    "except Exception as e:\n",
    "    print(f\"Error loading GoogleSerperAPIWrapper tool: {str(e)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "18687186-93b4-4662-b748-58e5b951de17",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "Using langchain Amadeus toolkit for fetching the flight related information.\n",
    "\"\"\"\n",
    "from amadeus import Client\n",
    "from langchain_community.agent_toolkits.amadeus.toolkit import AmadeusToolkit\n",
    "from langchain.tools.amadeus.closest_airport import AmadeusClosestAirport\n",
    "from langchain.tools.amadeus.flight_search import AmadeusFlightSearch\n",
    "\n",
    "\"\"\"\n",
    "Retrieving Amadeus API credentials from environment variables.\n",
    "Raises:\n",
    "    Exception: If there is any error during the loading of the Amadeus toolkit, an error is displayed.\n",
    "    The error may raise if the API keys are not defined in the environment and if not defining the Amadeus toolkit properly.\n",
    "\"\"\"\n",
    "\n",
    "try:\n",
    "    amadeus_client_secret = os.getenv(\"AMADEUS_CLIENT_SECRET\")\n",
    "    amadeus_client_id = os.getenv(\"AMADEUS_CLIENT_ID\")\n",
    "    \n",
    "    \"\"\"\n",
    "    Initialising the Amadeus client and toolkit here.\n",
    "    \"\"\"\n",
    "    amadeus = Client(client_id=amadeus_client_id, client_secret=amadeus_client_secret)\n",
    "    amadeus_toolkit = AmadeusToolkit(client=amadeus, llm=llm)\n",
    "    \n",
    "except Exception as e:\n",
    "    print(f\"Error: Invalid API keys of Amadeus :{str(e)}\")\n",
    "\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "Rebuilding the models for the Amadeus toolkit components here.\n",
    "Raises:\n",
    "    Exception: If there is any error during the rebuild of the Amadeus toolkit, an error is displayed.\n",
    "\"\"\"\n",
    "\n",
    "try:\n",
    "    AmadeusToolkit.model_rebuild()\n",
    "    AmadeusClosestAirport.model_rebuild()\n",
    "    AmadeusFlightSearch.model_rebuild()\n",
    "except Exception as e:\n",
    "    print(f\"Error: Amadeus toolkit Model rebuild failed: {str(e)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e6be1dc8-1b1e-4d69-a90e-acb9ebe5f1a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "Combining the tools for the agent.\n",
    "Adding Langchain SerpApi tool a real-time API to access Google search results.\n",
    "\n",
    "Raises:\n",
    "    Exception: If there is any error during the loading all the tools, an error is displayed.\n",
    "\"\"\"\n",
    "from langchain.agents import load_tools\n",
    "\n",
    "try:\n",
    "    tools = [google_search_tool] + amadeus_toolkit.get_tools() + load_tools([\"serpapi\"])\n",
    "except Exception as e:\n",
    "    print(f\"Error loading the tools: {str(e)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebe3b991",
   "metadata": {},
   "source": [
    "#### Prompt template"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "234018d1-98c4-492f-b4fb-3f76c1a3c02c",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "The following Prompt template is for the Structured chat agent and is customised to handle the travel related queries.\n",
    "\"\"\"\n",
    "\n",
    "PREFIX = \"\"\"[INST]Respond to the human as helpfully and accurately as possible. You have access to the following tools:\"\"\"\n",
    "\n",
    "FORMAT_INSTRUCTIONS = \"\"\"Use a json blob to specify a tool by providing an action key (tool name) and an action_input key (tool input).\n",
    "\n",
    "Use the closest_airport tool and single_flight_search tool for any flight related queries. Give all the flight details including Flight Number, Carrier, Departure time, Arrival time and Terminal details to the human.\n",
    "Use the Google Search tool and knowledge base for any itinerary-related queries. Give the day-wise itinerary to the human. Give all the detailed information on tourist attractions, must-visit places, and hotels with ratings to the human.\n",
    "Use the Google Search tool for distance calculations. Give all the web results to the human.\n",
    "Provide the complete Final Answer. Do not truncate the response.\n",
    "Always consider the traveler's preferences, budget constraints, and any specific requirements mentioned in their query.\n",
    "\n",
    "Valid \"action\" values: \"Final Answer\" or {tool_names}\n",
    "\n",
    "Provide only ONE action per $JSON_BLOB, as shown:\n",
    "\n",
    "```\n",
    "{{{{\n",
    "  \"action\": $TOOL_NAME,\n",
    "  \"action_input\": $INPUT\n",
    "}}}}\n",
    "```\n",
    "\n",
    "Follow this format:\n",
    "\n",
    "Question: input question to answer\n",
    "Thought: consider previous and subsequent steps\n",
    "Action:\n",
    "```\n",
    "$JSON_BLOB\n",
    "```\n",
    "Observation: action result\n",
    "... (repeat Thought/Action/Observation N times)\n",
    "Thought: I know what to respond\n",
    "Action:\n",
    "```\n",
    "{{{{\n",
    "  \"action\": \"Final Answer\",\n",
    "  \"action_input\": \"Provide the detailed Final Answer to the human\"\n",
    "}}}}\n",
    "```[/INST]\"\"\"\n",
    "\n",
    "SUFFIX = \"\"\"Begin! Reminder to ALWAYS respond with a valid json blob of a single action. Use tools if necessary. Respond directly if appropriate. Format is Action:```$JSON_BLOB```then Observation:.\n",
    "Thought:[INST]\"\"\"\n",
    "\n",
    "HUMAN_MESSAGE_TEMPLATE = \"{input}\\n\\n{agent_scratchpad}\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc98ed5d",
   "metadata": {},
   "source": [
    "#### Agent\n",
    "[**StructuredChatAgent**](https://api.python.langchain.com/en/latest/agents/langchain.agents.structured_chat.base.StructuredChatAgent.html): A specialized agent is capable of using multi-input tools and designed to handle structured conversations using the specified language model and tools.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "4d665fc9-0cf3-4212-97dd-445721df78ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "Creating and initialising a structured chat agent using the LLM and defined tools.\n",
    "\n",
    "    llm : LLM to be used\n",
    "    \n",
    "    tools : list\n",
    "        List of tools to use\n",
    "        \n",
    "    PREFIX : str\n",
    "        Prefix string prepended to the agent's input. \n",
    "        \n",
    "    SUFFIX : str\n",
    "        Suffix string appended to the agent's input. \n",
    "\n",
    "    HUMAN_MESSAGE_TEMPLATE : str\n",
    "        Template defining the structure of human messages.\n",
    "\n",
    "    FORMAT_INSTRUCTIONS : str\n",
    "        Format instructions for the agent\n",
    "\n",
    "    Raises:\n",
    "\t\tException: If there is any error during the agent creation, an error is displayed\n",
    "\n",
    "\"\"\"\n",
    "from langchain.agents import StructuredChatAgent\n",
    "\n",
    "try:\n",
    "    agent = StructuredChatAgent.from_llm_and_tools(\n",
    "        llm,                                           # LLM to use                            \n",
    "        tools,                                         # Tools available for the agent    \n",
    "        prefix=PREFIX,                                 # Prefix to prepend to the input\n",
    "        suffix=SUFFIX,                                 # Suffix to append to the input\n",
    "        human_message_template=HUMAN_MESSAGE_TEMPLATE, # Template for human messages\n",
    "        format_instructions=FORMAT_INSTRUCTIONS,       # Instructions for formatting responses\n",
    "    )\n",
    "except Exception as e:\n",
    "    print(f\"Error during agent creation :{str(e)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7a13f43",
   "metadata": {},
   "source": [
    "### 3. Run the agent"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "658c55f5-be41-4bc3-a2a7-229e0edb9165",
   "metadata": {},
   "source": [
    "#### Agent Executor\n",
    "\n",
    "[**AgentExecutor**](https://python.langchain.com/docs/how_to/agent_executor/): The agent executor is the runtime environment for an agent, facilitating the execution of actions and returning outputs for continuous processing.\\"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "24f8d007",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.agents import AgentExecutor\n",
    "\"\"\"\n",
    "Creating and configuring agent executor for managing interactions with the LLM model and available tools.\n",
    "    agent : structured chat agent to be used\n",
    "    \n",
    "    tools : list\n",
    "        List of tools to use by the agent\n",
    "        \n",
    "    verbose : bool\n",
    "        Used for detailed output\n",
    "        \n",
    "    handle_parsing_errors : bool\n",
    "        Handle the output parsing-related errors while generating the response\n",
    "        \n",
    "    max_iterations : int\n",
    "        Used to limit the number of agent iterations to prevent infinite loops. Here we are using 1 iteration, We can change based on the requirement.\n",
    "        \n",
    "    early_stopping_method : str\n",
    "        For stopping the agent execution early, we are using 'generate' here.\n",
    "        \n",
    "    Returns:\n",
    "        AgentExecutor instance for task execution.\n",
    "\n",
    "    Raises:\n",
    "        Exception: If there is any error during the agent executor's creation, an is displayed\n",
    "\n",
    "\"\"\"\n",
    "try:\n",
    "    agent_executor = AgentExecutor(\n",
    "        agent=agent,                     # The structured chat agent\n",
    "        tools=tools,                     # Tools to be used by the agent\n",
    "        verbose=True,                    # Enable verbose output for debugging\n",
    "        handle_parsing_errors=True,      # Allow error handling for parsing issues\n",
    "        max_iterations=1,                # Limit the number of iterations. Can change based on requirement\n",
    "        early_stopping_method='generate' # Method to use for agent early stopping\n",
    ")\n",
    "except Exception as e:\n",
    "    print(f\"Error during agent executor's creation :{str(e)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c509dd7",
   "metadata": {},
   "source": [
    "#### Testing scenarios"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f1d19848-d297-41a4-ad76-f1eda023a476",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "Thought: The human is asking for information about airlines that operate to London. I can use the Google Search tool to find this information.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Google Search tool\",\n",
      "  \"action_input\": \"airlines operating to London\"\n",
      "}\n",
      "```\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =   11479.93 ms /   958 tokens (   11.98 ms per token,    83.45 tokens per second)\n",
      "llama_perf_context_print:        eval time =   14994.18 ms /    58 runs   (  258.52 ms per token,     3.87 tokens per second)\n",
      "llama_perf_context_print:       total time =   26574.20 ms /  1016 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3mThought: The human is asking for information about airlines that operate to London. I can use the Google Search tool to find this information.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Google Search tool\",\n",
      "  \"action_input\": \"airlines operating to London\"\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mPopular Airlines Flying From United States to London ___Virgin Atlantic Airways. ___British Airways. ___JetBlue Airways. ___TAP Air Portugal.\u001b[0m\n",
      "Thought:"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 1012 prefix-match hit, remaining 49 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " the human asked about airlines operating to London, and I used the Google Search tool to find this information.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The major airlines that operate to London are Virgin Atlantic Airways, British Airways, JetBlue Airways, and TAP Air Portugal.\"\n",
      "}\n",
      "```[/INST] \n",
      "\n",
      "Final Answer: The final answer is The major airlines that operate to London are Virgin Atlantic Airways, British Airways, JetBlue Airways, and TAP Air Portugal..[/INST]  [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1761.36 ms /    49 tokens (   35.95 ms per token,    27.82 tokens per second)\n",
      "llama_perf_context_print:        eval time =   65850.65 ms /   255 runs   (  258.24 ms per token,     3.87 tokens per second)\n",
      "llama_perf_context_print:       total time =   68066.04 ms /   304 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3m the human asked about airlines operating to London, and I used the Google Search tool to find this information.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The major airlines that operate to London are Virgin Atlantic Airways, British Airways, JetBlue Airways, and TAP Air Portugal.\"\n",
      "}\n",
      "```[/INST] \n",
      "\n",
      "Final Answer: The final answer is The major airlines that operate to London are Virgin Atlantic Airways, British Airways, JetBlue Airways, and TAP Air Portugal..[/INST]  [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "The major airlines that operate to London are Virgin Atlantic Airways, British Airways, JetBlue Airways, and TAP Air Portugal.\n",
      "CPU times: total: 1min 34s\n",
      "Wall time: 1min 35s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    response = agent_executor.invoke({\"input\": \"What are the major airlines that operate to London?\"})\n",
    "    print(response['output'])\n",
    "except Exception as e:\n",
    "    print(f\"An error occurred :{str(e)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "57a046c5-071a-4835-b9aa-540b53906d5f",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 948 prefix-match hit, remaining 16 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Thought: The Lantern Festival is a popular event in Thailand, and it's usually held in January. However, I need to find the exact location of the festival in 2025.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Google Search tool\",\n",
      "  \"action_input\": \"Lantern Festival 2025 Thailand\"\n",
      "}\n",
      "```\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1605.40 ms /    16 tokens (  100.34 ms per token,     9.97 tokens per second)\n",
      "llama_perf_context_print:        eval time =   18320.85 ms /    71 runs   (  258.04 ms per token,     3.88 tokens per second)\n",
      "llama_perf_context_print:       total time =   20052.74 ms /    87 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3mThought: The Lantern Festival is a popular event in Thailand, and it's usually held in January. However, I need to find the exact location of the festival in 2025.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Google Search tool\",\n",
      "  \"action_input\": \"Lantern Festival 2025 Thailand\"\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mChiang Mai\u001b[0m\n",
      "Thought:"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 1031 prefix-match hit, remaining 25 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " finding the location of the Lantern Festival 2025 in Thailand.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The best place to see the Lantern Festival 2025 in Thailand is Chiang Mai.\"\n",
      "}\n",
      "```[/INST]\n",
      "\n",
      "Final Answer: The best place to see the Lantern Festival 2025 in Thailand is Chiang Mai. [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1512.40 ms /    25 tokens (   60.50 ms per token,    16.53 tokens per second)\n",
      "llama_perf_context_print:        eval time =   65222.96 ms /   255 runs   (  255.78 ms per token,     3.91 tokens per second)\n",
      "llama_perf_context_print:       total time =   67185.61 ms /   280 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3m finding the location of the Lantern Festival 2025 in Thailand.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The best place to see the Lantern Festival 2025 in Thailand is Chiang Mai.\"\n",
      "}\n",
      "```[/INST]\n",
      "\n",
      "Final Answer: The best place to see the Lantern Festival 2025 in Thailand is Chiang Mai. [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST]\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "The best place to see the Lantern Festival 2025 in Thailand is Chiang Mai.\n",
      "CPU times: total: 1min 27s\n",
      "Wall time: 1min 27s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    response = agent_executor.invoke({\"input\": \"Where is the best place to see the Lantern Festival 2025 in Thailand?\"})\n",
    "    print(response['output'])\n",
    "except Exception as e:\n",
    "    print(f\"An error occurred :{str(e)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "9b90be6a-a8e6-4453-a438-fdd2a0838975",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 948 prefix-match hit, remaining 24 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Thought: The human is asking for the cheapest flight details from Dubai to New York on 20th December 2025. I need to use the single_flight_search tool to find the cheapest flights.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"single_flight_search\",\n",
      "  \"action_input\": {\n",
      "    \"originLocationCode\": \"DXB\",\n",
      "    \"destinationLocationCode\": \"JFK\",\n",
      "    \"departureDateTimeEarliest\": \"2025-12-20T00:00:00\",\n",
      "    \"departureDateTimeLatest\": \"2025-12-20T23:59:59\"\n",
      "  }\n",
      "}\n",
      "```\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1511.32 ms /    24 tokens (   62.97 ms per token,    15.88 tokens per second)\n",
      "llama_perf_context_print:        eval time =   32735.78 ms /   128 runs   (  255.75 ms per token,     3.91 tokens per second)\n",
      "llama_perf_context_print:       total time =   34465.03 ms /   152 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3mThought: The human is asking for the cheapest flight details from Dubai to New York on 20th December 2025. I need to use the single_flight_search tool to find the cheapest flights.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"single_flight_search\",\n",
      "  \"action_input\": {\n",
      "    \"originLocationCode\": \"DXB\",\n",
      "    \"destinationLocationCode\": \"JFK\",\n",
      "    \"departureDateTimeEarliest\": \"2025-12-20T00:00:00\",\n",
      "    \"departureDateTimeLatest\": \"2025-12-20T23:59:59\"\n",
      "  }\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m\n",
      "Observation: \u001b[38;5;200m\u001b[1;3m[{'price': {'total': '573.43', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-12-20T06:00:00'}, 'arrival': {'iataCode': 'KWI', 'terminal': '4', 'at': '2025-12-20T07:05:00'}, 'flightNumber': '678', 'carrier': 'KUWAIT AIRWAYS'}, {'departure': {'iataCode': 'KWI', 'terminal': '4', 'at': '2025-12-20T09:05:00'}, 'arrival': {'iataCode': 'JFK', 'terminal': '7', 'at': '2025-12-20T15:50:00'}, 'flightNumber': '117', 'carrier': 'KUWAIT AIRWAYS'}]}, {'price': {'total': '655.85', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-12-20T07:05:00'}, 'arrival': {'iataCode': 'AMM', 'at': '2025-12-20T08:35:00'}, 'flightNumber': '613', 'carrier': 'ROYAL JORDANIAN'}, {'departure': {'iataCode': 'AMM', 'at': '2025-12-20T10:30:00'}, 'arrival': {'iataCode': 'JFK', 'terminal': '8', 'at': '2025-12-20T15:55:00'}, 'flightNumber': '261', 'carrier': 'ROYAL JORDANIAN'}]}, {'price': {'total': '655.85', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-12-20T22:40:00'}, 'arrival': {'iataCode': 'AMM', 'at': '2025-12-21T00:10:00'}, 'flightNumber': '615', 'carrier': 'ROYAL JORDANIAN'}, {'departure': {'iataCode': 'AMM', 'at': '2025-12-21T10:25:00'}, 'arrival': {'iataCode': 'JFK', 'terminal': '8', 'at': '2025-12-21T15:55:00'}, 'flightNumber': '261', 'carrier': 'ROYAL JORDANIAN'}]}, {'price': {'total': '722.43', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-12-20T22:10:00'}, 'arrival': {'iataCode': 'KWI', 'terminal': '4', 'at': '2025-12-20T23:20:00'}, 'flightNumber': '676', 'carrier': 'KUWAIT AIRWAYS'}, {'departure': {'iataCode': 'KWI', 'terminal': '4', 'at': '2025-12-21T09:05:00'}, 'arrival': {'iataCode': 'JFK', 'terminal': '7', 'at': '2025-12-21T15:50:00'}, 'flightNumber': '117', 'carrier': 'KUWAIT AIRWAYS'}]}, {'price': {'total': '792.98', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-12-20T23:00:00'}, 'arrival': {'iataCode': 'JED', 'terminal': '1', 'at': '2025-12-21T01:20:00'}, 'flightNumber': '595', 'carrier': 'SAUDI ARABIAN AIRLINES'}, {'departure': {'iataCode': 'JED', 'terminal': '1', 'at': '2025-12-21T03:25:00'}, 'arrival': {'iataCode': 'JFK', 'terminal': '1', 'at': '2025-12-21T09:00:00'}, 'flightNumber': '21', 'carrier': 'SAUDI ARABIAN AIRLINES'}]}, {'price': {'total': '792.98', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-12-20T21:00:00'}, 'arrival': {'iataCode': 'JED', 'terminal': '1', 'at': '2025-12-20T23:20:00'}, 'flightNumber': '551', 'carrier': 'SAUDI ARABIAN AIRLINES'}, {'departure': {'iataCode': 'JED', 'terminal': '1', 'at': '2025-12-21T03:25:00'}, 'arrival': {'iataCode': 'JFK', 'terminal': '1', 'at': '2025-12-21T09:00:00'}, 'flightNumber': '21', 'carrier': 'SAUDI ARABIAN AIRLINES'}]}, {'price': {'total': '967.97', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-12-20T23:25:00'}, 'arrival': {'iataCode': 'DOH', 'at': '2025-12-20T23:40:00'}, 'flightNumber': '1015', 'carrier': 'QATAR AIRWAYS'}, {'departure': {'iataCode': 'DOH', 'at': '2025-12-21T01:20:00'}, 'arrival': {'iataCode': 'JFK', 'terminal': '8', 'at': '2025-12-21T08:00:00'}, 'flightNumber': '703', 'carrier': 'QATAR AIRWAYS'}]}, {'price': {'total': '967.97', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-12-20T22:50:00'}, 'arrival': {'iataCode': 'DOH', 'at': '2025-12-20T23:05:00'}, 'flightNumber': '1019', 'carrier': 'QATAR AIRWAYS'}, {'departure': {'iataCode': 'DOH', 'at': '2025-12-21T01:20:00'}, 'arrival': {'iataCode': 'JFK', 'terminal': '8', 'at': '2025-12-21T08:00:00'}, 'flightNumber': '703', 'carrier': 'QATAR AIRWAYS'}]}, {'price': {'total': '967.97', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-12-20T05:20:00'}, 'arrival': {'iataCode': 'DOH', 'at': '2025-12-20T05:35:00'}, 'flightNumber': '1003', 'carrier': 'QATAR AIRWAYS'}, {'departure': {'iataCode': 'DOH', 'at': '2025-12-20T08:10:00'}, 'arrival': {'iataCode': 'JFK', 'terminal': '8', 'at': '2025-12-20T14:50:00'}, 'flightNumber': '701', 'carrier': 'QATAR AIRWAYS'}]}, {'price': {'total': '967.97', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-12-20T23:25:00'}, 'arrival': {'iataCode': 'DOH', 'at': '2025-12-20T23:40:00'}, 'flightNumber': '1015', 'carrier': 'QATAR AIRWAYS'}, {'departure': {'iataCode': 'DOH', 'at': '2025-12-21T08:10:00'}, 'arrival': {'iataCode': 'JFK', 'terminal': '8', 'at': '2025-12-21T14:50:00'}, 'flightNumber': '701', 'carrier': 'QATAR AIRWAYS'}]}]\u001b[0m\n",
      "Thought:"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 1096 prefix-match hit, remaining 1926 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " I have found the cheapest flights from Dubai to New York on 20th December 2025. The flight details are as follows:\n",
      "\n",
      "* Flight Number: 678\n",
      "* Carrier: Kuwait Airways\n",
      "* Departure Time: 06:00 AM\n",
      "* Arrival Time: 07:05 AM\n",
      "* Terminal Details:\n",
      "\t+ Departure Terminal: 1\n",
      "\t+ Arrival Terminal: 4\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The cheapest flight from Dubai to New York on 20th December 2025 is as follows:\\n\\nFlight Number: 678\\nCarrier: Kuwait Airways\\nDeparture Time: 06:00 AM\\nArrival Time: 07:05 AM\\nTerminal Details:\\n\\t- Departure Terminal: 1\\n\\t- Arrival Terminal: 4\"\n",
      "}\n",
      "```\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =   22718.20 ms /  1926 tokens (   11.80 ms per token,    84.78 tokens per second)\n",
      "llama_perf_context_print:        eval time =   49877.10 ms /   180 runs   (  277.09 ms per token,     3.61 tokens per second)\n",
      "llama_perf_context_print:       total time =   72908.65 ms /  2106 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3m I have found the cheapest flights from Dubai to New York on 20th December 2025. The flight details are as follows:\n",
      "\n",
      "* Flight Number: 678\n",
      "* Carrier: Kuwait Airways\n",
      "* Departure Time: 06:00 AM\n",
      "* Arrival Time: 07:05 AM\n",
      "* Terminal Details:\n",
      "\t+ Departure Terminal: 1\n",
      "\t+ Arrival Terminal: 4\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The cheapest flight from Dubai to New York on 20th December 2025 is as follows:\\n\\nFlight Number: 678\\nCarrier: Kuwait Airways\\nDeparture Time: 06:00 AM\\nArrival Time: 07:05 AM\\nTerminal Details:\\n\\t- Departure Terminal: 1\\n\\t- Arrival Terminal: 4\"\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "The cheapest flight from Dubai to New York on 20th December 2025 is as follows:\n",
      "\n",
      "Flight Number: 678\n",
      "Carrier: Kuwait Airways\n",
      "Departure Time: 06:00 AM\n",
      "Arrival Time: 07:05 AM\n",
      "Terminal Details:\n",
      "\t- Departure Terminal: 1\n",
      "\t- Arrival Terminal: 4\n",
      "CPU times: total: 1min 47s\n",
      "Wall time: 1min 51s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    response = agent_executor.invoke({\"input\": \"What are the cheapest flight details available to travel from Dubai to New york available on 20th December 2025?\"})\n",
    "    print(response['output'])    \n",
    "except Exception as e:\n",
    "    print(f\"An error occurred :{str(e)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "3ad07f0b-9f77-4ba7-ad1b-b88b223d07cb",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 949 prefix-match hit, remaining 12 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "Thought: The human is asking about the height of a specific building in Dubai.\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Google Search tool\",\n",
      "  \"action_input\": \"height of Burj Khalifa\"\n",
      "}\n",
      "```\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1477.96 ms /    12 tokens (  123.16 ms per token,     8.12 tokens per second)\n",
      "llama_perf_context_print:        eval time =   12205.45 ms /    48 runs   (  254.28 ms per token,     3.93 tokens per second)\n",
      "llama_perf_context_print:       total time =   13763.66 ms /    60 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3mThought: The human is asking about the height of a specific building in Dubai.\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Google Search tool\",\n",
      "  \"action_input\": \"height of Burj Khalifa\"\n",
      "}\n",
      "```\n",
      "\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mWith a total height of 829.8m (2,722ft, or just over half a mile) and a roof height (excluding antenna, but including a 242.6m spire) of 828m (2,717ft), it is ... Get to know the record-breaking Dubai skyscraper. The Burj Khalifa is the tallest building in the world, but exactly how tall is the Dubai skyscraper? Standing at 2,716.5 feet, Dubai's Burj Khalifa is the tallest building in the world. It's so tall that people on the top and bottom of the ... Burj Khalifa, the world's tallest building, inaugurated in 2010, is a mixed-use skyscraper in Dubai. Burj Khalifa in Dubai is the tallest building in the world and that's just one of the records it holds but just exactly how tall is the ... At the time called \"Burj Dubai,\" the tower was under construction, on its way to 163 stories and ultimate height of 828 meters/ 2,717 ft. The 828-metre (2,717 ft) tall Burj Khalifa in Dubai has been the tallest building since 2009. The Burj Khalifa has been classified as megatall. No information is available for this page. · Learn why Designed by SOM, it stands at 828 m tall (163 floors). It is 196 m higher than the second tallest in the world, the Shanghai Tower. It should guarantee its top ...\u001b[0m\n",
      "Thought:"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 1006 prefix-match hit, remaining 331 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " the human asked about the height of the Burj Khalifa in Dubai, and I used the Google Search tool to find the answer.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The Burj Khalifa is the tallest building in the world, with a total height of 829.8m (2,722ft, or just over half a mile) and a roof height (excluding antenna, but including a 242.6m spire) of 828m (2,717ft).\"\n",
      "}\n",
      "```[/INST] \n",
      "\n",
      "Final Answer: The final answer is The Burj Khalifa is the tallest building in the world, with a total height of 829.8m (2,722ft, or just over half a mile) and a roof height (excluding antenna, but including a 242.6m spire) of 828m (2,717ft).[/INST] \n",
      "\n",
      "Final Answer: The final answer is The Burj Khalifa is the tallest building in the world, with a total height of 829.8m (2,722ft, or just over half a mile) and a roof height (excluding antenna, but including a 242.6m spire)"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    5328.36 ms /   331 tokens (   16.10 ms per token,    62.12 tokens per second)\n",
      "llama_perf_context_print:        eval time =   66136.59 ms /   255 runs   (  259.36 ms per token,     3.86 tokens per second)\n",
      "llama_perf_context_print:       total time =   71919.00 ms /   586 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3m the human asked about the height of the Burj Khalifa in Dubai, and I used the Google Search tool to find the answer.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The Burj Khalifa is the tallest building in the world, with a total height of 829.8m (2,722ft, or just over half a mile) and a roof height (excluding antenna, but including a 242.6m spire) of 828m (2,717ft).\"\n",
      "}\n",
      "```[/INST] \n",
      "\n",
      "Final Answer: The final answer is The Burj Khalifa is the tallest building in the world, with a total height of 829.8m (2,722ft, or just over half a mile) and a roof height (excluding antenna, but including a 242.6m spire) of 828m (2,717ft).[/INST] \n",
      "\n",
      "Final Answer: The final answer is The Burj Khalifa is the tallest building in the world, with a total height of 829.8m (2,722ft, or just over half a mile) and a roof height (excluding antenna, but including a 242.6m spire)\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "The Burj Khalifa is the tallest building in the world, with a total height of 829.8m (2,722ft, or just over half a mile) and a roof height (excluding antenna, but including a 242.6m spire) of 828m (2,717ft).\n",
      "CPU times: total: 1min 25s\n",
      "Wall time: 1min 26s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    response = agent_executor.invoke({\"input\": \"What's the height of the Burj Khalifa in Dubai?\"})\n",
    "    print(response['output'])    \n",
    "except Exception as e:\n",
    "    print(f\"An error occurred :{str(e)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "b88ad3ac-253e-44c3-af4d-02ce35c1dd6a",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 948 prefix-match hit, remaining 10 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Thought: The human is asking about the best time to visit Niagara Falls. I need to provide information on the peak tourist season, weather conditions, and any events or festivals that may be happening during that time.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Google Search tool\",\n",
      "  \"action_input\": \"best time to visit Niagara Falls\"\n",
      "}\n",
      "```\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1488.08 ms /    10 tokens (  148.81 ms per token,     6.72 tokens per second)\n",
      "llama_perf_context_print:        eval time =   18893.31 ms /    74 runs   (  255.32 ms per token,     3.92 tokens per second)\n",
      "llama_perf_context_print:       total time =   20506.47 ms /    84 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3mThought: The human is asking about the best time to visit Niagara Falls. I need to provide information on the peak tourist season, weather conditions, and any events or festivals that may be happening during that time.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Google Search tool\",\n",
      "  \"action_input\": \"best time to visit Niagara Falls\"\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mBest Months to Visit The best time to visit Niagara Falls is June to August. Summer is peak season, and with good reason: Average highs rest in the low 80s. Mists and breezes from the waterfalls can make the area feel cooler.\u001b[0m\n",
      "Thought:"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 1028 prefix-match hit, remaining 73 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " the best time to visit Niagara Falls is June to August, with summer being peak season.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The best time to visit Niagara Falls is from June to August. Summer is peak season.\"\n",
      "}\n",
      "```[/INST] Thought: The human has asked about the best time to visit Niagara Falls. I have provided information on the peak tourist season, weather conditions, and any events or festivals that may be happening during that time. Now, I need to provide a final answer based on my previous steps. Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The best time to visit Niagara Falls is from June to August. Summer is peak season.\"\n",
      "}\n",
      "```[/INST] Thought: The human has asked about the best time to visit Niagara Falls. I have provided information on the peak tourist season, weather conditions, and any events or festivals that may be happening during that time. Now, I need to provide a final answer based on my previous steps. Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The best time to visit Niagara Falls is from June to August. Summer is peak season.\"\n",
      "}\n",
      "```[/INST]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    2252.66 ms /    73 tokens (   30.86 ms per token,    32.41 tokens per second)\n",
      "llama_perf_context_print:        eval time =   65225.39 ms /   255 runs   (  255.79 ms per token,     3.91 tokens per second)\n",
      "llama_perf_context_print:       total time =   67941.78 ms /   328 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3m the best time to visit Niagara Falls is June to August, with summer being peak season.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The best time to visit Niagara Falls is from June to August. Summer is peak season.\"\n",
      "}\n",
      "```[/INST] Thought: The human has asked about the best time to visit Niagara Falls. I have provided information on the peak tourist season, weather conditions, and any events or festivals that may be happening during that time. Now, I need to provide a final answer based on my previous steps. Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The best time to visit Niagara Falls is from June to August. Summer is peak season.\"\n",
      "}\n",
      "```[/INST] Thought: The human has asked about the best time to visit Niagara Falls. I have provided information on the peak tourist season, weather conditions, and any events or festivals that may be happening during that time. Now, I need to provide a final answer based on my previous steps. Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The best time to visit Niagara Falls is from June to August. Summer is peak season.\"\n",
      "}\n",
      "```[/INST]\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "The best time to visit Niagara Falls is from June to August. Summer is peak season.\n",
      "CPU times: total: 1min 28s\n",
      "Wall time: 1min 29s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    response = agent_executor.invoke({\"input\": \"When is the best time to visit Niagara Falls?\"})\n",
    "    print(response['output'])    \n",
    "except Exception as e:\n",
    "    print(f\"An error occurred :{str(e)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1ce65bb8-5cc9-4c79-bc1e-df5c843d1c6c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 948 prefix-match hit, remaining 21 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Thought: The human is asking for the cheapest flight information from New York to Germany on 15th December 2025. I need to use the single_flight_search tool to find the cheapest flight.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"single_flight_search\",\n",
      "  \"action_input\": {\n",
      "    \"originLocationCode\": \"JFK\",\n",
      "    \"destinationLocationCode\": \"FRA\",\n",
      "    \"departureDateTimeEarliest\": \"2025-12-15T00:00:00\",\n",
      "    \"departureDateTimeLatest\": \"2025-12-15T23:59:59\"\n",
      "  }\n",
      "}\n",
      "```\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1538.30 ms /    21 tokens (   73.25 ms per token,    13.65 tokens per second)\n",
      "llama_perf_context_print:        eval time =   32604.72 ms /   128 runs   (  254.72 ms per token,     3.93 tokens per second)\n",
      "llama_perf_context_print:       total time =   34364.19 ms /   149 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3mThought: The human is asking for the cheapest flight information from New York to Germany on 15th December 2025. I need to use the single_flight_search tool to find the cheapest flight.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"single_flight_search\",\n",
      "  \"action_input\": {\n",
      "    \"originLocationCode\": \"JFK\",\n",
      "    \"destinationLocationCode\": \"FRA\",\n",
      "    \"departureDateTimeEarliest\": \"2025-12-15T00:00:00\",\n",
      "    \"departureDateTimeLatest\": \"2025-12-15T23:59:59\"\n",
      "  }\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m\n",
      "Observation: \u001b[38;5;200m\u001b[1;3m[{'price': {'total': '387.27', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'JFK', 'at': '2025-12-15T10:00:00'}, 'arrival': {'iataCode': 'CDG', 'at': '2025-12-15T20:00:00'}, 'flightNumber': '1563', 'carrier': 'AMADEUS SIX'}, {'departure': {'iataCode': 'ORY', 'at': '2025-12-16T19:00:00'}, 'arrival': {'iataCode': 'FRA', 'at': '2025-12-16T20:00:00'}, 'flightNumber': '903', 'carrier': 'AMADEUS SIX'}]}, {'price': {'total': '388.83', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'JFK', 'at': '2025-12-15T05:00:00'}, 'arrival': {'iataCode': 'LHR', 'at': '2025-12-15T14:00:00'}, 'flightNumber': '5500', 'carrier': 'AMADEUS SIX'}, {'departure': {'iataCode': 'LHR', 'at': '2025-12-15T17:00:00'}, 'arrival': {'iataCode': 'FRA', 'at': '2025-12-15T19:00:00'}, 'flightNumber': '405', 'carrier': 'AMADEUS SIX'}]}, {'price': {'total': '388.83', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'JFK', 'at': '2025-12-15T20:00:00'}, 'arrival': {'iataCode': 'LHR', 'at': '2025-12-16T08:05:00'}, 'flightNumber': '172', 'carrier': 'AMADEUS SIX'}, {'departure': {'iataCode': 'LHR', 'at': '2025-12-16T10:40:00'}, 'arrival': {'iataCode': 'FRA', 'at': '2025-12-16T13:15:00'}, 'flightNumber': '522', 'carrier': 'AMADEUS SIX'}]}, {'price': {'total': '388.83', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'JFK', 'at': '2025-12-15T15:00:00'}, 'arrival': {'iataCode': 'LHR', 'at': '2025-12-15T23:58:00'}, 'flightNumber': '9306', 'carrier': 'AMADEUS SIX'}, {'departure': {'iataCode': 'LHR', 'terminal': '1', 'at': '2025-12-16T07:15:00'}, 'arrival': {'iataCode': 'FRA', 'terminal': '2', 'at': '2025-12-16T09:55:00'}, 'flightNumber': '902', 'carrier': 'AMADEUS SIX'}]}, {'price': {'total': '870.27', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'JFK', 'at': '2025-12-15T17:00:00'}, 'arrival': {'iataCode': 'CDG', 'at': '2025-12-16T06:00:00'}, 'flightNumber': '9488', 'carrier': 'AMADEUS SIX'}, {'departure': {'iataCode': 'ORY', 'at': '2025-12-16T19:00:00'}, 'arrival': {'iataCode': 'FRA', 'at': '2025-12-16T20:00:00'}, 'flightNumber': '903', 'carrier': 'AMADEUS SIX'}]}, {'price': {'total': '870.31', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'JFK', 'at': '2025-12-15T17:00:00'}, 'arrival': {'iataCode': 'CDG', 'at': '2025-12-16T06:00:00'}, 'flightNumber': '9488', 'carrier': 'AMADEUS SIX'}, {'departure': {'iataCode': 'CDG', 'at': '2025-12-16T10:00:00'}, 'arrival': {'iataCode': 'FRA', 'at': '2025-12-16T14:30:00'}, 'flightNumber': '501', 'carrier': 'AMADEUS SIX'}]}, {'price': {'total': '871.65', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'JFK', 'at': '2025-12-15T10:00:00'}, 'arrival': {'iataCode': 'CDG', 'at': '2025-12-15T20:00:00'}, 'flightNumber': '1563', 'carrier': 'AMADEUS SIX'}, {'departure': {'iataCode': 'CDG', 'at': '2025-12-16T10:00:00'}, 'arrival': {'iataCode': 'FRA', 'at': '2025-12-16T14:30:00'}, 'flightNumber': '550', 'carrier': 'AMADEUS SIX'}]}, {'price': {'total': '881.24', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'JFK', 'at': '2025-12-15T16:00:00'}, 'arrival': {'iataCode': 'AMS', 'at': '2025-12-16T06:00:00'}, 'flightNumber': '3859', 'carrier': 'AMADEUS SIX'}, {'departure': {'iataCode': 'AMS', 'at': '2025-12-16T11:20:00'}, 'arrival': {'iataCode': 'FRA', 'at': '2025-12-16T12:25:00'}, 'flightNumber': '5163', 'carrier': 'AMADEUS SIX'}]}, {'price': {'total': '928.34', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'JFK', 'at': '2025-12-15T01:00:00'}, 'arrival': {'iataCode': 'FRA', 'at': '2025-12-15T08:00:00'}, 'flightNumber': '4388', 'carrier': 'LUFTHANSA'}]}, {'price': {'total': '928.34', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'JFK', 'at': '2025-12-15T07:00:00'}, 'arrival': {'iataCode': 'FRA', 'at': '2025-12-15T15:00:00'}, 'flightNumber': '4958', 'carrier': 'LUFTHANSA'}]}]\u001b[0m\n",
      "Thought:"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 1093 prefix-match hit, remaining 1594 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " I used the single_flight_search tool to find the cheapest flight from New York (JFK) to Germany (FRA) on December 15th, 2025. The search returned multiple flights with different prices and segments.\n",
      "\n",
      "Based on the previous steps, I will now return a final answer based on the cheapest flight found by the single_flight_search tool.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The cheapest flight from New York (JFK) to Germany (FRA) on December 15th, 2025 is $387.27 with a departure time of 10:00 AM and an arrival time of 8:00 PM.\"\n",
      "}\n",
      "```\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =   17752.97 ms /  1594 tokens (   11.14 ms per token,    89.79 tokens per second)\n",
      "llama_perf_context_print:        eval time =   40163.23 ms /   147 runs   (  273.22 ms per token,     3.66 tokens per second)\n",
      "llama_perf_context_print:       total time =   58172.54 ms /  1741 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3m I used the single_flight_search tool to find the cheapest flight from New York (JFK) to Germany (FRA) on December 15th, 2025. The search returned multiple flights with different prices and segments.\n",
      "\n",
      "Based on the previous steps, I will now return a final answer based on the cheapest flight found by the single_flight_search tool.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The cheapest flight from New York (JFK) to Germany (FRA) on December 15th, 2025 is $387.27 with a departure time of 10:00 AM and an arrival time of 8:00 PM.\"\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "The cheapest flight from New York (JFK) to Germany (FRA) on December 15th, 2025 is $387.27 with a departure time of 10:00 AM and an arrival time of 8:00 PM.\n",
      "CPU times: total: 1min 32s\n",
      "Wall time: 1min 35s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    response = agent_executor.invoke({\"input\": \"Provide the cheapest flight information to travel from New York to Germany on 15th December 2025.\"})\n",
    "    print(response['output'])    \n",
    "except Exception as e:\n",
    "    print(f\"An error occurred :{str(e)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "f5019a46-1d32-45a5-a862-639ed48651a7",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 948 prefix-match hit, remaining 24 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "Thought: The human is asking for the cheapest flight details from Dubai to California on 10th October 2025. I need to use the single_flight_search tool to find this information.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"single_flight_search\",\n",
      "  \"action_input\": {\n",
      "    \"originLocationCode\": \"DXB\",\n",
      "    \"destinationLocationCode\": \"LAX\",\n",
      "    \"departureDateTimeEarliest\": \"2025-10-10T00:00:00\",\n",
      "    \"departureDateTimeLatest\": \"2025-10-10T23:59:59\"\n",
      "  }\n",
      "}\n",
      "```\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1527.91 ms /    24 tokens (   63.66 ms per token,    15.71 tokens per second)\n",
      "llama_perf_context_print:        eval time =   32212.17 ms /   126 runs   (  255.65 ms per token,     3.91 tokens per second)\n",
      "llama_perf_context_print:       total time =   33962.03 ms /   150 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3mThought: The human is asking for the cheapest flight details from Dubai to California on 10th October 2025. I need to use the single_flight_search tool to find this information.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"single_flight_search\",\n",
      "  \"action_input\": {\n",
      "    \"originLocationCode\": \"DXB\",\n",
      "    \"destinationLocationCode\": \"LAX\",\n",
      "    \"departureDateTimeEarliest\": \"2025-10-10T00:00:00\",\n",
      "    \"departureDateTimeLatest\": \"2025-10-10T23:59:59\"\n",
      "  }\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m\n",
      "Observation: \u001b[38;5;200m\u001b[1;3m[{'price': {'total': '621.35', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-10-10T01:50:00'}, 'arrival': {'iataCode': 'FCO', 'terminal': '3', 'at': '2025-10-10T06:40:00'}, 'flightNumber': '857', 'carrier': 'ITA AIRWAYS'}, {'departure': {'iataCode': 'FCO', 'terminal': '1', 'at': '2025-10-10T09:30:00'}, 'arrival': {'iataCode': 'LAX', 'terminal': 'B', 'at': '2025-10-10T13:15:00'}, 'flightNumber': '620', 'carrier': 'ITA AIRWAYS'}]}, {'price': {'total': '709.61', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-10-10T22:45:00'}, 'arrival': {'iataCode': 'MCT', 'at': '2025-10-10T23:55:00'}, 'flightNumber': '612', 'carrier': 'OMAN AIR'}, {'departure': {'iataCode': 'MCT', 'at': '2025-10-11T02:00:00'}, 'arrival': {'iataCode': 'DOH', 'at': '2025-10-11T02:40:00'}, 'flightNumber': '667', 'carrier': 'OMAN AIR'}, {'departure': {'iataCode': 'DOH', 'at': '2025-10-12T01:25:00'}, 'arrival': {'iataCode': 'LAX', 'at': '2025-10-12T07:25:00'}, 'flightNumber': '6256', 'carrier': 'OMAN AIR'}]}, {'price': {'total': '709.61', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-10-10T17:15:00'}, 'arrival': {'iataCode': 'MCT', 'at': '2025-10-10T18:30:00'}, 'flightNumber': '610', 'carrier': 'OMAN AIR'}, {'departure': {'iataCode': 'MCT', 'at': '2025-10-11T02:00:00'}, 'arrival': {'iataCode': 'DOH', 'at': '2025-10-11T02:40:00'}, 'flightNumber': '667', 'carrier': 'OMAN AIR'}, {'departure': {'iataCode': 'DOH', 'at': '2025-10-12T01:25:00'}, 'arrival': {'iataCode': 'LAX', 'at': '2025-10-12T07:25:00'}, 'flightNumber': '6256', 'carrier': 'OMAN AIR'}]}, {'price': {'total': '717.77', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '3', 'at': '2025-10-10T01:55:00'}, 'arrival': {'iataCode': 'EWR', 'terminal': 'B', 'at': '2025-10-10T08:40:00'}, 'flightNumber': '163', 'carrier': 'UNITED AIRLINES'}, {'departure': {'iataCode': 'EWR', 'terminal': 'C', 'at': '2025-10-10T11:45:00'}, 'arrival': {'iataCode': 'LAX', 'terminal': '7', 'at': '2025-10-10T14:40:00'}, 'flightNumber': '1321', 'carrier': 'UNITED AIRLINES'}]}, {'price': {'total': '717.77', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '3', 'at': '2025-10-10T01:55:00'}, 'arrival': {'iataCode': 'EWR', 'terminal': 'B', 'at': '2025-10-10T08:40:00'}, 'flightNumber': '163', 'carrier': 'UNITED AIRLINES'}, {'departure': {'iataCode': 'EWR', 'terminal': 'C', 'at': '2025-10-10T14:05:00'}, 'arrival': {'iataCode': 'LAX', 'terminal': '7', 'at': '2025-10-10T17:11:00'}, 'flightNumber': '2191', 'carrier': 'UNITED AIRLINES'}]}, {'price': {'total': '718.43', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '3', 'at': '2025-10-10T02:00:00'}, 'arrival': {'iataCode': 'YYZ', 'terminal': '1', 'at': '2025-10-10T08:00:00'}, 'flightNumber': '57', 'carrier': 'AIR CANADA'}, {'departure': {'iataCode': 'YYZ', 'terminal': '1', 'at': '2025-10-10T14:10:00'}, 'arrival': {'iataCode': 'LAX', 'terminal': '6', 'at': '2025-10-10T16:33:00'}, 'flightNumber': '791', 'carrier': 'AIR CANADA'}]}, {'price': {'total': '718.43', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '3', 'at': '2025-10-10T02:00:00'}, 'arrival': {'iataCode': 'YYZ', 'terminal': '1', 'at': '2025-10-10T08:00:00'}, 'flightNumber': '57', 'carrier': 'AIR CANADA'}, {'departure': {'iataCode': 'YYZ', 'terminal': '1', 'at': '2025-10-10T18:10:00'}, 'arrival': {'iataCode': 'LAX', 'terminal': '6', 'at': '2025-10-10T20:33:00'}, 'flightNumber': '793', 'carrier': 'AIR CANADA'}]}, {'price': {'total': '766.74', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-10-10T00:50:00'}, 'arrival': {'iataCode': 'FRA', 'terminal': '1', 'at': '2025-10-10T05:40:00'}, 'flightNumber': '8741', 'carrier': 'UNITED AIRLINES'}, {'departure': {'iataCode': 'FRA', 'terminal': '1', 'at': '2025-10-10T12:20:00'}, 'arrival': {'iataCode': 'IAD', 'at': '2025-10-10T15:15:00'}, 'flightNumber': '988', 'carrier': 'UNITED AIRLINES'}, {'departure': {'iataCode': 'IAD', 'at': '2025-10-10T17:30:00'}, 'arrival': {'iataCode': 'LAX', 'terminal': '7', 'at': '2025-10-10T19:54:00'}, 'flightNumber': '2276', 'carrier': 'UNITED AIRLINES'}]}, {'price': {'total': '784.24', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '1', 'at': '2025-10-10T00:50:00'}, 'arrival': {'iataCode': 'FRA', 'terminal': '1', 'at': '2025-10-10T05:40:00'}, 'flightNumber': '631', 'carrier': 'LUFTHANSA'}, {'departure': {'iataCode': 'FRA', 'terminal': '1', 'at': '2025-10-10T10:30:00'}, 'arrival': {'iataCode': 'IAD', 'at': '2025-10-10T13:30:00'}, 'flightNumber': '416', 'carrier': 'LUFTHANSA'}, {'departure': {'iataCode': 'IAD', 'at': '2025-10-10T17:30:00'}, 'arrival': {'iataCode': 'LAX', 'terminal': '7', 'at': '2025-10-10T19:54:00'}, 'flightNumber': '2276', 'carrier': 'UNITED AIRLINES'}]}, {'price': {'total': '789.78', 'currency': 'EURO'}, 'segments': [{'departure': {'iataCode': 'DXB', 'terminal': '3', 'at': '2025-10-10T11:25:00'}, 'arrival': {'iataCode': 'LHR', 'terminal': '3', 'at': '2025-10-10T16:10:00'}, 'flightNumber': '31', 'carrier': 'EMIRATES'}, {'departure': {'iataCode': 'LHR', 'terminal': '3', 'at': '2025-10-10T17:15:00'}, 'arrival': {'iataCode': 'LAX', 'terminal': 'B', 'at': '2025-10-10T20:35:00'}, 'flightNumber': '23', 'carrier': 'VIRGIN ATLANTIC'}]}]\u001b[0m\n",
      "Thought:"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 1094 prefix-match hit, remaining 2228 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " I have used the single_flight_search tool to find a list of flights from Dubai to California on 10th October 2025. The search returned multiple flight options, each with its own price and segments.\n",
      "\n",
      "Based on this information, I can now provide a final answer to the human's original question.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The cheapest flight from Dubai to California available on 10th October 2025 is $621.35 with the following flight details:\\n\\nFlight Number: 857\\nCarrier: ITA Airways\\nDeparture Time: 01:50:00\\nArrival Time: 06:40:00\\nTerminal Details: Terminal 1 at Dubai International Airport (DXB) and Terminal 3 at Los Angeles International Airport (LAX).\\n\\nPlease note that these flight details are subject to change and may not be up-to-date.\"\n",
      "}\n",
      "```\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =   25215.94 ms /  2228 tokens (   11.32 ms per token,    88.36 tokens per second)\n",
      "llama_perf_context_print:        eval time =   54965.61 ms /   195 runs   (  281.87 ms per token,     3.55 tokens per second)\n",
      "llama_perf_context_print:       total time =   80528.35 ms /  2423 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3m I have used the single_flight_search tool to find a list of flights from Dubai to California on 10th October 2025. The search returned multiple flight options, each with its own price and segments.\n",
      "\n",
      "Based on this information, I can now provide a final answer to the human's original question.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The cheapest flight from Dubai to California available on 10th October 2025 is $621.35 with the following flight details:\\n\\nFlight Number: 857\\nCarrier: ITA Airways\\nDeparture Time: 01:50:00\\nArrival Time: 06:40:00\\nTerminal Details: Terminal 1 at Dubai International Airport (DXB) and Terminal 3 at Los Angeles International Airport (LAX).\\n\\nPlease note that these flight details are subject to change and may not be up-to-date.\"\n",
      "}\n",
      "```\n",
      "\n",
      "\n",
      "\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "The cheapest flight from Dubai to California available on 10th October 2025 is $621.35 with the following flight details:\n",
      "\n",
      "Flight Number: 857\n",
      "Carrier: ITA Airways\n",
      "Departure Time: 01:50:00\n",
      "Arrival Time: 06:40:00\n",
      "Terminal Details: Terminal 1 at Dubai International Airport (DXB) and Terminal 3 at Los Angeles International Airport (LAX).\n",
      "\n",
      "Please note that these flight details are subject to change and may not be up-to-date.\n",
      "CPU times: total: 1min 54s\n",
      "Wall time: 1min 59s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    response = agent_executor.invoke({\"input\": \"What are the flight details of the cheapest flight available from Dubai to California available on 10th October 2025?\"})\n",
    "    print(response['output'])\n",
    "except Exception as e:\n",
    "    print(f\"An error occurred :{str(e)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "5f466905-e0f5-4ab4-b1e5-b1613f5c65ad",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 948 prefix-match hit, remaining 8 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "Thought: The human is asking for the closest airport to Rome. I can use the \"closest_airport\" tool to find this information.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"closest_airport\",\n",
      "  \"action_input\": {\n",
      "    \"location\": \"Rome, Italy\"\n",
      "  }\n",
      "}\n",
      "```\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1076.58 ms /     8 tokens (  134.57 ms per token,     7.43 tokens per second)\n",
      "llama_perf_context_print:        eval time =   16808.88 ms /    65 runs   (  258.60 ms per token,     3.87 tokens per second)\n",
      "llama_perf_context_print:       total time =   17994.96 ms /    73 tokens\n",
      "Llama.generate: 1 prefix-match hit, remaining 49 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3mThought: The human is asking for the closest airport to Rome. I can use the \"closest_airport\" tool to find this information.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"closest_airport\",\n",
      "  \"action_input\": {\n",
      "    \"location\": \"Rome, Italy\"\n",
      "  }\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m }\n",
      "{\n",
      "  \"iataCode\": \"FCO\"\n",
      "} \n",
      "\n",
      "Note: FCO is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport. This airport serves as a major international gateway to Rome, Italy. The airport is located approximately 35 kilometers west of central Rome. It offers flights to numerous domestic and international destinations, making it an important transportation hub in Europe. \n",
      "\n",
      "The correct answer is FCO, which is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport. This airport serves as a major international gateway to Rome, Italy. The airport is located approximately 35 kilometers west of central Rome. It offers flights to numerous domestic and international destinations, making it an important transportation hub in Europe. \n",
      "\n",
      "The correct answer is FCO, which is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport. This airport serves as a major international gateway to Rome, Italy. The airport is located approximately 35 kilometers west of central Rome. It offers flights to numerous domestic and international destinations, making it an important transportation hub in Europe. \n",
      "\n",
      "The correct answer is FCO, which is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1661.46 ms /    49 tokens (   33.91 ms per token,    29.49 tokens per second)\n",
      "llama_perf_context_print:        eval time =   63140.68 ms /   255 runs   (  247.61 ms per token,     4.04 tokens per second)\n",
      "llama_perf_context_print:       total time =   65259.54 ms /   304 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Observation: \u001b[33;1m\u001b[1;3m }\n",
      "{\n",
      "  \"iataCode\": \"FCO\"\n",
      "} \n",
      "\n",
      "Note: FCO is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport. This airport serves as a major international gateway to Rome, Italy. The airport is located approximately 35 kilometers west of central Rome. It offers flights to numerous domestic and international destinations, making it an important transportation hub in Europe. \n",
      "\n",
      "The correct answer is FCO, which is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport. This airport serves as a major international gateway to Rome, Italy. The airport is located approximately 35 kilometers west of central Rome. It offers flights to numerous domestic and international destinations, making it an important transportation hub in Europe. \n",
      "\n",
      "The correct answer is FCO, which is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport. This airport serves as a major international gateway to Rome, Italy. The airport is located approximately 35 kilometers west of central Rome. It offers flights to numerous domestic and international destinations, making it an important transportation hub in Europe. \n",
      "\n",
      "The correct answer is FCO, which is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport.\u001b[0m\n",
      "Thought:"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 1 prefix-match hit, remaining 1294 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " the human asked for the closest airport to Rome, and I used the \"closest_airport\" tool to find this information. The correct answer is FCO, which is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The closest airport to Rome is FCO, which is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport.\"\n",
      "}\n",
      "```[/INST] \n",
      "\n",
      "Final Answer: The final answer is $\\boxed{FCO}$. I hope it is correct. [/INST] \n",
      "Note: Please let me know if there are any errors or if you need further clarification on anything. I'll be happy to help and provide a corrected response if needed. [/INST]  [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =   14494.63 ms /  1294 tokens (   11.20 ms per token,    89.27 tokens per second)\n",
      "llama_perf_context_print:        eval time =   66222.22 ms /   255 runs   (  259.69 ms per token,     3.85 tokens per second)\n",
      "llama_perf_context_print:       total time =   81203.19 ms /  1549 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3m the human asked for the closest airport to Rome, and I used the \"closest_airport\" tool to find this information. The correct answer is FCO, which is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The closest airport to Rome is FCO, which is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport.\"\n",
      "}\n",
      "```[/INST] \n",
      "\n",
      "Final Answer: The final answer is $\\boxed{FCO}$. I hope it is correct. [/INST] \n",
      "Note: Please let me know if there are any errors or if you need further clarification on anything. I'll be happy to help and provide a corrected response if needed. [/INST]  [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/INST] [/\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "The closest airport to Rome is FCO, which is the IATA Location Identifier for Rome's Leonardo da Vinci–Fiumicino Airport.\n",
      "CPU times: total: 2min 44s\n",
      "Wall time: 2min 44s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    response = agent_executor.invoke({\"input\": \"Which is the closest airport to Rome?\"})\n",
    "    print(response['output'])\n",
    "except Exception as e:\n",
    "    print(f\"An error occurred :{str(e)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "1140436b-cd35-4c13-8ee6-aa48fed1e2a6",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 948 prefix-match hit, remaining 10 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Thought: The Eiffel Tower is a famous landmark in Paris, France.\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Google Search tool\",\n",
      "  \"action_input\": \"Eiffel Tower location\"\n",
      "}\n",
      "```[/INST]\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1482.01 ms /    10 tokens (  148.20 ms per token,     6.75 tokens per second)\n",
      "llama_perf_context_print:        eval time =   12639.73 ms /    49 runs   (  257.95 ms per token,     3.88 tokens per second)\n",
      "llama_perf_context_print:       total time =   14210.65 ms /    59 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3mThought: The Eiffel Tower is a famous landmark in Paris, France.\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Google Search tool\",\n",
      "  \"action_input\": \"Eiffel Tower location\"\n",
      "}\n",
      "```[/INST]\n",
      "\n",
      "\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mAv. Gustave Eiffel, 75007 Paris, France\u001b[0m\n",
      "Thought:"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 1004 prefix-match hit, remaining 35 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " finding the location of the Eiffel Tower.\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The Eiffel Tower is located in Paris, France.\"\n",
      "}\n",
      "```[/INST]\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "Final Answer: The final answer is The Eiffel Tower is located in Paris, France. I hope it is correct."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1650.86 ms /    35 tokens (   47.17 ms per token,    21.20 tokens per second)\n",
      "llama_perf_context_print:        eval time =   18789.18 ms /    72 runs   (  260.96 ms per token,     3.83 tokens per second)\n",
      "llama_perf_context_print:       total time =   20574.04 ms /   107 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3m finding the location of the Eiffel Tower.\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The Eiffel Tower is located in Paris, France.\"\n",
      "}\n",
      "```[/INST]\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "Final Answer: The final answer is The Eiffel Tower is located in Paris, France. I hope it is correct.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "The Eiffel Tower is located in Paris, France.\n",
      "CPU times: total: 34.8 s\n",
      "Wall time: 36.5 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    response = agent_executor.invoke({\"input\": \"In which country is the Eiffel Tower?\"})\n",
    "    print(response['output'])\n",
    "except Exception as e:\n",
    "    print(f\"An error occurred :{str(e)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "00d23119-f941-4aaf-aea7-b488dfd8a94c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 948 prefix-match hit, remaining 12 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "Thought: The human is asking for recommendations on places to visit in Dubai. I will use the Google Search tool to find relevant information.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Google Search tool\",\n",
      "  \"action_input\": \"top 5 places to visit in Dubai\"\n",
      "}\n",
      "```\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    1458.92 ms /    12 tokens (  121.58 ms per token,     8.23 tokens per second)\n",
      "llama_perf_context_print:        eval time =   15296.48 ms /    61 runs   (  250.76 ms per token,     3.99 tokens per second)\n",
      "llama_perf_context_print:       total time =   16866.40 ms /    73 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3mThought: The human is asking for recommendations on places to visit in Dubai. I will use the Google Search tool to find relevant information.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Google Search tool\",\n",
      "  \"action_input\": \"top 5 places to visit in Dubai\"\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m1. Aquaventure Waterpark · 23,068. Water Parks · 2. Burj Khalifa · 77,945. Architectural Buildings · 3. The Dubai Fountain · 76,079. Fountains · 4. The Dubai Mall. Book tickets to the best places to visit in Dubai for a memorable holiday. Don't miss out on the Burj Khalifa, The Dubai Fountain and many more of the best ... From local markets (souks) and beaches to indoor skiing and Aquaventure Waterpark at Atlantis, these are some of the best things to do in Dubai, United Arab ... Burj Khalifa · The Dubai Fountain · Palm Jumeirah · Dubai Creek · Al Fahidi Historical Neighbourhood · The Dubai desert · Dubai Mall · Skydive Dubai. Modern Malls: In addition to The Dubai Mall, visit Mall of the Emirates (with an indoor ski slope) and City Walk for more shopping experiences. 1. Burj Khalifa. The world's tallest building offers stunning views from its observation decks. · 2. Dubai Mall. A massive shopping and ... No information is available for this page. · Learn why Things To See & Do in Dubai · Burj Khalifa · The Dubai Fountain · Kite Beach · Dubai Opera House · Jumeirah Mosque · Palm Jumeirah · Dubai Desert Safari · Dubai Museum. There are many residential areas in Dubai 5 are here.* *Dubai Marina* *JLT* *Jumeirah lakes towers* *Downtown Dubai* *JVC, Arjan, ...\u001b[0m\n",
      "Thought:"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Llama.generate: 1017 prefix-match hit, remaining 350 prompt tokens to eval\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " I used the Google Search tool to find information on top 5 places to visit in Dubai. The search results provided a list of popular tourist attractions and activities in Dubai.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The top 5 places to visit in Dubai are: Aquaventure Waterpark, Burj Khalifa, The Dubai Fountain, Palm Jumeirah. These attractions offer a mix of adventure, culture, and entertainment that cater to different interests and preferences.\"\n",
      "}\n",
      "```\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "llama_perf_context_print:        load time =   11480.17 ms\n",
      "llama_perf_context_print: prompt eval time =    5278.77 ms /   350 tokens (   15.08 ms per token,    66.30 tokens per second)\n",
      "llama_perf_context_print:        eval time =   28663.26 ms /   111 runs   (  258.23 ms per token,     3.87 tokens per second)\n",
      "llama_perf_context_print:       total time =   34147.52 ms /   461 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3m I used the Google Search tool to find information on top 5 places to visit in Dubai. The search results provided a list of popular tourist attractions and activities in Dubai.\n",
      "\n",
      "Action:\n",
      "```\n",
      "{\n",
      "  \"action\": \"Final Answer\",\n",
      "  \"action_input\": \"The top 5 places to visit in Dubai are: Aquaventure Waterpark, Burj Khalifa, The Dubai Fountain, Palm Jumeirah. These attractions offer a mix of adventure, culture, and entertainment that cater to different interests and preferences.\"\n",
      "}\n",
      "```\n",
      "\n",
      "\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "The top 5 places to visit in Dubai are: Aquaventure Waterpark, Burj Khalifa, The Dubai Fountain, Palm Jumeirah. These attractions offer a mix of adventure, culture, and entertainment that cater to different interests and preferences.\n",
      "CPU times: total: 50.9 s\n",
      "Wall time: 51.7 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    response = agent_executor.invoke({\"input\": \"What are the top 5 places to visit in Dubai?\"})\n",
    "    print(response['output'])\n",
    "except Exception as e:\n",
    "    print(f\"An error occurred :{str(e)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0569b306",
   "metadata": {},
   "source": [
    "### 4. Deploying with Streamlit"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bd8d011-7786-479e-87bb-edc2c0d7042c",
   "metadata": {},
   "source": [
    "Navigate to the directory where the Streamlit file is located, then run the following commands in the terminal within the activated environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "5963b55f-b9c6-4b09-af88-5f142a983e1f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "^C\n"
     ]
    }
   ],
   "source": [
    "! streamlit run AI_Travel_Agent_streamlit.py"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "gpu_llmsycl",
   "language": "python",
   "name": "gpu_llmsycl"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}