Add walkthroughs and delete tutorial (substratusai#20)

* move content from tutorial to walkthrough
* update introduction

---------

Co-authored-by: Brandon J. Bjelland <[email protected]>

Author: samos123

.vscode/settings.json

@@ -0,0 +1,5 @@
+{
+    "cSpell.words": [
+        "finetune"
+    ]
+}

docs/introduction.md

@@ -9,11 +9,13 @@ Substratus is a cross-cloud substrate for training and serving ML models. Substr
 
 ## Why Substratus?
 
-* Train and serve models from within your cloud account, on a portable platform. Your data stays private.
+* Zero code needed to deploy and finetune OSS LLMs
+* Ability to deploy and finetune state of the art large language models easily within an hour.
+* Automatically provision all the required infrastructure (K8s cluster, buckets etc) in your cloud account using Terraform.
+* Scale from 0 and spot VMs supported by default (GPUs $$$).
+* Your data stays private and won't be used by the corporate overlords
 * Leverage containers to avoid library lock-in and dependency wrangling.
-* Let substratus calculate your resource requirements and automatically provision GPUs, CPUs, Storage, and Memory.
 * Adopt best practice conventions by default.
-* Train pre-packaged state of the art models on your own datasets.
 * Leverage GitOps out of the box.
 
 ## One Minute Intro Video

docs/quickstart.ipynb

@@ -229,54 +229,46 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": null,
    "id": "90825af1",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
-      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
-      "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0"
-     ]
-    },
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "100   458  100   326  100   132    171     69  0:00:01  0:00:01 --:--:--   241\n",
-      "\u001b[1;39m{\n",
-      "  \u001b[0m\u001b[34;1m\"id\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"cmpl-e42772faf58cd46c18a955f1\"\u001b[0m\u001b[1;39m,\n",
-      "  \u001b[0m\u001b[34;1m\"object\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"text_completion\"\u001b[0m\u001b[1;39m,\n",
-      "  \u001b[0m\u001b[34;1m\"created\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m1689485483\u001b[0m\u001b[1;39m,\n",
-      "  \u001b[0m\u001b[34;1m\"model\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"falcon-7b-instruct\"\u001b[0m\u001b[1;39m,\n",
-      "  \u001b[0m\u001b[34;1m\"choices\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m[\n",
-      "    \u001b[1;39m{\n",
-      "      \u001b[0m\u001b[34;1m\"text\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"\\nGeorge Washington was the first president of the United States.\"\u001b[0m\u001b[1;39m,\n",
-      "      \u001b[0m\u001b[34;1m\"index\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m0\u001b[0m\u001b[1;39m,\n",
-      "      \u001b[0m\u001b[34;1m\"logprobs\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;30mnull\u001b[0m\u001b[1;39m,\n",
-      "      \u001b[0m\u001b[34;1m\"finish_reason\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;32m\"length\"\u001b[0m\u001b[1;39m\n",
-      "    \u001b[1;39m}\u001b[0m\u001b[1;39m\n",
-      "  \u001b[1;39m]\u001b[0m\u001b[1;39m,\n",
-      "  \u001b[0m\u001b[34;1m\"usage\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[1;39m{\n",
-      "    \u001b[0m\u001b[34;1m\"prompt_tokens\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m11\u001b[0m\u001b[1;39m,\n",
-      "    \u001b[0m\u001b[34;1m\"completion_tokens\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m12\u001b[0m\u001b[1;39m,\n",
-      "    \u001b[0m\u001b[34;1m\"total_tokens\"\u001b[0m\u001b[1;39m: \u001b[0m\u001b[0;39m23\u001b[0m\u001b[1;39m\n",
-      "  \u001b[1;39m}\u001b[0m\u001b[1;39m\n",
-      "\u001b[1;39m}\u001b[0m\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "! curl http://localhost:8080/v1/completions \\\n",
     "  -H \"Content-Type: application/json\" \\\n",
     "  -d '{ \\\n",
     "    \"model\": \"falcon-7b-instruct\", \\\n",
     "    \"prompt\": \"Who was the first president of the United States? \", \\\n",
-    "    \"max_tokens\": 12\\\n",
-    "  }' | jq"
+    "    \"max_tokens\": 10\\\n",
+    "  }' "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "59cdbd6c",
+   "metadata": {},
+   "source": [
+    "```json\n",
+    "{\n",
+    "  \"id\": \"cmpl-e42772faf58cd46c18a955f1\",\n",
+    "  \"object\": \"text_completion\",\n",
+    "  \"created\": 1689485483,\n",
+    "  \"model\": \"falcon-7b-instruct\",\n",
+    "  \"choices\": [\n",
+    "    {\n",
+    "      \"text\": \"\\nGeorge Washington was the first president of the United States.\",\n",
+    "      \"index\": 0,\n",
+    "      \"logprobs\": null,\n",
+    "      \"finish_reason\": \"length\"\n",
+    "    }\n",
+    "  ],\n",
+    "  \"usage\": {\n",
+    "    \"prompt_tokens\": 11,\n",
+    "    \"completion_tokens\": 12,\n",
+    "    \"total_tokens\": 23\n",
+    "  }\n",
+    "}\n",
+    "```"
    ]
   },
   {
@@ -285,8 +277,7 @@
    "id": "95a9966d-5679-4208-9b28-0323aa80cd79",
    "metadata": {},
    "source": [
-    "If you are interested in continuing your journey through Substratus, take a look at the [Guided Walkthrough](./category/walkthrough) or\n",
-    "follow the [tutorial to finetune falcon-7b-instruct](./tutorials/deploying-finetuning-falcon-7b-instruct.md) with a custom dataset."
+    "If you are interested in continuing your journey through Substratus, take a look at the [Guided Walkthrough](./category/walkthrough) to learn how to finetune models with your own dataset and much more!"
    ]
   },
   {

docs/quickstart.md

@@ -103,36 +103,33 @@ All substratus Servers ship with an API and interactive frontend. Open up your b
   -d '{ \
     "model": "falcon-7b-instruct", \
     "prompt": "Who was the first president of the United States? ", \
-    "max_tokens": 12\
-  }' | jq
+    "max_tokens": 10\
+  }' 
 ```
 
-      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
-                                     Dload  Upload   Total   Spent    Left  Speed
-      0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0100   458  100   326  100   132    171     69  0:00:01  0:00:01 --:--:--   241
-    {
-      "id": "cmpl-e42772faf58cd46c18a955f1",
-      "object": "text_completion",
-      "created": 1689485483,
-      "model": "falcon-7b-instruct",
-      "choices": [
-        {
-          "text": "\nGeorge Washington was the first president of the United States.",
-          "index": 0,
-          "logprobs": null,
-          "finish_reason": "length"
-        }
-      ],
-      "usage": {
-        "prompt_tokens": 11,
-        "completion_tokens": 12,
-        "total_tokens": 23
-      }
-    }
-
-
-If you are interested in continuing your journey through Substratus, take a look at the [Guided Walkthrough](./category/walkthrough) or
-follow the [tutorial to finetune falcon-7b-instruct](./tutorials/deploying-finetuning-falcon-7b-instruct.md) with a custom dataset.
+```json
+{
+  "id": "cmpl-e42772faf58cd46c18a955f1",
+  "object": "text_completion",
+  "created": 1689485483,
+  "model": "falcon-7b-instruct",
+  "choices": [
+    {
+      "text": "\nGeorge Washington was the first president of the United States.",
+      "index": 0,
+      "logprobs": null,
+      "finish_reason": "length"
+    }
+  ],
+  "usage": {
+    "prompt_tokens": 11,
+    "completion_tokens": 12,
+    "total_tokens": 23
+  }
+}
+```
+
+If you are interested in continuing your journey through Substratus, take a look at the [Guided Walkthrough](./category/walkthrough) to learn how to finetune models with your own dataset and much more!
 
 ## Cleanup