Skip to content

Commit 5d6b954

Browse files
davidgao7davidgao7
committed
add MLX local LLM finetune tutorial
1 parent f3d7155 commit 5d6b954

File tree

172 files changed

+971
-583
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

172 files changed

+971
-583
lines changed

‎404.html

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,10 @@
11
<!doctype html>
22
<html lang="en-us">
33
<head><script src="/livereload.js?mindelay=10&amp;v=2&amp;port=1313&amp;path=livereload" data-no-instant defer></script>
4-
54
<title>404 Page not found // davidgao7 blog</title>
65
<link rel="shortcut icon" href="/favicon.ico" />
76
<meta charset="utf-8" />
8-
<meta name="generator" content="Hugo 0.145.0">
7+
<meta name="generator" content="Hugo 0.146.5">
98
<meta name="viewport" content="width=device-width, initial-scale=1" />
109
<meta name="author" content="David Gao" />
1110
<meta name="description" content="" />

‎_index_of_content/index.html

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,10 @@
11
<!doctype html>
22
<html lang="en-us">
33
<head><script src="/livereload.js?mindelay=10&amp;v=2&amp;port=1313&amp;path=livereload" data-no-instant defer></script>
4-
54
<title> // davidgao7 blog</title>
65
<link rel="shortcut icon" href="/favicon.ico" />
76
<meta charset="utf-8" />
8-
<meta name="generator" content="Hugo 0.145.0">
7+
<meta name="generator" content="Hugo 0.146.5">
98
<meta name="viewport" content="width=device-width, initial-scale=1" />
109
<meta name="author" content="David Gao" />
1110
<meta name="description" content="" />

‎categories/index.html

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,10 @@
11
<!doctype html>
22
<html lang="en-us">
33
<head><script src="/livereload.js?mindelay=10&amp;v=2&amp;port=1313&amp;path=livereload" data-no-instant defer></script>
4-
54
<title>Categories // davidgao7 blog</title>
65
<link rel="shortcut icon" href="/favicon.ico" />
76
<meta charset="utf-8" />
8-
<meta name="generator" content="Hugo 0.145.0">
7+
<meta name="generator" content="Hugo 0.146.5">
98
<meta name="viewport" content="width=device-width, initial-scale=1" />
109
<meta name="author" content="David Gao" />
1110
<meta name="description" content="" />

‎index.html

Lines changed: 24 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,10 @@
11
<!doctype html>
22
<html lang="en-us">
33
<head><script src="/livereload.js?mindelay=10&amp;v=2&amp;port=1313&amp;path=livereload" data-no-instant defer></script>
4-
54
<title>davidgao7 blog</title>
65
<link rel="shortcut icon" href="/favicon.ico" />
76
<meta charset="utf-8" />
8-
<meta name="generator" content="Hugo 0.145.0">
7+
<meta name="generator" content="Hugo 0.146.5">
98
<meta name="viewport" content="width=device-width, initial-scale=1" />
109
<meta name="author" content="David Gao" />
1110
<meta name="description" content="Note new findings every day to let the magic happen!" />
@@ -54,14 +53,36 @@ <h1>davidgao7 blog</h1>
5453

5554
<ul class="posts-list">
5655

56+
<li class="posts-list-item">
57+
<a class="posts-list-item-title" href="/posts/finetune-llm-locally-using-mlx/">Finetune LLM Locally Using MLX</a>
58+
<span class="posts-list-item-description">
59+
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
60+
Apr 19, 2025
61+
<span class="posts-list-item-separator">-</span>
62+
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
63+
5 min read
64+
</span>
65+
</li>
66+
67+
<li class="posts-list-item">
68+
<a class="posts-list-item-title" href="/posts/llm-production/">LLM Production</a>
69+
<span class="posts-list-item-description">
70+
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
71+
Apr 13, 2025
72+
<span class="posts-list-item-separator">-</span>
73+
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
74+
1 min read
75+
</span>
76+
</li>
77+
5778
<li class="posts-list-item">
5879
<a class="posts-list-item-title" href="/posts/evolution-of-route-planning-in-urban-env/">Evolution of Route Planning in Urban Environments</a>
5980
<span class="posts-list-item-description">
6081
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
6182
Mar 26, 2025
6283
<span class="posts-list-item-separator">-</span>
6384
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
64-
73 min read
85+
75 min read
6586
</span>
6687
</li>
6788

@@ -142,28 +163,6 @@ <h1>davidgao7 blog</h1>
142163
</span>
143164
</li>
144165

145-
<li class="posts-list-item">
146-
<a class="posts-list-item-title" href="/posts/virtual-machine/">Virtual Machine</a>
147-
<span class="posts-list-item-description">
148-
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
149-
Feb 8, 2025
150-
<span class="posts-list-item-separator">-</span>
151-
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
152-
10 min read
153-
</span>
154-
</li>
155-
156-
<li class="posts-list-item">
157-
<a class="posts-list-item-title" href="/posts/tcp-rust/">TCP Rust</a>
158-
<span class="posts-list-item-description">
159-
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
160-
Feb 7, 2025
161-
<span class="posts-list-item-separator">-</span>
162-
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
163-
18 min read
164-
</span>
165-
</li>
166-
167166
</ul>
168167

169168

‎index.xml

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,22 @@
66
<description>Recent content on davidgao7 blog</description>
77
<generator>Hugo</generator>
88
<language>en-us</language>
9-
<lastBuildDate>Wed, 26 Mar 2025 16:41:21 -0400</lastBuildDate>
9+
<lastBuildDate>Sat, 19 Apr 2025 16:59:26 -0400</lastBuildDate>
1010
<atom:link href="http://localhost:1313/index.xml" rel="self" type="application/rss+xml" />
11+
<item>
12+
<title>Finetune LLM Locally Using MLX</title>
13+
<link>http://localhost:1313/posts/finetune-llm-locally-using-mlx/</link>
14+
<pubDate>Sat, 19 Apr 2025 16:59:26 -0400</pubDate>
15+
<guid>http://localhost:1313/posts/finetune-llm-locally-using-mlx/</guid>
16+
<description>&lt;h1 id=&#34;why-this-matter&#34;&gt;Why this matter?&lt;/h1&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I want to make full use of apple silicon&amp;rsquo;s chip, it&amp;rsquo;s a powerhouse&lt;/li&gt;&#xA;&lt;li&gt;can&amp;rsquo;t let the cuda have all the fun&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h1 id=&#34;prerequisite&#34;&gt;Prerequisite&lt;/h1&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a laptop with apple silicon chip&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2 id=&#34;install-python&#34;&gt;Install python&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You can install python with any tools you like, for me I use &lt;code&gt;uv&lt;/code&gt;, a fast and lightweight python version manager built with Rust.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;uv venv --python 3.11.11&#xA;&lt;/span&gt;&lt;/span&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;source .venv/bin/activate&#xA;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ul&gt;&#xA;&lt;li&gt;Here are the necessary python packages in your &lt;code&gt;requirements.txt&lt;/code&gt;:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre tabindex=&#34;0&#34;&gt;&lt;code&gt;mlx&#xA;mlx_lm&#xA;huggingface_hub&#xA;requests&#xA;urllib3&#xA;idna&#xA;certifi&#xA;tqdm&#xA;pyyaml&#xA;filelock&#xA;transformers&#xA;packaging&#xA;torch&#xA;pytz&#xA;datetime&#xA;numpy&#xA;pandas&#xA;jupyter&#xA;ipykernel&#xA;jupyterlab&#xA;typing_extensions&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;uv pip sync requirements.txt&#xA;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ul&gt;&#xA;&lt;li&gt;You can go to jupyter lab by running this command&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;div class=&#34;highlight&#34;&gt;&lt;pre tabindex=&#34;0&#34; style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;&#34;&gt;&lt;code class=&#34;language-bash&#34; data-lang=&#34;bash&#34;&gt;&lt;span style=&#34;display:flex;&#34;&gt;&lt;span&gt;uv run --with jupyter jupyter lab&#xA;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h1 id=&#34;how-to-fine-tune-your-llm&#34;&gt;How to fine tune your LLM&lt;/h1&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&#xA;&lt;p&gt;There are two major part of finetuning your llm&lt;/p&gt;</description>
17+
</item>
18+
<item>
19+
<title>LLM Production</title>
20+
<link>http://localhost:1313/posts/llm-production/</link>
21+
<pubDate>Sun, 13 Apr 2025 15:13:29 -0400</pubDate>
22+
<guid>http://localhost:1313/posts/llm-production/</guid>
23+
<description>&lt;h1 id=&#34;deploying-llms-in-production&#34;&gt;Deploying LLMs in Production&lt;/h1&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In this post, I will share my experience deploying LLMs in production. I will cover the challenges I faced, the solutions I found, and the lessons I learned along the way.&lt;/p&gt;&lt;/blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&#34;https://github.com/vllm-project/production-stack/&#34;&gt; Reference repo &lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&#34;https://github.com/davidgao7/homelab&#34;&gt; Code for people in hurry &lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2 id=&#34;setup-automated-deployments-to-the-aks-cluster--optional-&#34;&gt;Setup automated deployments to the AKS cluster ( optional )&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&#34;https://learn.microsoft.com/en-us/azure/aks/automated-deployments&#34;&gt; Reference &lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;</description>
24+
</item>
1125
<item>
1226
<title>Evolution of Route Planning in Urban Environments</title>
1327
<link>http://localhost:1313/posts/evolution-of-route-planning-in-urban-env/</link>
@@ -202,7 +216,7 @@
202216
<link>http://localhost:1313/posts/_index_of_posts/</link>
203217
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
204218
<guid>http://localhost:1313/posts/_index_of_posts/</guid>
205-
<description>&lt;p&gt;%% Zoottelkeeper: Beginning of the autogenerated index file list %%&#xA;[[davidgao7blogs/content/posts/array-summary|array-summary]]&#xA;[[davidgao7blogs/content/posts/binary-tree-path-sum|binary-tree-path-sum]]&#xA;[[davidgao7blogs/content/posts/binaryTree-Output-levely|binaryTree-Output-levely]]&#xA;[[davidgao7blogs/content/posts/copy-random-list|copy-random-list]]&#xA;[[davidgao7blogs/content/posts/cout-vowel-strings|cout-vowel-strings]]&#xA;[[davidgao7blogs/content/posts/divide-arr-into-arr-with-max-diff|divide-arr-into-arr-with-max-diff]]&#xA;[[davidgao7blogs/content/posts/find-kth-largest|find-kth-largest]]&#xA;[[davidgao7blogs/content/posts/go-syntax-learning|go-syntax-learning]]&#xA;[[davidgao7blogs/content/posts/grafana-prometheus|grafana-prometheus]]&#xA;[[davidgao7blogs/content/posts/heap-bst|heap-bst]]&#xA;[[davidgao7blogs/content/posts/jump-game|jump-game]]&#xA;[[davidgao7blogs/content/posts/local-rag-ollama|local-rag-ollama]]&#xA;[[davidgao7blogs/content/posts/lowest-common-ancestor|lowest-common-ancestor]]&#xA;[[davidgao7blogs/content/posts/machine-learning-pipeline|machine-learning-pipeline]]&#xA;[[davidgao7blogs/content/posts/maxLength|maxLength]]&#xA;[[davidgao7blogs/content/posts/merge-k-sorted-lists|merge-k-sorted-lists]]&#xA;[[davidgao7blogs/content/posts/minimum-length-encoding|minimum-length-encoding]]&#xA;[[davidgao7blogs/content/posts/rob|rob]]&#xA;[[davidgao7blogs/content/posts/shortest-bridge|shortest-bridge]]&#xA;[[davidgao7blogs/content/posts/SVM|SVM]]&#xA;[[davidgao7blogs/content/posts/symmetric-tree|symmetric-tree]]&#xA;[[davidgao7blogs/content/posts/TCP-rust|TCP-rust]]&#xA;[[davidgao7blogs/content/posts/valid-word-abbr|valid-word-abbr]]&#xA;[[davidgao7blogs/content/posts/virtual-machine|virtual-machine]]&#xA;%% Zoottelkeeper: End of the autogenerated index file list %%&lt;/p&gt;</description>
219+
<description>&lt;p&gt;%% Zoottelkeeper: Beginning of the autogenerated index file list %%&#xA;[[davidgao7blogs/content/posts/array-summary|array-summary]]&#xA;[[davidgao7blogs/content/posts/binary-tree-path-sum|binary-tree-path-sum]]&#xA;[[davidgao7blogs/content/posts/binaryTree-Output-levely|binaryTree-Output-levely]]&#xA;[[davidgao7blogs/content/posts/copy-random-list|copy-random-list]]&#xA;[[davidgao7blogs/content/posts/cout-vowel-strings|cout-vowel-strings]]&#xA;[[davidgao7blogs/content/posts/divide-arr-into-arr-with-max-diff|divide-arr-into-arr-with-max-diff]]&#xA;[[davidgao7blogs/content/posts/evolution-of-route-planning-in-urban-env|evolution-of-route-planning-in-urban-env]]&#xA;[[davidgao7blogs/content/posts/find-kth-largest|find-kth-largest]]&#xA;[[davidgao7blogs/content/posts/Finetune-LLM-Locally-using-MLX|Finetune-LLM-Locally-using-MLX]]&#xA;[[davidgao7blogs/content/posts/go-syntax-learning|go-syntax-learning]]&#xA;[[davidgao7blogs/content/posts/grafana-prometheus|grafana-prometheus]]&#xA;[[davidgao7blogs/content/posts/heap-bst|heap-bst]]&#xA;[[davidgao7blogs/content/posts/how-to-build-n8n-agent-from-scratch|how-to-build-n8n-agent-from-scratch]]&#xA;[[davidgao7blogs/content/posts/jump-game|jump-game]]&#xA;[[davidgao7blogs/content/posts/LLM-Production|LLM-Production]]&#xA;[[davidgao7blogs/content/posts/local-rag-ollama|local-rag-ollama]]&#xA;[[davidgao7blogs/content/posts/lowest-common-ancestor|lowest-common-ancestor]]&#xA;[[davidgao7blogs/content/posts/machine-learning-pipeline|machine-learning-pipeline]]&#xA;[[davidgao7blogs/content/posts/maxLength|maxLength]]&#xA;[[davidgao7blogs/content/posts/merge-k-sorted-lists|merge-k-sorted-lists]]&#xA;[[davidgao7blogs/content/posts/minimum-length-encoding|minimum-length-encoding]]&#xA;[[davidgao7blogs/content/posts/rob|rob]]&#xA;[[davidgao7blogs/content/posts/shortest-bridge|shortest-bridge]]&#xA;[[davidgao7blogs/content/posts/SVM|SVM]]&#xA;[[davidgao7blogs/content/posts/symmetric-tree|symmetric-tree]]&#xA;[[davidgao7blogs/content/posts/TCP-rust|TCP-rust]]&#xA;[[davidgao7blogs/content/posts/valid-word-abbr|valid-word-abbr]]&#xA;[[davidgao7blogs/content/posts/virtual-machine|virtual-machine]]&#xA;%% Zoottelkeeper: End of the autogenerated index file list %%&lt;/p&gt;</description>
206220
</item>
207221
</channel>
208222
</rss>

0 commit comments

Comments
 (0)