Skip to content

Commit 8eaea1a

Browse files
davidgao7davidgao7
committed
test run on one machine succeed, but logically wrong when create cluster
1 parent 20b8ddb commit 8eaea1a

File tree

31 files changed

+791
-227
lines changed

31 files changed

+791
-227
lines changed

‎images/generate-ssh-public-private-keys.png

165 KB
Loading

‎index.html

+12-12
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,17 @@ <h1>davidgao7 blog</h1>
7676

7777
<ul class="posts-list">
7878

79+
<li class="posts-list-item">
80+
<a class="posts-list-item-title" href="/posts/wrong-vllm-production-stack-setup/">Wrong Vllm Production Stack Setup</a>
81+
<span class="posts-list-item-description">
82+
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
83+
May 10, 2025
84+
<span class="posts-list-item-separator">-</span>
85+
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
86+
2 min read
87+
</span>
88+
</li>
89+
7990
<li class="posts-list-item">
8091
<a class="posts-list-item-title" href="/posts/palindrome-linked-list/">Palindrome Linked List</a>
8192
<span class="posts-list-item-description">
@@ -138,7 +149,7 @@ <h1>davidgao7 blog</h1>
138149
Apr 13, 2025
139150
<span class="posts-list-item-separator">-</span>
140151
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
141-
14 min read
152+
15 min read
142153
</span>
143154
</li>
144155

@@ -175,17 +186,6 @@ <h1>davidgao7 blog</h1>
175186
</span>
176187
</li>
177188

178-
<li class="posts-list-item">
179-
<a class="posts-list-item-title" href="/posts/go-syntax-learning/">Go Syntax Learning</a>
180-
<span class="posts-list-item-description">
181-
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
182-
Mar 2, 2025
183-
<span class="posts-list-item-separator">-</span>
184-
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
185-
1 min read
186-
</span>
187-
</li>
188-
189189
</ul>
190190

191191

‎index.xml

+9-2
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,15 @@
66
<description>Recent content on davidgao7 blog</description>
77
<generator>Hugo</generator>
88
<language>en-us</language>
9-
<lastBuildDate>Sat, 03 May 2025 18:38:09 -0400</lastBuildDate>
9+
<lastBuildDate>Sat, 10 May 2025 02:30:34 -0400</lastBuildDate>
1010
<atom:link href="http://localhost:1313/index.xml" rel="self" type="application/rss+xml" />
11+
<item>
12+
<title>Wrong Vllm Production Stack Setup</title>
13+
<link>http://localhost:1313/posts/wrong-vllm-production-stack-setup/</link>
14+
<pubDate>Sat, 10 May 2025 02:30:34 -0400</pubDate>
15+
<guid>http://localhost:1313/posts/wrong-vllm-production-stack-setup/</guid>
16+
<description>&lt;h1 id=&#34;-lessons-learned-from-k3s-deployment-attempt-on-runpod&#34;&gt;📚 Lessons Learned from K3s Deployment Attempt on RunPod&lt;/h1&gt;&#xA;&lt;hr&gt;&#xA;&lt;h2 id=&#34;1-environment--real-server&#34;&gt;1. &lt;strong&gt;Environment ≠ Real Server&lt;/strong&gt;&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;RunPod &amp;ldquo;pods&amp;rdquo; are &lt;strong&gt;containerized environments&lt;/strong&gt;, not real VMs.&lt;/li&gt;&#xA;&lt;li&gt;Critical kernel features (mounting filesystems, &lt;code&gt;/dev/kmsg&lt;/code&gt;, privileged container operations) are &lt;strong&gt;restricted&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Even with root access inside the pod, the host kernel limits remain.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;🧠 Always verify if the environment allows privileged operations before deploying Kubernetes clusters.&lt;/p&gt;&lt;/blockquote&gt;&#xA;&lt;hr&gt;&#xA;&lt;h2 id=&#34;2-k3s-needs-more-than-just-cpu-and-ram&#34;&gt;2. &lt;strong&gt;K3s Needs More Than Just CPU and RAM&lt;/strong&gt;&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&#xA;&lt;p&gt;&lt;strong&gt;K3s&lt;/strong&gt; is lightweight but &lt;strong&gt;still needs&lt;/strong&gt;:&lt;/p&gt;</description>
17+
</item>
1118
<item>
1219
<title>Palindrome Linked List</title>
1320
<link>http://localhost:1313/posts/palindrome-linked-list/</link>
@@ -48,7 +55,7 @@
4855
<link>http://localhost:1313/posts/llm-production/</link>
4956
<pubDate>Sun, 13 Apr 2025 15:13:29 -0400</pubDate>
5057
<guid>http://localhost:1313/posts/llm-production/</guid>
51-
<description>&lt;h1 id=&#34;deploying-llms-in-production&#34;&gt;Deploying LLMs in Production&lt;/h1&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In this post, I will share my experience deploying LLMs in production. I will cover the challenges I faced, the solutions I found, and the lessons I learned along the way.&lt;/p&gt;&lt;/blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;first I managed to find a gpu from runpod to experience one complete conversation(multi-inferences).&lt;/li&gt;&#xA;&lt;li&gt;now I know that if you want to deploy this in production, you not only need multiple gpu like the one&#xA;I&amp;rsquo;m using in the following writing, you also need a system to help you manage resources (i.e. Kubernetes).&#xA;2.1. so they have the solution called &lt;code&gt;vllm production-stack&lt;/code&gt;&#xA;Got to find a cluster with powerful GPUs, OR I could just deploy the stack, since now I&amp;rsquo;ve experienced model inference. But it will be hard to tell if I deploy it successfully or not&amp;hellip; ~(- &amp;lt;_ -)&amp;gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&#xA;&lt;p&gt;&lt;a href=&#34;https://github.com/vllm-project/production-stack/&#34;&gt;Reference repo&lt;/a&gt;&lt;/p&gt;</description>
58+
<description>&lt;h1 id=&#34;deploying-llms-in-a-single-machine&#34;&gt;Deploying LLMs in a single Machine&lt;/h1&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In this post, I will share my experience deploying LLMs in production. I will cover the challenges I faced, the solutions I found, and the lessons I learned along the way.&lt;/p&gt;&lt;/blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;first I managed to find a gpu from runpod to experience one complete conversation(multi-inferences).&lt;/li&gt;&#xA;&lt;li&gt;now I know that if you want to deploy this in production, you not only need multiple gpu like the one&#xA;I&amp;rsquo;m using in the following writing, you also need a system to help you manage resources (i.e. Kubernetes).&#xA;2.1. so they have the solution called &lt;code&gt;vllm production-stack&lt;/code&gt;&#xA;Got to find a cluster with powerful GPUs, OR I could just deploy the stack, since now I&amp;rsquo;ve experienced model inference. But it will be hard to tell if I deploy it successfully or not&amp;hellip; ~(- &amp;lt;_ -)&amp;gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&#xA;&lt;p&gt;&lt;a href=&#34;https://github.com/vllm-project/production-stack/&#34;&gt;Reference repo&lt;/a&gt;&lt;/p&gt;</description>
5259
</item>
5360
<item>
5461
<title>Evolution of Route Planning in Urban Environments</title>

‎page/2/index.html

+11-11
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,17 @@ <h1>davidgao7 blog</h1>
7676

7777
<ul class="posts-list">
7878

79+
<li class="posts-list-item">
80+
<a class="posts-list-item-title" href="/posts/go-syntax-learning/">Go Syntax Learning</a>
81+
<span class="posts-list-item-description">
82+
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
83+
Mar 2, 2025
84+
<span class="posts-list-item-separator">-</span>
85+
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
86+
1 min read
87+
</span>
88+
</li>
89+
7990
<li class="posts-list-item">
8091
<a class="posts-list-item-title" href="/posts/divide-arr-into-arr-with-max-diff/">Divide Arr Into Arrs With Max Diff</a>
8192
<span class="posts-list-item-description">
@@ -175,17 +186,6 @@ <h1>davidgao7 blog</h1>
175186
</span>
176187
</li>
177188

178-
<li class="posts-list-item">
179-
<a class="posts-list-item-title" href="/posts/find-kth-largest/">Find Kth Largest</a>
180-
<span class="posts-list-item-description">
181-
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
182-
Feb 7, 2025
183-
<span class="posts-list-item-separator">-</span>
184-
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
185-
1 min read
186-
</span>
187-
</li>
188-
189189
</ul>
190190

191191

‎page/3/index.html

+11-11
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,17 @@ <h1>davidgao7 blog</h1>
7676

7777
<ul class="posts-list">
7878

79+
<li class="posts-list-item">
80+
<a class="posts-list-item-title" href="/posts/find-kth-largest/">Find Kth Largest</a>
81+
<span class="posts-list-item-description">
82+
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
83+
Feb 7, 2025
84+
<span class="posts-list-item-separator">-</span>
85+
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
86+
1 min read
87+
</span>
88+
</li>
89+
7990
<li class="posts-list-item">
8091
<a class="posts-list-item-title" href="/posts/rob/">Rob</a>
8192
<span class="posts-list-item-description">
@@ -175,17 +186,6 @@ <h1>davidgao7 blog</h1>
175186
</span>
176187
</li>
177188

178-
<li class="posts-list-item">
179-
<a class="posts-list-item-title" href="/posts/shortest-bridge/">Shortest Bridge</a>
180-
<span class="posts-list-item-description">
181-
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
182-
Feb 7, 2025
183-
<span class="posts-list-item-separator">-</span>
184-
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
185-
1 min read
186-
</span>
187-
</li>
188-
189189
</ul>
190190

191191

‎page/4/index.html

+11
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,17 @@ <h1>davidgao7 blog</h1>
7676

7777
<ul class="posts-list">
7878

79+
<li class="posts-list-item">
80+
<a class="posts-list-item-title" href="/posts/shortest-bridge/">Shortest Bridge</a>
81+
<span class="posts-list-item-description">
82+
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
83+
Feb 7, 2025
84+
<span class="posts-list-item-separator">-</span>
85+
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
86+
1 min read
87+
</span>
88+
</li>
89+
7990
<li class="posts-list-item">
8091
<a class="posts-list-item-title" href="/posts/cout-vowel-strings/">Cout Vowel Strings</a>
8192
<span class="posts-list-item-description">

‎posts/20fbd9df-d205-49d7-b632-fd4cbbc786b1-choosing-the-wrong-env-setups/index.html

+99
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
<!doctype html>
2+
<html lang="en-us">
3+
<head><script src="/livereload.js?mindelay=10&amp;v=2&amp;port=1313&amp;path=livereload" data-no-instant defer></script>
4+
<title> // davidgao7 blog</title>
5+
<link rel="shortcut icon" href="/images/favicon.jpg" />
6+
<meta charset="utf-8" />
7+
<meta name="generator" content="Hugo 0.147.2">
8+
<meta name="viewport" content="width=device-width, initial-scale=1" />
9+
<meta name="author" content="David Gao" />
10+
<meta name="description" content="" />
11+
<link rel="stylesheet" href="/css/main.min.5b1fcc8902588589c4767187402a3c29f8b8d7a6fdef6d9f8f77045bb0d14fee.css" />
12+
13+
14+
15+
16+
<meta name="twitter:card" content="summary">
17+
<meta name="twitter:title" content="davidgao7 blog">
18+
<meta name="twitter:description" content="choosing the wrong env setups">
19+
20+
<meta property="og:url" content="http://localhost:1313/posts/20fbd9df-d205-49d7-b632-fd4cbbc786b1-choosing-the-wrong-env-setups/">
21+
<meta property="og:site_name" content="davidgao7 blog">
22+
<meta property="og:title" content="davidgao7 blog">
23+
<meta property="og:description" content="choosing the wrong env setups">
24+
<meta property="og:locale" content="en_us">
25+
<meta property="og:type" content="article">
26+
<meta property="article:section" content="posts">
27+
28+
29+
30+
<script>
31+
window.MathJax = {
32+
tex: {
33+
inlineMath: [['\\(', '\\)']],
34+
displayMath: [['\\[', '\\]'], ['$$', '$$']]
35+
},
36+
svg: {
37+
fontCache: 'global'
38+
}
39+
};
40+
</script>
41+
<script type="text/javascript" async
42+
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/3.2.0/es5/tex-mml-chtml.js">
43+
</script>
44+
45+
46+
47+
48+
</head>
49+
<body>
50+
<header class="app-header">
51+
<a href="/"><img class="app-header-avatar" src="/images/avatar.jpg" alt="David Gao" /></a>
52+
<span class="app-header-title">davidgao7 blog</span>
53+
<nav class="app-header-menu">
54+
<a class="app-header-menu-item" href="/">Home</a>
55+
-
56+
57+
<a class="app-header-menu-item" href="/tags/">Tags</a>
58+
</nav>
59+
<p>Note new findings every day to let the magic happen!</p>
60+
<div class="app-header-social">
61+
62+
<a href="https://github.com/davidgao7" target="_blank" rel="noreferrer noopener me">
63+
<svg class="icon icon-brand-github" viewBox="0 0 24 24" fill="currentColor"><title>GitHub</title><path d="M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12"/></svg>
64+
</a>
65+
66+
<a href="https://x.com/AiiGen71976j" target="_blank" rel="noreferrer noopener me">
67+
<svg class="icon icon-brand-x" viewBox="0 0 24 24" fill="currentColor"><title>X</title><path d="M18.901 1.153h3.68l-8.04 9.19L24 22.846h-7.406l-5.8-7.584-6.638 7.584H.474l8.6-9.83L0 1.154h7.594l5.243 6.932ZM17.61 20.644h2.039L6.486 3.24H4.298Z"/></svg>
68+
</a>
69+
70+
</div>
71+
</header>
72+
<main class="app-container">
73+
74+
<article class="post">
75+
<header class="post-header">
76+
<h1 class ="post-title"></h1>
77+
<div class="post-meta">
78+
<div>
79+
<svg class="icon icon-calendar" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>calendar</title><rect x="3" y="4" width="18" height="18" rx="2" ry="2"></rect><line x1="16" y1="2" x2="16" y2="6"></line><line x1="8" y1="2" x2="8" y2="6"></line><line x1="3" y1="10" x2="21" y2="10"></line></svg>
80+
Jan 1, 0001
81+
</div>
82+
<div>
83+
<svg class="icon icon-clock" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><title>clock</title><circle cx="12" cy="12" r="10"></circle><polyline points="12 6 12 12 16 14"></polyline></svg>
84+
1 min read
85+
</div>
86+
</div>
87+
</header>
88+
<div class="post-content">
89+
<h1 id="choosing-the-wrong-env-setups">choosing the wrong env setups</h1>
90+
91+
</div>
92+
<div class="post-footer">
93+
94+
</div>
95+
</article>
96+
97+
</main>
98+
</body>
99+
</html>

0 commit comments

Comments
 (0)