You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This guide explains how to launch Tabby using docker.
@@ -13,14 +15,45 @@ import TabItem from '@theme/TabItem';
13
15
<TabItemvalue="cuda"label="CUDA"default>
14
16
15
17
16
-
For CUDA support in Tabby, install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html).
17
-
After installation, you can start Tabby with the following command:
18
+
To run Tabby with CUDA support in Docker, please install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html).
19
+
Once installed, you can launch Tabby using the command below:
18
20
19
21
```bash title="run.sh"
20
-
docker run -it --gpus all \
21
-
-p 8080:8080 -v $HOME/.tabby:/data \
22
+
docker run -d \
23
+
--name tabby \
24
+
--gpus all \
25
+
-p 8080:8080 \
26
+
-v $HOME/.tabby:/data \
27
+
registry.tabbyml.com/tabbyml/tabby \
28
+
serve \
29
+
--model StarCoder-1B \
30
+
--chat-model Qwen2-1.5B-Instruct \
31
+
--device cuda
32
+
```
33
+
34
+
<Collapsetitle="For the systems with SELinux enabled, you may need to add the `:Z` option to the volume mount">
35
+
36
+
```bash title="run.sh"
37
+
docker run -d \
38
+
--name tabby \
39
+
--gpus all \
40
+
-p 8080:8080 \
41
+
-v $HOME/.tabby:/data:Z \
22
42
registry.tabbyml.com/tabbyml/tabby \
23
-
serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct --device cuda
43
+
serve \
44
+
--model StarCoder-1B \
45
+
--chat-model Qwen2-1.5B-Instruct \
46
+
--device cuda
47
+
```
48
+
49
+
</Collapse>
50
+
51
+
After Tabby is running, you can access it at [http://localhost:8080](http://localhost:8080).
52
+
53
+
To view the logs, you can use the following command:
0 commit comments