diff --git a/docs/.vitepress/en.ts b/docs/.vitepress/en.ts index fed00c01..81af64ce 100644 --- a/docs/.vitepress/en.ts +++ b/docs/.vitepress/en.ts @@ -106,6 +106,10 @@ const side = { text: "Use ComfyUI in Krita", link: "/manual/tutorials/comfyui-for-krita", }, + { + text: "GPU passthrough for PVE", + link: "/manual/tutorials/set-gpu-passthrough", + }, ], }, { diff --git a/docs/.vitepress/zh.ts b/docs/.vitepress/zh.ts index e3fa1038..81b10d57 100644 --- a/docs/.vitepress/zh.ts +++ b/docs/.vitepress/zh.ts @@ -106,6 +106,10 @@ const side = { text: "Krita + ComfyUI 实时绘画", link: "/zh/manual/tutorials/comfyui-for-krita", }, + { + text: "配置 PVE 显卡直通", + link: "/zh/manual/tutorials/set-gpu-passthrough", + }, ], }, { diff --git a/docs/manual/get-started/install-olares-pve.md b/docs/manual/get-started/install-olares-pve.md index 753ef225..d05bc624 100644 --- a/docs/manual/get-started/install-olares-pve.md +++ b/docs/manual/get-started/install-olares-pve.md @@ -26,4 +26,8 @@ In PVE CLI, run the following command: - \ No newline at end of file + + +::: Enbale GPU passthrough for PVE +To enable GPU acceleration for Olares in a PVE virtual machine, you must first configure GPU passthrough on PVE before installing Olares. For detailed steps, see the [tutorial](../tutorials/set-gpu-passthrough.md). +::: \ No newline at end of file diff --git a/docs/manual/tutorials/index.md b/docs/manual/tutorials/index.md index 8c0a78e7..28406773 100644 --- a/docs/manual/tutorials/index.md +++ b/docs/manual/tutorials/index.md @@ -14,3 +14,4 @@ This section contains detailed, hands-on guides to help you accomplish specific ## Setup and scaling - [Install a multi-node Olares cluster](install-olares-multi-node.md) +- [Set up GPU passthrough in PVE](set-gpu-passthrough.md) diff --git a/docs/manual/tutorials/set-gpu-passthrough.md b/docs/manual/tutorials/set-gpu-passthrough.md new file mode 100644 index 00000000..495838c3 --- /dev/null +++ b/docs/manual/tutorials/set-gpu-passthrough.md @@ -0,0 +1,241 @@ +--- +outline: [2, 3] +description: Step-by-step tutorial on how to configure GPU passthrough for PVE and install Olares in the VM with GPU acceleration enabled. +--- + +# Set up GPU passthrough in PVE +GPU passthrough in Proxmox Virtual Environment (PVE) allows virtual machines (VMs) to directly access the physical GPU, enabling hardware-accelerated computing for tasks like hosting local AI models and AI applications. + +This tutorial walks you through the steps to configure GPU passthrough for PVE and install Olares in the VM with GPU acceleration enabled. + +## Objectives + +By the end of this tutorial, you will learn how to: + +- Configure the underlying GPU passthrough environment for PVE, which includes: + - Enable IO Memory Management Unit (IOMMU) + - Bind Virtual Function I/O (VFIO) drivers +- Install the virtual machine system and add PCI GPU devices +- Install Olares in the virtual machine system with GPU enabled + +## Prerequisites +Before proceeding, ensure that your hardware meets the following requirements: +- **CPU**: 4 cores or more, with IOMMU enabled in BIOS + - Intel: VT-d + - AMD: AMD-Vi or IOMMU +- **GPU**: An NVIDIA GPU that supports GPU passthrough +- **RAM**: Recommended 16GB or more +- **Storage**: At least 150GB on SSD if you need to run local AI models + +The configuration used in this tutorial is as follows: +- **PVE Version**: 8.3.2 +- **CPU**: Intel i5-12600KF +- **GPU**: NVIDIA RTX 4060 Ti 16GB +- **VM OS**: Ubuntu 22.04.5 LTS + +## Enabling IOMMU for PVE + +1. In the PVE command line (Shell), execute the following command to edit the GRUB configuration file: + + ```bash + nano /etc/default/grub + ``` + +2. Find `GRUB_CMDLINE_LINUX_DEFAULT="quiet"` and edit it as below: + + ::: code-group + + ```bash [Intel] + GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt" + ``` + + ```bash [AMD] + GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt" + ``` + ::: + + +3. Save and exit, and then execute the following command to update GRUB: + + ```bash + update-grub + ``` + +4. Reboot PVE to apply the changes. + +5. Verify if IOMMU has been successfully enabled: + + ```bash + dmesg | grep -e DMAR -e IOMMU + ``` + + If successful, you should see output similar to the following: + + ::: code-group + + ```bash [Intel] + [0.061644] DMAR: IOMMU enabled + ... + [0.408103] DMAR: Intel(R) Virtualization Technology for Directed I/O + ``` + + ```bash [AMD] + [1.219719] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40 + ``` + ::: + +## Add VFIO modules + +1. In the PVE command line, execute the following command to edit the `modules` file: + + ```bash + nano /etc/modules + ``` + +2. Append the following VFIO modules to the end of the file: + + ```bash + vfio + vfio_iommu_type1 + vfio_pci + vfio_virqfd + ``` + +3. Save and exit the editor. + +## Bind GPU to VFIO + +1. In the PVE command line, execute the following command to query the GPU information: + + ```bash + lspci | grep NVIDIA + + # Example output: + 01:00.0 VGA compatible controller: NVIDIA Corporation AD106 [GeForce RTX 4060 Ti] (rev a1) + 01:00.1 Audio device: NVIDIA Corporation AD106M High Definition Audio Controller (rev a1) + ``` + + Here, the GPU’s PCI address is `01:00`, with two functions listed. + +2. Execute the following command to get the IDs of the GPU on PCI `01:00`: + + ```bash + lspci -n -s 01:00 + + # Example output: + 01:00.0 0300: 10de:2803 (rev a1) + 01:00.1 0403: 10de:22bd (rev a1) + ``` + The corresponding IDs of the GPU are `10de:2803` and `10de:22bd`. + +3. Add the GPU ID information to the VFIO configuration file: + + ```bash + echo "options vfio-pci ids=10de:2803, 10de:22bd" > /etc/modprobe.d/vfio.conf + ``` + +4. Update the initial RAM file system (initramfs) and reboot PVE: + + ```bash + update-initramfs -u + reboot + ``` + +5. In the PVE command line, execute the following command to check if the GPU is bound to the `vfio-pci` driver: + + ```bash + lspci -v + ``` + + If GPU passthrough is successful, the output will show something like: + + ```bash + 01:00.0 VGA compatible controller: NVIDIA Corporation AD106 [GeForce RTX 4060 Ti] (rev a1) (prog-if 00 [VGA controller]) + Subsystem: Gigabyte Technology Co., Ltd AD106 [GeForce RTX 4060 Ti] + Flags: fast devsel, IRQ 255, IOMMU group 11 + ... + Kernel driver in use: vfio-pci + ``` + +## Create the VM + +1. In the PVE management interface, click **Create VM** in the top-right corner. The key configurations for this tutorial are as follows: + - **Operating System** + - ISO: `ubuntu-22.04.5-desktop-amd64.iso` + - **System** + - Machine: `q35` + - BIOS: `OVMF (UEFI)` + - EFI Storage: `local-lvm` + - **Storage** + - Disk size: `150 GB` + - **CPU** + - Category: `x86-64-v3` + - **Memory** + - Memory: `10240 MiB` (Adjust as necessary based on your setup) + + ::: tip Note + - You can leave other configurations at their default values. + - This tutorial verifies the operating system as `Ubuntu 22.04.5-desktop`, and CPU category as `x86-64-v3` or `host`. Using other configurations may cause pass-through failures or application issues. + ::: + +2. Click **Finish** to create the virtual machine. + +## Install and configure the VM system + +1. In the PVE server view menu, double-click the newly created VM to enter the console, and click **Start Now** to install Ubuntu. + +2. After installation completes, reboot the VM, perform the following initial setups in Ubuntu terminal: + + ```bash + sudo apt update + sudo apt install -y curl openssh-server + sudo systemctl enable ssh + sudo systemctl start ssh + ``` + + ::: warning Important + Take note of the virtual machine’s IP address, login credentials, and password. After configuring GPU passthrough, you will not be able to retrieve this information from the PVE console. + ::: + +3. Disable Secure Boot. + + ![Disable Secure Boot](/images/manual/tutorials/pve-bios.png#bordered) + + a. In the VM console, restart the virtual machine and immediately press **ESC** to enter the VM’s BIOS setup. + + b. In BIOS, go to **Device Manager** > **Secure Boot Configuration**. + + c. Press the space bar to uncheck **Attempt Secure Boot**. + + d. Press **F10** to save the configuration and exit BIOS. + + e. Reboot into Ubuntu and then shut down the virtual machine. + +## Add GPU to the VM + +1. In the PVE server view menu, click on the virtual machine to enter the VM details page. +2. Click **Hardware** > **Add** > **PCI Device**. + ![Add PCI](/images/manual/tutorials/pve-add-pci.png#bordered) +3. Select **Raw Device**. From the dropdown list, choose the corresponding GPU by the PCI address (`01:00`). + ![Add GPU](/images/manual/tutorials/pve-add-pci-gpu.png#bordered) +4. In the bottom-right corner, select the **Advanced** options and check **PCI-Express**. +5. Click **Add** to add the GPU to the VM system. + +At this point, you have successfully configured GPU passthrough for your virtual machine. + +## Install Olares + +1. After restarting the VM, log in to it via SSH. + + ::: tip Using SSH to log in + The VM will lose graphical display after GPU passthrough is enabled, so you need to access it via SSH. + ::: + +2. Execute the following command to install Olares: + + ```bash + export LOCAL_GPU_ENABLE=1 \ + && curl -sSfL https://olares.sh | bash - + ``` + +Follow the on-screen prompts to complete the installation and activation. For detailed installation steps, refer to [Install Olares on Linux](../get-started/install-olares-linux.md). \ No newline at end of file diff --git a/docs/public/images/manual/tutorials/pve-add-pci-gpu.png b/docs/public/images/manual/tutorials/pve-add-pci-gpu.png new file mode 100644 index 00000000..d53783e9 Binary files /dev/null and b/docs/public/images/manual/tutorials/pve-add-pci-gpu.png differ diff --git a/docs/public/images/manual/tutorials/pve-add-pci.png b/docs/public/images/manual/tutorials/pve-add-pci.png new file mode 100644 index 00000000..03cf0a15 Binary files /dev/null and b/docs/public/images/manual/tutorials/pve-add-pci.png differ diff --git a/docs/public/images/manual/tutorials/pve-bios.png b/docs/public/images/manual/tutorials/pve-bios.png new file mode 100644 index 00000000..6ba4968b Binary files /dev/null and b/docs/public/images/manual/tutorials/pve-bios.png differ diff --git a/docs/public/images/zh/manual/tutorials/pve-add-pci-gpu.png b/docs/public/images/zh/manual/tutorials/pve-add-pci-gpu.png new file mode 100644 index 00000000..7ce7c2f0 Binary files /dev/null and b/docs/public/images/zh/manual/tutorials/pve-add-pci-gpu.png differ diff --git a/docs/public/images/zh/manual/tutorials/pve-add-pci.png b/docs/public/images/zh/manual/tutorials/pve-add-pci.png new file mode 100644 index 00000000..abc89ebe Binary files /dev/null and b/docs/public/images/zh/manual/tutorials/pve-add-pci.png differ diff --git a/docs/public/images/zh/manual/tutorials/pve-bios.png b/docs/public/images/zh/manual/tutorials/pve-bios.png new file mode 100644 index 00000000..d4c99a83 Binary files /dev/null and b/docs/public/images/zh/manual/tutorials/pve-bios.png differ diff --git a/docs/public/images/zh/manual/tutorials/pve-gpu-create-vm.png b/docs/public/images/zh/manual/tutorials/pve-gpu-create-vm.png new file mode 100644 index 00000000..45fee237 Binary files /dev/null and b/docs/public/images/zh/manual/tutorials/pve-gpu-create-vm.png differ diff --git a/docs/zh/manual/get-started/install-olares-pve.md b/docs/zh/manual/get-started/install-olares-pve.md index d22b67f1..8a926e33 100644 --- a/docs/zh/manual/get-started/install-olares-pve.md +++ b/docs/zh/manual/get-started/install-olares-pve.md @@ -26,4 +26,8 @@ Olares 主要运行在基于 Linux 的系统上。目前以验证可运行的系 - \ No newline at end of file + + +::: 为 PVE 启用显卡直通 +如果你想为 PVE 中的 Olares 启用 GPU 加速功能，需在安装 Olares 前完成 PVE 的 GPU 直通配置。详情请参考[教程](../tutorials/set-gpu-passthrough.md)。 +::: \ No newline at end of file diff --git a/docs/zh/manual/tutorials/index.md b/docs/zh/manual/tutorials/index.md index 5af4e5f3..032aa969 100644 --- a/docs/zh/manual/tutorials/index.md +++ b/docs/zh/manual/tutorials/index.md @@ -13,4 +13,5 @@ ## 部署和扩展 -- [安装多节点 Olares 集群](install-olares-multi-node.md) \ No newline at end of file +- [安装多节点 Olares 集群](install-olares-multi-node.md) +- [为 PVE 设置显卡直通](set-pve-passthrough.md) \ No newline at end of file diff --git a/docs/zh/manual/tutorials/set-gpu-passthrough.md b/docs/zh/manual/tutorials/set-gpu-passthrough.md new file mode 100644 index 00000000..781f178a --- /dev/null +++ b/docs/zh/manual/tutorials/set-gpu-passthrough.md @@ -0,0 +1,239 @@ +--- +outline: [2, 3] +description: 如何在PVE中配置GPU直通并在启用 GPU 加速的虚拟机中安装 Olares。 +--- + +# 为 PVE 设置显卡直通 + +为 Proxmox Virtual Environment (PVE) 启用显卡直通（GPU Passthrough）可以让虚拟机（VM）直接访问物理 GPU，从而实现硬件加速计算，如托管本地 AI 模型、AI 应用等。 + +本文将指导您如何在 PVE 上配置显卡直通，并在启用 GPU 加速的虚拟机中安装 Olares。 + + +## 目标 +通过本教程，你将学习： +- 配置 PVE 显卡直通底层环境： + - 为 PVE 配置 IO 内存管理单元（IOMMU） + - 绑定 VFIO（Virtual Function I/O）驱动 +- 安装虚拟机系统并为其添加 PCI 显卡设备 +- 在虚拟机系统里安装 Olares 并启用 GPU 加速 + +## 开始之前 +在开始操作之前，确保你的硬件符合以下条件： +- CPU: 4 核以上，并在 BIOS 中启用 IOMMU。 + - Intel: VT-d + - AMD：AMD-Vi 或 IOMMU +- 显卡：支持 GPU 直通的 NVIDIA 显卡 +- 内存：建议 16GB 及以上 +- 存储：如果运行本地大模型，建议至少 150GB 的 SSD 空间 + +本教程使用配置如下： +- PVE 版本：8.3.2 +- CPU：Intel i5-12600KF +- 显卡：NVIDIA RTX 4060 Ti 16GB +- 虚拟机系统：Ubuntu 22.04.5 LTS + +## 为 PVE 启用 IOMMU +1. 在 PVE 命令行 (Shell) 界面里，执行以下命令编辑 GRUB 配置文件： + + ```bash + nano /etc/default/grub + ``` + +2. 找到 `GRUB_CMDLINE_LINUX_DEFAULT="quiet"` 并编辑如下： + + ::: code-group + + ```bash [Intel] + GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt" + ``` + + ```bash [AMD] + GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt" + ``` + ::: + +3. 保存并退出编辑模式，执行以下命令以更新 GRUB： + + ```bash + update-grub + ``` + +4. 重启 PVE 主机以使配置生效。 +5. 在 PVE 中执行以下命令以验证 IOMMU 是否启用成功： + + ```bash + dmesg | grep -e DMAR -e IOMMU + ``` + + 如成功启用，你会看到类似以下的输出： + + ::: code-group + + ```bash [Intel] + [0.061644] DMAR: IOMMU enabled + ... + [0.408103] DMAR: Intel(R) Virtualization Technology for Directed I/O + ``` + + ```bash [AMD] + + ```bash + [1.219719] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40 + ``` + ::: + +## 添加 VFIO 模块 + +1. 在 PVE 命令行中，执行以下命令编辑 modules 文件： + + ```bash + nano /etc/modules + ``` + +2. 在文件末尾处添加以下 VFIO 模块： + + ```bash + vfio + vfio_iommu_type1 + vfio_pci + vfio_virqfd + ``` + +3. 保存并退出。 + +## 绑定显卡信息至 VFIO + +1. 在 PVE 命令行中，执行以下命令查询显卡信息： + ```bash {4,6} + lspci |grep NVIDIA + + # 返回示例如下： + 01:00.0 VGA compatible controller: NVIDIA Corporation AD106 [GeForce RTX 4060 Ti] (rev a1) + 01:00.1 Audio device: NVIDIA Corporation AD106M High Definition Audio Controller (rev a1) + ``` + + 此处我们获取到了显卡的 PCI 地址 `01:00`，并列出了该显卡的两个功能。 + +2. 执行以下命令获取该 PCI 上对应的显卡设备 ID： + + ```bash {4,6} + lspci -n -s 01:00 + + # 返回 ID 信息如下: + 01:00.0 0300: 10de:2803 (rev a1) + 01:00.1 0403: 10de:22bd (rev a1) + ``` + 对应设备 ID 分别为 `10de:2803` 与 `10de:22bd`。 + +3. 将显卡 ID 信息添加到 VFIO 配置文件中： + + ```bash + echo "options vfio-pci ids=10de:2803, 10de:22bd" > /etc/modprobe.d/vfio.conf + ``` + +4. 更新 initramfs （初始化内存文件系统）并重启 PVE： + + ```bash + update-initramfs -u + reboot + ``` + +5. 在 PVE 命令行中，运行以下命令以检查显卡是否绑定至 `vfio-pci` 驱动： + + ```bash + lspci -v + ``` + + 如显卡直通成功，则会返回如下显卡信息： + + ```bash + 01:00.0 VGA compatible controller: NVIDIA Corporation AD106 [GeForce RTX 4060 Ti] (rev a1) (prog-if 00 [VGA controller]) + Subsystem: Gigabyte Technology Co., Ltd AD106 [GeForce RTX 4060 Ti] + Flags: fast devsel, IRQ 255, IOMMU group 11 + ... + Kernel driver in use: vfio-pci + ``` + +## 创建虚拟机 + +1. 在 PVE 管理界面右上角点击**创建虚拟机**。本教程关键配置如下： + + - 操作系统 + - ISO: ubuntu-22.04.5-desktop-amd64.iso + - 系统 + - Machine: q35 + - BIOS: OVMF(UEFI) + - EFI 存储：local-lvm + - Storage + - 磁盘大小：150 GB + - CPU + - 类别：x86-64-v3 + - 内存 + - 内存：10240MiB （可根据实际配置设置最大可用内存） + + ::: tip 注意 + - 以上项目之外的配置可使用默认值。 + - 本教程验证的操作系统为 `Ubuntu 22.04.5-desktop`，CPU 类别为 `x86-64-v3` 或 `host`。选择其他配置可能导致直通失败或 AI 应用运行问题。 + ::: + +2. 点击**完成**以创建虚拟机。 + +## 安装并设置虚拟机系统 + +1. 在 PVE 服务器视图里，双击刚创建的虚拟机进入控制台，并点击 **Start Now** 以安装 Ubuntu 系统。 +2. 安装完成后重启虚拟机，然后进入 Linux 系统执行以下初始设置： + + ```bash + sudo apt update + sudo apt install -y curl openssh-server + sudo systemctl enable ssh + sudo systemctl start ssh + ``` + ::: Warning 重要 + 记下虚拟机的 IP 地址、登录账号和密码。配置显卡直通后，你将无法通过 PVE 控制台获取这些信息。 + ::: + +3. 禁用安全启动设置。 + + ![禁用安全启动](/images/zh/manual/tutorials/pve-bios.png#bordered) + + a. 在 PVE 控制台里重启虚拟机，并立即按 **ESC** 键进入虚拟机 BIOS 设置。 + + b. 在 BIOS 中，进入 **Device Manager** > **Secure Boot Configuration**。 + + c. 按空格键取消选中 **Attempt Secure Boot**。 + + d. 按 F10 保存配置并推出 BIOS。 + + e. 重启进入 Ubuntu 系统后，关闭虚拟机。 + +## 为虚拟机添加 PCI 设备 + +1. 在 PVE 服务器视图里，点击虚拟机进入虚拟机详情页。 +2. 点击**硬件** > **添加** > **PCI 设备**。 + ![添加 PCI](/images/zh/manual/tutorials/pve-add-pci.png#bordered) +3. 选择**原始设备**。在下拉列表里，通过 PCI 地址 `01:00` 选取对应的显卡。 + ![添加 GPU](/images/zh/manual/tutorials/pve-add-pci-gpu.png#bordered) +4. 在页面右下方选中**高级**选项，并勾选 **PCI-Express**。 +5. 点击**添加**。 + +至此，你已成功为虚拟机配置了显卡直通。 + +## 安装 Olares + +1. 重启后，通过 SSH 方式登陆虚拟机。 + + ::: tip 使用 SSH 登陆 + 启用 GPU 直通后，虚拟机将失去图形界面显示，因此你需要通过 SSH 访问虚拟机系统。 + ::: + +2. 在命令行中，执行以下指令安装 Olares: + + ```bash + export LOCAL_GPU_ENABLE=1 \ + && curl -sSfL https://olares.sh | bash - + ``` + + 按照命令行提示完成安装，激活。详细安装步骤请参考[在 Linux 上安装 Olares](../get-started/install-olares-linux.md)。 +