sd3.5-large

Serving Stable Diffusion 3.5 Large with BentoML

Stable Diffusion 3.5 Large is a Multimodal Diffusion Transformer (MMDiT) text-to-image model that features improved performance in image quality, typography, complex prompt understanding, and resource-efficiency.

This is a BentoML example project, demonstrating how to build an image generation inference API server, using the Stable Diffusion 3.5 Large model. See here for a full list of BentoML example projects.

Prerequisites

• Accept the conditions to gain access to Stable Diffusion 3.5 Large on Hugging Face.
• To run the Service locally, you need an Nvidia GPU with at least 20G VRAM.

Install dependencies

git clone https://github.com/bentoml/BentoDiffusion.git
cd BentoDiffusion/sd3.5-large
pip install -r requirements.txt

export HF_TOKEN=<your-api-key>

Run the BentoML Service

We have defined a BentoML Service in service.py. Run bentoml serve in your project directory to start the Service.

$ bentoml serve

2024-01-18T18:31:49+0800 [INFO] [cli] Starting production HTTP BentoServer from "service:SD35Large" listening on http://localhost:3000 (Press CTRL+C to quit)
Loading pipeline components...: 100%

The server is now active at http://localhost:3000. You can interact with it using the Swagger UI or in other different ways.

CURL

curl -X 'POST' \
  'http://localhost:3000/txt2img' \
  -H 'accept: image/*' \
  -H 'Content-Type: application/json' \
  -d '{
  "prompt": "A cat holding a sign that says hello world",
  "num_inference_steps": 40,
  "guidance_scale": 4.5
}'

Python client

import bentoml

with bentoml.SyncHTTPClient("http://localhost:3000") as client:
        result = client.txt2img(
            prompt="A cat holding a sign that says hello world",
            num_inference_steps=40,
            guidance_scale=4.5
        )

Deploy to BentoCloud

After the Service is ready, you can deploy the application to BentoCloud for better management and scalability. Sign up if you haven't got a BentoCloud account.

Make sure you have logged in to BentoCloud.

bentoml cloud login

Create a BentoCloud secret to store the required environment variable and reference it for deployment.

bentoml secret create huggingface HF_TOKEN=$HF_TOKEN

bentoml deploy --secret huggingface

Once the application is up and running on BentoCloud, you can access it via the exposed URL.

Note: For custom deployment in your own infrastructure, use BentoML to generate an OCI-compliant image.