Skip to content

Commit 80e956d

Browse files
Sma1lboywsxiaoys
andauthored
docs: adding deepinfra model HTTP api use case doc (#3677)
* docs: adding deepinfra model HTTP api use case support * chore: Update website/docs/references/models-http-api/deepinfra.md Co-authored-by: Meng Zhang <[email protected]> * docs: update DeepInfra API documentation for clarity and conciseness --------- Co-authored-by: Meng Zhang <[email protected]>
1 parent e93e28f commit 80e956d

File tree

1 file changed

+39
-0
lines changed

1 file changed

+39
-0
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# DeepInfra
2+
3+
[DeepInfra](https://deepinfra.com/) is a cloud platform providing efficient and scalable model inference services, offering access to various open-source models like [Llama 3](https://deepinfra.com/meta-llama/Llama-3.3-70B-Instruct), [Mixtral](https://deepinfra.com/mistralai/Mixtral-8x7B-Instruct-v0.1), and [Qwen](https://deepinfra.com/Qwen/Qwen2.5-Coder-32B-Instruct).
4+
5+
## Chat model
6+
7+
DeepInfra provides an OpenAI-compatible chat API interface.
8+
9+
```toml title="~/.tabby/config.toml"
10+
[model.chat.http]
11+
kind = "openai/chat"
12+
model_name = "meta-llama/Llama-3.3-70B-Instruct"
13+
api_endpoint = "https://api.deepinfra.com/v1/openai"
14+
api_key = "your-api-key"
15+
```
16+
17+
## Completion model
18+
19+
DeepInfra provides an OpenAI-compatible completion API interface.
20+
21+
```toml title="~/.tabby/config.toml"
22+
[model.completion.http]
23+
kind = "openai/completion"
24+
model_name = "Qwen/Qwen2.5-Coder-32B-Instruct"
25+
api_endpoint = "https://api.deepinfra.com/v1/openai"
26+
api_key = "your-api-key"
27+
```
28+
29+
## Embeddings model
30+
31+
DeepInfra also provides an OpenAI-compatible embeddings API interface.
32+
33+
```toml title="~/.tabby/config.toml"
34+
[model.embedding.http]
35+
kind = "openai/embedding"
36+
model_name = "BAAI/bge-base-en-v1.5"
37+
api_endpoint = "https://api.deepinfra.com/v1/openai"
38+
api_key = "your-api-key"
39+
```

0 commit comments

Comments
 (0)