Skip to content

Commit 7253be3

Browse files
authored
Update client SDK snippets (#3207)
* Update client SDK snippets * good catch from copilot
1 parent d303c1e commit 7253be3

File tree

3 files changed

+19
-17
lines changed

3 files changed

+19
-17
lines changed

README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
</a>
1515

1616
A Rust, Python and gRPC server for text generation inference. Used in production at [Hugging Face](https://huggingface.co)
17-
to power Hugging Chat, the Inference API and Inference Endpoint.
17+
to power Hugging Chat, the Inference API and Inference Endpoints.
1818

1919
</div>
2020

docs/source/basic_tutorials/visual_language_models.md

+8-8
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ To infer with vision language models through Python, you can use the [`huggingfa
2222
```python
2323
from huggingface_hub import InferenceClient
2424

25-
client = InferenceClient("http://127.0.0.1:3000")
25+
client = InferenceClient(base_url="http://127.0.0.1:3000")
2626
image = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rabbit.png"
2727
prompt = f"![]({image})What is this a picture of?\n\n"
2828
for token in client.text_generation(prompt, max_new_tokens=16, stream=True):
@@ -37,7 +37,7 @@ import base64
3737
import requests
3838
import io
3939

40-
client = InferenceClient("http://127.0.0.1:3000")
40+
client = InferenceClient(base_url="http://127.0.0.1:3000")
4141

4242
# read image from local file
4343
image_path = "rabbit.png"
@@ -58,7 +58,7 @@ or via the `chat_completion` endpoint:
5858
```python
5959
from huggingface_hub import InferenceClient
6060

61-
client = InferenceClient("http://127.0.0.1:3000")
61+
client = InferenceClient(base_url="http://127.0.0.1:3000")
6262

6363
chat = client.chat_completion(
6464
messages=[
@@ -137,19 +137,19 @@ First, we need to install the `@huggingface/inference` library.
137137
npm install @huggingface/inference
138138
```
139139

140-
If you're using the free Inference API, you can use [Huggingface.js](https://huggingface.co/docs/huggingface.js/inference/README)'s `HfInference`. If you're using inference endpoints, you can use `HfInferenceEndpoint` class to easily interact with the Inference API.
140+
Whether you use Inference Providers (our serverless API), or Inference Endpoints, you can call `InferenceClient`.
141141

142-
We can create a `HfInferenceEndpoint` providing our endpoint URL and We can create a `HfInferenceEndpoint` providing our endpoint URL and [Hugging Face access token](https://huggingface.co/settings/tokens).
142+
We can create a `InferenceClient` providing our endpoint URL and [Hugging Face access token](https://huggingface.co/settings/tokens).
143143

144144
```js
145-
import { HfInferenceEndpoint } from "@huggingface/inference";
145+
import { InferenceClient } from "@huggingface/inference";
146146

147-
const hf = new HfInferenceEndpoint("http://127.0.0.1:3000", "HF_TOKEN");
147+
const client = new InferenceClient('hf_YOUR_TOKEN', { endpointUrl: 'https://YOUR_ENDPOINT.endpoints.huggingface.cloud' });
148148

149149
const prompt =
150150
"![](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rabbit.png)What is this a picture of?\n\n";
151151

152-
const stream = hf.textGenerationStream({
152+
const stream = client.textGenerationStream({
153153
inputs: prompt,
154154
parameters: { max_new_tokens: 16, seed: 42 },
155155
});

docs/source/conceptual/streaming.md

+10-8
Original file line numberDiff line numberDiff line change
@@ -125,24 +125,26 @@ curl localhost:8080/v1/chat/completions \
125125
### Streaming with JavaScript
126126

127127
First, we need to install the `@huggingface/inference` library.
128-
`npm install @huggingface/inference`
129128

130-
If you're using the free Inference API, you can use `HfInference`. If you're using inference endpoints, you can use `HfInferenceEndpoint`.
129+
```bash
130+
npm install @huggingface/inference
131+
```
132+
133+
Whether you use Inference Providers (our serverless API), or Inference Endpoints, you can call `InferenceClient`.
131134

132-
We can create a `HfInferenceEndpoint` providing our endpoint URL and credential.
133135

134136
```js
135-
import { HfInferenceEndpoint } from '@huggingface/inference'
137+
import { InferenceClient } from '@huggingface/inference';
136138

137-
const hf = new HfInferenceEndpoint('https://YOUR_ENDPOINT.endpoints.huggingface.cloud', 'hf_YOUR_TOKEN')
139+
const client = new InferenceClient('hf_YOUR_TOKEN', { endpointUrl: 'https://YOUR_ENDPOINT.endpoints.huggingface.cloud' });
138140

139141
// prompt
140-
const prompt = 'What can you do in Nuremberg, Germany? Give me 3 Tips'
142+
const prompt = 'What can you do in Nuremberg, Germany? Give me 3 Tips';
141143

142-
const stream = hf.textGenerationStream({ inputs: prompt })
144+
const stream = client.textGenerationStream({ inputs: prompt });
143145
for await (const r of stream) {
144146
// yield the generated token
145-
process.stdout.write(r.token.text)
147+
process.stdout.write(r.token.text);
146148
}
147149
```
148150

0 commit comments

Comments
 (0)