Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

repeated "content-type" in headers when send https request to models api #4089

Open
chench53 opened this issue Mar 31, 2025 · 1 comment
Open
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@chench53
Copy link

chench53 commented Mar 31, 2025

Describe the bug
When sending a request to the api point of a model, the "content-type" will be repeated twice, like this following:

POST /compatible-mode/v1/embeddings HTTP/1.1
content-type: application/json
content-type: application/json
authorization: Bearer sk-abc
accept: /
host: dashscope.aliyuncs.com
content-length: 53

{"input":["hello Tabby"],"model":"text-embedding-v3"}

This may cause the model http api providers return an error.

Information about your version
tabby 0.26.0

Information about your GPU
(I use remote model http api, so I think it is irrelevant)
NVIDIA-SMI 470.141.03 Driver Version: 470.141.03 CUDA Version: 11.4

Additional context
Some model http api providers may require very strict http schema, even they claim the apis are compatible with openai.
This could happen when use alibaba cloud model studio. I found the alibaba embedding model api keep returning errors. After remove the redundant content-type in requests, the api works fine.

In this code snippet in file /crates/http-api-bindings/src/embedding/openai.rs:

        let request_builder = self
            .client
            .post(&self.api_endpoint)
            .json(&request)
            .header("content-type", "application/json")
            .bearer_auth(&self.api_key);

I don't think the line .header("content-type", "application/json") is necessary, because the header "content-type" is already set to "application/json" by the previous .json(&request), Check this json function in reqwest:

    pub fn json<T: Serialize + ?Sized>(mut self, json: &T) -> RequestBuilder {
        let mut error = None;
        if let Ok(ref mut req) = self.request {
            match serde_json::to_vec(json) {
                Ok(body) => {
                    if !req.headers().contains_key(CONTENT_TYPE) {
                        req.headers_mut()
                            .insert(CONTENT_TYPE, HeaderValue::from_static("application/json"));
                    }
                    *req.body_mut() = Some(body.into());
                }
                Err(err) => error = Some(crate::error::builder(err)),
            }
        }
        if let Some(err) = error {
            self.request = Err(err);
        }
        self
    }

Or you can put .header("content-type", "application/json") before the .json function.

Maybe it exist for some compatibility reasons?

@wsxiaoys
Copy link
Member

Thank you for reporting the bug. It appears that the simplest solution is to remove the line .header("content-type", "application/json")

@wsxiaoys wsxiaoys added bug Something isn't working good first issue Good for newcomers and removed bug-unconfirmed labels Mar 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants